Validation of a Self-Administered Computerized System to Detect Cognitive Impairment in Older Adults

PubMed Central

Brinkman, Samuel D.; Reese, Robert J.; Norsworthy, Larry A.; Dellaria, Donna K.; Kinkade, Jacob W.; Benge, Jared; Brown, Kimberly; Ratka, Anna; Simpkins, James W.

2015-01-01

There is increasing interest in the development of economical and accurate approaches to identifying persons in the community who have mild, undetected cognitive impairments. Computerized assessment systems have been suggested as a viable approach to identifying these persons. The validity of a computerized assessment system for identification of memory and executive deficits in older individuals was evaluated in the current study. Volunteers (N = 235) completed a 3-hr battery of neuropsychological tests and a computerized cognitive assessment system. Participants were classified as impaired (n = 78) or unimpaired (n = 157) on the basis of the Mini Mental State Exam, Wechsler Memory Scale-III and the Trail Making Test (TMT), Part B. All six variables (three memory variables and three executive variables) derived from the computerized assessment differed significantly between groups in the expected direction. There was also evidence of temporal stability and concurrent validity. Application of computerized assessment systems for clinical practice and for identification of research participants is discussed in this article. PMID:25332303

The Use of Authentic Assessment to Report Accountability Data on Young Children's Language, Literacy and Pre-Math Competency

ERIC Educational Resources Information Center

Gao, Xin; Grisham-Brown, Jennifer

2011-01-01

This validity study examined the validity of Assessment, Evaluation, and Programming System, 2nd Edition (AEPS®), a curriculum-based, authentic assessment for infants and young children. The primary purposes were to: a) examine whether the AEPS® is a concurrently valid tool for measuring young children's language, literacy and pre-math skills for…

Development and validation of a composite scoring system for robot-assisted surgical training--the Robotic Skills Assessment Score.

PubMed

Chowriappa, Ashirwad J; Shi, Yi; Raza, Syed Johar; Ahmed, Kamran; Stegemann, Andrew; Wilding, Gregory; Kaouk, Jihad; Peabody, James O; Menon, Mani; Hassett, James M; Kesavadas, Thenkurussi; Guru, Khurshid A

2013-12-01

A standardized scoring system does not exist in virtual reality-based assessment metrics to describe safe and crucial surgical skills in robot-assisted surgery. This study aims to develop an assessment score along with its construct validation. All subjects performed key tasks on previously validated Fundamental Skills of Robotic Surgery curriculum, which were recorded, and metrics were stored. After an expert consensus for the purpose of content validation (Delphi), critical safety determining procedural steps were identified from the Fundamental Skills of Robotic Surgery curriculum and a hierarchical task decomposition of multiple parameters using a variety of metrics was used to develop Robotic Skills Assessment Score (RSA-Score). Robotic Skills Assessment mainly focuses on safety in operative field, critical error, economy, bimanual dexterity, and time. Following, the RSA-Score was further evaluated for construct validation and feasibility. Spearman correlation tests performed between tasks using the RSA-Scores indicate no cross correlation. Wilcoxon rank sum tests were performed between the two groups. The proposed RSA-Score was evaluated on non-robotic surgeons (n = 15) and on expert-robotic surgeons (n = 12). The expert group demonstrated significantly better performance on all four tasks in comparison to the novice group. Validation of the RSA-Score in this study was carried out on the Robotic Surgical Simulator. The RSA-Score is a valid scoring system that could be incorporated in any virtual reality-based surgical simulator to achieve standardized assessment of fundamental surgical tents during robot-assisted surgery. Copyright © 2013 Elsevier Inc. All rights reserved.

The Communication Function Classification System: cultural adaptation, validity, and reliability of the Farsi version for patients with cerebral palsy.

PubMed

Soleymani, Zahra; Joveini, Ghodsiye; Baghestani, Ahmad Reza

2015-03-01

This study developed a Farsi language Communication Function Classification System and then tested its reliability and validity. Communication Function Classification System is designed to classify the communication functions of individuals with cerebral palsy. Up until now, there has been no instrument for assessment of this communication function in Iran. The English Communication Function Classification System was translated into Farsi and cross-culturally modified by a panel of experts. Professionals and parents then assessed the content validity of the modified version. A backtranslation of the Farsi version was confirmed by the developer of the English Communication Function Classification System. Face validity was assessed by therapists and parents of 10 patients. The Farsi Communication Function Classification System was administered to 152 individuals with cerebral palsy (age, 2 to 18 years; median age, 10 years; mean age, 9.9 years; standard deviation, 4.3 years). Inter-rater reliability was analyzed between parents, occupational therapists, and speech and language pathologists. The test-retest reliability was assessed for 75 patients with a 14 day interval between tests. The inter-rater reliability of the Communication Function Classification System was 0.81 between speech and language pathologists and occupational therapists, 0.74 between parents and occupational therapists, and 0.88 between parents and speech and language pathologists. The test-retest reliability was 0.96 for occupational therapists, 0.98 for speech and language pathologists, and 0.94 for parents. The findings suggest that the Farsi version of Communication Function Classification System is a reliable and valid measure that can be used in clinical settings to assess communication function in patients with cerebral palsy. Copyright © 2015 Elsevier Inc. All rights reserved.

Elbow-specific clinical rating systems: extent of established validity, reliability, and responsiveness.

PubMed

The, Bertram; Reininga, Inge H F; El Moumni, Mostafa; Eygendaal, Denise

2013-10-01

The modern standard of evaluating treatment results includes the use of rating systems. Elbow-specific rating systems are frequently used in studies aiming at elbow-specific pathology. However, proper validation studies seem to be relatively sparse. In addition, these scoring systems might not always be used for appropriate populations of interest. Both of these issues might give rise to invalid conclusions being reported in the literature. Our aim was to investigate the extent to which the available elbow-specific outcome measurement tools have been validated and the quality of the validation itself. We also aimed to provide characteristics of the populations used for validation of these scales to enable clinicians to use them appropriately. A literature search identified 17 studies of 12 different elbow-specific scoring systems. These were assessed for validity, reliability, and responsiveness characteristics. The quality of these assessments was rated according to the Consensus Based Standards for the Selection of Health Measurement Instruments (COSMIN) checklist criteria, a standardized and validated tool developed specifically for this purpose. Currently, the only elbow-specific rating system that is validated using high-quality methodology is the Oxford Elbow Score, a patient-administered outcome measure tool that has been validated on heterogeneous study populations. Other rating systems still have to be proven in the future to be as good as the Oxford Elbow Score for clinical or research purposes. Additional validation studies are needed. Copyright © 2013 Journal of Shoulder and Elbow Surgery Board of Trustees. Published by Mosby, Inc. All rights reserved.

Reliability of human-supervised formant-trajectory measurement for forensic voice comparison.

PubMed

Zhang, Cuiling; Morrison, Geoffrey Stewart; Ochoa, Felipe; Enzinger, Ewald

2013-01-01

Acoustic-phonetic approaches to forensic voice comparison often include human-supervised measurement of vowel formants, but the reliability of such measurements is a matter of concern. This study assesses the within- and between-supervisor variability of three sets of formant-trajectory measurements made by each of four human supervisors. It also assesses the validity and reliability of forensic-voice-comparison systems based on these measurements. Each supervisor's formant-trajectory system was fused with a baseline mel-frequency cepstral-coefficient system, and performance was assessed relative to the baseline system. Substantial improvements in validity were found for all supervisors' systems, but some supervisors' systems were more reliable than others.

A Validation of the Classroom Assessment Scoring System in Finnish Kindergartens

ERIC Educational Resources Information Center

Pakarinen, Eija; Lerkkanen, Marja-Kristiina; Poikkeus, Anna-Maija; Kiuru, Noona; Siekkinen, Martti; Rasku-Puttonen, Helena; Nurmi, Jari-Erik

2010-01-01

Research Findings: This study examined the validity and reliability of the Classroom Assessment Scoring System (CLASS; R. C. Pianta, K. M. La Paro, & B. K. Hamre, 2008) in Finnish kindergartens. A pair of trained observers used the CLASS to observe 49 kindergarten teachers (47 female, 2 male) on two different days. Questionnaires measuring…

Meaningful Understanding and Systems Thinking in Organic Chemistry: Validating Measurement and Exploring Relationships

ERIC Educational Resources Information Center

Vachliotis, Theodoros; Salta, Katerina; Tzougraki, Chryssa

2014-01-01

The purpose of this study was dual: First, to develop and validate assessment schemes for assessing 11th grade students' meaningful understanding of organic chemistry concepts, as well as their systems thinking skills in the domain. Second, to explore the relationship between the two constructs of interest based on students' performance…

Validating the Use of pPerformance Risk Indices for System-Level Risk and Maturity Assessments

NASA Astrophysics Data System (ADS)

Holloman, Sherrica S.

With pressure on the U.S. Defense Acquisition System (DAS) to reduce cost overruns and schedule delays, system engineers' performance is only as good as their tools. Recent literature details a need for 1) objective, analytical risk quantification methodologies over traditional subjective qualitative methods -- such as, expert judgment, and 2) mathematically rigorous system-level maturity assessments. The Mahafza, Componation, and Tippett (2005) Technology Performance Risk Index (TPRI) ties the assessment of technical performance to the quantification of risk of unmet performance; however, it is structured for component- level data as input. This study's aim is to establish a modified TPRI with systems-level data as model input, and then validate the modified index with actual system-level data from the Department of Defense's (DoD) Major Defense Acquisition Programs (MDAPs). This work's contribution is the establishment and validation of the System-level Performance Risk Index (SPRI). With the introduction of the SPRI, system-level metrics are better aligned, allowing for better assessment, tradeoff and balance of time, performance and cost constraints. This will allow system engineers and program managers to ultimately make better-informed system-level technical decisions throughout the development phase.

Development and feasibility of the misuse, abuse, and diversion drug event reporting system (MADDERS®).

PubMed

Treister, Roi; Trudeau, Jeremiah J; Van Inwegen, Richard; Jones, Judith K; Katz, Nathaniel P

2016-12-01

Inappropriate use of analgesic drugs has become increasingly pervasive over the past decade. Currently, drug abuse potential is primarily assessed post-marketing; no validated tools are available to assess this potential in phase II and III clinical trials. This paper describes the development and feasibility testing of a Misuse, Abuse, and Diversion Drug Event Reporting System (MADDERS), which aims to identify potentially abuse-related events and classify them according to a recently developed classification scheme, allowing the quantification of these events in clinical trials. The system was initially conceived and designed with input from experts and patients, followed by field-testing to assess its feasibility and content validity in both completed and ongoing clinical trials. The results suggest that MADDERS is a feasible system with initial validity. It showed higher rates of the triggering events in subjects taking medications with known abuse potential than in patients taking medications without abuse potential. Additionally, experts agreed on the classification of most abuse-related events in MADDERS. MADDERS is a new systematic approach to collect information on potentially abuse-related events in clinical trials and classify them. The system has demonstrated feasibility for implementation. Additional research is ongoing to further evaluate its validity. Currently, there are no validated tools to assess drug abuse potential during clinical trials. Because of its ease of implementation, its systematic approach, and its preliminary validation results, MADDERS could provide such a tool for clinical trials. (Am J Addict 2016;25:641-651). © 2016 American Academy of Addiction Psychiatry.

Concurrent Validity of the Classroom Strategies Scale-Teacher Form: A Preliminary Investigation

ERIC Educational Resources Information Center

Reddy, Linda A.; Dudek, Christopher M.; Rualo, Angelique J.; Fabiano, Gregory A.

2016-01-01

The present study investigated the concurrent validity of the Classroom Strategies Scale-Teacher Form (CSS-T), a multidimensional teacher formative assessment of instructional and behavioral management practices. The CSS-T is compared with the Classroom Assessment Scoring System (CLASS), a well-known teacher assessment of overall classroom…

Is Learner Self-Assessment Reliable and Valid in a Web-Based Portfolio Environment for High School Students?

ERIC Educational Resources Information Center

Chang, Chi-Cheng; Liang, Chaoyun; Chen, Yi-Hui

2013-01-01

This study explored the reliability and validity of Web-based portfolio self-assessment. Participants were 72 senior high school students enrolled in a computer application course. The students created learning portfolios, viewed peers' work, and performed self-assessment on the Web-based portfolio assessment system. The results indicated: 1)…

Predictive Validity of a Student Self-Report Screener of Behavioral and Emotional Risk in an Urban High School

ERIC Educational Resources Information Center

Dowdy, Erin; Harrell-Williams, Leigh; Dever, Bridget V.; Furlong, Michael J.; Moore, Stephanie; Raines, Tara; Kamphaus, Randy W.

2016-01-01

Increasingly, schools are implementing school-based screening for risk of behavioral and emotional problems; hence, foundational evidence supporting the predictive validity of screening instruments is important to assess. This study examined the predictive validity of the Behavior Assessment System for Children-2 Behavioral and Emotional Screening…

Construct Validity of the Behavior Assessment System for Children (BASC) Self-Report of Personality: Evidence from Adolescents Referred to Residential Treatment

ERIC Educational Resources Information Center

Weis, Robert; Smenner, Lindsey

2007-01-01

The authors investigate the construct validity of the Behavior Assessment System for Children Self-Report of Personality (BASC-SRP; Reynolds & Kamphaus, 1998). A sample of 970 adolescents (16-18 years) with histories of disruptive behavior problems and truancy complete the SRP; a subsample of 290 adolescents also completed the Minnesota…

Translation, cultural adaptation and validation into portuguese (Brazil) in Systemic Sclerosis Questionnaire (SySQ).

PubMed

Machado, Roberta Ismael Lacerda; Souto, Lais Medeiros; Freire, Eutilia Andrade Medeiros

2014-01-01

Systemic sclerosis (SSc) is a multisystem disease, autoimmune disorder characterized by a fibroblastic disfunction, with significant impact on quality of life (QoL), measured by instruments or questionnaires that usually were formulated in other languages and in different cultural contexts. Translate into Brazilian Portuguese, cross cultural adaptation and assess the reliability and validity of the Systemic Sclerosis Questionnaire (SySQ). Translation and adaptation: into Portuguese and cross-cultural adaptation was performed in accordance with studies on questionnaire translation methodology into other languages. Reliability: it was analyzed using three interviews with different interviewers, two on the same day (interobserver) and the third within 14 days of the first assessment (intraobserver).Validity was assessed by correlating clinical and quality of life parameters with the domain scores of Sysc. a descriptive analysis of the study sample. Reproducibility was assessed using an intraclass correlation coefficient (ICC). Internal consistency was assessed using Cronbach's alpha coefficient. To assess validity we used Spearman correlation coefficient. Five percent was the level of significance adopted for all statistical tests. In the evaluation of the questionnaires, the results were similar to the original questionnaire, the internal consistency ranging between 0.73 and 0.93 for each item. The interobserver reproducibility was very good for all domains (α = 0.786 to 0.983) and intraobserver agreement was considered very good for general symptoms domain (ICC = 0.916), good for musculoskeletal symptoms domain (ICC = 0.897) and cardiopulmonary domain (ICC = 0.842) and reasonable for gastrointestinal symptoms domain (ICC = 0.686). The Brazilian Portuguese version of SySQ proved to be reproducible and valid for our population, using a recognized methodology for translation and cultural adaptation of questionnaires, as well as to assess the reproducibility and validity.

Five-level emergency triage systems: variation in assessment of validity.

PubMed

Kuriyama, Akira; Urushidani, Seigo; Nakayama, Takeo

2017-11-01

Triage systems are scales developed to rate the degree of urgency among patients who arrive at EDs. A number of different scales are in use; however, the way in which they have been validated is inconsistent. Also, it is difficult to define a surrogate that accurately predicts urgency. This systematic review described reference standards and measures used in previous validation studies of five-level triage systems. We searched PubMed, EMBASE and CINAHL to identify studies that had assessed the validity of five-level triage systems and described the reference standards and measures applied in these studies. Studies were divided into those using criterion validity (reference standards developed by expert panels or triage systems already in use) and those using construct validity (prognosis, costs and resource use). A total of 57 studies examined criterion and construct validity of 14 five-level triage systems. Criterion validity was examined by evaluating (1) agreement between the assigned degree of urgency with objective standard criteria (12 studies), (2) overtriage and undertriage (9 studies) and (3) sensitivity and specificity of triage systems (7 studies). Construct validity was examined by looking at (4) the associations between the assigned degree of urgency and measures gauged in EDs (48 studies) and (5) the associations between the assigned degree of urgency and measures gauged after hospitalisation (13 studies). Particularly, among 46 validation studies of the most commonly used triages (Canadian Triage and Acuity Scale, Emergency Severity Index and Manchester Triage System), 13 and 39 studies examined criterion and construct validity, respectively. Previous studies applied various reference standards and measures to validate five-level triage systems. They either created their own reference standard or used a combination of severity/resource measures. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

NASA Aerospace Flight Battery Systems Program Update

NASA Technical Reports Server (NTRS)

Manzo, Michelle; ODonnell, Patricia

1997-01-01

The objectives of NASA's Aerospace Flight Battery Systems Program is to: develop, maintain and provide tools for the validation and assessment of aerospace battery technologies; accelerate the readiness of technology advances and provide infusion paths for emerging technologies; provide NASA projects with the required database and validation guidelines for technology selection of hardware and processes relating to aerospace batteries; disseminate validation and assessment tools, quality assurance, reliability, and availability information to the NASA and aerospace battery communities; and ensure that safe, reliable batteries are available for NASA's future missions.

Experimental investigations into visual and electronic tooth color measurement.

PubMed

Ratzmann, Anja; Treichel, Anja; Langforth, Gabriele; Gedrange, Tomasz; Welk, Alexander

2011-04-01

The present study aimed to examine the validity of the visual color assessment and an electronic tooth color measurement system by means of Shade Inspector™ in comparison with a gold standard. Additionally, reproducibility of electronic measurements was demonstrated by means of two reference systems. Ceramic specimens of two thicknesses (h=1.6 mm, h=2.6 mm) were used. Three experienced dental technicians using the VITAPAN Classical(®) color scale carried out all visual tests. Validity of the visual assessment and the electronic measurements was confirmed separately for both thicknesses by means of lightness and hue of the VITAPAN Classical(®) color scale. Reproducibility of electronic measurements was confirmed by means of the VITAPAN Classical(®) and 3D-Master(®). The 3D-Master(®) data were calculated according to lightness, hue and chroma. Intraclass correlation coefficient (ICC) was used in assessing validity/reproducibility for lightness and chroma, Kappa statistics were used for hue. A level ≥0.75 was pre-established for ICC and ≥0.60 for the Kappa index. RESULTS OF VISUAL COLOR ASSESSMENT: Validity for lightness was good for both thicknesses; agreement rates for hue were inconsistent. ELECTRONIC MEASUREMENT: Validity for lightness was fair to good, hue values were below 0.60. Reproducibility of lightness was good to very good for both reference systems. Hue values (VITAPAN Classical(®)) for 1.6 mm test specimens were upside, for 2.6 mm below 0.60, Kappa values for 3D-Master(®) were ≥0.60 for all measurements, reproducibility of chroma was very good. Validity was better for visual than for electronic color assessment. Reproducibility of the electronic device by means of the Shade Inspector™ was given for the VITAPAN Classical(®) and 3D-Master(®) systems.

Educational Assessment Using Intelligent Systems. Research Report. ETS RR-08-68

ERIC Educational Resources Information Center

Shute, Valerie J.; Zapata-Rivera, Diego

2008-01-01

Recent advances in educational assessment, cognitive science, and artificial intelligence have made it possible to integrate valid assessment and instruction in the form of modern computer-based intelligent systems. These intelligent systems leverage assessment information that is gathered from various sources (e.g., summative and formative). This…

A Metric-Based Validation Process to Assess the Realism of Synthetic Power Grids

DOE Office of Scientific and Technical Information (OSTI.GOV)

Birchfield, Adam; Schweitzer, Eran; Athari, Mir

Public power system test cases that are of high quality benefit the power systems research community with expanded resources for testing, demonstrating, and cross-validating new innovations. Building synthetic grid models for this purpose is a relatively new problem, for which a challenge is to show that created cases are sufficiently realistic. This paper puts forth a validation process based on a set of metrics observed from actual power system cases. These metrics follow the structure, proportions, and parameters of key power system elements, which can be used in assessing and validating the quality of synthetic power grids. Though wide diversitymore » exists in the characteristics of power systems, the paper focuses on an initial set of common quantitative metrics to capture the distribution of typical values from real power systems. The process is applied to two new public test cases, which are shown to meet the criteria specified in the metrics of this paper.« less

A Metric-Based Validation Process to Assess the Realism of Synthetic Power Grids

DOE PAGES

Birchfield, Adam; Schweitzer, Eran; Athari, Mir; ...

2017-08-19

Public power system test cases that are of high quality benefit the power systems research community with expanded resources for testing, demonstrating, and cross-validating new innovations. Building synthetic grid models for this purpose is a relatively new problem, for which a challenge is to show that created cases are sufficiently realistic. This paper puts forth a validation process based on a set of metrics observed from actual power system cases. These metrics follow the structure, proportions, and parameters of key power system elements, which can be used in assessing and validating the quality of synthetic power grids. Though wide diversitymore » exists in the characteristics of power systems, the paper focuses on an initial set of common quantitative metrics to capture the distribution of typical values from real power systems. The process is applied to two new public test cases, which are shown to meet the criteria specified in the metrics of this paper.« less

A Comprehensive, Multi-modal Evaluation of the Assessment System of an Undergraduate Research Methodology Course: Translating Theory into Practice.

PubMed

Mohammad Abdulghani, Hamza; G Ponnamperuma, Gominda; Ahmad, Farah; Amin, Zubair

2014-03-01

To evaluate assessment system of the 'Research Methodology Course' using utility criteria (i.e. validity, reliability, acceptability, educational impact, and cost-effectiveness). This study demonstrates comprehensive evaluation of assessment system and suggests a framework for similar courses. Qualitative and quantitative methods used for evaluation of the course assessment components (50 MCQ, 3 Short Answer Questions (SAQ) and research project) using the utility criteria. RESULTS of multiple evaluation methods for all the assessment components were collected and interpreted together to arrive at holistic judgments, rather than judgments based on individual methods or individual assessment. Face validity, evaluated using a self-administered questionnaire (response rate-88.7%) disclosed that the students perceived that there was an imbalance in the contents covered by the assessment. This was confirmed by the assessment blueprint. Construct validity was affected by the low correlation between MCQ and SAQ scores (r=0.326). There was a higher correlation between the project and MCQ (r=0.466)/SAQ (r=0.463) scores. Construct validity was also affected by the presence of recall type of MCQs (70%; 35/50), item construction flaws and non-functioning distractors. High discriminating indices (>0.35) were found in MCQs with moderate difficulty indices (0.3-0.7). Reliability of the MCQs was 0.75 which could be improved up to 0.8 by increasing the number of MCQs to at least 70. A positive educational impact was found in the form of the research project assessment driving students to present/publish their work in conferences/peer reviewed journals. Cost per student to complete the course was US$164.50. The multi-modal evaluation of an assessment system is feasible and provides thorough and diagnostic information. Utility of the assessment system could be further improved by modifying the psychometrically inappropriate assessment items.

A Comprehensive, Multi-modal Evaluation of the Assessment System of an Undergraduate Research Methodology Course: Translating Theory into Practice

PubMed Central

Mohammad Abdulghani, Hamza; G. Ponnamperuma, Gominda; Ahmad, Farah; Amin, Zubair

2014-01-01

Objective: To evaluate assessment system of the 'Research Methodology Course' using utility criteria (i.e. validity, reliability, acceptability, educational impact, and cost-effectiveness). This study demonstrates comprehensive evaluation of assessment system and suggests a framework for similar courses. Methods: Qualitative and quantitative methods used for evaluation of the course assessment components (50 MCQ, 3 Short Answer Questions (SAQ) and research project) using the utility criteria. Results of multiple evaluation methods for all the assessment components were collected and interpreted together to arrive at holistic judgments, rather than judgments based on individual methods or individual assessment. Results: Face validity, evaluated using a self-administered questionnaire (response rate-88.7%) disclosed that the students perceived that there was an imbalance in the contents covered by the assessment. This was confirmed by the assessment blueprint. Construct validity was affected by the low correlation between MCQ and SAQ scores (r=0.326). There was a higher correlation between the project and MCQ (r=0.466)/SAQ (r=0.463) scores. Construct validity was also affected by the presence of recall type of MCQs (70%; 35/50), item construction flaws and non-functioning distractors. High discriminating indices (>0.35) were found in MCQs with moderate difficulty indices (0.3-0.7). Reliability of the MCQs was 0.75 which could be improved up to 0.8 by increasing the number of MCQs to at least 70. A positive educational impact was found in the form of the research project assessment driving students to present/publish their work in conferences/peer reviewed journals. Cost per student to complete the course was US$164.50. Conclusions: The multi-modal evaluation of an assessment system is feasible and provides thorough and diagnostic information. Utility of the assessment system could be further improved by modifying the psychometrically inappropriate assessment items. PMID:24772117

System for assessing Aviation's Global Emissions (SAGE). Version 1.5 : validation assessment, model assumptions and uncertainties

DOT National Transportation Integrated Search

2005-09-01

The United States (US) Federal Aviation Administration (FAA) Office of Environment and Energy (AEE) has : developed the System for assessing Aviations Global Emissions (SAGE) with support from the Volpe National : Transportation Systems Center (Vo...

Design and validation of a portable, inexpensive and multi-beam timing light system using the Nintendo Wii hand controllers.

PubMed

Clark, Ross A; Paterson, Kade; Ritchie, Callan; Blundell, Simon; Bryant, Adam L

2011-03-01

Commercial timing light systems (CTLS) provide precise measurement of athletes running velocity, however they are often expensive and difficult to transport. In this study an inexpensive, wireless and portable timing light system was created using the infrared camera in Nintendo Wii hand controllers (NWHC). System creation with gold-standard validation. A Windows-based software program using NWHC to replicate a dual-beam timing gate was created. Firstly, data collected during 2m walking and running trials were validated against a 3D kinematic system. Secondly, data recorded during 5m running trials at various intensities from standing or flying starts were compared to a single beam CTLS and the independent and average scores of three handheld stopwatch (HS) operators. Intraclass correlation coefficient and Bland-Altman plots were used to assess validity. Absolute error quartiles and percentage of trials in absolute error threshold ranges were used to determine accuracy. The NWHC system was valid when compared against the 3D kinematic system (ICC=0.99, median absolute error (MAR)=2.95%). For the flying 5m trials the NWHC system possessed excellent validity and precision (ICC=0.97, MAR<3%) when compared with the CTLS. In contrast, the NWHC system and the HS values during standing start trials possessed only modest validity (ICC<0.75) and accuracy (MAR>8%). A NWHC timing light system is inexpensive, portable and valid for assessing running velocity. Errors in the 5m standing start trials may have been due to erroneous event detection by either the commercial or NWHC-based timing light systems. Copyright © 2010 Sports Medicine Australia. Published by Elsevier Ltd. All rights reserved.

Gait assessment using the Microsoft Xbox One Kinect: Concurrent validity and inter-day reliability of spatiotemporal and kinematic variables.

PubMed

Mentiplay, Benjamin F; Perraton, Luke G; Bower, Kelly J; Pua, Yong-Hao; McGaw, Rebekah; Heywood, Sophie; Clark, Ross A

2015-07-16

The revised Xbox One Kinect, also known as the Microsoft Kinect V2 for Windows, includes enhanced hardware which may improve its utility as a gait assessment tool. This study examined the concurrent validity and inter-day reliability of spatiotemporal and kinematic gait parameters estimated using the Kinect V2 automated body tracking system and a criterion reference three-dimensional motion analysis (3DMA) marker-based camera system. Thirty healthy adults performed two testing sessions consisting of comfortable and fast paced walking trials. Spatiotemporal outcome measures related to gait speed, speed variability, step length, width and time, foot swing velocity and medial-lateral and vertical pelvis displacement were examined. Kinematic outcome measures including ankle flexion, knee flexion and adduction and hip flexion were examined. To assess the agreement between Kinect and 3DMA systems, Bland-Altman plots, relative agreement (Pearson's correlation) and overall agreement (concordance correlation coefficients) were determined. Reliability was assessed using intraclass correlation coefficients, Cronbach's alpha and standard error of measurement. The spatiotemporal measurements had consistently excellent (r≥0.75) concurrent validity, with the exception of modest validity for medial-lateral pelvis sway (r=0.45-0.46) and fast paced gait speed variability (r=0.73). In contrast kinematic validity was consistently poor to modest, with all associations between the systems weak (r<0.50). In those measures with acceptable validity, the inter-day reliability was similar between systems. In conclusion, while the Kinect V2 body tracking may not accurately obtain lower body kinematic data, it shows great potential as a tool for measuring spatiotemporal aspects of gait. Copyright © 2015 Elsevier Ltd. All rights reserved.

Validity evidence for an OSCE to assess competency in systems-based practice and practice-based learning and improvement: a preliminary investigation.

PubMed

Varkey, Prathibha; Natt, Neena; Lesnick, Timothy; Downing, Steven; Yudkowsky, Rachel

2008-08-01

To determine the psychometric properties and validity of an OSCE to assess the competencies of Practice-Based Learning and Improvement (PBLI) and Systems-Based Practice (SBP) in graduate medical education. An eight-station OSCE was piloted at the end of a three-week Quality Improvement elective for nine preventive medicine and endocrinology fellows at Mayo Clinic. The stations assessed performance in quality measurement, root cause analysis, evidence-based medicine, insurance systems, team collaboration, prescription errors, Nolan's model, and negotiation. Fellows' performance in each of the stations was assessed by three faculty experts using checklists and a five-point global competency scale. A modified Angoff procedure was used to set standards. Evidence for the OSCE's validity, feasibility, and acceptability was gathered. Evidence for content and response process validity was judged as excellent by institutional content experts. Interrater reliability of scores ranged from 0.85 to 1 for most stations. Interstation correlation coefficients ranged from -0.62 to 0.99, reflecting case specificity. Implementation cost was approximately $255 per fellow. All faculty members agreed that the OSCE was realistic and capable of providing accurate assessments. The OSCE provides an opportunity to systematically sample the different subdomains of Quality Improvement. Furthermore, the OSCE provides an opportunity for the demonstration of skills rather than the testing of knowledge alone, thus making it a potentially powerful assessment tool for SBP and PBLI. The study OSCE was well suited to assess SBP and PBLI. The evidence gathered through this study lays the foundation for future validation work.

Posturography using the Wii Balance Board™: A feasibility study with healthy adults and adults post-stroke.

PubMed

Llorens, Roberto; Latorre, Jorge; Noé, Enrique; Keshner, Emily A

2016-01-01

Posturography systems that incorporate force platforms are considered to assess balance and postural control with greater sensitivity and objectivity than conventional clinical tests. The Wii Balance Board (WBB) system has been shown to have similar performance characteristics as other force platforms, but with lower cost and size. To determine the validity and reliability of a freely available WBB-based posturography system that combined the WBB with several traditional balance assessments, and to assess the performance of a cohort of stroke individuals with respect to healthy individuals. Healthy subjects and individuals with stroke were recruited. Both groups were assessed using the WBB-based posturography system. Individuals with stroke were also assessed using a laboratory grade posturography system and a battery of clinical tests to determine the concurrent validity of the system. A group of subjects were assessed twice with the WBB-based system to determine its reliability. A total of 144 healthy individuals and 53 individuals with stroke participated in the study. Concurrent validity with another posturography system was moderate to high. Correlations with clinical scales were consistent with previous research. The reliability of the system was excellent in almost all measures. In addition, the system successfully characterized individuals with stroke with respect to the healthy population. The WBB-based posturography system exhibited excellent psychometric properties and sensitivity for identifying balance performance of individuals with stroke in comparison with healthy subjects, which supports feasibility of the system as a clinical tool. Copyright © 2015 Elsevier B.V. All rights reserved.

Validation of a physically based catchment model for application in post-closure radiological safety assessments of deep geological repositories for solid radioactive wastes.

PubMed

Thorne, M C; Degnan, P; Ewen, J; Parkin, G

2000-12-01

The physically based river catchment modelling system SHETRAN incorporates components representing water flow, sediment transport and radionuclide transport both in solution and bound to sediments. The system has been applied to simulate hypothetical future catchments in the context of post-closure radiological safety assessments of a potential site for a deep geological disposal facility for intermediate and certain low-level radioactive wastes at Sellafield, west Cumbria. In order to have confidence in the application of SHETRAN for this purpose, various blind validation studies have been undertaken. In earlier studies, the validation was undertaken against uncertainty bounds in model output predictions set by the modelling team on the basis of how well they expected the model to perform. However, validation can also be carried out with bounds set on the basis of how well the model is required to perform in order to constitute a useful assessment tool. Herein, such an assessment-based validation exercise is reported. This exercise related to a field plot experiment conducted at Calder Hollow, west Cumbria, in which the migration of strontium and lanthanum in subsurface Quaternary deposits was studied on a length scale of a few metres. Blind predictions of tracer migration were compared with experimental results using bounds set by a small group of assessment experts independent of the modelling team. Overall, the SHETRAN system performed well, failing only two out of seven of the imposed tests. Furthermore, of the five tests that were not failed, three were positively passed even when a pessimistic view was taken as to how measurement errors should be taken into account. It is concluded that the SHETRAN system, which is still being developed further, is a powerful tool for application in post-closure radiological safety assessments.

Development and validation of a scale for mouth handicap in systemic sclerosis: the Mouth Handicap in Systemic Sclerosis scale

PubMed Central

Mouthon, L; Rannou, F; Bérezné, A; Pagnoux, C; Arène, J‐P; Foïs, E; Cabane, J; Guillevin, L; Revel, M; Fermanian, J; Poiraudeau, S

2007-01-01

Objective To develop and assess the reliability and construct validity of a scale assessing disability involving the mouth in systemic sclerosis (SSc). Methods We generated a 34‐item provisional scale from mailed responses of patients (n = 74), expert consensus (n = 10) and literature analysis. A total of 71 other SSc patients were recruited. The test–retest reliability was assessed using the intraclass coefficient correlation and divergent validity using the Spearman correlation coefficient. Factor analysis followed by varimax rotation was performed to assess the factorial structure of the scale. Results The item reduction process retained 12 items with 5 levels of answers (total score range 0–48). The mean total score of the scale was 20.3 (SD 9.7). The test–retest reliability was 0.96. Divergent validity was confirmed for global disability (Health Assessment Questionnaire (HAQ), r = 0.33), hand function (Cochin Hand Function Scale, r = 0.37), inter‐incisor distance (r = −0.34), handicap (McMaster‐Toronto Arthritis questionnaire (MACTAR), r = 0.24), depression (Hospital Anxiety and Depression (HAD); HADd, r = 0.26) and anxiety (HADa, r = 0.17). Factor analysis extracted 3 factors with eigenvalues of 4.26, 1.76 and 1.47, explaining 63% of the variance. These 3 factors could be clinically characterised. The first factor (5 items) represents handicap induced by the reduction in mouth opening, the second (5 items) handicap induced by sicca syndrome and the third (2 items) aesthetic concerns. Conclusion We propose a new scale, the Mouth Handicap in Systemic Sclerosis (MHISS) scale, which has excellent reliability and good construct validity, and assesses specifically disability involving the mouth in patients with SSc. PMID:17502364

A Calculus of Occupational Skill Attainment: Building More Validity into a Valid Assessment System

ERIC Educational Resources Information Center

Munyofu, Paul; Kohr, Richard

2009-01-01

This study investigated several aspects of occupational skill assessment as implemented in one state: (1) What is the extent to which student achievement on the cognitive component was related to their achievement on the psychomotor component of the technical skill assessments? (2) How efficiently was their overall composite attainment calculated?…

Validity of the OSU Post-Traumatic Stress Disorder Scale and the Behavior Assessment System for Children Self-Report of Personality with Child Tornado Survivors

ERIC Educational Resources Information Center

Evans, Linda Garner; Oehler-Stinnett, Judy

2008-01-01

Tornadoes and other natural disasters can lead to anxiety and posttraumatic stress disorder (PTSD) in children. This study provides further validity for the Oklahoma State University Post-Traumatic Stress Disorder Scale-Child Form (OSU PTSDS-CF) by comparing it to the Behavior Assessment System for Children Self-Report of Personality (BASC-SRP).…

Measuring Quality in Rural Kindergarten Classrooms: Reliability and Validity Evidence for the Classroom Assessment Scoring System, Kindergarten-Third Grade (CLASS K-3)

ERIC Educational Resources Information Center

Sandilos, Lia E.

2012-01-01

The purpose of the current study was to evaluate the structural validity and stability of scores on a measure of global classroom quality, the Classroom Assessment Scoring System, Kindergarten-Third Grade (CLASS K-3; Pianta, La Paro, & Hamre, 2008). Using data from a sample of 417 kindergarten classrooms in the rural Southern and Mid-Atlantic…

45 CFR 95.626 - Independent Verification and Validation.

Code of Federal Regulations, 2013 CFR

2013-10-01

... 45 Public Welfare 1 2013-10-01 2013-10-01 false Independent Verification and Validation. 95.626... (FFP) Specific Conditions for Ffp § 95.626 Independent Verification and Validation. (a) An assessment for independent verification and validation (IV&V) analysis of a State's system development effort may...

45 CFR 95.626 - Independent Verification and Validation.

Code of Federal Regulations, 2014 CFR

2014-10-01

... 45 Public Welfare 1 2014-10-01 2014-10-01 false Independent Verification and Validation. 95.626... (FFP) Specific Conditions for Ffp § 95.626 Independent Verification and Validation. (a) An assessment for independent verification and validation (IV&V) analysis of a State's system development effort may...

45 CFR 95.626 - Independent Verification and Validation.

Code of Federal Regulations, 2011 CFR

2011-10-01

... 45 Public Welfare 1 2011-10-01 2011-10-01 false Independent Verification and Validation. 95.626... (FFP) Specific Conditions for Ffp § 95.626 Independent Verification and Validation. (a) An assessment for independent verification and validation (IV&V) analysis of a State's system development effort may...

45 CFR 95.626 - Independent Verification and Validation.

Code of Federal Regulations, 2012 CFR

2012-10-01

... 45 Public Welfare 1 2012-10-01 2012-10-01 false Independent Verification and Validation. 95.626... (FFP) Specific Conditions for Ffp § 95.626 Independent Verification and Validation. (a) An assessment for independent verification and validation (IV&V) analysis of a State's system development effort may...

Agreement between the spatio-temporal gait parameters from treadmill-based photoelectric cell and the instrumented treadmill system in healthy young adults and stroke patients.

PubMed

Lee, Myungmo; Song, Changho; Lee, Kyoungjin; Shin, Doochul; Shin, Seungho

2014-07-14

Treadmill gait analysis was more advantageous than over-ground walking because it allowed continuous measurements of the gait parameters. The purpose of this study was to investigate the concurrent validity and the test-retest reliability of the OPTOGait photoelectric cell system against the treadmill-based gait analysis system by assessing spatio-temporal gait parameters. Twenty-six stroke patients and 18 healthy adults were asked to walk on the treadmill at their preferred speed. The concurrent validity was assessed by comparing data obtained from the 2 systems, and the test-retest reliability was determined by comparing data obtained from the 1st and the 2nd session of the OPTOGait system. The concurrent validity, identified by the intra-class correlation coefficients (ICC [2, 1]), coefficients of variation (CVME), and 95% limits of agreement (LOA) for the spatial-temporal gait parameters, were excellent but the temporal parameters expressed as a percentage of the gait cycle were poor. The test-retest reliability of the OPTOGait System, identified by ICC (3, 1), CVME, 95% LOA, standard error of measurement (SEM), and minimum detectable change (MDC95%) for the spatio-temporal gait parameters, was high. These findings indicated that the treadmill-based OPTOGait System had strong concurrent validity and test-retest reliability. This portable system could be useful for clinical assessments.

British isles lupus assessment group 2004 index is valid for assessment of disease activity in systemic lupus erythematosus

PubMed Central

Yee, Chee-Seng; Farewell, Vernon; Isenberg, David A; Rahman, Anisur; Teh, Lee-Suan; Griffiths, Bridget; Bruce, Ian N; Ahmad, Yasmeen; Prabu, Athiveeraramapandian; Akil, Mohammed; McHugh, Neil; D'Cruz, David; Khamashta, Munther A; Maddison, Peter; Gordon, Caroline

2007-01-01

Objective To determine the construct and criterion validity of the British Isles Lupus Assessment Group 2004 (BILAG-2004) index for assessing disease activity in systemic lupus erythematosus (SLE). Methods Patients with SLE were recruited into a multicenter cross-sectional study. Data on SLE disease activity (scores on the BILAG-2004 index, Classic BILAG index, and Systemic Lupus Erythematosus Disease Activity Index 2000 [SLEDAI-2K]), investigations, and therapy were collected. Overall BILAG-2004 and overall Classic BILAG scores were determined by the highest score achieved in any of the individual systems in the respective index. Erythrocyte sedimentation rates (ESRs), C3 levels, C4 levels, anti–double-stranded DNA (anti-dsDNA) levels, and SLEDAI-2K scores were used in the analysis of construct validity, and increase in therapy was used as the criterion for active disease in the analysis of criterion validity. Statistical analyses were performed using ordinal logistic regression for construct validity and logistic regression for criterion validity. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated. Results Of the 369 patients with SLE, 92.7% were women, 59.9% were white, 18.4% were Afro-Caribbean and 18.4% were South Asian. Their mean ± SD age was 41.6 ± 13.2 years and mean disease duration was 8.8 ± 7.7 years. More than 1 assessment was obtained on 88.6% of the patients, and a total of 1,510 assessments were obtained. Increasing overall scores on the BILAG-2004 index were associated with increasing ESRs, decreasing C3 levels, decreasing C4 levels, elevated anti-dsDNA levels, and increasing SLEDAI-2K scores (all P < 0.01). Increase in therapy was observed more frequently in patients with overall BILAG-2004 scores reflecting higher disease activity. Scores indicating active disease (overall BILAG-2004 scores of A and B) were significantly associated with increase in therapy (odds ratio [OR] 19.3, P < 0.01). The BILAG-2004 and Classic BILAG indices had comparable sensitivity, specificity, PPV, and NPV. Conclusion These findings show that the BILAG-2004 index has construct and criterion validity. PMID:18050213

Reliability and validity of an accele-rometric system for assessing vertical jumping performance.

PubMed

Choukou, M-A; Laffaye, G; Taiar, R

2014-03-01

The validity of an accelerometric system (Myotest©) for assessing vertical jump height, vertical force and power, leg stiffness and reactivity index was examined. 20 healthy males performed 3×"5 hops in place", 3×"1 squat jump" and 3× "1 countermovement jump" during 2 test-retest sessions. The variables were simultaneously assessed using an accelerometer and a force platform at a frequency of 0.5 and 1 kHz, respectively. Both reliability and validity of the accelerometric system were studied. No significant differences between test and retest data were found (p < 0.05), showing a high level of reliability. Besides, moderate to high intraclass correlation coefficients (ICCs) (from 0.74 to 0.96) were obtained for all variables whereas weak to moderate ICCs (from 0.29 to 0.79) were obtained for force and power during the countermovement jump. With regards to validity, the difference between the two devices was not significant for 5 hops in place height (1.8 cm), force during squat (-1.4 N · kg(-1)) and countermovement (0.1 N · kg(-1)) jumps, leg stiffness (7.8 kN · m(-1)) and reactivity index (0.4). So, the measurements of these variables with this accelerometer are valid, which is not the case for the other variables. The main causes of non-validity for velocity, power and contact time assessment are temporal biases of the takeoff and touchdown moments detection.

RELIABILITY AND VALIDITY OF AN ACCELEROMETRIC SYSTEM FOR ASSESSING VERTICAL JUMPING PERFORMANCE

PubMed Central

Laffaye, G.; Taiar, R.

2014-01-01

The validity of an accelerometric system (Myotest©) for assessing vertical jump height, vertical force and power, leg stiffness and reactivity index was examined. 20 healthy males performed 3ד5 hops in place”, 3ד1 squat jump” and 3× “1 countermovement jump” during 2 test-retest sessions. The variables were simultaneously assessed using an accelerometer and a force platform at a frequency of 0.5 and 1 kHz, respectively. Both reliability and validity of the accelerometric system were studied. No significant differences between test and retest data were found (p < 0.05), showing a high level of reliability. Besides, moderate to high intraclass correlation coefficients (ICCs) (from 0.74 to 0.96) were obtained for all variables whereas weak to moderate ICCs (from 0.29 to 0.79) were obtained for force and power during the countermovement jump. With regards to validity, the difference between the two devices was not significant for 5 hops in place height (1.8 cm), force during squat (-1.4 N · kg−1) and countermovement (0.1 N · kg−1) jumps, leg stiffness (7.8 kN · m−1) and reactivity index (0.4). So, the measurements of these variables with this accelerometer are valid, which is not the case for the other variables. The main causes of non-validity for velocity, power and contact time assessment are temporal biases of the takeoff and touchdown moments detection. PMID:24917690

Longitudinal construct validity of the minimum data set health status index.

PubMed

Jones, Aaron; Feeny, David; Costa, Andrew P

2018-05-24

The Minimum Data Set Health Status Index (MDS-HSI) is a generic, preference-based health-related quality of life (HRQOL) measure derived by mapping items from the Resident Assessment Instrument - Minimum Data Set (RAI-MDS) assessment onto the Health Utilities Index Mark 2 classification system. While the validity of the MDS-HSI has been examined in cross-sectional settings, the longitudinal validity has not been explored. The objective of this study was to investigate the longitudinal construct validity of the MDS-HSI in a home care population. This study utilized a retrospective cohort of home care patients in the Hamilton-Niagara-Haldimand-Brant health region of Ontario, Canada with at least two RAI-MDS Home Care assessments between January 2010 and December 2014. Convergent validity was assessed by calculating Spearman rank correlations between the change in MDS-HSI and changes in six validated indices of health domains that can be calculated from the RAI-MDS assessment. Known-groups validity was investigated by fitting multivariable linear regression models to estimate the mean change in MDS-HSI associated with clinically important changes in the six health domain indices and 15 disease symptoms from the RAI-MDS Home Care assessment, controlling for age and sex. The cohort contained 25,182 patients with two RAI-MDS Home Care assessments. Spearman correlations between the MDS-HSI change and changes in the health domain indices were all statistically significant and in the hypothesized small to moderate range [0.1 < ρ < 0.5]. Clinically important changes in all of the health domain indices and 13 of the 15 disease symptoms were significantly associated with clinically important changes in the MDS-HSI. The findings of this study support the longitudinal construct validity of the MDS-HSI in home care populations. In addition to evaluating changes in HRQOL among home care patients in clinical research, economic evaluation, and health technology assessment, the MDS-HSI may be used in system-level applications using routinely collected population-level data.

The Validity of Individual Rorschach Variables: Systematic Reviews and Meta-Analyses of the Comprehensive System

ERIC Educational Resources Information Center

Mihura, Joni L.; Meyer, Gregory J.; Dumitrascu, Nicolae; Bombel, George

2013-01-01

We systematically evaluated the peer-reviewed Rorschach validity literature for the 65 main variables in the popular Comprehensive System (CS). Across 53 meta-analyses examining variables against externally assessed criteria (e.g., observer ratings, psychiatric diagnosis), the mean validity was r = 0.27 (k = 770) as compared to r = 0.08 (k = 386)…

Validity and reliability of the Myotest accelerometric system for the assessment of vertical jump height.

PubMed

Casartelli, Nicola; Müller, Roland; Maffiuletti, Nicola A

2010-11-01

The aim of the present study was to verify the validity and reliability of the Myotest accelerometric system (Myotest SA, Sion, Switzerland) for the assessment of vertical jump height. Forty-four male basketball players (age range: 9-25 years) performed series of squat, countermovement and repeated jumps during 2 identical test sessions separated by 2-15 days. Flight height was simultaneously quantified with the Myotest system and validated photoelectric cells (Optojump). Two calculation methods were used to estimate the jump height from Myotest recordings: flight time (Myotest-T) and vertical takeoff velocity (Myotest-V). Concurrent validity was investigated comparing Myotest-T and Myotest-V to the criterion method (Optojump), and test-retest reliability was also examined. As regards validity, Myotest-T overestimated jumping height compared to Optojump (p < 0.001) with a systematic bias of approximately 7 cm, even though random errors were low (2.7 cm) and intraclass correlation coefficients (ICCs) where high (>0.98), that is, excellent validity. Myotest-V overestimated jumping height compared to Optojump (p < 0.001), with high random errors (>12 cm), high limits of agreement ratios (>36%), and low ICCs (<0.75), that is, poor validity. As regards reliability, Myotest-T showed high ICCs (range: 0.92-0.96), whereas Myotest-V showed low ICCs (range: 0.56-0.89), and high random errors (>9 cm). In conclusion, Myotest-T is a valid and reliable method for the assessment of vertical jump height, and its use is legitimate for field-based evaluations, whereas Myotest-V is neither valid nor reliable.

Validity and reliability of a novel immunosuppressive adverse effects scoring system in renal transplant recipients.

PubMed

Meaney, Calvin J; Arabi, Ziad; Venuto, Rocco C; Consiglio, Joseph D; Wilding, Gregory E; Tornatore, Kathleen M

2014-06-12

After renal transplantation, many patients experience adverse effects from maintenance immunosuppressive drugs. When these adverse effects occur, patient adherence with immunosuppression may be reduced and impact allograft survival. If these adverse effects could be prospectively monitored in an objective manner and possibly prevented, adherence to immunosuppressive regimens could be optimized and allograft survival improved. Prospective, standardized clinical approaches to assess immunosuppressive adverse effects by health care providers are limited. Therefore, we developed and evaluated the application, reliability and validity of a novel adverse effects scoring system in renal transplant recipients receiving calcineurin inhibitor (cyclosporine or tacrolimus) and mycophenolic acid based immunosuppressive therapy. The scoring system included 18 non-renal adverse effects organized into gastrointestinal, central nervous system and aesthetic domains developed by a multidisciplinary physician group. Nephrologists employed this standardized adverse effect evaluation in stable renal transplant patients using physical exam, review of systems, recent laboratory results, and medication adherence assessment during a clinic visit. Stable renal transplant recipients in two clinical studies were evaluated and received immunosuppressive regimens comprised of either cyclosporine or tacrolimus with mycophenolic acid. Face, content, and construct validity were assessed to document these adverse effect evaluations. Inter-rater reliability was determined using the Kappa statistic and intra-class correlation. A total of 58 renal transplant recipients were assessed using the adverse effects scoring system confirming face validity. Nephrologists (subject matter experts) rated the 18 adverse effects as: 3.1 ± 0.75 out of 4 (maximum) regarding clinical importance to verify content validity. The adverse effects scoring system distinguished 1.75-fold increased gastrointestinal adverse effects (p=0.008) in renal transplant recipients receiving tacrolimus and mycophenolic acid compared to the cyclosporine regimen. This finding demonstrated construct validity. Intra-class correlation was 0.81 (95% confidence interval: 0.65-0.90) and Kappa statistic of 0.68 ± 0.25 for all 18 adverse effects and verified substantial inter-rater reliability. This immunosuppressive adverse effects scoring system in stable renal transplant recipients was evaluated and substantiated face, content and construct validity with inter-rater reliability. The scoring system may facilitate prospective, standardized clinical monitoring of immunosuppressive adverse drug effects in stable renal transplant recipients and improve medication adherence.

Incremental Validity of Test Session and Classroom Observations in a Multimethod Assessment of Attention Deficit/Hyperactivity Disorder

ERIC Educational Resources Information Center

McConaughy, Stephanie H.; Harder, Valerie S.; Antshel, Kevin M.; Gordon, Michael; Eiraldi, Ricardo; Dumenci, Levent

2010-01-01

This study tested the incremental validity of behavioral observations, over and above parent and teacher reports, for assessing symptoms of Attention Deficit/Hyperactivity Disorder (ADHD) in children ages 6 to 12, using the Test Observation Form (TOF) and Direct Observation Form (DOF) from the Achenbach System of Empirically Based Assessment. The…

Institutional Effectiveness: A Model for Planning, Assessment & Validation.

ERIC Educational Resources Information Center

Truckee Meadows Community Coll., Sparks, NV.

The report presents Truckee Meadows Community College's (Colorado) model for assessing institutional effectiveness and validating the College's mission and vision, and the strategic plan for carrying out the institutional effectiveness model. It also outlines strategic goals for the years 1999-2001. From the system-wide directive that education…

Assessing Meritorious Teacher Performance: A Differential Validity Study.

ERIC Educational Resources Information Center

Ellett, Chad D; Capie, William

The Teacher Assessment and Development System (TADS) - Meritorious Teacher Program (MTP) FORM instrument is used in the Dade County Public Schools, Miami, Florida, to evaluate teachers. Its validity for decisions concerning merit pay for master teachers was examined in this study. Specifically, its ability to discriminate between high performing…

Earth Science Enterprise Scientific Data Purchase Project: Verification and Validation

NASA Technical Reports Server (NTRS)

Jenner, Jeff; Policelli, Fritz; Fletcher, Rosea; Holecamp, Kara; Owen, Carolyn; Nicholson, Lamar; Dartez, Deanna

2000-01-01

This paper presents viewgraphs on the Earth Science Enterprise Scientific Data Purchase Project's verification,and validation process. The topics include: 1) What is Verification and Validation? 2) Why Verification and Validation? 3) Background; 4) ESE Data Purchas Validation Process; 5) Data Validation System and Ingest Queue; 6) Shipment Verification; 7) Tracking and Metrics; 8) Validation of Contract Specifications; 9) Earth Watch Data Validation; 10) Validation of Vertical Accuracy; and 11) Results of Vertical Accuracy Assessment.

A low-cost contact system to assess load displacement velocity in a resistance training machine.

PubMed

Buscà, Bernat; Font, Anna

2011-01-01

This study sought to determine the validity of a new system for assessing the displacement and average velocity within machine-based resistance training exercise using the Chronojump System. The new design is based on a contact bar and a simple, low-cost mechanism that detects the conductivity of electrical potentials with a precision chronograph. This system allows coaches to assess velocity to control the strength training process. A validation study was performed by assessing the concentric phase parameters of a leg press exercise. Output time data from the Chronojump System in combination with the pre-established range of movement was compared with data from a position sensor connected to a Biopac System. A subset of 87 actions from 11 professional tennis players was recorded and, using the two methods, average velocity and displacement variables in the same action were compared. A t-test for dependent samples and a correlation analysis were undertaken. The r value derived from the correlation between the Biopac System and the contact Chronojump System was >0.94 for all measures of displacement and velocity on all loads (p < 0.01). The Effect Size (ES) was 0.18 in displacement and 0.14 in velocity and ranged from 0.09 to 0.31 and from 0.07 to 0.34, respectively. The magnitude of the difference between the two methods in all parameters and the correlation values provided certain evidence of validity of the Chronojump System to assess the average displacement velocity of loads in a resistance training machine. Key pointsThe assessment of speed in resistance machines is a valuable source of information for strength training.Many commercial systems used to assess velocity, power and force are expensive thereby preventing widespread use by coaches and athletes.The system is intended to be a low-cost device for assessing and controlling the velocity exerted on each repetition in any resistance training machine.The system could be easily adapted in any vertical displacement barbell exercise.

A Content Validity Study of AIMIT (Assessing Interpersonal Motivation in Transcripts).

PubMed

Fassone, Giovanni; Lo Reto, Floriana; Foggetti, Paola; Santomassimo, Chiara; D'Onofrio, Maria Rita; Ivaldi, Antonella; Liotti, Giovanni; Trincia, Valeria; Picardi, Angelo

2016-07-01

Multi-motivational theories of human relatedness state that different motivational systems with an evolutionary basis modulate interpersonal relationships. The reliable assessment of their dynamics may usefully inform the understanding of the therapeutic relationship. The coding system of the Assessing Interpersonal Motivation in Transcripts (AIMIT) allows to identify in the clinical the activity of five main interpersonal motivational systems (IMSs): attachment (care-seeking), caregiving, ranking, sexuality and peer cooperation. To assess whether the criteria currently used to score the AIMIT are consistently correlated with the conceptual formulation of the interpersonal multi-motivational theory, two different studies were designed. Study 1: Content validity as assessed by highly qualified independent raters. Study 2: Content validity as assessed by unqualified raters. Results of study 1 show that out of the total 60 AIMIT verbal criteria, 52 (86.7%) met the required minimum degree of correspondence. The average semantic correspondence scores between these items and the related IMSs were quite good (overall mean: 3.74, standard deviation: 0.61). In study 2, a group of 20 naïve raters had to identify each prevalent motivation (IMS) in a random sequence of 1000 utterances drawn from therapy sessions. Cohen's Kappa coefficient was calculated for each rater with reference to each IMS and then calculated the average Kappa for all raters for each IMS. All average Kappa values were satisfactory (>0.60) and ranged between 0.63 (ranking system) and 0.83 (sexuality system). Data confirmed the overall soundness of AIMIT's theoretical-applicative approach. Results are discussed, corroborating the hypothesis that the AIMIT possesses the required criteria for content validity. Copyright © 2015 John Wiley & Sons, Ltd. Assessing Interpersonal Motivations in psychotherapy transcripts as a useful tool to better understand links between motivational systems and intersubjectivity. A step forward in the knowledge of evolutionary cognitivism and a contribution to the bio-psycho-social model of human relatedness and interpersonal neurobiology. Copyright © 2015 John Wiley & Sons, Ltd.

Organizational Systems Questionnaire (OSQ) Validity Study

ERIC Educational Resources Information Center

Billings, James C.; Kimball, Thomas G.; Shumway, Sterling T.; Korinek, Alan W.

2007-01-01

Marriage and family therapists (MFTs), who are trained in systems theory and consult with complex and difficult systems (e.g., couples and families), are uniquely suited to both assess and intervene in broader organizational systems. However, MFTs are in need of more systemically designed assessment tools to guide and inform their interventions…

A Systems-Level Approach to Building Sustainable Assessment Cultures: Moderation, Quality Task Design and Dependability of Judgement

ERIC Educational Resources Information Center

Colbert, Peta; Wyatt-Smith, Claire; Klenowski, Val

2012-01-01

This article considers the conditions that are necessary at system and local levels for teacher assessment to be valid, reliable and rigorous. With sustainable assessment cultures as a goal, the article examines how education systems can support local-level efforts for quality learning and dependable teacher assessment. This is achieved through…

Psychometrics of the MHSIP Adult Consumer Survey.

PubMed

Jerrell, Jeanette M

2006-10-01

The reliability and validity of the Mental Health Statistics Improvement Program (MHSIP) Adult Consumer Survey were assessed in a statewide convenience sample of 459 persons with severe mental illness served through a public mental health system. Consistent with previous findings and the intent of its developers, three factors were identified that demonstrate good internal consistency, moderate test-retest reliability, and good convergent validity with consumer perceptions of other aspects of their care. The reliability and validity of the MHSIP Adult Consumer Survey documented in this study underscore its scientific and practical utility as an abbreviated tool for assessing access, quality and appropriateness, and outcome in mental health service systems.

The Predictive Validity of Interim Assessment Scores Based on the Full-Information Bifactor Model for the Prediction of End-of-Grade Test Performance

ERIC Educational Resources Information Center

Immekus, Jason C.; Atitya, Ben

2016-01-01

Interim tests are a central component of district-wide assessment systems, yet their technical quality to guide decisions (e.g., instructional) has been repeatedly questioned. In response, the study purpose was to investigate the validity of a series of English Language Arts (ELA) interim assessments in terms of dimensionality and prediction of…

LAnd surface remote sensing Products VAlidation System (LAPVAS) and its preliminary application

NASA Astrophysics Data System (ADS)

Lin, Xingwen; Wen, Jianguang; Tang, Yong; Ma, Mingguo; Dou, Baocheng; Wu, Xiaodan; Meng, Lumin

2014-11-01

The long term record of remote sensing product shows the land surface parameters with spatial and temporal change to support regional and global scientific research widely. Remote sensing product with different sensors and different algorithms is necessary to be validated to ensure the high quality remote sensing product. Investigation about the remote sensing product validation shows that it is a complex processing both the quality of in-situ data requirement and method of precision assessment. A comprehensive validation should be needed with long time series and multiple land surface types. So a system named as land surface remote sensing product is designed in this paper to assess the uncertainty information of the remote sensing products based on a amount of in situ data and the validation techniques. The designed validation system platform consists of three parts: Validation databases Precision analysis subsystem, Inter-external interface of system. These three parts are built by some essential service modules, such as Data-Read service modules, Data-Insert service modules, Data-Associated service modules, Precision-Analysis service modules, Scale-Change service modules and so on. To run the validation system platform, users could order these service modules and choreograph them by the user interactive and then compete the validation tasks of remote sensing products (such as LAI ,ALBEDO ,VI etc.) . Taking SOA-based architecture as the framework of this system. The benefit of this architecture is the good service modules which could be independent of any development environment by standards such as the Web-Service Description Language(WSDL). The standard language: C++ and java will used as the primary programming language to create service modules. One of the key land surface parameter, albedo, is selected as an example of the system application. It is illustrated that the LAPVAS has a good performance to implement the land surface remote sensing product validation.

Pilot feasibility of an mHealth system for conducting ecological momentary assessment of mood-related symptoms following traumatic brain injury.

PubMed

Juengst, Shannon B; Graham, Kristin M; Pulantara, I Wayan; McCue, Michael; Whyte, Ellen M; Dicianno, Brad E; Parmanto, Bambang; Arenth, Patricia M; Skidmore, Elizabeth R D; Wagner, Amy K

2015-01-01

This study assessed pilot feasibility and validity of a mobile health (mHealth) system for tracking mood-related symptoms after traumatic brain injury (TBI). A prospective, repeated measures design was used to assess compliance with daily ecological momentary assessments (EMA) conducted via a smartphone application over an 8-week period. An mHealth system was developed specifically for individuals with TBI and utilized previously validated tools for depressive and anxiety symptoms (Patient Health Questionnaire-9, Generalized Anxiety Disorder-7). Feasibility was assessed in 20 community-dwelling adults with TBI via an assessment of compliance, satisfaction and usability of the smartphone applications. The authors also developed and implemented a clinical patient safety management mechanism for those endorsing suicidality. Participants correctly completed 73.4% of all scheduled assessments, demonstrating good compliance. Daily assessments took <2 minutes to complete. Participants reported high satisfaction with smartphone applications (6.3 of 7) and found them easy to use (6.2 of 7). Comparison of assessments obtained via telephone-based interview and EMA demonstrated high correlations (r = 0.81-0.97), supporting the validity of conducting these assessments via smartphone application in this population. EMA conducted via smartphone demonstrates initial feasibility among adults with TBI and presents numerous opportunities for long-term monitoring of mood-related symptoms in real-world settings.

Validation of Bioreactor and Human-on-a-Chip Devices for Chemical Safety Assessment.

PubMed

Rebelo, Sofia P; Dehne, Eva-Maria; Brito, Catarina; Horland, Reyk; Alves, Paula M; Marx, Uwe

2016-01-01

Equipment and device qualification and test assay validation in the field of tissue engineered human organs for substance assessment remain formidable tasks with only a few successful examples so far. The hurdles seem to increase with the growing complexity of the biological systems, emulated by the respective models. Controlled single tissue or organ culture in bioreactors improves the organ-specific functions and maintains their phenotypic stability for longer periods of time. The reproducibility attained with bioreactor operations is, per se, an advantage for the validation of safety assessment. Regulatory agencies have gradually altered the validation concept from exhaustive "product" to rigorous and detailed process characterization, valuing reproducibility as a standard for validation. "Human-on-a-chip" technologies applying micro-physiological systems to the in vitro combination of miniaturized human organ equivalents into functional human micro-organisms are nowadays thought to be the most elaborate solution created to date. They target the replacement of the current most complex models-laboratory animals. Therefore, we provide here a road map towards the validation of such "human-on-a-chip" models and qualification of their respective bioreactor and microchip equipment along a path currently used for the respective animal models.

The Development of a Web-Based Assessment System to Identify Students' Misconception Automatically on Linear Kinematics with a Four-Tier Instrument Test

ERIC Educational Resources Information Center

Pujayanto, Pujayanto; Budiharti, Rini; Adhitama, Egy; Nuraini, Niken Rizky Amalia; Putri, Hanung Vernanda

2018-01-01

This research proposes the development of a web-based assessment system to identify students' misconception. The system, named WAS (web-based assessment system), can identify students' misconception profile on linear kinematics automatically after the student has finished the test. The test instrument was developed and validated. Items were…

The Adult Attachment Projective Picture System: integrating attachment into clinical assessment.

PubMed

George, Carol; West, Malcolm

2011-01-01

This article summarizes the development and validation of the Adult Attachment Projective System (AAP), a measure we developed from the Bowlby-Ainsworth developmental tradition to assess adult attachment status. The AAP has demonstrated excellent concurrent validity with the Adult Attachment Interview (George, Kaplan, & Main, 1984/1985/1996; Main & Goldwyn, 1985-1994; Main, Goldwyn, & Hesse, 2003), interjudge reliability, and test-retest reliability, with no effects of verbal intelligence or social desirability. The AAP coding and classification system and application in clinical and community samples are summarized. Finally, we introduce the 3 other articles that are part of this Special Section and discuss the use of the AAP in therapeutic assessment and treatment.

Reliability and Validity of Finger Strength and Endurance Measurements in Rock Climbing

ERIC Educational Resources Information Center

Michailov, Michail Lubomirov; Baláš, Jirí; Tanev, Stoyan Kolev; Andonov, Hristo Stoyanov; Kodejška, Jan; Brown, Lee

2018-01-01

Purpose: An advanced system for the assessment of climbing-specific performance was developed and used to: (a) investigate the effect of arm fixation (AF) on construct validity evidence and reliability of climbing-specific finger-strength measurement; (b) assess reliability of finger-strength and endurance measurements; and (c) evaluate the…

Reliability and concurrent validity of the Microsoft Xbox One Kinect for assessment of standing balance and postural control.

PubMed

Clark, Ross A; Pua, Yong-Hao; Oliveira, Cristino C; Bower, Kelly J; Thilarajah, Shamala; McGaw, Rebekah; Hasanki, Ksaniel; Mentiplay, Benjamin F

2015-07-01

The Microsoft Kinect V2 for Windows, also known as the Xbox One Kinect, includes new and potentially far improved depth and image sensors which may increase its accuracy for assessing postural control and balance. The aim of this study was to assess the concurrent validity and reliability of kinematic data recorded using a marker-based three dimensional motion analysis (3DMA) system and the Kinect V2 during a variety of static and dynamic balance assessments. Thirty healthy adults performed two sessions, separated by one week, consisting of static standing balance tests under different visual (eyes open vs. closed) and supportive (single limb vs. double limb) conditions, and dynamic balance tests consisting of forward and lateral reach and an assessment of limits of stability. Marker coordinate and joint angle data were concurrently recorded using the Kinect V2 skeletal tracking algorithm and the 3DMA system. Task-specific outcome measures from each system on Day 1 and 2 were compared. Concurrent validity of trunk angle data during the dynamic tasks and anterior-posterior range and path length in the static balance tasks was excellent (Pearson's r>0.75). In contrast, concurrent validity for medial-lateral range and path length was poor to modest for all trials except single leg eyes closed balance. Within device test-retest reliability was variable; however, the results were generally comparable between devices. In conclusion, the Kinect V2 has the potential to be used as a reliable and valid tool for the assessment of some aspects of balance performance. Copyright © 2015 Elsevier B.V. All rights reserved.

Uncertainty Analysis of OC5-DeepCwind Floating Semisubmersible Offshore Wind Test Campaign

DOE Office of Scientific and Technical Information (OSTI.GOV)

Robertson, Amy N

This paper examines how to assess the uncertainty levels for test measurements of the Offshore Code Comparison, Continued, with Correlation (OC5)-DeepCwind floating offshore wind system, examined within the OC5 project. The goal of the OC5 project was to validate the accuracy of ultimate and fatigue load estimates from a numerical model of the floating semisubmersible using data measured during scaled tank testing of the system under wind and wave loading. The examination of uncertainty was done after the test, and it was found that the limited amount of data available did not allow for an acceptable uncertainty assessment. Therefore, thismore » paper instead qualitatively examines the sources of uncertainty associated with this test to start a discussion of how to assess uncertainty for these types of experiments and to summarize what should be done during future testing to acquire the information needed for a proper uncertainty assessment. Foremost, future validation campaigns should initiate numerical modeling before testing to guide the test campaign, which should include a rigorous assessment of uncertainty, and perform validation during testing to ensure that the tests address all of the validation needs.« less

Uncertainty Analysis of OC5-DeepCwind Floating Semisubmersible Offshore Wind Test Campaign: Preprint

DOE Office of Scientific and Technical Information (OSTI.GOV)

Robertson, Amy N

This paper examines how to assess the uncertainty levels for test measurements of the Offshore Code Comparison, Continued, with Correlation (OC5)-DeepCwind floating offshore wind system, examined within the OC5 project. The goal of the OC5 project was to validate the accuracy of ultimate and fatigue load estimates from a numerical model of the floating semisubmersible using data measured during scaled tank testing of the system under wind and wave loading. The examination of uncertainty was done after the test, and it was found that the limited amount of data available did not allow for an acceptable uncertainty assessment. Therefore, thismore » paper instead qualitatively examines the sources of uncertainty associated with this test to start a discussion of how to assess uncertainty for these types of experiments and to summarize what should be done during future testing to acquire the information needed for a proper uncertainty assessment. Foremost, future validation campaigns should initiate numerical modeling before testing to guide the test campaign, which should include a rigorous assessment of uncertainty, and perform validation during testing to ensure that the tests address all of the validation needs.« less

Educational Milestone Development in the First 7 Specialties to Enter the Next Accreditation System

PubMed Central

Swing, Susan R.; Beeson, Michael S.; Carraccio, Carol; Coburn, Michael; Iobst, William; Selden, Nathan R.; Stern, Peter J.; Vydareny, Kay

2013-01-01

Background The Accreditation Council for Graduate Medical Education (ACGME) Outcome Project introduced 6 general competencies relevant to medical practice but fell short of its goal to create a robust assessment system that would allow program accreditation based on outcomes. In response, the ACGME, the specialty boards, and other stakeholders collaborated to develop educational milestones, observable steps in residents' professional development that describe progress from entry to graduation and beyond. Objectives We summarize the development of the milestones, focusing on 7 specialties, moving to the next accreditation system in July 2013, and offer evidence of their validity. Methods Specialty workgroups with broad representation used a 5-level developmental framework and incorporated information from literature reviews, specialty curricula, dialogue with constituents, and pilot testing. Results The workgroups produced richly diverse sets of milestones that reflect the community's consideration of attributes of competence relevant to practice in the given specialty. Both their development process and the milestones themselves establish a validity argument, when contemporary views of validity for complex performance assessment are used. Conclusions Initial evidence for validity emerges from the development processes and the resulting milestones. Further advancing a validity argument will require research on the use of milestone data in resident assessment and program accreditation. PMID:24404235

Validation of Procedures for Monitoring Crewmember Immune Function - Short Duration Biological Investigation

NASA Technical Reports Server (NTRS)

Sams, Clarence; Crucian, Brian; Stowe, Raymond; Pierson, Duane; Mehta, Satish; Morukov, Boris; Uchakin, Peter; Nehlsen-Cannarella, Sandra

2008-01-01

Validation of Procedures for Monitoring Crew Member Immune Function - Short Duration Biological Investigation (Integrated Immune-SDBI) will assess the clinical risks resulting from the adverse effects of space flight on the human immune system and will validate a flightcompatible immune monitoring strategy. Immune system changes will be monitored by collecting and analyzing blood, urine and saliva samples from crewmembers before, during and after space flight.

Validating the Octave Allegro Information Systems Risk Assessment Methodology: A Case Study

ERIC Educational Resources Information Center

Keating, Corland G.

2014-01-01

An information system (IS) risk assessment is an important part of any successful security management strategy. Risk assessments help organizations to identify mission-critical IS assets and prioritize risk mitigation efforts. Many risk assessment methodologies, however, are complex and can only be completed successfully by highly qualified and…

Assessment of validity with polytrauma Veteran populations.

PubMed

Bush, Shane S; Bass, Carmela

2015-01-01

Veterans with polytrauma have suffered injuries to multiple body parts and organs systems, including the brain. The injuries can generate a triad of physical, neurologic/cognitive, and emotional symptoms. Accurate diagnosis is essential for the treatment of these conditions and for fair allocation of benefits. To accurately diagnose polytrauma disorders and their related problems, clinicians take into account the validity of reported history and symptoms, as well as clinical presentations. The purpose of this article is to describe the assessment of validity with polytrauma Veteran populations. Review of scholarly and other relevant literature and clinical experience are utilized. A multimethod approach to validity assessment that includes objective, standardized measures increases the confidence that can be placed in the accuracy of self-reported symptoms and physical, cognitive, and emotional test results. Due to the multivariate nature of polytrauma and the multiple disciplines that play a role in diagnosis and treatment, an ideal model of validity assessment with polytrauma Veteran populations utilizes neurocognitive, neurological, neuropsychiatric, and behavioral measures of validity. An overview of these validity assessment approaches as applied to polytrauma Veteran populations is presented. Veterans, the VA, and society are best served when accurate diagnoses are made.

Development and initial validation of an endoscopic part-task training box.

PubMed

Thompson, Christopher C; Jirapinyo, Pichamol; Kumar, Nitin; Ou, Amy; Camacho, Andrew; Lengyel, Balazs; Ryan, Michele B

2014-09-01

There is currently no objective and validated methodology available to assess the progress of endoscopy trainees or to determine when technical competence has been achieved. The aims of the current study were to develop an endoscopic part-task simulator and to assess scoring system validity. Fundamental endoscopic skills were determined via kinematic analysis, literature review, and expert interviews. Simulator prototypes and scoring systems were developed to reflect these skills. Validity evidence for content, internal structure, and response process was evaluated. The final training box consisted of five modules (knob control, torque, retroflexion, polypectomy, and navigation and loop reduction). A total of 5 minutes were permitted per module with extra points for early completion. Content validity index (CVI)-realism was 0.88, CVI-relevance was 1.00, and CVI-representativeness was 0.88, giving a composite CVI of 0.92. Overall, 82 % of participants considered the simulator to be capable of differentiating between ability levels, and 93 % thought the simulator should be used to assess ability prior to performing procedures in patients. Inter-item assessment revealed correlations from 0.67 to 0.93, suggesting that tasks were sufficiently correlated to assess the same underlying construct, with each task remaining independent. Each module represented 16.0 % - 26.1 % of the total score, suggesting that no module contributed disproportionately to the composite score. Average box scores were 272.6 and 284.4 (P = 0.94) when performed sequentially, and average score for all participants with proctor 1 was 297.6 and 308.1 with proctor 2 (P = 0.94), suggesting reproducibility and minimal error associated with test administration. A part-task training box and scoring system were developed to assess fundamental endoscopic skills, and validity evidence regarding content, internal structure, and response process was demonstrated. © Georg Thieme Verlag KG Stuttgart · New York.

Seeking Empirical Validity in an Assurance of Learning System

ERIC Educational Resources Information Center

Avery, Sherry L.; McWhorter, Rochell R.; Lirely, Roger; Doty, H. Harold

2014-01-01

Business schools have established measurement tools to support their assurance of learning (AoL) systems and to assess student achievement of learning objectives. However, business schools have not required their tools to be empirically validated, thus ensuring that they measure what they are intended to measure. The authors propose confirmatory…

The Modified Cognitive Constructions Coding System: Reliability and Validity Assessments

ERIC Educational Resources Information Center

Moran, Galia S.; Diamond, Gary M.

2006-01-01

The cognitive constructions coding system (CCCS) was designed for coding client's expressed problem constructions on four dimensions: intrapersonal-interpersonal, internal-external, responsible-not responsible, and linear-circular. This study introduces, and examines the reliability and validity of, a modified version of the CCCS--a version that…

The Motivational Value Systems Questionnaire (MVSQ): Psychometric Analysis Using a Forced Choice Thurstonian IRT Model

PubMed Central

Merk, Josef; Schlotz, Wolff; Falter, Thomas

2017-01-01

This study presents a new measure of value systems, the Motivational Value Systems Questionnaire (MVSQ), which is based on a theory of value systems by psychologist Clare W. Graves. The purpose of the instrument is to help people identify their personal hierarchies of value systems and thus become more aware of what motivates and demotivates them in work-related contexts. The MVSQ is a forced-choice (FC) measure, making it quicker to complete and more difficult to intentionally distort, but also more difficult to assess its psychometric properties due to ipsativity of FC data compared to rating scales. To overcome limitations of ipsative data, a Thurstonian IRT (TIRT) model was fitted to the questionnaire data, based on a broad sample of N = 1,217 professionals and students. Comparison of normative (IRT) scale scores and ipsative scores suggested that MVSQ IRT scores are largely freed from restrictions due to ipsativity and thus allow interindividual comparison of scale scores. Empirical reliability was estimated using a sample-based simulation approach which showed acceptable and good estimates and, on average, slightly higher test-retest reliabilities. Further, validation studies provided evidence on both construct validity and criterion-related validity. Scale score correlations and associations of scores with both age and gender were largely in line with theoretically- and empirically-based expectations, and results of a multitrait-multimethod analysis supports convergent and discriminant construct validity. Criterion validity was assessed by examining the relation of value system preferences to departmental affiliation which revealed significant relations in line with prior hypothesizing. These findings demonstrate the good psychometric properties of the MVSQ and support its application in the assessment of value systems in work-related contexts. PMID:28979228

The Motivational Value Systems Questionnaire (MVSQ): Psychometric Analysis Using a Forced Choice Thurstonian IRT Model.

PubMed

Merk, Josef; Schlotz, Wolff; Falter, Thomas

2017-01-01

This study presents a new measure of value systems, the Motivational Value Systems Questionnaire (MVSQ), which is based on a theory of value systems by psychologist Clare W. Graves. The purpose of the instrument is to help people identify their personal hierarchies of value systems and thus become more aware of what motivates and demotivates them in work-related contexts. The MVSQ is a forced-choice (FC) measure, making it quicker to complete and more difficult to intentionally distort, but also more difficult to assess its psychometric properties due to ipsativity of FC data compared to rating scales. To overcome limitations of ipsative data, a Thurstonian IRT (TIRT) model was fitted to the questionnaire data, based on a broad sample of N = 1,217 professionals and students. Comparison of normative (IRT) scale scores and ipsative scores suggested that MVSQ IRT scores are largely freed from restrictions due to ipsativity and thus allow interindividual comparison of scale scores. Empirical reliability was estimated using a sample-based simulation approach which showed acceptable and good estimates and, on average, slightly higher test-retest reliabilities. Further, validation studies provided evidence on both construct validity and criterion-related validity. Scale score correlations and associations of scores with both age and gender were largely in line with theoretically- and empirically-based expectations, and results of a multitrait-multimethod analysis supports convergent and discriminant construct validity. Criterion validity was assessed by examining the relation of value system preferences to departmental affiliation which revealed significant relations in line with prior hypothesizing. These findings demonstrate the good psychometric properties of the MVSQ and support its application in the assessment of value systems in work-related contexts.

Psychometric properties including reliability, validity and responsiveness of the Majeed pelvic score in patients with chronic sacroiliac joint pain.

PubMed

Bajada, Stefan; Mohanty, Khitish

2016-06-01

The Majeed scoring system is a disease-specific outcome measure that was originally designed to assess pelvic injuries. The aim of this study was to determine the psychometric properties of the Majeed scoring system for chronic sacroiliac joint pain. Internal consistency, content validity, criterion validity, construct validity and responsiveness to change was assessed prospectively for the Majeed scoring system in a cohort of 60 patients diagnosed with sacroiliac joint pain. This diagnosis was confirmed with CT-guided sacroiliac joint anaesthetic block. The overall Majeed score showed acceptable internal consistency (Cronbach alpha = 0.63). Similarly, it showed acceptable floor (0 %) and ceiling (0 %) effects. On the other hand, the domains of pain, work, sitting and sexual intercourse had high (>30 %) floor effects. Significant correlation with the physical component of the Short Form-36 (p = 0.005) and Oswestry disability index (p ≤ 0.001) was found indicating acceptable criterion validity. The overall Majeed score showed acceptable construct validity with all five developed hypotheses showing significance (p ≤ 0.05). The overall Majeed score showed acceptable responsiveness to change with a large (≥0.80) effect size and standardized response mean. Overall the Majeed scoring system demonstrated acceptable psychometric properties for outcome assessment in chronic sacroiliac joint pain. Thus, its use in this condition is adequate. However, some domains demonstrated suboptimal performance indicating that improvement might be achieved with the development of an outcome measure specific for sacroiliac joint dysfunction and degeneration.

Development of Internal System of Education Quality Assessment at a University

ERIC Educational Resources Information Center

Kalimullin, Aydar M.; Khodyreva, Elena ?.; Koinova-Zoellner, Julia

2016-01-01

The urgency of the research is determined by the need to ensure the quality of higher education an essential factor of which is development of the internal assessment system for educational activities at universities. The aim of the article is validation of the model of development of the internal assessment system for educational activities at…

Validation of the Behavioral Risk Factor Surveillance System Sleep Questions

PubMed Central

Jungquist, Carla R.; Mund, Jaime; Aquilina, Alan T.; Klingman, Karen; Pender, John; Ochs-Balcom, Heather; van Wijngaarden, Edwin; Dickerson, Suzanne S.

2016-01-01

Study Objective: Sleep problems may constitute a risk for health problems, including cardiovascular disease, depression, diabetes, poor work performance, and motor vehicle accidents. The primary purpose of this study was to assess the validity of the current Behavioral Risk Factor Surveillance System (BRFSS) sleep questions by establishing the sensitivity and specificity for detection of sleep/ wake disturbance. Methods: Repeated cross-sectional assessment of 300 community dwelling adults over the age of 18 who did not wear CPAP or oxygen during sleep. Reliability and validity testing of the BRFSS sleep questions was performed comparing to BFRSS responses to data from home sleep study, actigraphy for 14 days, Insomnia Severity Index, Epworth Sleepiness Scale, and PROMIS-57. Results: Only two of the five BRFSS sleep questions were found valid and reliable in determining total sleep time and excessive daytime sleepiness. Conclusions: Refinement of the BRFSS questions is recommended. Citation: Jungquist CR, Mund J, Aquilina AT, Klingman K, Pender J, Ochs-Balcom H, van Wijngaarden E, Dickerson SS. Validation of the behavioral risk factor surveillance system sleep questions. J Clin Sleep Med 2016;12(3):301–310. PMID:26446246

Reliability and validity of the Microsoft Kinect for assessment of manual wheelchair propulsion.

PubMed

Milgrom, Rachel; Foreman, Matthew; Standeven, John; Engsberg, Jack R; Morgan, Kerri A

2016-01-01

Concurrent validity and test-retest reliability of the Microsoft Kinect in quantification of manual wheelchair propulsion were examined. Data were collected from five manual wheelchair users on a roller system. Three Kinect sensors were used to assess test-retest reliability with a still pose. Three systems were used to assess concurrent validity of the Kinect to measure propulsion kinematics (joint angles, push loop characteristics): Kinect, Motion Analysis, and Dartfish ProSuite (Dartfish joint angles were limited to shoulder and elbow flexion). Intraclass correlation coefficients revealed good reliability (0.87-0.99) between five of the six joint angles (neck flexion, shoulder flexion, shoulder abduction, elbow flexion, wrist flexion). ICCs suggested good concurrent validity for elbow flexion between the Kinect and Dartfish and between the Kinect and Motion Analysis. Good concurrent validity was revealed for maximum height, hand-axle relationship, and maximum area (0.92-0.95) between the Kinect and Dartfish and maximum height and hand-axle relationship (0.89-0.96) between the Kinect and Motion Analysis. Analysis of variance revealed significant differences (p < 0.05) in maximum length between Dartfish (mean 58.76 cm) and the Kinect (40.16 cm). Results pose promising research and clinical implications for propulsion assessment and overuse injury prevention with the application of current findings to future technology.

Evaluating the Diagnostic Validity of a Facet-Based Formative Assessment System

ERIC Educational Resources Information Center

DeBarger, Angela Haydel; DiBello, Louis; Minstrell, Jim; Feng, Mingyu; Stout, William; Pellegrino, James; Haertel, Geneva; Harris, Christopher; Ructinger, Liliana

2011-01-01

This paper describes methods for an alignment study and psychometric analyses of a formative assessment system, Diagnoser Tools for physics. Diagnoser Tools begin with facet clusters as the interpretive framework for designing questions and instructional activities. Thus each question in the diagnostic assessments includes distractors that…

ARM Radiosondes for National Polar-Orbiting Operational Environmental Satellite System Preparatory Project Validation Field Campaign Report

DOE Office of Scientific and Technical Information (OSTI.GOV)

Borg, Lori; Tobin, David; Reale, Anthony

This IOP has been a coordinated effort involving the U.S. Department of Energy (DOE) Atmospheric Radiation (ARM) Climate Research Facility, the University of Wisconsin (UW)-Madison, and the JPSS project to validate SNPP NOAA Unique Combined Atmospheric Processing System (NUCAPS) temperature and moisture sounding products from the Cross-track Infrared Sounder (CrIS) and the Advanced Technology Microwave Sounder (ATMS). In this arrangement, funding for radiosondes was provided by the JPSS project to ARM. These radiosondes were launched coincident with the SNPP satellite overpasses (OP) at four of the ARM field sites beginning in July 2012 and running through September 2017. Combined withmore » other ARM data, an assessment of the radiosonde data quality was performed and post-processing corrections applied producing an ARM site Best Estimate (BE) product. The SNPP targeted radiosondes were integrated into the NOAA Products Validation System (NPROVS+) system, which collocated the radiosondes with satellite products (NOAA, National Aeronautics and Space Administration [NASA], European Organisation for the Exploitation of Meteorological Satellites [EUMETSAT], Geostationary Operational Environmental Satellite [GOES], Constellation Observing System for Meteorology, Ionosphere, and Climate [COSMIC]) and Numerical Weather Prediction (NWP forecasts for use in product assessment and algorithm development. This work was a fundamental, integral, and cost-effective part of the SNPP validation effort and provided critical accuracy assessments of the SNPP temperature and water vapor soundings.« less

Validity of an ultra-wideband local positioning system to measure locomotion in indoor sports.

PubMed

Serpiello, F R; Hopkins, W G; Barnes, S; Tavrou, J; Duthie, G M; Aughey, R J; Ball, K

2018-08-01

The validity of an Ultra-wideband (UWB) positioning system was investigated during linear and change-of-direction (COD) running drills. Six recreationally-active men performed ten repetitions of four activities (walking, jogging, maximal acceleration, and 45º COD) on an indoor court. Activities were repeated twice, in the centre of the court and on the side. Participants wore a receiver tag (Clearsky T6, Catapult Sports) and two reflective markers placed on the tag to allow for comparisons with the criterion system (Vicon). Distance, mean and peak velocity, acceleration, and deceleration were assessed. Validity was assessed via percentage least-square means difference (Clearsky-Vicon) with 90% confidence interval and magnitude-based inference; typical error was expressed as within-subject standard deviation. The mean differences for distance, mean/peak speed, and mean/peak accelerations in the linear drills were in the range of 0.2-12%, with typical errors between 1.2 and 9.3%. Mean and peak deceleration had larger differences and errors between systems. In the COD drill, moderate-to-large differences were detected for the activity performed in the centre of the court, increasing to large/very large on the side. When filtered and smoothed following a similar process, the UWB-based positioning system had acceptable validity, compared to Vicon, to assess movements representative of indoor sports.

Development of an instrument to measure medical students' perceptions of the assessment environment: initial validation.

PubMed

Sim, Joong Hiong; Tong, Wen Ting; Hong, Wei-Han; Vadivelu, Jamuna; Hassan, Hamimah

2015-01-01

Assessment environment, synonymous with climate or atmosphere, is multifaceted. Although there are valid and reliable instruments for measuring the educational environment, there is no validated instrument for measuring the assessment environment in medical programs. This study aimed to develop an instrument for measuring students' perceptions of the assessment environment in an undergraduate medical program and to examine the psychometric properties of the new instrument. The Assessment Environment Questionnaire (AEQ), a 40-item, four-point (1=Strongly Disagree to 4=Strongly Agree) Likert scale instrument designed by the authors, was administered to medical undergraduates from the authors' institution. The response rate was 626/794 (78.84%). To establish construct validity, exploratory factor analysis (EFA) with principal component analysis and varimax rotation was conducted. To examine the internal consistency reliability of the instrument, Cronbach's α was computed. Mean scores for the entire AEQ and for each factor/subscale were calculated. Mean AEQ scores of students from different academic years and sex were examined. Six hundred and eleven completed questionnaires were analysed. EFA extracted four factors: feedback mechanism (seven items), learning and performance (five items), information on assessment (five items), and assessment system/procedure (three items), which together explained 56.72% of the variance. Based on the four extracted factors/subscales, the AEQ was reduced to 20 items. Cronbach's α for the 20-item AEQ was 0.89, whereas Cronbach's α for the four factors/subscales ranged from 0.71 to 0.87. Mean score for the AEQ was 2.68/4.00. The factor/subscale of 'feedback mechanism' recorded the lowest mean (2.39/4.00), whereas the factor/subscale of 'assessment system/procedure' scored the highest mean (2.92/4.00). Significant differences were found among the AEQ scores of students from different academic years. The AEQ is a valid and reliable instrument. Initial validation supports its use to measure students' perceptions of the assessment environment in an undergraduate medical program.

Selecting postoperative adjuvant systemic therapy for early stage breast cancer: A critical assessment of commercially available gene expression assays

PubMed Central

Schuur, Eric; Angel Aristizabal, Javier; Bargallo Rocha, Juan Enrique; Cabello, Cesar; Elizalde, Roberto; García‐Estévez, Laura; Gomez, Henry L.; Katz, Artur; Nuñez De Pierro, Aníbal

2017-01-01

Risk stratification of patients with early stage breast cancer may support adjuvant chemotherapy decision‐making. This review details the development and validation of six multi‐gene classifiers, each of which claims to provide useful prognostic and possibly predictive information for early stage breast cancer patients. A careful assessment is presented of each test's analytical validity, clinical validity, and clinical utility, as well as the quality of evidence supporting its use. PMID:28211064

Technical skills assessment toolbox: a review using the unitary framework of validity.

PubMed

Ghaderi, Iman; Manji, Farouq; Park, Yoon Soo; Juul, Dorthea; Ott, Michael; Harris, Ilene; Farrell, Timothy M

2015-02-01

The purpose of this study was to create a technical skills assessment toolbox for 35 basic and advanced skills/procedures that comprise the American College of Surgeons (ACS)/Association of Program Directors in Surgery (APDS) surgical skills curriculum and to provide a critical appraisal of the included tools, using contemporary framework of validity. Competency-based training has become the predominant model in surgical education and assessment of performance is an essential component. Assessment methods must produce valid results to accurately determine the level of competency. A search was performed, using PubMed and Google Scholar, to identify tools that have been developed for assessment of the targeted technical skills. A total of 23 assessment tools for the 35 ACS/APDS skills modules were identified. Some tools, such as Operative Performance Rating System (OSATS) and Objective Structured Assessment of Technical Skill (OPRS), have been tested for more than 1 procedure. Therefore, 30 modules had at least 1 assessment tool, with some common surgical procedures being addressed by several tools. Five modules had none. Only 3 studies used Messick's framework to design their validity studies. The remaining studies used an outdated framework on the basis of "types of validity." When analyzed using the contemporary framework, few of these studies demonstrated validity for content, internal structure, and relationship to other variables. This study provides an assessment toolbox for common surgical skills/procedures. Our review shows that few authors have used the contemporary unitary concept of validity for development of their assessment tools. As we progress toward competency-based training, future studies should provide evidence for various sources of validity using the contemporary framework.

A Case Study: Follow-Up Assessment of Facilitated Communication.

ERIC Educational Resources Information Center

Simon, Elliott W.; And Others

1996-01-01

This study of an adolescent with multiple disabilities, including moderate mental retardation, who was reported to engage in validated facilitated communication (FC) found he did not engage in validated FC; performance was equivalent whether food or nonfood reinforcers were used; and the Picture Exchange Communication System was a valid and…

The military social health index: a partial multicultural validation.

PubMed

Van Breda, Adrian D

2008-05-01

Routine military deployments place great stress on military families. Before South African soldiers can be deployed, they undergo a comprehensive health assessment, which includes a social work assessment. The assessment focuses on the resilience of the family system to estimate how well the family will cope when exposed to the stress of deployments. This article reports on the development and validation of a new measuring tool, the Military Social Health Index, or MSHI. The MSHI is made up of four scales, each comprising 14 items, viz social support, problem solving, stressor appraisal, and generalized resistance resources. An initial, large-scale, multicultural validation of the MSHI revealed strong levels of reliability (Cronbach a and standard error of measurement) and validity (factorial, construct, convergent, and discriminant).

Development, qualification, validation and application of the neutral red uptake assay in Chinese Hamster Ovary (CHO) cells using a VITROCELL® VC10® smoke exposure system.

PubMed

Fields, Wanda; Fowler, Kathy; Hargreaves, Victoria; Reeve, Lesley; Bombick, Betsy

2017-04-01

Cytotoxicity assessment of combustible tobacco products by neutral red uptake (NRU) has historically used total particulate matter (TPM) or solvent captured gas vapor phase (GVP), rather than fresh whole smoke. Here, the development, validation and application of the NRU assay in Chinese Hamster Ovary (CHO) cells, following exposure to fresh whole smoke generated with the VITROCELL® VC10® system is described. Whole smoke exposure is particularly important as both particulate and vapor phases of tobacco smoke show cytotoxicity in vitro. The VITROCELL® VC10® system provides exposure at the air liquid interface (ALI) to mimic in vivo conditions for assessing the toxicological impact of smoke in vitro. Instrument and assay validations are crucial for comparative analyses. 1) demonstrate functionality of the VITROCELL® VC10® system by installation, operational and performance qualification, 2) develop and validate a cellular system for assessing cytotoxicity following whole smoke exposure and 3) assess the whole smoke NRU assay sensitivity for statistical differentiation between a reference combustible cigarette (3R4F) and a primarily "heat-not-burn" cigarette (Eclipse). The VITROCELL® VC10® provided consistent generation and delivery of whole smoke; exposure-related changes in in vitro cytotoxicity were observed with reproducible IC 50 values; comparative analysis showed that the heat-not-burn cigarette was significantly (P<0.001) less cytotoxic than the 3R4F combustible cigarette, consistent with the lower levels of chemical constituents liberated by primarily-heating the cigarette versus burning. Copyright © 2017. Published by Elsevier Ltd.

Convergent, discriminant, and criterion validity of DSM-5 traits.

PubMed

Yalch, Matthew M; Hopwood, Christopher J

2016-10-01

Section III of the Diagnostic and Statistical Manual of Mental Disorders (5th edi.; DSM-5; American Psychiatric Association, 2013) contains a system for diagnosing personality disorder based in part on assessing 25 maladaptive traits. Initial research suggests that this aspect of the system improves the validity and clinical utility of the Section II Model. The Computer Adaptive Test of Personality Disorder (CAT-PD; Simms et al., 2011) contains many similar traits as the DSM-5, as well as several additional traits seemingly not covered in the DSM-5. In this study we evaluate the convergent and discriminant validity between the DSM-5 traits, as assessed by the Personality Inventory for DSM-5 (PID-5; Krueger et al., 2012), and CAT-PD in an undergraduate sample, and test whether traits included in the CAT-PD but not the DSM-5 provide incremental validity in association with clinically relevant criterion variables. Results supported the convergent and discriminant validity of the PID-5 and CAT-PD scales in their assessment of 23 out of 25 DSM-5 traits. DSM-5 traits were consistently associated with 11 criterion variables, despite our having intentionally selected clinically relevant criterion constructs not directly assessed by DSM-5 traits. However, the additional CAT-PD traits provided incremental information above and beyond the DSM-5 traits for all criterion variables examined. These findings support the validity of pathological trait models in general and the DSM-5 and CAT-PD models in particular, while also suggesting that the CAT-PD may include additional traits for consideration in future iterations of the DSM-5 system. (PsycINFO Database Record (c) 2016 APA, all rights reserved).

Box-ticking and Olympic high jumping - Physicians' perceptions and acceptance of national physician validation systems.

PubMed

Sehlbach, Carolin; Govaerts, Marjan J B; Mitchell, Sharon; Rohde, Gernot G U; Smeenk, Frank W J M; Driessen, Erik W

2018-05-24

National physician validation systems aim to ensure lifelong learning through periodic appraisals of physicians' competence. Their effectiveness is determined by physicians' acceptance of and commitment to the system. This study, therefore, sought to explore physicians' perceptions and self-reported acceptance of validation across three different physician validation systems in Europe. Using a constructivist grounded-theory approach, we conducted semi-structured interviews with 32 respiratory specialists from three countries with markedly different validation systems: Germany, which has a mandatory, credit-based system oriented to continuing professional development; Denmark, with mandatory annual dialogs and ensuing, non-compulsory activities; and the UK, with a mandatory, portfolio-based revalidation system. We analyzed interview data with a view to identifying factors influencing physicians' perceptions and acceptance. Factors that influenced acceptance were the assessment's authenticity and alignment of its requirements with clinical practice, physicians' beliefs about learning, perceived autonomy, and organizational support. Users' acceptance levels determine any system's effectiveness. To support lifelong learning effectively, national physician validation systems must be carefully designed and integrated into daily practice. Involving physicians in their design may render systems more authentic and improve alignment between individual ambitions and the systems' goals, thereby promoting acceptance.

Validation of measures from the smartphone sway balance application: a pilot study.

PubMed

Patterson, Jeremy A; Amick, Ryan Z; Thummar, Tarunkumar; Rogers, Michael E

2014-04-01

A number of different balance assessment techniques are currently available and widely used. These include both subjective and objective assessments. The ability to provide quantitative measures of balance and posture is the benefit of objective tools, however these instruments are not generally utilized outside of research laboratory settings due to cost, complexity of operation, size, duration of assessment, and general practicality. The purpose of this pilot study was to assess the value and validity of using software developed to access the iPod and iPhone accelerometers output and translate that to the measurement of human balance. Thirty healthy college-aged individuals (13 male, 17 female; age = 26.1 ± 8.5 years) volunteered. Participants performed a static Athlete's Single Leg Test protocol for 10 sec, on a Biodex Balance System SD while concurrently utilizing a mobile device with balance software. Anterior/posterior stability was recorded using both devices, described as the displacement in degrees from level, and was termed the "balance score." There were no significant differences between the two reported balance scores (p = 0.818. Mean balance score on the balance platform was 1.41 ± 0.90, as compared to 1.38 ± 0.72 using the mobile device. There is a need for a valid, convenient, and cost-effective tool to objectively measure balance. Results of this study are promising, as balance score derived from the Smartphone accelerometers were consistent with balance scores obtained from a previously validated balance system. However, further investigation is necessary as this version of the mobile software only assessed balance in the anterior/posterior direction. Additionally, further testing is necessary on a healthy populations and as well as those with impairment of the motor control system. Level 2b (Observational study of validity)(1.)

An alternative to the balance error scoring system: using a low-cost balance board to improve the validity/reliability of sports-related concussion balance testing.

PubMed

Chang, Jasper O; Levy, Susan S; Seay, Seth W; Goble, Daniel J

2014-05-01

Recent guidelines advocate sports medicine professionals to use balance tests to assess sensorimotor status in the management of concussions. The present study sought to determine whether a low-cost balance board could provide a valid, reliable, and objective means of performing this balance testing. Criterion validity testing relative to a gold standard and 7 day test-retest reliability. University biomechanics laboratory. Thirty healthy young adults. Balance ability was assessed on 2 days separated by 1 week using (1) a gold standard measure (ie, scientific grade force plate), (2) a low-cost Nintendo Wii Balance Board (WBB), and (3) the Balance Error Scoring System (BESS). Validity of the WBB center of pressure path length and BESS scores were determined relative to the force plate data. Test-retest reliability was established based on intraclass correlation coefficients. Composite scores for the WBB had excellent validity (r = 0.99) and test-retest reliability (R = 0.88). Both the validity (r = 0.10-0.52) and test-retest reliability (r = 0.61-0.78) were lower for the BESS. These findings demonstrate that a low-cost balance board can provide improved balance testing accuracy/reliability compared with the BESS. This approach provides a potentially more valid/reliable, yet affordable, means of assessing sports-related concussion compared with current methods.

A Low-Cost Contact System to Assess Load Displacement Velocity in a Resistance Training Machine

PubMed Central

Buscà, Bernat; Font, Anna

2011-01-01

This study sought to determine the validity of a new system for assessing the displacement and average velocity within machine-based resistance training exercise using the Chronojump System. The new design is based on a contact bar and a simple, low-cost mechanism that detects the conductivity of electrical potentials with a precision chronograph. This system allows coaches to assess velocity to control the strength training process. A validation study was performed by assessing the concentric phase parameters of a leg press exercise. Output time data from the Chronojump System in combination with the pre-established range of movement was compared with data from a position sensor connected to a Biopac System. A subset of 87 actions from 11 professional tennis players was recorded and, using the two methods, average velocity and displacement variables in the same action were compared. A t-test for dependent samples and a correlation analysis were undertaken. The r value derived from the correlation between the Biopac System and the contact Chronojump System was >0.94 for all measures of displacement and velocity on all loads (p < 0.01). The Effect Size (ES) was 0.18 in displacement and 0.14 in velocity and ranged from 0.09 to 0.31 and from 0.07 to 0.34, respectively. The magnitude of the difference between the two methods in all parameters and the correlation values provided certain evidence of validity of the Chronojump System to assess the average displacement velocity of loads in a resistance training machine. Key points The assessment of speed in resistance machines is a valuable source of information for strength training. Many commercial systems used to assess velocity, power and force are expensive thereby preventing widespread use by coaches and athletes. The system is intended to be a low-cost device for assessing and controlling the velocity exerted on each repetition in any resistance training machine. The system could be easily adapted in any vertical displacement barbell exercise. PMID:24150620

Is the Television Rating System Valid? Indirect, Verbal, and Physical Aggression in Programs Viewed by Fifth Grade Girls and Associations with Behavior

ERIC Educational Resources Information Center

Linder, Jennifer Ruh; Gentile, Douglas A.

2009-01-01

This study had two goals: first, to examine the validity of the television rating system for assessing aggression in programs popular among girls; second, to evaluate the importance of inclusion of non-physical forms of aggression in the ratings system by examining associations between television aggression exposure and behavior. Ninety-nine fifth…

Assessment and classification of cancer breakthrough pain: a systematic literature review.

PubMed

Haugen, Dagny Faksvåg; Hjermstad, Marianne Jensen; Hagen, Neil; Caraceni, Augusto; Kaasa, Stein

2010-06-01

Temporal variations in cancer pain intensity are highly prevalent, and are often difficult to manage. However, the phenomenon is not well understood: several definitions and approaches to classification and bedside assessment of cancer breakthrough pain (BTP) have been described. The present study is a systematic review of published literature on cancer BTP to answer the following questions: which terms and definitions have been used; are there validated assessment tools; which domains of BTP do the tools delineate, and which items do they contain; how have assessment tools been applied within clinical studies; and are there validated classification systems for BTP. A systematic search of the peer-reviewed literature was performed using five major databases. Of 375 titles and abstracts initially identified, 51 articles were examined in detail. Analysis of these publications indicates a range of overlapping but distinct definitions have been used to characterize BTP; 42 of the included papers presented one or more ways of classifying BTP; and while 10 tools to assess patients' experience of BTP were identified, only 2 have been partially validated. We conclude that there is no widely accepted definition, classification system or well-validated assessment tool for cancer-related breakthrough pain, but there is strong concurrence on most of its key attributes. With further work in this area, an internationally agreed upon definition and classification system for cancer-related breakthrough pain, and a standard approach on how to measure it, hold the promise to improve patient care and support research in this poor-prognosis cancer pain syndrome.

The Integral Theory System Questionnaire: an anatomically directed questionnaire to determine pelvic floor dysfunctions in women.

PubMed

Wagenlehner, Florian Martin Erich; Fröhlich, Oliver; Bschleipfer, Thomas; Weidner, Wolfgang; Perletti, Gianpaolo

2014-06-01

Anatomical damage to pelvic floor structures may cause multiple symptoms. The Integral Theory System Questionnaire (ITSQ) is a holistic questionnaire that uses symptoms to help locate damage in specific connective tissue structures as a guide to reconstructive surgery. It is based on the integral theory, which states that pelvic floor symptoms and prolapse are both caused by lax suspensory ligaments. The aim of the present study was to psychometrically validate the ITSQ. Established psychometric properties including validity, reliability, and responsiveness were considered for evaluation. Criterion validity was assessed in a cohort of 110 women with pelvic floor dysfunctions by analyzing the correlation of questionnaire responses with objective clinical data. Test-retest was performed with questionnaires from 47 patients. Cronbach's alpha and "split-half" reliability coefficients were calculated for inner consistency analysis. Psychometric properties of ITSQ were comparable to the ones of previously validated Pelvic Floor Questionnaires. Face validity and content validity were approved by an expert group of the International Collaboration of Pelvic Floor surgeons. Convergent validity assessed using Bayesian method was at least as accurate as the expert assessment of anatomical defects. Objective data measurement in patients demonstrated significant correlations with ITSQ domains fulfilling criterion validity. Internal consistency values ranked from 0.85 to 0.89 in different scenarios. The ITSQ proofed accurate and is able to serve as a holistic Pelvic Floor Questionnaire directing symptoms to site-specific pelvic floor reconstructive surgery.

Validation of a global scale to assess the quality of interprofessional teamwork in mental health settings.

PubMed

Tomizawa, Ryoko; Yamano, Mayumi; Osako, Mitue; Hirabayashi, Naotugu; Oshima, Nobuo; Sigeta, Masahiro; Reeves, Scott

2017-12-01

Few scales currently exist to assess the quality of interprofessional teamwork through team members' perceptions of working together in mental health settings. The purpose of this study was to revise and validate an interprofessional scale to assess the quality of teamwork in inpatient psychiatric units and to use it multi-nationally. A literature review was undertaken to identify evaluative teamwork tools and develop an additional 12 items to ensure a broad global focus. Focus group discussions considered adaptation to different care systems using subjective judgements from 11 participants in a pre-test of items. Data quality, construct validity, reproducibility, and internal consistency were investigated in the survey using an international comparative design. Exploratory factor analysis yielded five factors with 21 items: 'patient/community centred care', 'collaborative communication', 'interprofessional conflict', 'role clarification', and 'environment'. High overall internal consistency, reproducibility, adequate face validity, and reasonable construct validity were shown in the USA and Japan. The revised Collaborative Practice Assessment Tool (CPAT) is a valid measure to assess the quality of interprofessional teamwork in psychiatry and identifies the best strategies to improve team performance. Furthermore, the revised scale will generate more rigorous evidence for collaborative practice in psychiatry internationally.

Evaluating the Diagnostic Validity of the Facet-Based Formative Assessment System

ERIC Educational Resources Information Center

DeBarger, Angela H.; DiBello, Louis; Minstrell, Jim; Stout, William; Pellegrino, James; Haertel, Geneva; Feng, Mingyu

2011-01-01

The research design and team constitute a multidisciplinary attack on problems of educational and assessment design in physics instruction. Components of the research include: (a) an Evidence-Centered Design analysis of Diagnoser instructional materials and assessments that provides a view of the evidentiary coherence of the existing system; (b)…

Towards a Framework for the Validation of Early Childhood Assessment Systems

ERIC Educational Resources Information Center

Goldstein, Jessica; Flake, Jessica Kay

2016-01-01

American early childhood education is in the midst of drastic change. In recent years, states have begun the process of overhauling early childhood education systems in response to federal grant competitions, bringing an increased focus on assessment and accountability for early learning programs. The assessment of young children is fraught with…

Measuring Teacher Effectiveness with the Pennsylvania Value-Added Assessment System

ERIC Educational Resources Information Center

Bowen, Naomi

2017-01-01

The purpose of this research was to determine if the Pennsylvania Value-Added Assessment System Average Growth Index (PVAAS AGI) scores, derived from standardized tests and calculated for Pennsylvania schools, provide a valid and reliable assessment of teacher effectiveness, as these scores are currently used to derive 15% of the annual…

Validity and reliability of a pilot scale for assessment of multiple system atrophy symptoms.

PubMed

Matsushima, Masaaki; Yabe, Ichiro; Takahashi, Ikuko; Hirotani, Makoto; Kano, Takahiro; Horiuchi, Kazuhiro; Houzen, Hideki; Sasaki, Hidenao

2017-01-01

Multiple system atrophy (MSA) is a rare progressive neurodegenerative disorder for which brief yet sensitive scale is required in order for use in clinical trials and general screening. We previously compared several scales for the assessment of MSA symptoms and devised an eight-item pilot scale with large standardized response mean [handwriting, finger taps, transfers, standing with feet together, turning trunk, turning 360°, gait, body sway]. The aim of the present study is to investigate the validity and reliability of a simple pilot scale for assessment of multiple system atrophy symptoms. Thirty-two patients with MSA (15 male/17 female; 20 cerebellar subtype [MSA-C]/12 parkinsonian subtype [MSA-P]) were prospectively registered between January 1, 2014 and February 28, 2015. Patients were evaluated by two independent raters using the Unified MSA Rating Scale (UMSARS), Scale for Assessment and Rating of Ataxia (SARA), and the pilot scale. Correlations between UMSARS, SARA, pilot scale scores, intraclass correlation coefficients (ICCs), and Cronbach's alpha coefficients were calculated. Pilot scale scores significantly correlated with scores for UMSARS Parts I, II, and IV as well as with SARA scores. Intra-rater and inter-rater ICCs and Cronbach's alpha coefficients remained high (> 0.94) for all measures. The results of the present study indicate the validity and reliability of the eight-item pilot scale, particularly for the assessment of symptoms in patients with early state multiple system atrophy.

Recommendations for standardizing validation procedures assessing physical activity of older persons by monitoring body postures and movements.

PubMed

Lindemann, Ulrich; Zijlstra, Wiebren; Aminian, Kamiar; Chastin, Sebastien F M; de Bruin, Eling D; Helbostad, Jorunn L; Bussmann, Johannes B J

2014-01-10

Physical activity is an important determinant of health and well-being in older persons and contributes to their social participation and quality of life. Hence, assessment tools are needed to study this physical activity in free-living conditions. Wearable motion sensing technology is used to assess physical activity. However, there is a lack of harmonisation of validation protocols and applied statistics, which make it hard to compare available and future studies. Therefore, the aim of this paper is to formulate recommendations for assessing the validity of sensor-based activity monitoring in older persons with focus on the measurement of body postures and movements. Validation studies of body-worn devices providing parameters on body postures and movements were identified and summarized and an extensive inter-active process between authors resulted in recommendations about: information on the assessed persons, the technical system, and the analysis of relevant parameters of physical activity, based on a standardized and semi-structured protocol. The recommended protocols can be regarded as a first attempt to standardize validity studies in the area of monitoring physical activity.

Evaluation of the Performance of Routine Information System Management (PRISM) framework: evidence from Uganda.

PubMed

Hotchkiss, David R; Aqil, Anwer; Lippeveld, Theo; Mukooyo, Edward

2010-07-03

Sound policy, resource allocation and day-to-day management decisions in the health sector require timely information from routine health information systems (RHIS). In most low- and middle-income countries, the RHIS is viewed as being inadequate in providing quality data and continuous information that can be used to help improve health system performance. In addition, there is limited evidence on the effectiveness of RHIS strengthening interventions in improving data quality and use. The purpose of this study is to evaluate the usefulness of the newly developed Performance of Routine Information System Management (PRISM) framework, which consists of a conceptual framework and associated data collection and analysis tools to assess, design, strengthen and evaluate RHIS. The specific objectives of the study are: a) to assess the reliability and validity of the PRISM instruments and b) to assess the validity of the PRISM conceptual framework. Facility- and worker-level data were collected from 110 health care facilities in twelve districts in Uganda in 2004 and 2007 using records reviews, structured interviews and self-administered questionnaires. The analysis procedures include Cronbach's alpha to assess internal consistency of selected instruments, test-retest analysis to assess the reliability and sensitivity of the instruments, and bivariate and multivariate statistical techniques to assess validity of the PRISM instruments and conceptual framework. Cronbach's alpha analysis suggests high reliability (0.7 or greater) for the indices measuring a promotion of a culture of information, RHIS tasks self-efficacy and motivation. The study results also suggest that a promotion of a culture of information influences RHIS tasks self-efficacy, RHIS tasks competence and motivation, and that self-efficacy and the presence of RHIS staff have a direct influence on the use of RHIS information, a key aspect of RHIS performance. The study results provide some empirical support for the reliability and validity of the PRISM instruments and the validity of the PRISM conceptual framework, suggesting that the PRISM approach can be effectively used by RHIS policy makers and practitioners to assess the RHIS and evaluate RHIS strengthening interventions. However, additional studies with larger sample sizes are needed to further investigate the value of the PRISM instruments in exploring the linkages between RHIS data quality and use, and health systems performance.

Protocol and Demonstrations of Probabilistic Reliability Assessment for Structural Health Monitoring Systems (Preprint)

DTIC Science & Technology

2011-11-01

assessment to quality of localization/characterization estimates. This protocol includes four critical components: (1) a procedure to identify the...critical factors impacting SHM system performance; (2) a multistage or hierarchical approach to SHM system validation; (3) a model -assisted evaluation...Lindgren, E. A ., Buynak, C. F., Steffes, G., Derriso, M., “ Model -assisted Probabilistic Reliability Assessment for Structural Health Monitoring

Select Methodology for Validating Advanced Satellite Measurement Systems

NASA Technical Reports Server (NTRS)

Larar, Allen M.; Zhou, Daniel K.; Liu, Xi; Smith, William L.

2008-01-01

Advanced satellite sensors are tasked with improving global measurements of the Earth's atmosphere, clouds, and surface to enable enhancements in weather prediction, climate monitoring capability, and environmental change detection. Measurement system validation is crucial to achieving this goal and maximizing research and operational utility of resultant data. Field campaigns including satellite under-flights with well calibrated FTS sensors aboard high-altitude aircraft are an essential part of the validation task. This presentation focuses on an overview of validation methodology developed for assessment of high spectral resolution infrared systems, and includes results of preliminary studies performed to investigate the performance of the Infrared Atmospheric Sounding Interferometer (IASI) instrument aboard the MetOp-A satellite.

Helicopter simulation validation using flight data

NASA Technical Reports Server (NTRS)

Key, D. L.; Hansen, R. S.; Cleveland, W. B.; Abbott, W. Y.

1982-01-01

A joint NASA/Army effort to perform a systematic ground-based piloted simulation validation assessment is described. The best available mathematical model for the subject helicopter (UH-60A Black Hawk) was programmed for real-time operation. Flight data were obtained to validate the math model, and to develop models for the pilot control strategy while performing mission-type tasks. The validated math model is to be combined with motion and visual systems to perform ground based simulation. Comparisons of the control strategy obtained in flight with that obtained on the simulator are to be used as the basis for assessing the fidelity of the results obtained in the simulator.

IV&V Project Assessment Process Validation

NASA Technical Reports Server (NTRS)

Driskell, Stephen

2012-01-01

The Space Launch System (SLS) will launch NASA's Multi-Purpose Crew Vehicle (MPCV). This launch vehicle will provide American launch capability for human exploration and travelling beyond Earth orbit. SLS is designed to be flexible for crew or cargo missions. The first test flight is scheduled for December 2017. The SLS SRR/SDR provided insight into the project development life cycle. NASA IV&V ran the standard Risk Based Assessment and Portfolio Based Risk Assessment to identify analysis tasking for the SLS program. This presentation examines the SLS System Requirements Review/System Definition Review (SRR/SDR), IV&V findings for IV&V process validation correlation to/from the selected IV&V tasking and capabilities. It also provides a reusable IEEE 1012 scorecard for programmatic completeness across the software development life cycle.

The accuracy of Internet search engines to predict diagnoses from symptoms can be assessed with a validated scoring system.

PubMed

Shenker, Bennett S

2014-02-01

To validate a scoring system that evaluates the ability of Internet search engines to correctly predict diagnoses when symptoms are used as search terms. We developed a five point scoring system to evaluate the diagnostic accuracy of Internet search engines. We identified twenty diagnoses common to a primary care setting to validate the scoring system. One investigator entered the symptoms for each diagnosis into three Internet search engines (Google, Bing, and Ask) and saved the first five webpages from each search. Other investigators reviewed the webpages and assigned a diagnostic accuracy score. They rescored a random sample of webpages two weeks later. To validate the five point scoring system, we calculated convergent validity and test-retest reliability using Kendall's W and Spearman's rho, respectively. We used the Kruskal-Wallis test to look for differences in accuracy scores for the three Internet search engines. A total of 600 webpages were reviewed. Kendall's W for the raters was 0.71 (p<0.0001). Spearman's rho for test-retest reliability was 0.72 (p<0.0001). There was no difference in scores based on Internet search engine. We found a significant difference in scores based on the webpage's order on the Internet search engine webpage (p=0.007). Pairwise comparisons revealed higher scores in the first webpages vs. the fourth (corr p=0.009) and fifth (corr p=0.017). However, this significance was lost when creating composite scores. The five point scoring system to assess diagnostic accuracy of Internet search engines is a valid and reliable instrument. The scoring system may be used in future Internet research. Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.

The Predictive Validity of a Gender-Responsive Needs Assessment: An Exploratory Study

ERIC Educational Resources Information Center

Salisbury, Emily J.; Van Voorhis, Patricia; Spiropoulos, Georgia V.

2009-01-01

Risk assessment and classification systems for women have been largely derived from male-based systems. As a result, many of the needs unique to women are not formally assessed or treated. Emerging research advocating a gender-responsive approach to the supervision and treatment of women offenders suggests that needs such as abuse, mental health,…

Validation of the measure automobile emissions model : a statistical analysis

DOT National Transportation Integrated Search

2000-09-01

The Mobile Emissions Assessment System for Urban and Regional Evaluation (MEASURE) model provides an external validation capability for hot stabilized option; the model is one of several new modal emissions models designed to predict hot stabilized e...

Skeletal age assessment in children using an open compact MRI system.

PubMed

Terada, Yasuhiko; Kono, Saki; Tamada, Daiki; Uchiumi, Tomomi; Kose, Katsumi; Miyagi, Ryo; Yamabe, Eiko; Yoshioka, Hiroshi

2013-06-01

MRI may be a noninvasive and alternative tool for skeletal age assessment in children, although few studies have reported on this topic. In this article, skeletal age was assessed over a wide range of ages using an open, compact MRI optimized for the imaging of a child's hand and wrist, and its validity was evaluated. MR images and their three-dimensional segmentation visualized detailed skeletal features of each bone in the hand and wrist. Skeletal age was then independently scored from the MR images by two raters, according to the Tanner-Whitehouse Japan system. The skeletal age assessed by MR rating demonstrated a strong positive correlation with chronological age. The intrarater and inter-rater reproducibilities were significantly high. These results demonstrate the validity and reliability of skeletal age assessment using MRI. Copyright © 2012 Wiley Periodicals, Inc.

Validation of Procedures for Monitoring Crewmember Immune Function SDBI-1900, SMO-015 - Integrated Immune

NASA Technical Reports Server (NTRS)

Crucian, Brian; Stowe, Raymond; Mehta, Satish; Uchakin, Peter; Nehlsen-Cannarella, Sandra; Morukov, Boris; Pierson, Duane; Sams, Clarence

2007-01-01

There is ample evidence to suggest that space flight leads to immune system dysregulation. This may be a result of microgravity, confinement, physiological stress, radiation, environment or other mission-associated factors. The clinical risk from prolonged immune dysregulation during space flight are not yet determined, but may include increased incidence of infection, allergy, hypersensitivity, hematological malignancy or altered wound healing. Each of the clinical events resulting from immune dysfunction has the potential to impact mission critical objectives during exploration-class missions. To date, precious little in-flight immune data has been generated to assess this phenomenon. The majority of recent flight immune studies have been post-flight assessments, which may not accurately reflect the in-flight condition. There are no procedures currently in place to monitor immune function or its effect on crew health. The objective of this Supplemental Medical Objective (SMO) is to develop and validate an immune monitoring strategy consistent with operational flight requirements and constraints. This SMO will assess the clinical risks resulting from the adverse effects of space flight on the human immune system and will validate a flight-compatible immune monitoring strategy. Characterization of the clinical risk and the development of a monitoring strategy are necessary prerequisite activities prior to validating countermeasures. This study will determine, to the best level allowed by current technology, the in-flight status of crewmembers immune system. Pre-flight, in-flight and post-flight assessments of immune status, immune function, viral reactivation and physiological stress will be performed. The in-flight samples will allow a distinction between legitimate in-flight alterations and the physiological stresses of landing and readaptation which are believed to alter landing day assessments. The overall status of the immune system during flight (activation, deficiency, dysregulation) and the response of the immune system to specific latent virus reactivation (known to occur during space flight) will be thoroughly assessed. Following completion of the SMO the data will be evaluated to determine the optimal set of assays for routine monitoring of crewmember immune system function, should the clinical risk warrant such monitoring.

Assessing the competences associated with a nursing Bachelor thesis by means of rubrics.

PubMed

Llaurado-Serra, M; Rodríguez, E; Gallart, A; Fuster, P; Monforte-Royo, C; De Juan, M Á

2018-07-01

Writing a Bachelor thesis is the last step in obtaining a university degree. The thesis may be job- or research-orientated, but it must demonstrate certain degree-level competences. Rubrics are a useful way of unifying the assessment criteria. To design a system of rubrics for assessing the competences associated with the Bachelor thesis of a nursing degree, to examine the system's reliability and validity and to analyse results in relation to the final thesis mark. Cross-sectional and psychometric study conducted between 2012 and 2014. Nursing degree at a Spanish university. Twelve tutors who designed the system of rubrics. Students (n = 76) who wrote their Bachelor thesis during the 2013-2014 academic year. After deciding which aspects would be assessed, who would assess them and when, the tutors developed seven rubrics (drafting process, assessment of the written thesis by the supervisor and by a panel, student self-assessment, peer assessment, tutor evaluation of the peer assessment and panel assessment of the viva). We analysed the reliability (inter-rater and internal consistency) and validity (convergent and discriminant) of the rubrics, and also the relationship between the competences assessed and the final thesis mark. All the rubrics had internal consistency coefficients >0.80. The rubric for oral communication skills (viva) yielded inter-rater reliability of 0.95. Factor analysis indicated a unidimensional structure for all but one of the rubrics, the exception being the rubric for peer assessment, which had a two-factor structure. The main competences associated with a good quality Bachelor thesis were written communication skills and the ability to work independently. The assessment system based on seven rubrics is shown to be valid and reliable. Writing a Bachelor thesis requires a range of degree-level competences and it offers nursing students the opportunity to develop their evidence-based practice skills. Copyright © 2018 Elsevier Ltd. All rights reserved.

Validity of the Microsoft Kinect for assessment of postural control.

PubMed

Clark, Ross A; Pua, Yong-Hao; Fortin, Karine; Ritchie, Callan; Webster, Kate E; Denehy, Linda; Bryant, Adam L

2012-07-01

Clinically feasible methods of assessing postural control such as timed standing balance and functional reach tests provide important information, however, they cannot accurately quantify specific postural control mechanisms. The Microsoft Kinect™ system provides real-time anatomical landmark position data in three dimensions (3D), and given that it is inexpensive, portable and simple to setup it may bridge this gap. This study assessed the concurrent validity of the Microsoft Kinect™ against a benchmark reference, a multiple-camera 3D motion analysis system, in 20 healthy subjects during three postural control tests: (i) forward reach, (ii) lateral reach, and (iii) single-leg eyes-closed standing balance. For the reach tests, the outcome measures consisted of distance reached and trunk flexion angle in the sagittal (forward reach) and coronal (lateral reach) planes. For the standing balance test the range and deviation of movement in the anatomical landmark positions for the sternum, pelvis, knee and ankle and the lateral and anterior trunk flexion angle were assessed. The Microsoft Kinect™ and 3D motion analysis systems had comparable inter-trial reliability (ICC difference=0.06±0.05; range, 0.00-0.16) and excellent concurrent validity, with Pearson's r-values >0.90 for the majority of measurements (r=0.96±0.04; range, 0.84-0.99). However, ordinary least products analyses demonstrated proportional biases for some outcome measures associated with the pelvis and sternum. These findings suggest that the Microsoft Kinect™ can validly assess kinematic strategies of postural control. Given the potential benefits it could therefore become a useful tool for assessing postural control in the clinical setting. Copyright © 2012 Elsevier B.V. All rights reserved.

Development and validation of the Sports Athlete Foot and Ankle Score: an instrument for sports-related ankle injuries.

PubMed

Morssinkhof, M L A; Wang, O; James, L; van der Heide, H J L; Winson, I G

2013-09-01

Many existing scoring systems assess ankle function, but there is no evidence that any of them has been validated in a group of patients with a higher demand on their ankle function. Problems include ceiling effects, not being able to detect change or they do not contain a sports-subscale. The aim of this study was to create a validated self-administered scoring system for ankle injuries in the higher performing athlete. First, 26 patients were interviewed to solicit opinions needed to create the final score, which is modified from the Foot and Ankle Outcome Score (FAOS). Second, SAFAS was validated in a group of 25 athletes with and 14 athletes without ankle injury. It is a self-administered region specific sports foot and ankle score that contains four subscales assessing the levels of symptoms, pain, daily living and sports. The Spearman correlation coefficients between SAFAS and the Foot and Ankle Ability Measure (FAAM) ranged from 0.78 to 0.88. Content validity is established by key informant interviews, expert opinions and a high satisfaction rate of 75%. Cronbach's alpha indicated good internal consistency of each subscale ranging from 0.77 to 0.92. SAFAS has shown good evidence for being a valid instrudent for assessing sports-related ankle injuries in high-performing athletes. Copyright © 2013 European Foot and Ankle Society. Published by Elsevier Ltd. All rights reserved.

The CMEMS-Med-MFC-Biogeochemistry operational system: implementation of NRT and Multi-Year validation tools

NASA Astrophysics Data System (ADS)

Salon, Stefano; Cossarini, Gianpiero; Bolzon, Giorgio; Teruzzi, Anna

2017-04-01

The Mediterranean Monitoring and Forecasting Centre (Med-MFC) is one of the regional production centres of the EU Copernicus Marine Environment Monitoring Service (CMEMS). Med-MFC manages a suite of numerical model systems for the operational delivery of the CMEMS products, providing continuous monitoring and forecasting of the Mediterranean marine environment. The CMEMS products of fundamental biogeochemical variables (chlorophyll, nitrate, phosphate, oxygen, phytoplankton biomass, primary productivity, pH, pCO2) are organised as gridded datasets and are available at the marine.copernicus.eu web portal. Quantitative estimates of CMEMS products accuracy are prerequisites to release reliable information to intermediate users, end users and to other downstream services. In particular, validation activities aim to deliver accuracy information of the model products and to serve as a long term monitoring of the performance of the modelling systems. The quality assessment of model output is implemented using a multiple-stages approach, basically inspired to the classic "GODAE 4 Classes" metrics and criteria (consistency, quality, performance and benefit). Firstly, pre-operational runs qualify the operational model system against historical data, also providing a verification of the improvements of the new model system release with respect to the previous version. Then, the near real time (NRT) validation aims at delivering a sustained on-line skill assessment of the model analysis and forecast, relying on the NRT available relevant observations (e.g. in situ, Bio Argo and satellite observations). NRT validation results are operated on weekly basis and published on the MEDEAF web portal (www.medeaf.inogs.it). On a quarterly basis, the integration of the NRT validation activities delivers a comprehensive view of the accuracy of model forecast through the official CMEMS validation webpage. Multi-Year production (e.g. reanalysis runs) follows a similar procedure, and the validation is achieved using the same metrics on available historical observations (e.g. the World Ocean Atlas 2013 dataset). Results of the validation activities show that the comparison of the different variables of the CMEMS products with experimental data is feasible at different levels (i.e. either as skill assessment of the short-term forecast and as model consistency through different system versions) and at different spatial and temporal scales. In particular, the accuracy of some variables (chlorophyll, nitrate, oxygen) can be provided at weekly scale and sub-mesoscale, others (carbonate system, phosphate) at quarterly/annual and sub-basin scale, and others (phytoplankton biomass, primary production) only at the level of consistency of model functioning (e.g. literature- or climatology-based). In spite of a wide literature on model validation has been produced so far, maintaining a validation framework in the biogeochemical operational contest that fulfils GODAE criteria is still a challenge. Recent results of the validation activities and new potential validation framework at the Med-MFC will be presented in our contribution.

A framework to assess management performance in district health systems: a qualitative and quantitative case study in Iran.

PubMed

Tabrizi, Jafar Sadegh; Gholipour, Kamal; Iezadi, Shabnam; Farahbakhsh, Mostafa; Ghiasi, Akbar

2018-01-01

The aim was to design a district health management performance framework for Iran's healthcare system. The mixed-method study was conducted between September 2015 and May 2016 in Tabriz, Iran. In this study, the indicators of district health management performance were obtained by analyzing the 45 semi-structured surveys of experts in the public health system. Content validity of performance indicators which were generated in qualitative part were reviewed and confirmed based on content validity index (CVI). Also content validity ratio (CVR) was calculated using data acquired from a survey of 21 experts in quantitative part. The result of this study indicated that, initially, 81 indicators were considered in framework of district health management performance and, at the end, 53 indicators were validated and confirmed. These indicators were classified in 11 categories which include: human resources and organizational creativity, management and leadership, rules and ethics, planning and evaluation, district managing, health resources management and economics, community participation, quality improvement, research in health system, health information management, epidemiology and situation analysis. The designed framework model can be used to assess the district health management and facilitates performance improvement at the district level.

Evaluation of Urinary Tract Dilation Classification System for Grading Postnatal Hydronephrosis.

PubMed

Hodhod, Amr; Capolicchio, John-Paul; Jednak, Roman; El-Sherif, Eid; El-Doray, Abd El-Alim; El-Sherbiny, Mohamed

2016-03-01

We assessed the reliability and validity of the Urinary Tract Dilation classification system as a new grading system for postnatal hydronephrosis. We retrospectively reviewed charts of patients who presented with hydronephrosis from 2008 to 2013. We included patients diagnosed prenatally and those with hydronephrosis discovered incidentally during the first year of life. We excluded cases involving urinary tract infection, neurogenic bladder and chromosomal anomalies, those associated with extraurinary congenital malformations and those with followup of less than 24 months without resolution. Hydronephrosis was graded postnatally using the Society for Fetal Urology system, and then the management protocol was chosen. All units were regraded using the Urinary Tract Dilation classification system and compared to the Society for Fetal Urology system to assess reliability. Univariate and multivariate analyses were performed to assess the validity of the Urinary Tract Dilation classification system in predicting hydronephrosis resolution and surgical intervention. A total of 490 patients (730 renal units) were eligible to participate. The Urinary Tract Dilation classification system was reliable in the assessment of hydronephrosis (parallel forms 0.92). Hydronephrosis resolved in 357 units (49%), and 86 units (12%) were managed by surgical intervention. The remainder of renal units demonstrated stable or improved hydronephrosis. Multivariate analysis revealed that the likelihood of surgical intervention was predicted independently by Urinary Tract Dilation classification system risk group, while Society for Fetal Urology grades were predictive of likelihood of resolution. The Urinary Tract Dilation classification system is reliable for evaluation of postnatal hydronephrosis and is valid in predicting surgical intervention. Copyright © 2016 American Urological Association Education and Research, Inc. Published by Elsevier Inc. All rights reserved.

Sandia National Laboratories: Fabrication, Testing and Validation

Science.gov Websites

; Technology Defense Systems & Assessments About Defense Systems & Assessments Program Areas safe, secure, reliable, and can fully support the Nation's deterrence policy. Employing only the most support of this mission, Sandia National Laboratories has a significant role in advancing the "state

Evaluation of Two Observational Assessment Systems for Children's Development and Learning

ERIC Educational Resources Information Center

Kim, Do-Hong; Smith, JaneDiane

2010-01-01

This study provided preliminary evidence for the reliability and validity of "Teaching Strategies GOLD", a recently developed observational system for assessing young children's development and learning. The measurement properties of "Teaching Strategies GOLD" were compared with those of an older instrument, "The Creative…

Procedure-specific assessment tool for flexible pharyngo-laryngoscopy: gathering validity evidence and setting pass-fail standards.

PubMed

Melchiors, Jacob; Petersen, K; Todsen, T; Bohr, A; Konge, Lars; von Buchwald, Christian

2018-06-01

The attainment of specific identifiable competencies is the primary measure of progress in the modern medical education system. The system, therefore, requires a method for accurately assessing competence to be feasible. Evidence of validity needs to be gathered before an assessment tool can be implemented in the training and assessment of physicians. This evidence of validity must according to the contemporary theory on validity be gathered from specific sources in a structured and rigorous manner. The flexible pharyngo-laryngoscopy (FPL) is central to the otorhinolaryngologist. We aim to evaluate the flexible pharyngo-laryngoscopy assessment tool (FLEXPAT) created in a previous study and to establish a pass-fail level for proficiency. Eighteen physicians with different levels of experience (novices, intermediates, and experienced) were recruited to the study. Each performed an FPL on two patients. These procedures were video recorded, blinded, and assessed by two specialists. The score was expressed as the percentage of a possible max score. Cronbach's α was used to analyze internal consistency of the data, and a generalizability analysis was performed. The scores of the three different groups were explored, and a pass-fail level was determined using the contrasting groups' standard setting method. Internal consistency was strong with a Cronbach's α of 0.86. We found a generalizability coefficient of 0.72 sufficient for moderate stakes assessment. We found a significant difference between the novice and experienced groups (p < 0.001) and strong correlation between experience and score (Pearson's r = 0.75). The pass/fail level was established at 72% of the maximum score. Applying this pass-fail level in the test population resulted in half of the intermediary group receiving a failing score. We gathered validity evidence for the FLEXPAT according to the contemporary framework as described by Messick. Our results support a claim of validity and are comparable to other studies exploring clinical assessment tools. The high rate of physicians underperforming in the intermediary group demonstrates the need for continued educational intervention. Based on our work, we recommend the use of the FLEXPAT in clinical assessment of FPL and the application of a pass-fail level of 72% for proficiency.

Development of an Encompassing Questionnaire for Evaluating the Outcomes Following Total Knee Arthroplasty.

PubMed

Chughtai, Morad; Khlopas, Anton; Thomas, Melbin; Gwam, Chukwuweike U; Jauregui, Julio J; Elmallah, Randa K; Roche, Martin; Delanois, Ronald E

2017-01-10

There are many standardized scales and questionnaires used to evaluate TKA patients; however, individually they do not always assess patients adequately. Consequently, many are used in combinations to provide a thorough evaluation. However, this leads to redundancy, confusion, and an excessive patient time-burden. Therefore, the purpose of this study was to develop a usable combined knee questionnaire that combines questions in a non-redundant manner. Specifically, we aimed to: 1) create a combined knee questionnaire that encompasses questions from multiple systems, while eliminating redundancy; 2) correlate the new system with the existing validated questionnaires; and 3) determine the length of time it takes to administer this new questionnaire. In a previous study, it was determined that the six most commonly cited validated systems to assess the knee were the: Knee Society Score (KSS), The Western Ontario and McMaster Universities Arthritis Index (WOMAC), Knee injury and Osteoarthritis Outcome Score (KOOS), Lower Extremity Functional Scale (LEFS), Activity Rating Scale (ARS), and Short-Form-36 (SF-36). Therefore, we ensured that the new questionnaire encompassed all elements of these systems. After development of the combined questionnaire, we co-administered it to 20 subjects alongside the above validated questionnaires. We then transposed the corresponding answers from the combined questionnaire to each selected validated system to perform an intra-class correlation analysis. In addition, we recorded the length of time it took to administer the new questionnaire and compared it to the time it took to administer the individual validated questionnaires. Intra-class correlation analysis demonstrated statistically significant positive correlations between the KSS, WOMAC, KOOS, LEFS, ARS, SF-36, and the corresponding questions in the combined questionnaire. The mean length of time it took to administer the combined questionnaire (mean, 10.1 minutes, range, 6.6 to 12.6 minutes) was significantly shorter than the time it took to administer the selected validated questionnaires (mean, 21.3 minutes, range, 17.3 to 24.1 minutes). We have proposed an all-encompassing combined knee questionnaire that eliminates redundancy and inefficiency during the evaluation of TKA patients. It is a reliable, time-efficient system that can be utilized to fill out the most commonly used questionnaires for assessing TKA. Standardization and uniform use of this questionnaire may simplify future patient assessment following TKA.

Validation in Support of Internationally Harmonised OECD Test Guidelines for Assessing the Safety of Chemicals.

PubMed

Gourmelon, Anne; Delrue, Nathalie

Ten years elapsed since the OECD published the Guidance document on the validation and international regulatory acceptance of test methods for hazard assessment. Much experience has been gained since then in validation centres, in countries and at the OECD on a variety of test methods that were subjected to validation studies. This chapter reviews validation principles and highlights common features that appear to be important for further regulatory acceptance across studies. Existing OECD-agreed validation principles will most likely generally remain relevant and applicable to address challenges associated with the validation of future test methods. Some adaptations may be needed to take into account the level of technique introduced in test systems, but demonstration of relevance and reliability will continue to play a central role as pre-requisite for the regulatory acceptance. Demonstration of relevance will become more challenging for test methods that form part of a set of predictive tools and methods, and that do not stand alone. OECD is keen on ensuring that while these concepts evolve, countries can continue to rely on valid methods and harmonised approaches for an efficient testing and assessment of chemicals.

Automatic, semi-automatic and manual validation of urban drainage data.

PubMed

Branisavljević, N; Prodanović, D; Pavlović, D

2010-01-01

Advances in sensor technology and the possibility of automated long distance data transmission have made continuous measurements the preferable way of monitoring urban drainage processes. Usually, the collected data have to be processed by an expert in order to detect and mark the wrong data, remove them and replace them with interpolated data. In general, the first step in detecting the wrong, anomaly data is called the data quality assessment or data validation. Data validation consists of three parts: data preparation, validation scores generation and scores interpretation. This paper will present the overall framework for the data quality improvement system, suitable for automatic, semi-automatic or manual operation. The first two steps of the validation process are explained in more detail, using several validation methods on the same set of real-case data from the Belgrade sewer system. The final part of the validation process, which is the scores interpretation, needs to be further investigated on the developed system.

Assessment of abdominal muscle function using the Biodex System-4. Validity and reliability in healthy volunteers and patients with giant ventral hernia.

PubMed

Gunnarsson, U; Johansson, M; Strigård, K

2011-08-01

The decrease in recurrence rates in ventral hernia surgery have led to a redirection of focus towards other important patient-related endpoints. One such endpoint is abdominal wall function. The aim of the present study was to evaluate the reliability and external validity of abdominal wall strength measurement using the Biodex System-4 with a back abdomen unit. Ten healthy volunteers and ten patients with ventral hernias exceeding 10 cm were recruited. Test-retest reliability, both with and without girdle, was evaluated by comparison of measurements at two test occasions 1 week apart. Reliability was calculated by the interclass correlation coefficients (ICC) method. Validity was evaluated by correlation with the well-established International Physical Activity Questionnaire (IPAQ) and a self-assessment of abdominal wall strength. One person in the healthy group was excluded after the first test due to neck problems following minor trauma. The reliability was excellent (>0.75), with ICC values between 0.92 and 0.97 for the different modalities tested. No differences were seen between testing with and without a girdle. Validity was also excellent both when calculated as correlation to self-assessment of abdominal wall strength, and to IPAQ, giving Kendall tau values of 0.51 and 0.47, respectively, and corresponding P values of 0.002 and 0.004. Measurement of abdominal muscle function using the Biodex System-4 is a reliable and valid method to assess this important patient-related endpoint. Further investigations will be made to explore the potential of this technique in the evaluation of the results of ventral hernia surgery, and to compare muscle function after different abdominal wall reconstruction techniques.

Reliability and Validity of the Arthroscopic International Cartilage Repair Society Classification System: Correlation With Histological Assessment of Depth.

PubMed

Dwyer, Tim; Martin, C Ryan; Kendra, Rita; Sermer, Corey; Chahal, Jaskarndip; Ogilvie-Harris, Darrell; Whelan, Daniel; Murnaghan, Lucas; Nauth, Aaron; Theodoropoulos, John

2017-06-01

To determine the interobserver reliability of the International Cartilage Repair Society (ICRS) grading system of chondral lesions in cadavers, to determine the intraobserver reliability of the ICRS grading system comparing arthroscopy and video assessment, and to compare the arthroscopic ICRS grading system with histological grading of lesion depth. Eighteen lesions in 5 cadaveric knee specimens were arthroscopically graded by 7 fellowship-trained arthroscopic surgeons using the ICRS classification system. The arthroscopic video of each lesion was sent to the surgeons 6 weeks later for repeat grading and determination of intraobserver reliability. Lesions were biopsied, and the depth of the cartilage lesion was assessed. Reliability was calculated using intraclass correlations. The interobserver reliability was 0.67 (95% confidence interval, 0.5-0.89) for the arthroscopic grading, and the intraobserver reliability with the video grading was 0.8 (95% confidence interval, 0.67-0.9). A high correlation was seen between the arthroscopic grading of depth and the histological grading of depth (0.91); on average, surgeons graded lesions using arthroscopy a mean of 0.37 (range, 0-0.86) deeper than the histological grade. The arthroscopic ICRS classification system has good interobserver and intraobserver reliability. A high correlation with histological assessment of depth provides evidence of validity for this classification system. As cartilage lesions are treated on the basis of the arthroscopic ICRS classification, it is important to ascertain the reliability and validity of this method. Copyright © 2016 Arthroscopy Association of North America. Published by Elsevier Inc. All rights reserved.

Validating Neuro-QoL short forms and targeted scales with people who have multiple sclerosis.

PubMed

Miller, Deborah M; Bethoux, Francois; Victorson, David; Nowinski, Cindy J; Buono, Sarah; Lai, Jin-Shei; Wortman, Katy; Burns, James L; Moy, Claudia; Cella, David

2016-05-01

Multiple sclerosis (MS) is a chronic, progressive, and disabling disease of the central nervous system with dramatic variations in the combination and severity of symptoms it can produce. The lack of reliable disease-specific health-related quality of life (HRQL) measures for use in clinical trials prompted the development of the Neurology Quality of Life (Neuro-QOL) instrument, which includes 13 scales that assess physical, emotional, cognitive, and social domains, for use in a variety of neurological illnesses. The objective of this research paper is to conduct an initial assessment of the reliability and validation of the Neuro-QOL short forms (SFs) in MS. We assessed reliability, concurrent validity, known groups validity, and responsiveness between cross-sectional and longitudinal data in 161 recruited MS patients. Internal consistency was high for all measures (α = 0.81-0.95) and ICCs were within the acceptable range (0.76-0.91); concurrent and known groups validity were highest with the Global HRQL question. Longitudinal assessment was limited by the lack of disease progression in the group. The Neuro-QOL SFs demonstrate good internal consistency, test-re-test reliability, and concurrent and known groups validity in this MS population, supporting the validity of Neuro-QOL in adults with MS. © The Author(s), 2015.

Cyber Selection Test Research Effort for U.S. Army New Accessions

DTIC Science & Technology

2017-10-12

assessment game 3. Develop an operational version of the STA game which incorporates assessments from phase 1 and (through game -play) examines...3 more STA abilities •5 STA behaviors 4. Validate the system thinking assessment game in an operational setting C O M PL ET ED PL AN N ED Research...Information Identifies Elements of Systems Models Relationships Understands System Dynamics Evaluates & Revises Model Applies Understanding to Problem STA Game

Translation into Brazilian Portuguese and validation of the "Quantitative Global Scarring Grading System for Post-acne Scarring" *

PubMed Central

Cachafeiro, Thais Hofmann; Escobar, Gabriela Fortes; Maldonado, Gabriela; Cestari, Tania Ferreira

2014-01-01

The "Quantitative Global Scarring Grading System for Postacne Scarring" was developed in English for acne scar grading, based on the number and severity of each type of scar. The aims of this study were to translate this scale into Brazilian Portuguese and verify its reliability and validity. The study followed five steps: Translation, Expert Panel, Back Translation, Approval of authors and Validation. The translated scale showed high internal consistency and high test-retest reliability, confirming its reproducibility. Therefore, it has been validated for our population and can be recommended as a reliable instrument to assess acne scarring. PMID:25184939

Methodologies for semiquantitative evaluation of hip osteoarthritis by magnetic resonance imaging: approaches based on the whole organ and focused on active lesions.

PubMed

Jaremko, Jacob L; Lambert, Robert G W; Zubler, Veronika; Weber, Ulrich; Loeuille, Damien; Roemer, Frank W; Cibere, Jolanda; Pianta, Marcus; Gracey, David; Conaghan, Philip; Ostergaard, Mikkel; Maksymowych, Walter P

2014-02-01

As a wider variety of therapeutic options for osteoarthritis (OA) becomes available, there is an increasing need to objectively evaluate disease severity on magnetic resonance imaging (MRI). This is more technically challenging at the hip than at the knee, and as a result, few systematic scoring systems exist. The OMERACT (Outcome Measures in Rheumatology) filter of truth, discrimination, and feasibility can be used to validate image-based scoring systems. Our objective was (1) to review the imaging features relevant to the assessment of severity and progression of hip OA; and (2) to review currently used methods to grade these features in existing hip OA scoring systems. A systematic literature review was conducted. MEDLINE keyword search was performed for features of arthropathy (such as hip + bone marrow edema or lesion, synovitis, cyst, effusion, cartilage, etc.) and scoring system (hip + OA + MRI + score or grade), with a secondary manual search for additional references in the retrieved publications. Findings relevant to the severity of hip OA include imaging markers associated with inflammation (bone marrow lesion, synovitis, effusion), structural damage (cartilage loss, osteophytes, subchondral cysts, labral tears), and predisposing geometric factors (hip dysplasia, femoral-acetabular impingement). Two approaches to the semiquantitative assessment of hip OA are represented by Hip OA MRI Scoring System (HOAMS), a comprehensive whole organ assessment of nearly all findings, and the Hip Inflammation MRI Scoring System (HIMRISS), which selectively scores only active lesions (bone marrow lesion, synovitis/effusion). Validation is presently confined to limited assessment of reliability. Two methods for semiquantitative assessment of hip OA on MRI have been described and validation according to the OMERACT Filter is limited to evaluation of reliability.

A Model-Based Approach to Support Validation of Medical Cyber-Physical Systems.

PubMed

Silva, Lenardo C; Almeida, Hyggo O; Perkusich, Angelo; Perkusich, Mirko

2015-10-30

Medical Cyber-Physical Systems (MCPS) are context-aware, life-critical systems with patient safety as the main concern, demanding rigorous processes for validation to guarantee user requirement compliance and specification-oriented correctness. In this article, we propose a model-based approach for early validation of MCPS, focusing on promoting reusability and productivity. It enables system developers to build MCPS formal models based on a library of patient and medical device models, and simulate the MCPS to identify undesirable behaviors at design time. Our approach has been applied to three different clinical scenarios to evaluate its reusability potential for different contexts. We have also validated our approach through an empirical evaluation with developers to assess productivity and reusability. Finally, our models have been formally verified considering functional and safety requirements and model coverage.

A Model-Based Approach to Support Validation of Medical Cyber-Physical Systems

PubMed Central

Silva, Lenardo C.; Almeida, Hyggo O.; Perkusich, Angelo; Perkusich, Mirko

2015-01-01

Medical Cyber-Physical Systems (MCPS) are context-aware, life-critical systems with patient safety as the main concern, demanding rigorous processes for validation to guarantee user requirement compliance and specification-oriented correctness. In this article, we propose a model-based approach for early validation of MCPS, focusing on promoting reusability and productivity. It enables system developers to build MCPS formal models based on a library of patient and medical device models, and simulate the MCPS to identify undesirable behaviors at design time. Our approach has been applied to three different clinical scenarios to evaluate its reusability potential for different contexts. We have also validated our approach through an empirical evaluation with developers to assess productivity and reusability. Finally, our models have been formally verified considering functional and safety requirements and model coverage. PMID:26528982

Comparison of seven fall risk assessment tools in community-dwelling Korean older women.

PubMed

Kim, Taekyoung; Xiong, Shuping

2017-03-01

This study aimed to compare seven widely used fall risk assessment tools in terms of validity and practicality, and to provide a guideline for choosing appropriate fall risk assessment tools for elderly Koreans. Sixty community-dwelling Korean older women (30 fallers and 30 matched non-fallers) were evaluated. Performance measures of all tools were compared between the faller and non-faller groups through two sample t-tests. Receiver Operating Characteristic curves were generated with odds ratios for discriminant analysis. Results showed that four tools had significant discriminative power, and the shortened version of Falls Efficacy Scale (SFES) showed excellent discriminant validity, followed by Berg Balance Scale (BBS) with acceptable discriminant validity. The Mini Balance Evaluation System Test and Timed Up and Go, however, had limited discriminant validities. In terms of practicality, SFES was also excellent. These findings suggest that SFES is the most suitable tool for assessing the fall risks of community-dwelling Korean older women, followed by BBS. Practitioner Summary: There is no general guideline on which fall risk assessment tools are suitable for community-dwelling Korean older women. This study compared seven widely used assessment tools in terms of validity and practicality. Results suggested that the short Falls Efficacy Scale is the most suitable tool, followed by Berg Balance Scale.

12 CFR 217.101 - Definitions.

Code of Federal Regulations, 2014 CFR

2014-01-01

... management and maintenance system; and control, oversight, and validation system for credit risk of wholesale... advanced IRB systems, operational risk management processes, operational risk data and assessment systems... the seller and the obligor (intercompany accounts receivable and receivables subject to contra...

Simulation-based assessment to identify critical gaps in safe anesthesia resident performance.

PubMed

Blum, Richard H; Boulet, John R; Cooper, Jeffrey B; Muret-Wagstaff, Sharon L

2014-01-01

Valid methods are needed to identify anesthesia resident performance gaps early in training. However, many assessment tools in medicine have not been properly validated. The authors designed and tested use of a behaviorally anchored scale, as part of a multiscenario simulation-based assessment system, to identify high- and low-performing residents with regard to domains of greatest concern to expert anesthesiology faculty. An expert faculty panel derived five key behavioral domains of interest by using a Delphi process (1) Synthesizes information to formulate a clear anesthetic plan; (2) Implements a plan based on changing conditions; (3) Demonstrates effective interpersonal and communication skills with patients and staff; (4) Identifies ways to improve performance; and (5) Recognizes own limits. Seven simulation scenarios spanning pre-to-postoperative encounters were used to assess performances of 22 first-year residents and 8 fellows from two institutions. Two of 10 trained faculty raters blinded to trainee program and training level scored each performance independently by using a behaviorally anchored rating scale. Residents, fellows, facilitators, and raters completed surveys. Evidence supporting the reliability and validity of the assessment scores was procured, including a high generalizability coefficient (ρ = 0.81) and expected performance differences between first-year resident and fellow participants. A majority of trainees, facilitators, and raters judged the assessment to be useful, realistic, and representative of critical skills required for safe practice. The study provides initial evidence to support the validity of a simulation-based performance assessment system for identifying critical gaps in safe anesthesia resident performance early in training.

Assessing preschoolers interactive behaviour: A validation study of the "Coding System for Mother-Child Interaction".

PubMed

Baiao, R; Baptista, J; Carneiro, A; Pinto, R; Toscano, C; Fearon, P; Soares, I; Mesquita, A R

2018-07-01

The preschool years are a period of great developmental achievements, which impact critically on a child's interactive skills. Having valid and reliable measures to assess interactive behaviour at this stage is therefore crucial. The aim of this study was to describe the adaptation and validation of the child coding of the Coding System for Mother-Child Interactions and discuss its applications and implications in future research and practice. Two hundred twenty Portuguese preschoolers and their mothers were videotaped during a structured task. Child and mother interactive behaviours were coded based on the task. Maternal reports on the child's temperament and emotional and behaviour problems were also collected, along with family psychosocial information. Interrater agreement was confirmed. The use of child Cooperation, Enthusiasm, and Negativity as subscales was supported by their correlations across tasks. Moreover, these subscales were correlated with each other, which supports the use of a global child interactive behaviour score. Convergent validity with a measure of emotional and behavioural problems (Child Behaviour Checklist 1 ½-5) was established, as well as divergent validity with a measure of temperament (Children's Behaviour Questionnaire-Short Form). Regarding associations with family variables, child interactive behaviour was only associated with maternal behaviour. Findings suggest that this coding system is a valid and reliable measure for assessing child interactive behaviour in preschool age children. It therefore represents an important alternative to this area of research and practice, with reduced costs and with more flexible training requirements. Attention should be given in future research to expanding this work to clinical populations and different age groups. © 2018 John Wiley & Sons Ltd.

Establishment of a VISAR Measurement System for Material Model Validation in DSTO

DTIC Science & Technology

2013-02-01

advancements published in the works by L.M. Baker, E.R. Hollenbach and W.F. Hemsing [1-3] and results in the user-friendly interface and configuration of the...VISAR system [4] used in the current work . VISAR tests are among the mandatory instrumentation techniques when validating material models and...The present work reports on preliminary tests using the recently commissioned DSTO VISAR system, providing an assessment of the experimental set-up

Development and validation of an exercise performance support system for people with lower extremity impairment.

PubMed

Minor, M A; Reid, J C; Griffin, J Z; Pittman, C B; Patrick, T B; Cutts, J H

1998-02-01

To identify innovative strategies to support appropriate, self-directed exercise that increase physical activity levels of people with arthritis. This article reports on one interactive, multimedia exercise performance support system (PSS) for people with lower extremity impairments in strength or flexibility. An interdisciplinary team developed the PSS using self-report of lower extremity musculoskeletal impairments (flexibility and strength) to produce an individualized exercise program with video and print educational materials. Initial evaluation has investigated the validity and reliability of program assessments and recommendations. PSS self-report and professional assessments were similar, with more impairments indicated by self-report. PSS exercise recommendations were similar to those made by 3 expert physical therapists using the same exercise data base. Results of PSS impairment assessments were stable over a 1-week period. PSS exercise recommendations appear to be reliable and a valid reflection of current exercise knowledge in rheumatology. Furthermore, users were able to complete the computer-based program with minimal assistance and reported it to be enjoyable and informative.

A radiation-free mixed-reality training environment and assessment concept for C-arm-based surgery.

PubMed

Stefan, Philipp; Habert, Séverine; Winkler, Alexander; Lazarovici, Marc; Fürmetz, Julian; Eck, Ulrich; Navab, Nassir

2018-06-25

The discrepancy of continuously decreasing opportunities for clinical training and assessment and the increasing complexity of interventions in surgery has led to the development of different training and assessment options like anatomical models, computer-based simulators or cadaver trainings. However, trainees, following training, assessment and ultimately performing patient treatment, still face a steep learning curve. To address this problem for C-arm-based surgery, we introduce a realistic radiation-free simulation system that combines patient-based 3D printed anatomy and simulated X-ray imaging using a physical C-arm. To explore the fidelity and usefulness of the proposed mixed-reality system for training and assessment, we conducted a user study with six surgical experts performing a facet joint injection on the simulator. In a technical evaluation, we show that our system simulates X-ray images accurately with an RMSE of 1.85 mm compared to real X-ray imaging. The participants expressed agreement with the overall realism of the simulation, the usefulness of the system for assessment and strong agreement with the usefulness of such a mixed-reality system for training of novices and experts. In a quantitative analysis, we furthermore evaluated the suitability of the system for the assessment of surgical skills and gather preliminary evidence for validity. The proposed mixed-reality simulation system facilitates a transition to C-arm-based surgery and has the potential to complement or even replace large parts of cadaver training, to provide a safe assessment environment and to reduce the risk for errors when proceeding to patient treatment. We propose an assessment concept and outline the steps necessary to expand the system into a test instrument that provides reliable and justified assessments scores indicative of surgical proficiency with sufficient evidence for validity.

Validity and reliability of wii fit balance board for the assessment of balance of healthy young adults and the elderly.

PubMed

Chang, Wen-Dien; Chang, Wan-Yi; Lee, Chia-Lun; Feng, Chi-Yen

2013-10-01

[Purpose] Balance is an integral part of human ability. The smart balance master system (SBM) is a balance test instrument with good reliability and validity, but it is expensive. Therefore, we modified a Wii Fit balance board, which is a convenient balance assessment tool, and analyzed its reliability and validity. [Subjects and Methods] We recruited 20 healthy young adults and 20 elderly people, and administered 3 balance tests. The correlation coefficient and intraclass correlation of both instruments were analyzed. [Results] There were no statistically significant differences in the 3 tests between the Wii Fit balance board and the SBM. The Wii Fit balance board had a good intraclass correlation (0.86-0.99) for the elderly people and positive correlations (r = 0.58-0.86) with the SBM. [Conclusions] The Wii Fit balance board is a balance assessment tool with good reliability and high validity for elderly people, and we recommend it as an alternative tool for assessing balance ability.

Caries Risk Assessment for Determination of Focus and Intensity of Prevention in a Dental School Clinic.

ERIC Educational Resources Information Center

Dodds, Michael W. J.; Suddick, Richard P.

1995-01-01

A study at the University of Texas, San Antonio's dental school resulted in development of a system of caries risk assessment, applied to all undergraduate clinic patients. The rationale, structure, elements, and application of the system are outlined, and course content supporting the system is noted. Need for validation and other improvements is…

Additional Support for the Information Systems Analyst Exam as a Valid Program Assessment Tool

ERIC Educational Resources Information Center

Carpenter, Donald A.; Snyder, Johnny; Slauson, Gayla Jo; Bridge, Morgan K.

2011-01-01

This paper presents a statistical analysis to support the notion that the Information Systems Analyst (ISA) exam can be used as a program assessment tool in addition to measuring student performance. It compares ISA exam scores earned by students in one particular Computer Information Systems program with scores earned by the same students on the…

Stroke Risk Stratification and its Validation using Ultrasonic Echolucent Carotid Wall Plaque Morphology: A Machine Learning Paradigm.

PubMed

Araki, Tadashi; Jain, Pankaj K; Suri, Harman S; Londhe, Narendra D; Ikeda, Nobutaka; El-Baz, Ayman; Shrivastava, Vimal K; Saba, Luca; Nicolaides, Andrew; Shafique, Shoaib; Laird, John R; Gupta, Ajay; Suri, Jasjit S

2017-01-01

Stroke risk stratification based on grayscale morphology of the ultrasound carotid wall has recently been shown to have a promise in classification of high risk versus low risk plaque or symptomatic versus asymptomatic plaques. In previous studies, this stratification has been mainly based on analysis of the far wall of the carotid artery. Due to the multifocal nature of atherosclerotic disease, the plaque growth is not restricted to the far wall alone. This paper presents a new approach for stroke risk assessment by integrating assessment of both the near and far walls of the carotid artery using grayscale morphology of the plaque. Further, this paper presents a scientific validation system for stroke risk assessment. Both these innovations have never been presented before. The methodology consists of an automated segmentation system of the near wall and far wall regions in grayscale carotid B-mode ultrasound scans. Sixteen grayscale texture features are computed, and fed into the machine learning system. The training system utilizes the lumen diameter to create ground truth labels for the stratification of stroke risk. The cross-validation procedure is adapted in order to obtain the machine learning testing classification accuracy through the use of three sets of partition protocols: (5, 10, and Jack Knife). The mean classification accuracy over all the sets of partition protocols for the automated system in the far and near walls is 95.08% and 93.47%, respectively. The corresponding accuracies for the manual system are 94.06% and 92.02%, respectively. The precision of merit of the automated machine learning system when compared against manual risk assessment system are 98.05% and 97.53% for the far and near walls, respectively. The ROC of the risk assessment system for the far and near walls is close to 1.0 demonstrating high accuracy. Copyright © 2016 Elsevier Ltd. All rights reserved.

Evidence on existing caries risk assessment systems: are they predictive of future caries?

PubMed

Tellez, M; Gomez, J; Pretty, I; Ellwood, R; Ismail, A I

2013-02-01

To critically appraise evidence for the prediction of caries using four caries risk assessment (CRA) systems/guidelines (Cariogram, Caries Management by Risk Assessment (CAMBRA), American Dental Association (ADA), and American Academy of Pediatric Dentistry (AAPD)). This review focused on prospective cohort studies or randomized controlled trials. A systematic search strategy was developed to locate papers published in Medline Ovid and Cochrane databases. The search identified 539 scientific reports, and after title and abstract review, 137 were selected for full review and 14 met the following inclusion criteria: (i) used as validating criterion caries incidence/increment, (ii) involved human subjects and natural carious lesions, and (iii) published in peer-reviewed journals. In addition, papers were excluded if they met one or more of the following criteria: (i) incomplete description of sample selection, outcomes, or small sample size and (ii) not meeting the criteria for best evidence under the prognosis category of the Oxford Centre for Evidence-Based Medicine. There are wide variations among the systems in terms of definitions of caries risk categories, type and number of risk factors/markers, and disease indicators. The Cariogram combined sensitivity and specificity for predicting caries in permanent dentition ranges from 110 to 139 and is the only system for which prospective studies have been conducted to assess its validity. The Cariogram had limited prediction utility in preschool children, and a moderate to good performance for sorting out elderly individuals into caries risk groups. One retrospective analysis on CAMBRA's CRA reported higher incidence of cavitated lesions among those assessed as extreme-risk patients when compared with those at low risk. The evidence on the validity for existing systems for CRA is limited. It is unknown if the identification of high-risk individuals can lead to more effective long-term patient management that prevents caries initiation and arrests or reverses the progression of lesions. There is an urgent need to develop valid and reliable methods for caries risk assessment that are based on best evidence for prediction and disease management rather than opinions of experts.

Assessing Arthroscopic Skills Using Wireless Elbow-Worn Motion Sensors.

PubMed

Kirby, Georgina S J; Guyver, Paul; Strickland, Louise; Alvand, Abtin; Yang, Guang-Zhong; Hargrove, Caroline; Lo, Benny P L; Rees, Jonathan L

2015-07-01

Assessment of surgical skill is a critical component of surgical training. Approaches to assessment remain predominantly subjective, although more objective measures such as Global Rating Scales are in use. This study aimed to validate the use of elbow-worn, wireless, miniaturized motion sensors to assess the technical skill of trainees performing arthroscopic procedures in a simulated environment. Thirty participants were divided into three groups on the basis of their surgical experience: novices (n = 15), intermediates (n = 10), and experts (n = 5). All participants performed three standardized tasks on an arthroscopic virtual reality simulator while wearing wireless wrist and elbow motion sensors. Video output was recorded and a validated Global Rating Scale was used to assess performance; dexterity metrics were recorded from the simulator. Finally, live motion data were recorded via Bluetooth from the wireless wrist and elbow motion sensors and custom algorithms produced an arthroscopic performance score. Construct validity was demonstrated for all tasks, with Global Rating Scale scores and virtual reality output metrics showing significant differences between novices, intermediates, and experts (p < 0.001). The correlation of the virtual reality path length to the number of hand movements calculated from the wireless sensors was very high (p < 0.001). A comparison of the arthroscopic performance score levels with virtual reality output metrics also showed highly significant differences (p < 0.01). Comparisons of the arthroscopic performance score levels with the Global Rating Scale scores showed strong and highly significant correlations (p < 0.001) for both sensor locations, but those of the elbow-worn sensors were stronger and more significant (p < 0.001) than those of the wrist-worn sensors. A new wireless assessment of surgical performance system for objective assessment of surgical skills has proven valid for assessing arthroscopic skills. The elbow-worn sensors were shown to achieve an accurate assessment of surgical dexterity and performance. The validation of an entirely objective assessment of arthroscopic skill with wireless elbow-worn motion sensors introduces, for the first time, a feasible assessment system for the live operating theater with the added potential to be applied to other surgical and interventional specialties. Copyright © 2015 by The Journal of Bone and Joint Surgery, Incorporated.

EVA: laparoscopic instrument tracking based on Endoscopic Video Analysis for psychomotor skills assessment.

PubMed

Oropesa, Ignacio; Sánchez-González, Patricia; Chmarra, Magdalena K; Lamata, Pablo; Fernández, Alvaro; Sánchez-Margallo, Juan A; Jansen, Frank Willem; Dankelman, Jenny; Sánchez-Margallo, Francisco M; Gómez, Enrique J

2013-03-01

The EVA (Endoscopic Video Analysis) tracking system is a new system for extracting motions of laparoscopic instruments based on nonobtrusive video tracking. The feasibility of using EVA in laparoscopic settings has been tested in a box trainer setup. EVA makes use of an algorithm that employs information of the laparoscopic instrument's shaft edges in the image, the instrument's insertion point, and the camera's optical center to track the three-dimensional position of the instrument tip. A validation study of EVA comprised a comparison of the measurements achieved with EVA and the TrEndo tracking system. To this end, 42 participants (16 novices, 22 residents, and 4 experts) were asked to perform a peg transfer task in a box trainer. Ten motion-based metrics were used to assess their performance. Construct validation of the EVA has been obtained for seven motion-based metrics. Concurrent validation revealed that there is a strong correlation between the results obtained by EVA and the TrEndo for metrics, such as path length (ρ = 0.97), average speed (ρ = 0.94), or economy of volume (ρ = 0.85), proving the viability of EVA. EVA has been successfully validated in a box trainer setup, showing the potential of endoscopic video analysis to assess laparoscopic psychomotor skills. The results encourage further implementation of video tracking in training setups and image-guided surgery.

Assessing Attachment in Psychotherapy: Validation of the Patient Attachment Coding System (PACS).

PubMed

Talia, Alessandro; Miller-Bottome, Madeleine; Daniel, Sarah I F

2017-01-01

The authors present and validate the Patient Attachment Coding System (PACS), a transcript-based instrument that assesses clients' in-session attachment based on any session of psychotherapy, in multiple treatment modalities. One-hundred and sixty clients in different types of psychotherapy (cognitive-behavioural, cognitive-behavioural-enhanced, psychodynamic, relational, supportive) and from three different countries were administered the Adult Attachment Interview (AAI) prior to treatment, and one session for each client was rated with the PACS by independent coders. Results indicate strong inter-rater reliability, and high convergent validity of the PACS scales and classifications with the AAI. These results present the PACS as a practical alternative to the AAI in psychotherapy research and suggest that clinicians using the PACS can assess clients' attachment status on an ongoing basis by monitoring clients' verbal activity. These results also provide information regarding the ways in which differences in attachment status play out in therapy sessions and further the study of attachment in psychotherapy from a pre-treatment client factor to a process variable. Copyright © 2015 John Wiley & Sons, Ltd. The Patient Attachment Coding System is a valid measure of attachment that can classify clients' attachment based on any single psychotherapy transcript, in many therapeutic modalities Client differences in attachment manifest in part independently of the therapist's contributions Client adult attachment patterns are likely to affect psychotherapeutic processes. Copyright © 2015 John Wiley & Sons, Ltd.

Testing the Predictive Validity of the Hendrich II Fall Risk Model.

PubMed

Jung, Hyesil; Park, Hyeoun-Ae

2018-03-01

Cumulative data on patient fall risk have been compiled in electronic medical records systems, and it is possible to test the validity of fall-risk assessment tools using these data between the times of admission and occurrence of a fall. The Hendrich II Fall Risk Model scores assessed during three time points of hospital stays were extracted and used for testing the predictive validity: (a) upon admission, (b) when the maximum fall-risk score from admission to falling or discharge, and (c) immediately before falling or discharge. Predictive validity was examined using seven predictive indicators. In addition, logistic regression analysis was used to identify factors that significantly affect the occurrence of a fall. Among the different time points, the maximum fall-risk score assessed between admission and falling or discharge showed the best predictive performance. Confusion or disorientation and having a poor ability to rise from a sitting position were significant risk factors for a fall.

Using Teacher Ratings to Track the Growth and Development of Young Children Using the "Teaching Strategies GOLD"® Assessment System

ERIC Educational Resources Information Center

Lambert, Richard G.; Kim, Do-Hong; Burts, Diane C.

2014-01-01

An important consideration in determining the validity of an observational assessment measure for young children is the variability attributed to the child versus that ascribed to the assessor or to some other factor such as classroom context. The "Teaching Strategies GOLD"® assessment system was used to elicit teacher ratings of a…

Assessing the Validity of the Qualistar Early Learning Quality Rating and Improvement System as a Tool for Improving Child-Care Quality

ERIC Educational Resources Information Center

Zellman, Gail L.; Perlman, Michal; Le, Vi-Nhuan; Setodji, Claude Messan

2008-01-01

As a result of the generally low quality of child care in the United States and the increased emphasis on accountability in education policy, quality rating systems (QRSs) are proliferating in the child-care arena. QRSs assess child-care providers on multiple dimensions of quality and integrate these assessments into an easily understood summary…

The Stroop test as a measure of performance validity in adults clinically referred for neuropsychological assessment.

PubMed

Erdodi, Laszlo A; Sagar, Sanya; Seke, Kristian; Zuccato, Brandon G; Schwartz, Eben S; Roth, Robert M

2018-06-01

This study was designed to develop performance validity indicators embedded within the Delis-Kaplan Executive Function Systems (D-KEFS) version of the Stroop task. Archival data from a mixed clinical sample of 132 patients (50% male; M Age = 43.4; M Education = 14.1) clinically referred for neuropsychological assessment were analyzed. Criterion measures included the Warrington Recognition Memory Test-Words and 2 composites based on several independent validity indicators. An age-corrected scaled score ≤6 on any of the 4 trials reliably differentiated psychometrically defined credible and noncredible response sets with high specificity (.87-.94) and variable sensitivity (.34-.71). An inverted Stroop effect was less sensitive (.14-.29), but comparably specific (.85-90) to invalid performance. Aggregating the newly developed D-KEFS Stroop validity indicators further improved classification accuracy. Failing the validity cutoffs was unrelated to self-reported depression or anxiety. However, it was associated with elevated somatic symptom report. In addition to processing speed and executive function, the D-KEFS version of the Stroop task can function as a measure of performance validity. A multivariate approach to performance validity assessment is generally superior to univariate models. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Reliability and validity of the Microsoft Kinect for evaluating static foot posture

PubMed Central

2013-01-01

Background The evaluation of foot posture in a clinical setting is useful to screen for potential injury, however disagreement remains as to which method has the greatest clinical utility. An inexpensive and widely available imaging system, the Microsoft Kinect™, may possess the characteristics to objectively evaluate static foot posture in a clinical setting with high accuracy. The aim of this study was to assess the intra-rater reliability and validity of this system for assessing static foot posture. Methods Three measures were used to assess static foot posture; traditional visual observation using the Foot Posture Index (FPI), a 3D motion analysis (3DMA) system and software designed to collect and analyse image and depth data from the Kinect. Spearman’s rho was used to assess intra-rater reliability and concurrent validity of the Kinect to evaluate foot posture, and a linear regression was used to examine the ability of the Kinect to predict total visual FPI score. Results The Kinect demonstrated moderate to good intra-rater reliability for four FPI items of foot posture (ρ = 0.62 to 0.78) and moderate to good correlations with the 3DMA system for four items of foot posture (ρ = 0.51 to 0.85). In contrast, intra-rater reliability of visual FPI items was poor to moderate (ρ = 0.17 to 0.63), and correlations with the Kinect and 3DMA systems were poor (absolute ρ = 0.01 to 0.44). Kinect FPI items with moderate to good reliability predicted 61% of the variance in total visual FPI score. Conclusions The majority of the foot posture items derived using the Kinect were more reliable than the traditional visual assessment of FPI, and were valid when compared to a 3DMA system. Individual foot posture items recorded using the Kinect were also shown to predict a moderate degree of variance in the total visual FPI score. Combined, these results support the future potential of the Kinect to accurately evaluate static foot posture in a clinical setting. PMID:23566934

Objective assessment of laparoscopic skills using a virtual reality stimulator.

PubMed

Eriksen, J R; Grantcharov, T

2005-09-01

Virtual reality simulation has a great potential as a training and assessment tool of laparoscopic skills. The study was carried out to investigate whether the LapSim system (Surgical Science Ltd., Gothenburg, Sweden) was able to differentiate between subjects with different laparoscopic experience and thus to demonstrate its construct validity. Subjects 24 were divided into two groups: experienced (performed > 100 laparoscopic procedures, n = 10) and beginners (performed <10 laparoscopic procedures, n = 14). Assessment of laparoscopic skills was based on parameters measured by the computer system. Experienced surgeons performed consistently better than the residents. Significant differences in the parameters time and economy of motion existed between the two groups in seven of seven tasks. Regarding error parameters, differences existed in most but not all tasks. LapSim was able to differentiate between subjects with different laparoscopic experience. This indicates that the system measures skills relevant for laparoscopic surgery and can be used in training programs as a valid assessment tool.

Reliability and validity evidence of the Assessment of Language Use in Social Contexts for Adults (ALUSCA).

PubMed

Valente, Ana Rita S; Hall, Andreia; Alvelos, Helena; Leahy, Margaret; Jesus, Luis M T

2018-04-12

The appropriate use of language in context depends on the speaker's pragmatic language competencies. A coding system was used to develop a specific and adult-focused self-administered questionnaire to adults who stutter and adults who do not stutter, The Assessment of Language Use in Social Contexts for Adults, with three categories: precursors, basic exchanges, and extended literal/non-literal discourse. This paper presents the content validity, item analysis, reliability coefficients and evidences of construct validity of the instrument. Content validity analysis was based on a two-stage process: first, 11 pragmatic questionnaires were assessed to identify items that probe each pragmatic competency and to create the first version of the instrument; second, items were assessed qualitatively by an expert panel composed by adults who stutter and controls, and quantitatively and qualitatively by an expert panel composed by clinicians. A pilot study was conducted with five adults who stutter and five controls to analyse items and calculate reliability. Construct validity evidences were obtained using the hypothesized relationships method and factor analysis with 28 adults who stutter and 28 controls. Concerning content validity, the questionnaires assessed up to 13 pragmatic competencies. Qualitative and quantitative analysis revealed ambiguities in items construction. Disagreement between experts was solved through item modification. The pilot study showed that the instrument presented internal consistency and temporal stability. Significant differences between adults who stutter and controls and different response profiles revealed the instrument's underlying construct. The instrument is reliable and presented evidences of construct validity.

Development and validation of a patient-reported questionnaire assessing systemic therapy induced diarrhea in oncology patients.

PubMed

Lui, Michelle; Gallo-Hershberg, Daniela; DeAngelis, Carlo

2017-12-22

Systemic therapy-induced diarrhea (STID) is a common side effect experienced by more than half of cancer patients. Despite STID-associated complications and poorer quality of life (QoL), no validated assessment tools exist to accurately assess STID occurrence and severity to guide clinical management. Therefore, we developed and validated a patient-reported questionnaire (STIDAT). The STIDAT was developed using the FDA iterative process for patient-reported outcomes. A literature search uncovered potential items and questions for questionnaire construction used by oncology clinicians to develop questions for the preliminary instrument. The instrument was evaluated on its face validity and content validity by patient interviews. Repetitive, similar and different themes uncovered from patient interviews were implemented to revise the instrument to the version used for validation. Patients starting high-risk STID treatments were monitored using the STIDAT, bowel diaries and EORTC QLQ-C30. The STIDAT was evaluated for construct validity using exploratory factor analysis (EFA) using minimal residual method with Promax rotation, reliability and consistency. A weighted scoring system was developed and a receiver-operating characteristic (ROC) curve evaluated the tool's ability to detect STID occurrence. Median scores and variability were analysed to determine how well it differentiates between diarrhea severities. A post-hoc analysis determined how diarrhea severity impacted QoL of cancer patients. Patients defined diarrhea based on presence of watery stool. The STIDAT assessed patient's perception of having diarrhea, daily number of bowel movements, daily number of diarrhea episodes, antidiarrheal medication use, the presence of urgency, abdominal pain, abdominal spasms or fecal incontinence, patient's perception of diarrhea severity, and QoL. These dimensions were sorted into four clusters using EFA - patient's perception of diarrhea, frequency of diarrhea, fecal incontinence and abdominal symptoms. Cronbach's alpha was 0.78; kappa ranged from 0.934-0.952, except for abdominal spasms (κ = 0.0455). The positive predictive value was 96.4%, with the minimum score of 1.35 predicting a positive STID occurrence. Patients with moderate or severe diarrhea experience significant decreases in QoL compared to those with no diarrhea. This is the first patient-reported questionnaire that accurately predicts the occurrence and severity of diarrhea in oncology patients via assessing several bowel habit dimensions.

Technical Report Series on Global Modeling and Data Assimilation. Volume 40; Soil Moisture Active Passive (SMAP) Project Assessment Report for the Beta-Release L4_SM Data Product

NASA Technical Reports Server (NTRS)

Koster, Randal D.; Reichle, Rolf H.; De Lannoy, Gabrielle J. M.; Liu, Qing; Colliander, Andreas; Conaty, Austin; Jackson, Thomas; Kimball, John

2015-01-01

During the post-launch SMAP calibration and validation (Cal/Val) phase there are two objectives for each science data product team: 1) calibrate, verify, and improve the performance of the science algorithm, and 2) validate the accuracy of the science data product as specified in the science requirements and according to the Cal/Val schedule. This report provides an assessment of the SMAP Level 4 Surface and Root Zone Soil Moisture Passive (L4_SM) product specifically for the product's public beta release scheduled for 30 October 2015. The primary objective of the beta release is to allow users to familiarize themselves with the data product before the validated product becomes available. The beta release also allows users to conduct their own assessment of the data and to provide feedback to the L4_SM science data product team. The assessment of the L4_SM data product includes comparisons of SMAP L4_SM soil moisture estimates with in situ soil moisture observations from core validation sites and sparse networks. The assessment further includes a global evaluation of the internal diagnostics from the ensemble-based data assimilation system that is used to generate the L4_SM product. This evaluation focuses on the statistics of the observation-minus-forecast (O-F) residuals and the analysis increments. Together, the core validation site comparisons and the statistics of the assimilation diagnostics are considered primary validation methodologies for the L4_SM product. Comparisons against in situ measurements from regional-scale sparse networks are considered a secondary validation methodology because such in situ measurements are subject to upscaling errors from the point-scale to the grid cell scale of the data product. Based on the limited set of core validation sites, the assessment presented here meets the criteria established by the Committee on Earth Observing Satellites for Stage 1 validation and supports the beta release of the data. The validation against sparse network measurements and the evaluation of the assimilation diagnostics address Stage 2 validation criteria by expanding the assessment to regional and global scales.

Advancing implementation science through measure development and evaluation: a study protocol.

PubMed

Lewis, Cara C; Weiner, Bryan J; Stanick, Cameo; Fischer, Sarah M

2015-07-22

Significant gaps related to measurement issues are among the most critical barriers to advancing implementation science. Three issues motivated the study aims: (a) the lack of stakeholder involvement in defining pragmatic measure qualities; (b) the dearth of measures, particularly for implementation outcomes; and (c) unknown psychometric and pragmatic strength of existing measures. Aim 1: Establish a stakeholder-driven operationalization of pragmatic measures and develop reliable, valid rating criteria for assessing the construct. Aim 2: Develop reliable, valid, and pragmatic measures of three critical implementation outcomes, acceptability, appropriateness, and feasibility. Aim 3: Identify Consolidated Framework for Implementation Research and Implementation Outcome Framework-linked measures that demonstrate both psychometric and pragmatic strength. For Aim 1, we will conduct (a) interviews with stakeholder panelists (N = 7) and complete a literature review to populate pragmatic measure construct criteria, (b) Q-sort activities (N = 20) to clarify the internal structure of the definition, (c) Delphi activities (N = 20) to achieve consensus on the dimension priorities, (d) test-retest and inter-rater reliability assessments of the emergent rating system, and (e) known-groups validity testing of the top three prioritized pragmatic criteria. For Aim 2, our systematic development process involves domain delineation, item generation, substantive validity assessment, structural validity assessment, reliability assessment, and predictive validity assessment. We will also assess discriminant validity, known-groups validity, structural invariance, sensitivity to change, and other pragmatic features. For Aim 3, we will refine our established evidence-based assessment (EBA) criteria, extract the relevant data from the literature, rate each measure using the EBA criteria, and summarize the data. The study outputs of each aim are expected to have a positive impact as they will establish and guide a comprehensive measurement-focused research agenda for implementation science and provide empirically supported measures, tools, and methods for accomplishing this work.

Stakes Matter: Student Motivation and the Validity of Student Assessments for Teacher Evaluation

ERIC Educational Resources Information Center

Rutkowski, David; Wild, Justin

2015-01-01

In 2011, Indiana lawmakers established a system to evaluate teachers using existing standardized assessments as an indicator of student learning. In this study we examined one component of Indiana's evaluation system to determine whether student knowledge of the test's consequences is predictive of test performance. Using an experimental design,…

An Empirical Validation of the Instrument: Student Perceptions of Teaching Effectiveness.

ERIC Educational Resources Information Center

West, Sandra S.; Denton, Jon J.

An assessment of the Student Perception of Teaching Effectiveness instrument (SPTE) is presented. The SPTE was developed to assess teaching interns' performance from the student's viewpoint. The Texas Teacher Appraisal System was selected as the content base for the development of the SPTE. The Texas system covers instructional strategies,…

An Evaluation Framework and Instrument for Evaluating e-Assessment Tools

ERIC Educational Resources Information Center

Singh, Upasana Gitanjali; de Villiers, Mary Ruth

2017-01-01

e-Assessment, in the form of tools and systems that deliver and administer multiple choice questions (MCQs), is used increasingly, raising the need for evaluation and validation of such systems. This research uses literature and a series of six empirical action research studies to develop an evaluation framework of categories and criteria called…

A Mobile Food Record For Integrated Dietary Assessment*

PubMed Central

Ahmad, Ziad; Kerr, Deborah A.; Bosch, Marc; Boushey, Carol J.; Delp, Edward J.; Khanna, Nitin; Zhu, Fengqing

2017-01-01

This paper presents an integrated dietary assessment system based on food image analysis that uses mobile devices or smartphones. We describe two components of our integrated system: a mobile application and an image-based food nutrient database that is connected to the mobile application. An easy-to-use mobile application user interface is described that was designed based on user preferences as well as the requirements of the image analysis methods. The user interface is validated by user feedback collected from several studies. Food nutrient and image databases are also described which facilitates image-based dietary assessment and enable dietitians and other healthcare professionals to monitor patients dietary intake in real-time. The system has been tested and validated in several user studies involving more than 500 users who took more than 60,000 food images under controlled and community-dwelling conditions. PMID:28691119

Developing the Polish Educational Needs Assessment Tool (Pol-ENAT) in rheumatoid arthritis and systemic sclerosis: a cross-cultural validation study using Rasch analysis.

PubMed

Sierakowska, Matylda; Sierakowski, Stanisław; Sierakowska, Justyna; Horton, Mike; Ndosi, Mwidimi

2015-03-01

To undertake cross-cultural adaptation and validation of the educational needs assessment tool (ENAT) for use with people with rheumatoid arthritis (RA) and systemic sclerosis (SSc) in Poland. The study involved two main phases: (1) cross-cultural adaptation of the ENAT from English into Polish and (2) Cross-cultural validation of Polish Educational Needs Assessment Tool (Pol-ENAT). The first phase followed an established process of cross-cultural adaptation of self-report measures. The second phase involved completion of the Pol-ENAT by patients and subjecting the data to Rasch analysis to assess the construct validity, unidimensionality, internal consistency and cross-cultural invariance. An adequate conceptual equivalence was achieved following the adaptation process. The dataset for validation comprised a total of 278 patients, 237 (85.3 %) of which were female. In each disease group (145, RA and 133, SSc), the 7 domains of the Pol-ENAT were found to fit the Rasch model, X (2)(df) = 16.953(14), p = 0.259 and 8.132(14), p = 0.882 for RA and SSc, respectively. Internal consistency of the Pol-ENAT was high (patient separation index = 0.85 and 0.89 for SSc and RA, respectively), and unidimensionality was confirmed. Cross-cultural differential item functioning (DIF) was detected in some subscales, and DIF-adjusted conversion tables were calibrated to enable cross-cultural comparison of data between Poland and the UK. Using a standard process in cross-cultural adaptation, conceptual equivalence was achieved between the original (UK) ENAT and the adapted Pol-ENAT. Fit to the Rasch model, confirmed that the construct validity, unidimensionality and internal consistency of the ENAT have been preserved.

Virtual temporal bone dissection system: OSU virtual temporal bone system: development and testing.

PubMed

Wiet, Gregory J; Stredney, Don; Kerwin, Thomas; Hittle, Bradley; Fernandez, Soledad A; Abdel-Rasoul, Mahmoud; Welling, D Bradley

2012-03-01

The objective of this project was to develop a virtual temporal bone dissection system that would provide an enhanced educational experience for the training of otologic surgeons. A randomized, controlled, multi-institutional, single-blinded validation study. The project encompassed four areas of emphasis: structural data acquisition, integration of the system, dissemination of the system, and validation. Structural acquisition was performed on multiple imaging platforms. Integration achieved a cost-effective system. Dissemination was achieved on different levels including casual interest, downloading of software, and full involvement in development and validation studies. A validation study was performed at eight different training institutions across the country using a two-arm randomized trial where study subjects were randomized to a 2-week practice session using either the virtual temporal bone or standard cadaveric temporal bones. Eighty subjects were enrolled and randomized to one of the two treatment arms; 65 completed the study. There was no difference between the two groups using a blinded rating tool to assess performance after training. A virtual temporal bone dissection system has been developed and compared to cadaveric temporal bones for practice using a multicenter trial. There was no statistical difference between practice on the current simulator compared to practice on human cadaveric temporal bones. Further refinements in structural acquisition and interface design have been identified, which can be implemented prior to full incorporation into training programs and used for objective skills assessment. Copyright © 2012 The American Laryngological, Rhinological, and Otological Society, Inc.

Classification in childhood disability: focusing on function in the 21st century.

PubMed

Rosenbaum, Peter; Eliasson, Ann-Christin; Hidecker, Mary Jo Cooley; Palisano, Robert J

2014-08-01

Classification systems in health care are usually based on current understanding of the condition. They are often derived empirically and adopted applying sound principles of measurement science to assess whether they are reliable (consistent) and valid (true) for the purposes to which they are applied. In the past 15 years, the authors have developed and validated classification systems for specific aspects of everyday function in people with cerebral palsy--gross motor function, manual abilities, and communicative function. This article describes the approaches used to conceptualize each aspect of function, develop the tools, and assess their reliability and validity. We report on the utility of each system with respect to clinical applicability, use of these tools for research, and the uptake and impact that they have had around the world. We hope that readers will find these accounts interesting, relevant, and applicable to their daily work with children and youth with disabilities. © The Author(s) 2014.

[Assessment of an Evaluation System for Psychiatry Learning].

PubMed

Campo-Cabal, Gerardo

2012-01-01

Through the analysis of a teaching evaluation system for a Psychiatry course aimed at Medicine students, the author reviews the basic elements taken into account in a teaching assessment process. Analysis was carried out of the assessment methods used as well as of the grades obtained by the students from four groups into which the they were divided. The selected assessment methods are appropriate to evaluate educational objectives; the contents are selected by means of a specification matrix; there is a high correlation coefficient between the grades obtained in previous academic periods and the ones obtained in the course, thus demonstrating the validity of the results (both considering the whole exam or just a part of it). Most of the students are on the right side of the grading curve, which means that the majority of them acquire the knowledge expected. The assessment system used in the Psychopathology course is fair, valid and reliable, specifically concerning the objective methods used, but the conceptual evaluation should be improved or, preferably, eliminated as a constituernt part of the evaluation system. Copyright © 2012 Asociación Colombiana de Psiquiatría. Publicado por Elsevier España. All rights reserved.

Development of reference practices for the calibration and validation of atmospheric composition satellites

NASA Astrophysics Data System (ADS)

Lambert, Jean-Christopher; Bojkov, Bojan

The Committee on Earth Observation Satellites (CEOS)/Working Group on Calibration and Validation (WGCV) is developing a global data quality strategy for the Global Earth Obser-vation System of Systems (GEOSS). In this context, CEOS WGCV elaborated the GEOSS Quality Assurance framework for Earth Observation (QA4EO, http://qa4eo.org). QA4EO en-compasses a documentary framework and a set of ten guidelines, which describe the top-level approach of QA activities and key requirements that drive the QA process. QA4EO is appli-cable virtually to all Earth Observation data. Calibration and validation activities are a cornerstone of the GEOSS data quality strategy. Proper uncertainty assessment of the satellite measurements and their derived data products is essential, and needs to be continuously monitored and traceable to standards. As a practical application of QA4EO, CEOS WGCV has undertaken to establish a set of best practices, methodologies and guidelines for satellite calibration and validation. The present paper reviews current developments of best practices and guidelines for the vali-dation of atmospheric composition satellites. Aimed as a community effort, the approach is to start with current practices that could be improved with time. The present review addresses current validation capabilities, achievements, caveats, harmonization efforts, and challenges. Terminologies and general principles of validation are reminded. Going beyond elementary def-initions of validation like the assessment of uncertainties, the specific GEOSS context requires considering also the validation of individual service components and against user requirements.

Validity and Reliability Testing of an e-learning Questionnaire for Chemistry Instruction

NASA Astrophysics Data System (ADS)

Guspatni, G.; Kurniawati, Y.

2018-04-01

The aim of this paper is to examine validity and reliability of a questionnaire used to evaluate e-learning implementation in chemistry instruction. 48 questionnaires were filled in by students who had studied chemistry through e-learning system. The questionnaire consisted of 20 indicators evaluating students’ perception on using e-learning. Parametric testing was done as data were assumed to follow normal distribution. Item validity of the questionnaire was examined through item-total correlation using Pearson’s formula while its reliability was assessed with Cronbach’s alpha formula. Moreover, convergent validity was assessed to see whether indicators building a factor had theoretically the same underlying construct. The result of validity testing revealed 19 valid indicators while the result of reliability testing revealed Cronbach’s alpha value of .886. The result of factor analysis showed that questionnaire consisted of five factors, and each of them had indicators building the same construct. This article shows the importance of factor analysis to get a construct valid questionnaire before it is used as research instrument.

Validation of Alternative In Vitro Methods to Animal Testing: Concepts, Challenges, Processes and Tools.

PubMed

Griesinger, Claudius; Desprez, Bertrand; Coecke, Sandra; Casey, Warren; Zuang, Valérie

This chapter explores the concepts, processes, tools and challenges relating to the validation of alternative methods for toxicity and safety testing. In general terms, validation is the process of assessing the appropriateness and usefulness of a tool for its intended purpose. Validation is routinely used in various contexts in science, technology, the manufacturing and services sectors. It serves to assess the fitness-for-purpose of devices, systems, software up to entire methodologies. In the area of toxicity testing, validation plays an indispensable role: "alternative approaches" are increasingly replacing animal models as predictive tools and it needs to be demonstrated that these novel methods are fit for purpose. Alternative approaches include in vitro test methods, non-testing approaches such as predictive computer models up to entire testing and assessment strategies composed of method suites, data sources and decision-aiding tools. Data generated with alternative approaches are ultimately used for decision-making on public health and the protection of the environment. It is therefore essential that the underlying methods and methodologies are thoroughly characterised, assessed and transparently documented through validation studies involving impartial actors. Importantly, validation serves as a filter to ensure that only test methods able to produce data that help to address legislative requirements (e.g. EU's REACH legislation) are accepted as official testing tools and, owing to the globalisation of markets, recognised on international level (e.g. through inclusion in OECD test guidelines). Since validation creates a credible and transparent evidence base on test methods, it provides a quality stamp, supporting companies developing and marketing alternative methods and creating considerable business opportunities. Validation of alternative methods is conducted through scientific studies assessing two key hypotheses, reliability and relevance of the test method for a given purpose. Relevance encapsulates the scientific basis of the test method, its capacity to predict adverse effects in the "target system" (i.e. human health or the environment) as well as its applicability for the intended purpose. In this chapter we focus on the validation of non-animal in vitro alternative testing methods and review the concepts, challenges, processes and tools fundamental to the validation of in vitro methods intended for hazard testing of chemicals. We explore major challenges and peculiarities of validation in this area. Based on the notion that validation per se is a scientific endeavour that needs to adhere to key scientific principles, namely objectivity and appropriate choice of methodology, we examine basic aspects of study design and management, and provide illustrations of statistical approaches to describe predictive performance of validated test methods as well as their reliability.

Ecological Validity and Clinical Utility of Patient-Reported Outcomes Measurement Information System (PROMIS®) instruments for detecting premenstrual symptoms of depression, anger, and fatigue

PubMed Central

Junghaenel, Doerte U.; Schneider, Stefan; Stone, Arthur A.; Christodoulou, Christopher; Broderick, Joan E.

2014-01-01

Objective This study examined the ecological validity and clinical utility of NIH Patient Reported-Outcomes Measurement Information System (PROMIS®) instruments for anger, depression, and fatigue in women with premenstrual symptoms. Methods One-hundred women completed daily diaries and weekly PROMIS assessments over 4 weeks. Weekly assessments were administered through Computerized Adaptive Testing (CAT). Weekly CATs and corresponding daily scores were compared to evaluate ecological validity. To test clinical utility, we examined if CATs could detect changes in symptom levels, if these changes mirrored those obtained from daily scores, and if CATs could identify clinically meaningful premenstrual symptom change. Results PROMIS CAT scores were higher in the pre-menstrual than the baseline (ps < .0001) and post-menstrual (ps < .0001) weeks. The correlations between CATs and aggregated daily scores ranged from .73 to .88 supporting ecological validity. Mean CAT scores showed systematic changes in accordance with the menstrual cycle and the magnitudes of the changes were similar to those obtained from the daily scores. Finally, Receiver Operating Characteristic (ROC) analyses demonstrated the ability of the CATs to discriminate between women with and without clinically meaningful premenstrual symptom change. Conclusions PROMIS CAT instruments for anger, depression, and fatigue demonstrated validity and utility in premenstrual symptom assessment. The results provide encouraging initial evidence of the utility of PROMIS instruments for the measurement of affective premenstrual symptoms. PMID:24630180

Skill assessment of the coupled physical-biogeochemical operational Mediterranean Forecasting System

NASA Astrophysics Data System (ADS)

Cossarini, Gianpiero; Clementi, Emanuela; Salon, Stefano; Grandi, Alessandro; Bolzon, Giorgio; Solidoro, Cosimo

2016-04-01

The Mediterranean Monitoring and Forecasting Centre (Med-MFC) is one of the regional production centres of the European Marine Environment Monitoring Service (CMEMS-Copernicus). Med-MFC operatively manages a suite of numerical model systems (3DVAR-NEMO-WW3 and 3DVAR-OGSTM-BFM) that provides gridded datasets of physical and biogeochemical variables for the Mediterranean marine environment with a horizontal resolution of about 6.5 km. At the present stage, the operational Med-MFC produces ten-day forecast: daily for physical parameters and bi-weekly for biogeochemical variables. The validation of the coupled model system and the estimate of the accuracy of model products are key issues to ensure reliable information to the users and the downstream services. Product quality activities at Med-MFC consist of two levels of validation and skill analysis procedures. Pre-operational qualification activities focus on testing the improvement of the quality of a new release of the model system and relays on past simulation and historical data. Then, near real time (NRT) validation activities aim at the routinely and on-line skill assessment of the model forecast and relays on the NRT available observations. Med-MFC validation framework uses both independent (i.e. Bio-Argo float data, in-situ mooring and vessel data of oxygen, nutrients and chlorophyll, moored buoys, tide-gauges and ADCP of temperature, salinity, sea level and velocity) and semi-independent data (i.e. data already used for assimilation, such as satellite chlorophyll, Satellite SLA and SST and in situ vertical profiles of temperature and salinity from XBT, Argo and Gliders) We give evidence that different variables (e.g. CMEMS-products) can be validated at different levels (i.e. at the forecast level or at the level of model consistency) and at different spatial and temporal scales. The fundamental physical parameters temperature, salinity and sea level are routinely validated on daily, weekly and quarterly base at regional and sub-regional scale and along specific vertical layers (temperature and salinity); while velocity fields are daily validated against in situ coastal moorings. Since the velocity skill cannot be accurately assessed through coastal measurements due to the actual model horizontal resolution (~6.5 km), new validation metrics and procedures are under investigation. Chlorophyll is the only biogeochemical variable that can be validated routinely at the temporal and spatial scale of the weekly forecast, while nutrients and oxygen predictions can be validated locally or at sub-basin and seasonal scales. For the other biogeochemical variables (i.e. primary production, carbonate system variables) only the accuracy of the average dynamics and model consistency can be evaluated. Then, we discuss the limiting factors of the present validation framework, and the quality and extension of the observing system that would be needed for improving the reliability of the physical and biogeochemical Mediterranean forecast services.

National programmes for validating physician competence and fitness for practice: a scoping review.

PubMed

Horsley, Tanya; Lockyer, Jocelyn; Cogo, Elise; Zeiter, Jeanie; Bursey, Ford; Campbell, Craig

2016-04-15

To explore and categorise the state of existing literature for national programmes designed to affirm or establish the continuing competence of physicians. Scoping review. MEDLINE, ERIC, Sociological Abstracts, web/grey literature (2000-2014). Included when a record described a (1) national-level physician validation system, (2) recognised as a system for affirming competence and (3) reported relevant data. Using bibliographic software, title and abstracts were reviewed using an assessment matrix to ensure duplicate, paired screening. Dyads included both a methodologist and content expert on each assessment, reflective of evidence-informed best practices to decrease errors. 45 reports were included. Publication dates ranged from 2002 to 2014 with the majority of publications occurring in the previous six years (n=35). Country of origin--defined as that of the primary author--included the USA (N=32), the UK (N=8), Canada (N=3), Kuwait (N=1) and Australia (N=1). Three broad themes emerged from this heterogeneous data set: contemporary national programmes, contextual factors and terminological consistency. Four national physician validation systems emerged from the data: the American Board of Medical Specialties Maintenance of Certification Program, the Federation of State Medical Boards Maintenance of Licensure Program, the Canadian Revalidation Program and the UK Revalidation Program. Three contextual factors emerged as stimuli for the implementation of national validation systems: medical regulation, quality of care and professional competence. Finally, great variation among the definitions of key terms was identified. There is an emerging literature focusing on national physician validation systems. Four major systems have been implemented in recent years and it is anticipated that more will follow. Much of this work is descriptive, and gaps exist for the extent to which systems build on current evidence or theory. Terminology is highly variable across programmes for validating physician competence and fitness for practice. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

National programmes for validating physician competence and fitness for practice: a scoping review

PubMed Central

Horsley, Tanya; Lockyer, Jocelyn; Cogo, Elise; Zeiter, Jeanie; Bursey, Ford; Campbell, Craig

2016-01-01

Objective To explore and categorise the state of existing literature for national programmes designed to affirm or establish the continuing competence of physicians. Design Scoping review. Data sources MEDLINE, ERIC, Sociological Abstracts, web/grey literature (2000–2014). Selection Included when a record described a (1) national-level physician validation system, (2) recognised as a system for affirming competence and (3) reported relevant data. Data extraction Using bibliographic software, title and abstracts were reviewed using an assessment matrix to ensure duplicate, paired screening. Dyads included both a methodologist and content expert on each assessment, reflective of evidence-informed best practices to decrease errors. Results 45 reports were included. Publication dates ranged from 2002 to 2014 with the majority of publications occurring in the previous six years (n=35). Country of origin—defined as that of the primary author—included the USA (N=32), the UK (N=8), Canada (N=3), Kuwait (N=1) and Australia (N=1). Three broad themes emerged from this heterogeneous data set: contemporary national programmes, contextual factors and terminological consistency. Four national physician validation systems emerged from the data: the American Board of Medical Specialties Maintenance of Certification Program, the Federation of State Medical Boards Maintenance of Licensure Program, the Canadian Revalidation Program and the UK Revalidation Program. Three contextual factors emerged as stimuli for the implementation of national validation systems: medical regulation, quality of care and professional competence. Finally, great variation among the definitions of key terms was identified. Conclusions There is an emerging literature focusing on national physician validation systems. Four major systems have been implemented in recent years and it is anticipated that more will follow. Much of this work is descriptive, and gaps exist for the extent to which systems build on current evidence or theory. Terminology is highly variable across programmes for validating physician competence and fitness for practice. PMID:27084276

A validation study of an alternate state science assessment: Alignment of the Pennsylvania Alternate System of Assessment (PASA) science assessment

NASA Astrophysics Data System (ADS)

Heh, Peter

The current study examined the validation and alignment of the PASA-Science by determining whether the alternate science assessment anchors linked to the regular education science anchors; whether the PASA-Science assessment items are science; whether the PASA-Science assessment items linked to the alternate science eligible content, and what PASA-Science assessment content was considered important by parents and teachers. Special education and science education university faculty determined all but one alternate science assessment anchor linked to the regular science assessment anchors. Special education and science education teachers determined that the PASA-Science assessment items are indeed science and linked to the alternate science eligible content. Finally, parents and teachers indicated the most important science content assessed in the PASA-Science involved safety and independence.

Cross-cultural adaptation and validation of Systemic Lupus Erythematosus Quality of Life questionnaire into Arabic.

PubMed

Aziz, M M; Galal, M A A; Elzohri, M H; El-Nouby, F; Leong, K P

2018-04-01

Systemic lupus erythematosus (SLE) is a chronic autoimmune disease which affects all aspects of quality of life (QoL) of the patients. Comprehensive patient assessment should include QoL measures in addition to the objective clinical measures of the disease. There is no specific Arabic instrument for assessment of QoL of SLE patients. The objective of this study was to translate and cross culturally adapt the SLEQOL questionnaire into Arabic and test its reliability and validity. The SLEQOL questionnaire was translated into Arabic based on the Guidelines for Translation and Cross-cultural Adaptation into other languages. Reliability was assessed by interviewing patients three times: two interviews on the same day by different interviewers and the third interview 14 days later by one of the first interviewers. Validity was assessed by correlating SLEQOL scores of 91 patients with 36-item Short Form Health Survey (SF-36) scores and clinical parameters of the patients. We found that the Arabic version of SLEQOL has a Cronbach's alpha of 0.936, interobserver and intraobserver correlation coefficients of 0.809 and 0.886 respectively. Strong correlations were also found between SLEQOL scores and SF-36 Physical and Mental Component summaries. In conclusion, the Arabic version of SLEQOL is a reliable and valid instrument for measuring QoL of Egyptian SLE patients.

Concurrent validity of persian version of wechsler intelligence scale for children - fourth edition and cognitive assessment system in patients with learning disorder.

PubMed

Rostami, Reza; Sadeghi, Vahid; Zarei, Jamileh; Haddadi, Parvaneh; Mohazzab-Torabi, Saman; Salamati, Payman

2013-04-01

The aim of this study was to compare the Persian version of the wechsler intelligence scale for children - fourth edition (WISC-IV) and cognitive assessment system (CAS) tests, to determine the correlation between their scales and to evaluate the probable concurrent validity of these tests in patients with learning disorders. One-hundered-sixty-two children with learning disorder who were presented at Atieh Comprehensive Psychiatry Center were selected in a consecutive non-randomized order. All of the patients were assessed based on WISC-IV and CAS scores questionnaires. Pearson correlation coefficient was used to analyze the correlation between the data and to assess the concurrent validity of the two tests. Linear regression was used for statistical modeling. The type one error was considered 5% in maximum. There was a strong correlation between total score of WISC-IV test and total score of CAS test in the patients (r=0.75, P<0.001). The correlations among the other scales were mostly high and all of them were statistically significant (P<0.001). A linear regression model was obtained (α = 0.51, β = 0.81 and P<0.001). There is an acceptable correlation between the WISC-IV scales and CAS test in children with learning disorders. A concurrent validity is established between the two tests and their scales.

Concurrent Validity of Persian Version of Wechsler Intelligence Scale for Children - Fourth Edition and Cognitive Assessment System in Patients with Learning Disorder

PubMed Central

Rostami, Reza; Sadeghi, Vahid; Zarei, Jamileh; Haddadi, Parvaneh; Mohazzab-Torabi, Saman; Salamati, Payman

2013-01-01

Objective The aim of this study was to compare the Persian version of the wechsler intelligence scale for children - fourth edition (WISC-IV) and cognitive assessment system (CAS) tests, to determine the correlation between their scales and to evaluate the probable concurrent validity of these tests in patients with learning disorders. Methods One-hundered-sixty-two children with learning disorder who were presented at Atieh Comprehensive Psychiatry Center were selected in a consecutive non-randomized order. All of the patients were assessed based on WISC-IV and CAS scores questionnaires. Pearson correlation coefficient was used to analyze the correlation between the data and to assess the concurrent validity of the two tests. Linear regression was used for statistical modeling. The type one error was considered 5% in maximum. Findings There was a strong correlation between total score of WISC-IV test and total score of CAS test in the patients (r=0.75, P<0.001). The correlations among the other scales were mostly high and all of them were statistically significant (P<0.001). A linear regression model was obtained (α = 0.51, β = 0.81 and P<0.001). Conclusion There is an acceptable correlation between the WISC-IV scales and CAS test in children with learning disorders. A concurrent validity is established between the two tests and their scales. PMID:23724180

A Technical Note on the PainChek™ System: A Web Portal and Mobile Medical Device for Assessing Pain in People With Dementia.

PubMed

Atee, Mustafa; Hoti, Kreshnik; Hughes, Jeffery D

2018-01-01

Background: Pain in dementia is predominant particularly in the advanced stages or in those who are unable to verbalize. Uncontrolled pain alters the course of behaviors in patients with dementia making them perturbed, unsettled, and devitalized. Current measures of assessing pain in this population group are inadequate and underutilized in clinical practice because they lack systematic evaluation and innovative design. Objective: To describe a novel method and system of pain assessment using a combination of technologies: automated facial recognition and analysis (AFRA), smart computing, affective computing, and cloud computing (Internet of Things) for people with advanced dementia. Methods and Results: Cognification and affective computing were used to conceptualize the system. A computerized clinical system was developed to address the challenging problem of identifying pain in non-verbal patients with dementia. The system is composed of a smart device enabled app (App) linked to a web admin portal (WAP). The App "PainChek™" uses AFRA to identify facial action units indicative of pain presence, and user-fed clinical information to calculate a pain intensity score. The App has various functionalities including: pain assessment, pain monitoring, patient profiling, and data synchronization (into the WAP). The WAP serves as a database that collects the data obtained through the App in the clinical setting. These technologies can assist in addressing the various characteristics of pain (e.g., subjectivity, multidimensionality, and dynamicity). With over 750 paired assessments conducted, the App has been validated in two clinical studies ( n = 74, age: 60-98 y), which showed sound psychometric properties: excellent concurrent validity ( r = 0.882-0.911), interrater reliability (Kw = 0.74-0.86), internal consistency (α = 0.925-0.950), and excellent test-retest reliability (ICC = 0.904), while it possesses good predictive validity and discriminant validity. Clinimetric data revealed high accuracy (95.0%), sensitivity (96.1%), and specificity (91.4%) as well as excellent clinical utility (0.95). Conclusions: PainChek™ is a comprehensive and evidence-based pain management system. This novel approach has the potential to transform pain assessment in people who are unable to verbalize because it can be used by clinicians and carers in everyday clinical practice.

Feasibility of using a tablet computer survey for parental assessment of resident communication skills.

PubMed

Co, John Patrick T; Mohamed, Hodon; Kelleher, Mary Louise; Edgman-Levitan, Susan; Perrin, James M

2008-01-01

The Accreditation Council for Graduate Medical Education recommends using patient surveys for assessing resident competency in interpersonal and communication skills. Despite the existence of several validated patient surveys for communication assessment, no system has been developed for their sustained use in resident assessment. We developed and pilot tested a system to collect surveys from parents of hospitalized children on the day of discharge. We used a 28-item, tablet computer-based survey that measures individual provider and team communication. The computer displays resident photographs to ensure accurate identification and offers the survey in multiple languages. We assessed parental acceptance of the system by analyzing response rate, as well as reasons for response and nonresponse. Of the 98 eligible parents that were approached, 62 (63%) completed the survey. Only 2 (2%) of the eligible families refused to participate, and only 5 (5%) refused participation because of the survey not being available in a language they were familiar with. Use of a tablet computer parent survey for resident assessment is feasible, with response rates comparable to those of mailed surveys. The low rate of parental refusal indicates our system could be used to attain sufficient numbers of survey responses to help validly measure resident communication skills.

About the Cancer Biomarkers Research Group | Division of Cancer Prevention

Cancer.gov

The Cancer Biomarkers Research Group promotes research to identify, develop, and validate biological markers for early cancer detection and cancer risk assessment. Activities include development and validation of promising cancer biomarkers, collaborative databases and informatics systems, and new technologies or the refinement of existing technologies. NCI DCP News Note

Personality disorder assessment: the challenge of construct validity.

PubMed

Clark, L A; Livesley, W J; Morey, L

1997-01-01

We begin with a review of the data that challenge the current categorical system for classifying personality disorder, focusing on the central assessment issues of convergent and discriminant validity. These data indicate that while there is room for improvement in assessment, even greater change is needed in conceptualization than in instrumentation. Accordingly, we then refocus the categorical-dimensional debate in assessment terms, and place it in the broader context of such issues as the hierarchical structure of personality, overlap and distinctions between normal and abnormal personality, sources of information in personality disorder assessment, and overlap and discrimination of trait and state assessment. We conclude that more complex conceptual models that can incorporate both biological and environmental influences on the development of adaptive and maladaptive personality are needed.

Validation of Patient-Reported Outcomes Measurement Information System Short Forms for Use in Childhood-Onset Systemic Lupus Erythematosus.

PubMed

Jones, Jordan T; Carle, Adam C; Wootton, Janet; Liberio, Brianna; Lee, Jiha; Schanberg, Laura E; Ying, Jun; Morgan DeWitt, Esi; Brunner, Hermine I

2017-01-01

To validate the pediatric Patient-Reported Outcomes Measurement Information System short forms (PROMIS-SFs) in childhood-onset systemic lupus erythematosus (SLE) in a clinical setting. At 3 study visits, childhood-onset SLE patients completed the PROMIS-SFs (anger, anxiety, depressive symptoms, fatigue, physical function-mobility, physical function-upper extremity, pain interference, and peer relationships) using the PROMIS assessment center, and health-related quality of life (HRQoL) legacy measures (Pediatric Quality of Life Inventory, Childhood Health Assessment Questionnaire, Simple Measure of Impact of Lupus Erythematosus in Youngsters [SMILEY], and visual analog scales [VAS] of pain and well-being). Physicians rated childhood-onset SLE activity on a VAS and completed the Systemic Lupus Erythematosus Disease Activity Index 2000. Using a global rating scale of change (GRC) between study visits, physicians rated change of childhood-onset SLE activity (GRC-MD1: better/same/worse) and change of patient overall health (GRC-MD2: better/same/worse). Questionnaire scores were compared in support of validity and responsiveness to change (external standards: GRC-MD1, GRC-MD2). In this population-based cohort (n = 100) with a mean age of 15.8 years (range 10-20 years), the PROMIS-SFs were completed in less than 5 minutes in a clinical setting. The PROMIS-SF scores correlated at least moderately (Pearson's r ≥ 0.5) with those of legacy HRQoL measures, except for the SMILEY. Measures of childhood-onset SLE activity did not correlate with the PROMIS-SFs. Responsiveness to change of the PROMIS-SFs was supported by path, mixed-model, and correlation analyses. To assess HRQoL in childhood-onset SLE, the PROMIS-SFs demonstrated feasibility, internal consistency, construct validity, and responsiveness to change in a clinical setting. © 2016, American College of Rheumatology.

Criterion-Referenced Testing for College-Level General Education: Some Problems and Recommendations.

ERIC Educational Resources Information Center

Benoist, Howard

1979-01-01

The adoption of a criterion-referenced assessment system and the resulting disadvantages of this form of evaluation for the college general education program are discussed, including problems in identifying assessment validation procedures. (RAO)

An automated system using spatial oversampling for optical mapping in murine atria. Development and validation with monophasic and transmembrane action potentials.

PubMed

Yu, Ting Yue; Syeda, Fahima; Holmes, Andrew P; Osborne, Benjamin; Dehghani, Hamid; Brain, Keith L; Kirchhof, Paulus; Fabritz, Larissa

2014-08-01

We developed and validated a new optical mapping system for quantification of electrical activation and repolarisation in murine atria. The system makes use of a novel 2nd generation complementary metal-oxide-semiconductor (CMOS) camera with deliberate oversampling to allow both assessment of electrical activation with high spatial and temporal resolution (128 × 2048 pixels) and reliable assessment of atrial murine repolarisation using post-processing of signals. Optical recordings were taken from isolated, superfused and electrically stimulated murine left atria. The system reliably describes activation sequences, identifies areas of functional block, and allows quantification of conduction velocities and vectors. Furthermore, the system records murine atrial action potentials with comparable duration to both monophasic and transmembrane action potentials in murine atria. Copyright © 2014 The Authors. Published by Elsevier Ltd.. All rights reserved.

A Critical Review of Some Qualitative Research Methods Used to Explore Rater Cognition

ERIC Educational Resources Information Center

Suto, Irenka

2012-01-01

Internationally, many assessment systems rely predominantly on human raters to score examinations. Arguably, this facilitates the assessment of multiple sophisticated educational constructs, strengthening assessment validity. It can introduce subjectivity into the scoring process, however, engendering threats to accuracy. The present objectives…

Assessing the validity and reliability of three indicators self-reported on the pregnancy risk assessment monitoring system survey.

PubMed

Ahluwalia, Indu B; Helms, Kristen; Morrow, Brian

2013-01-01

We investigated the reliability and validity of three self-reported indicators from the Pregnancy Risk Assessment Monitoring System (PRAMS) survey. We used 2008 PRAMS (n=15,646) data from 12 states that had implemented the 2003 revised U.S. Certificate of Live Birth. We estimated reliability by kappa coefficient and validity by sensitivity and specificity using the birth certificate data as the reference for the following: prenatal participation in the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC); Medicaid payment for delivery; and breastfeeding initiation. These indicators were examined across several demographic subgroups. The reliability was high for all three measures: 0.81 for WIC participation, 0.67 for Medicaid payment of delivery, and 0.72 for breastfeeding initiation. The validity of PRAMS indicators was also high: WIC participation (sensitivity = 90.8%, specificity = 90.6%), Medicaid payment for delivery (sensitivity = 82.4%, specificity = 85.6%), and breastfeeding initiation (sensitivity = 94.3%, specificity = 76.0%). The prevalence estimates were higher on PRAMS than the birth certificate for each of the indicators except Medicaid-paid delivery among non-Hispanic black women. Kappa values within most subgroups remained in the moderate range (0.40-0.80). Sensitivity and specificity values were lower for Hispanic women who responded to the PRAMS survey in Spanish and for breastfeeding initiation among women who delivered very low birthweight and very preterm infants. The validity and reliability of the PRAMS data for measures assessed were high. Our findings support the use of PRAMS data for epidemiological surveillance, research, and planning.

Health Information Technology Usability Evaluation Scale (Health-ITUES) for Usability Assessment of Mobile Health Technology: Validation Study.

PubMed

Schnall, Rebecca; Cho, Hwayoung; Liu, Jianfang

2018-01-05

Mobile technology has become a ubiquitous technology and can be particularly useful in the delivery of health interventions. This technology can allow us to deliver interventions to scale, cover broad geographic areas, and deliver technologies in highly tailored ways based on the preferences or characteristics of users. The broad use of mobile technologies supports the need for usability assessments of these tools. Although there have been a number of usability assessment instruments developed, none have been validated for use with mobile technologies. The goal of this work was to validate the Health Information Technology Usability Evaluation Scale (Health-ITUES), a customizable usability assessment instrument in a sample of community-dwelling adults who were testing the use of a new mobile health (mHealth) technology. A sample of 92 community-dwelling adults living with HIV used a new mobile app for symptom self-management and completed the Health-ITUES to assess the usability of the app. They also completed the Post-Study System Usability Questionnaire (PSSUQ), a widely used and well-validated usability assessment tool. Correlations between these scales and each of the subscales were assessed. The subscales of the Health-ITUES showed high internal consistency reliability (Cronbach alpha=.85-.92). Each of the Health-ITUES subscales and the overall scale was moderately to strongly correlated with the PSSUQ scales (r=.46-.70), demonstrating the criterion validity of the Health-ITUES. The Health-ITUES has demonstrated reliability and validity for use in assessing the usability of mHealth technologies in community-dwelling adults living with a chronic illness. ©Rebecca Schnall, Hwayoung Cho, Jianfang Liu. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 05.01.2018.

Health Information Technology Usability Evaluation Scale (Health-ITUES) for Usability Assessment of Mobile Health Technology: Validation Study

PubMed Central

Cho, Hwayoung; Liu, Jianfang

2018-01-01

Background Mobile technology has become a ubiquitous technology and can be particularly useful in the delivery of health interventions. This technology can allow us to deliver interventions to scale, cover broad geographic areas, and deliver technologies in highly tailored ways based on the preferences or characteristics of users. The broad use of mobile technologies supports the need for usability assessments of these tools. Although there have been a number of usability assessment instruments developed, none have been validated for use with mobile technologies. Objective The goal of this work was to validate the Health Information Technology Usability Evaluation Scale (Health-ITUES), a customizable usability assessment instrument in a sample of community-dwelling adults who were testing the use of a new mobile health (mHealth) technology. Methods A sample of 92 community-dwelling adults living with HIV used a new mobile app for symptom self-management and completed the Health-ITUES to assess the usability of the app. They also completed the Post-Study System Usability Questionnaire (PSSUQ), a widely used and well-validated usability assessment tool. Correlations between these scales and each of the subscales were assessed. Results The subscales of the Health-ITUES showed high internal consistency reliability (Cronbach alpha=.85-.92). Each of the Health-ITUES subscales and the overall scale was moderately to strongly correlated with the PSSUQ scales (r=.46-.70), demonstrating the criterion validity of the Health-ITUES. Conclusions The Health-ITUES has demonstrated reliability and validity for use in assessing the usability of mHealth technologies in community-dwelling adults living with a chronic illness. PMID:29305343

Plaque Tissue Morphology-Based Stroke Risk Stratification Using Carotid Ultrasound: A Polling-Based PCA Learning Paradigm.

PubMed

Saba, Luca; Jain, Pankaj K; Suri, Harman S; Ikeda, Nobutaka; Araki, Tadashi; Singh, Bikesh K; Nicolaides, Andrew; Shafique, Shoaib; Gupta, Ajay; Laird, John R; Suri, Jasjit S

2017-06-01

Severe atherosclerosis disease in carotid arteries causes stenosis which in turn leads to stroke. Machine learning systems have been previously developed for plaque wall risk assessment using morphology-based characterization. The fundamental assumption in such systems is the extraction of the grayscale features of the plaque region. Even though these systems have the ability to perform risk stratification, they lack the ability to achieve higher performance due their inability to select and retain dominant features. This paper introduces a polling-based principal component analysis (PCA) strategy embedded in the machine learning framework to select and retain dominant features, resulting in superior performance. This leads to more stability and reliability. The automated system uses offline image data along with the ground truth labels to generate the parameters, which are then used to transform the online grayscale features to predict the risk of stroke. A set of sixteen grayscale plaque features is computed. Utilizing the cross-validation protocol (K = 10), and the PCA cutoff of 0.995, the machine learning system is able to achieve an accuracy of 98.55 and 98.83%corresponding to the carotidfar wall and near wall plaques, respectively. The corresponding reliability of the system was 94.56 and 95.63%, respectively. The automated system was validated against the manual risk assessment system and the precision of merit for same cross-validation settings and PCA cutoffs are 98.28 and 93.92%for the far and the near wall, respectively.PCA-embedded morphology-based plaque characterization shows a powerful strategy for risk assessment and can be adapted in clinical settings.

A systematic review of publications assessing reliability and validity of the Behavioral Risk Factor Surveillance System (BRFSS), 2004–2011

PubMed Central

2013-01-01

Background In recent years response rates on telephone surveys have been declining. Rates for the behavioral risk factor surveillance system (BRFSS) have also declined, prompting the use of new methods of weighting and the inclusion of cell phone sampling frames. A number of scholars and researchers have conducted studies of the reliability and validity of the BRFSS estimates in the context of these changes. As the BRFSS makes changes in its methods of sampling and weighting, a review of reliability and validity studies of the BRFSS is needed. Methods In order to assess the reliability and validity of prevalence estimates taken from the BRFSS, scholarship published from 2004–2011 dealing with tests of reliability and validity of BRFSS measures was compiled and presented by topics of health risk behavior. Assessments of the quality of each publication were undertaken using a categorical rubric. Higher rankings were achieved by authors who conducted reliability tests using repeated test/retest measures, or who conducted tests using multiple samples. A similar rubric was used to rank validity assessments. Validity tests which compared the BRFSS to physical measures were ranked higher than those comparing the BRFSS to other self-reported data. Literature which undertook more sophisticated statistical comparisons was also ranked higher. Results Overall findings indicated that BRFSS prevalence rates were comparable to other national surveys which rely on self-reports, although specific differences are noted for some categories of response. BRFSS prevalence rates were less similar to surveys which utilize physical measures in addition to self-reported data. There is very little research on reliability and validity for some health topics, but a great deal of information supporting the validity of the BRFSS data for others. Conclusions Limitations of the examination of the BRFSS were due to question differences among surveys used as comparisons, as well as mode of data collection differences. As the BRFSS moves to incorporating cell phone data and changing weighting methods, a review of reliability and validity research indicated that past BRFSS landline only data were reliable and valid as measured against other surveys. New analyses and comparisons of BRFSS data which include the new methodologies and cell phone data will be needed to ascertain the impact of these changes on estimates in the future. PMID:23522349

A Study on Critical Thinking Assessment System of College English Writing

ERIC Educational Resources Information Center

Dong, Tian; Yue, Lu

2015-01-01

This research attempts to discuss the validity of introducing the evaluation of students' critical thinking skills (CTS) into the assessment system of college English writing through an empirical study. In this paper, 30 College English Test Band 4 (CET-4) writing samples were collected and analyzed. Students' CTS and the final scores of collected…

Evaluating the Validity of Classroom Observations in the Head Start Designation Renewal System

ERIC Educational Resources Information Center

Mashburn, Andrew J.

2017-01-01

Classroom observations are increasingly common in education policies as a means to assess the quality of teachers and/or education programs for purposes of making high-stakes decisions. This article considers one policy, the Head Start Designation Renewal System (DRS), which involves classroom observations to assess the quality of Head Start…

Applied Virtual Reality Research and Applications at NASA/Marshall Space Flight Center

NASA Technical Reports Server (NTRS)

Hale, Joseph P.

1995-01-01

A Virtual Reality (VR) applications program has been under development at NASA/Marshall Space Flight Center (MSFC) since 1989. The objectives of the MSFC VR Applications Program are to develop, assess, validate, and utilize VR in hardware development, operations development and support, mission operations training and science training. Before this technology can be utilized with confidence in these applications, it must be validated for each particular class of application. That is, the precision and reliability with which it maps onto real settings and scenarios, representative of a class, must be calculated and assessed. The approach of the MSFC VR Applications Program is to develop and validate appropriate virtual environments and associated object kinematic and behavior attributes for specific classes of applications. These application-specific environments and associated simulations will be validated, where possible, through empirical comparisons with existing, accepted tools and methodologies. These validated VR analytical tools will then be available for use in the design and development of space systems and operations and in training and mission support systems. Specific validation studies for selected classes of applications have been completed or are currently underway. These include macro-ergonomic "control-room class" design analysis, Spacelab stowage reconfiguration training, a full-body micro-gravity functional reach simulator, and a gross anatomy teaching simulator. This paper describes the MSFC VR Applications Program and the validation studies.

Validation of a method for assessing resident physicians' quality improvement proposals.

PubMed

Leenstra, James L; Beckman, Thomas J; Reed, Darcy A; Mundell, William C; Thomas, Kris G; Krajicek, Bryan J; Cha, Stephen S; Kolars, Joseph C; McDonald, Furman S

2007-09-01

Residency programs involve trainees in quality improvement (QI) projects to evaluate competency in systems-based practice and practice-based learning and improvement. Valid approaches to assess QI proposals are lacking. We developed an instrument for assessing resident QI proposals--the Quality Improvement Proposal Assessment Tool (QIPAT-7)-and determined its validity and reliability. QIPAT-7 content was initially obtained from a national panel of QI experts. Through an iterative process, the instrument was refined, pilot-tested, and revised. Seven raters used the instrument to assess 45 resident QI proposals. Principal factor analysis was used to explore the dimensionality of instrument scores. Cronbach's alpha and intraclass correlations were calculated to determine internal consistency and interrater reliability, respectively. QIPAT-7 items comprised a single factor (eigenvalue = 3.4) suggesting a single assessment dimension. Interrater reliability for each item (range 0.79 to 0.93) and internal consistency reliability among the items (Cronbach's alpha = 0.87) were high. This method for assessing resident physician QI proposals is supported by content and internal structure validity evidence. QIPAT-7 is a useful tool for assessing resident QI proposals. Future research should determine the reliability of QIPAT-7 scores in other residency and fellowship training programs. Correlations should also be made between assessment scores and criteria for QI proposal success such as implementation of QI proposals, resident scholarly productivity, and improved patient outcomes.

Designing and Validating Assessments of Complex Thinking in Science

ERIC Educational Resources Information Center

Ryoo, Kihyun; Linn, Marcia C.

2015-01-01

Typical assessment systems often measure isolated ideas rather than the coherent understanding valued in current science classrooms. Such assessments may motivate students to memorize, rather than to use new ideas to solve complex problems. To meet the requirements of the Next Generation Science Standards, instruction needs to emphasize sustained…

From Cognitive-Domain Theory to Assessment Practice

ERIC Educational Resources Information Center

Bennett, Randy E.; Deane, Paul; van Rijn, Peter W.

2016-01-01

This article exemplifies how assessment design might be grounded in theory, thereby helping to strengthen validity claims. Spanning work across multiple related projects, the article first briefly summarizes an assessment system model for the elementary and secondary levels. Next the article describes how cognitive-domain theory and principles are…

A Case Study of the Alignment between Curriculum and Assessment in the New York State Earth Science Standards-Based System

ERIC Educational Resources Information Center

Contino, Julie

2013-01-01

In a standards-based system, it is important for all components of the system to align in order to achieve the intended goals. No Child Left Behind law mandates that assessments be fully aligned with state standards, be valid, reliable and fair, be reported to all stakeholders, and provide evidence that all students in the state are meeting the…

Validity and validation of expert (Q)SAR systems.

PubMed

Hulzebos, E; Sijm, D; Traas, T; Posthumus, R; Maslankiewicz, L

2005-08-01

At a recent workshop in Setubal (Portugal) principles were drafted to assess the suitability of (quantitative) structure-activity relationships ((Q)SARs) for assessing the hazards and risks of chemicals. In the present study we applied some of the Setubal principles to test the validity of three (Q)SAR expert systems and validate the results. These principles include a mechanistic basis, the availability of a training set and validation. ECOSAR, BIOWIN and DEREK for Windows have a mechanistic or empirical basis. ECOSAR has a training set for each QSAR. For half of the structural fragments the number of chemicals in the training set is >4. Based on structural fragments and log Kow, ECOSAR uses linear regression to predict ecotoxicity. Validating ECOSAR for three 'valid' classes results in predictivity of > or = 64%. BIOWIN uses (non-)linear regressions to predict the probability of biodegradability based on fragments and molecular weight. It has a large training set and predicts non-ready biodegradability well. DEREK for Windows predictions are supported by a mechanistic rationale and literature references. The structural alerts in this program have been developed with a training set of positive and negative toxicity data. However, to support the prediction only a limited number of chemicals in the training set is presented to the user. DEREK for Windows predicts effects by 'if-then' reasoning. The program predicts best for mutagenicity and carcinogenicity. Each structural fragment in ECOSAR and DEREK for Windows needs to be evaluated and validated separately.

Systematic review of systemic sclerosis-specific instruments for the EULAR Outcome Measures Library: An evolutional database model of validated patient-reported outcomes.

PubMed

Ingegnoli, Francesca; Carmona, Loreto; Castrejon, Isabel

2017-04-01

The EULAR Outcome Measures Library (OML) is a freely available database of validated patient-reported outcomes (PROs). The aim of this study was to provide a comprehensive review of validated PROs specifically developed for systemic sclerosis (SSc) to feed the EULAR OML. A sensitive search was developed in Medline and Embase to identify all validation studies, cohort studies, reviews, or meta-analyses in which the objective were the development or validation of specific PROs evaluating organ involvement, disease activity or damage in SSc. A reviewer screened title and abstracts, selected the studies, and collected data concerning validation using ad hoc forms based on the COSMIN checklist. From 13,140 articles captured, 74 met the predefined criteria. After excluding two instruments as they were unavailable in English the selected 23 studies provided information on seven SSc-specific PROs on different SSc domains: burden of illness (symptom burden index), functional status (Scleroderma Assessment Questionnaire), functional ability (scleroderma Functional Score), Raynaud's phenomenon (Raynaud's condition score), mouth involvement (Mouth Handicap in SSc), gastro-intestinal involvement (University of California Los Angeles-Scleroderma Clinical Trial Consortium Gastro-Intestinal tract 2.0), and skin involvement (skin self-assessment). Each of them is partially validated and has different psychometric requirements. Seven SSc-specific PROs have a minimum validation and were included in the EULAR OML. Further development in the area of disease-specific PROs in SSc is warranted. Copyright © 2017 Elsevier Inc. All rights reserved.

Design and evaluation of a miniature laser speckle imaging device to assess gingival health

PubMed Central

Regan, Caitlin; White, Sean M.; Yang, Bruce Y.; Takesh, Thair; Ho, Jessica; Wink, Cherie; Wilder-Smith, Petra; Choi, Bernard

2016-01-01

Abstract. Current methods used to assess gingivitis are qualitative and subjective. We hypothesized that gingival perfusion measurements could provide a quantitative metric of disease severity. We constructed a compact laser speckle imaging (LSI) system that could be mounted in custom-made oral molds. Rigid fixation of the LSI system in the oral cavity enabled measurement of blood flow in the gingiva. In vitro validation performed in controlled flow phantoms demonstrated that the compact LSI system had comparable accuracy and linearity compared to a conventional bench-top LSI setup. In vivo validation demonstrated that the compact LSI system was capable of measuring expected blood flow dynamics during a standard postocclusive reactive hyperemia and that the compact LSI system could be used to measure gingival blood flow repeatedly without significant variation in measured blood flow values (p<0.05). Finally, compact LSI system measurements were collected from the interdental papilla of nine subjects and compared to a clinical assessment of gingival bleeding on probing. A statistically significant correlation (ρ=0.53; p<0.005) was found between these variables, indicating that quantitative gingival perfusion measurements performed using our system may aid in the diagnosis and prognosis of periodontal disease. PMID:27787545

Design and evaluation of a miniature laser speckle imaging device to assess gingival health

NASA Astrophysics Data System (ADS)

Regan, Caitlin; White, Sean M.; Yang, Bruce Y.; Takesh, Thair; Ho, Jessica; Wink, Cherie; Wilder-Smith, Petra; Choi, Bernard

2016-10-01

Current methods used to assess gingivitis are qualitative and subjective. We hypothesized that gingival perfusion measurements could provide a quantitative metric of disease severity. We constructed a compact laser speckle imaging (LSI) system that could be mounted in custom-made oral molds. Rigid fixation of the LSI system in the oral cavity enabled measurement of blood flow in the gingiva. In vitro validation performed in controlled flow phantoms demonstrated that the compact LSI system had comparable accuracy and linearity compared to a conventional bench-top LSI setup. In vivo validation demonstrated that the compact LSI system was capable of measuring expected blood flow dynamics during a standard postocclusive reactive hyperemia and that the compact LSI system could be used to measure gingival blood flow repeatedly without significant variation in measured blood flow values (p<0.05). Finally, compact LSI system measurements were collected from the interdental papilla of nine subjects and compared to a clinical assessment of gingival bleeding on probing. A statistically significant correlation (ρ=0.53 p<0.005) was found between these variables, indicating that quantitative gingival perfusion measurements performed using our system may aid in the diagnosis and prognosis of periodontal disease.

Methods for assessing the quality of data in public health information systems: a critical review.

PubMed

Chen, Hong; Yu, Ping; Hailey, David; Wang, Ning

2014-01-01

The quality of data in public health information systems can be ensured by effective data quality assessment. In order to conduct effective data quality assessment, measurable data attributes have to be precisely defined. Then reliable and valid measurement methods for data attributes have to be used to measure each attribute. We conducted a systematic review of data quality assessment methods for public health using major databases and well-known institutional websites. 35 studies were eligible for inclusion in the study. A total of 49 attributes of data quality were identified from the literature. Completeness, accuracy and timeliness were the three most frequently assessed attributes of data quality. Most studies directly examined data values. This is complemented by exploring either data users' perception or documentation quality. However, there are limitations of current data quality assessment methods: a lack of consensus on attributes measured; inconsistent definition of the data quality attributes; a lack of mixed methods for assessing data quality; and inadequate attention to reliability and validity. Removal of these limitations is an opportunity for further improvement.

Patient-centered technological assessment and monitoring of depression for low-income patients.

PubMed

Wu, Shinyi; Vidyanti, Irene; Liu, Pai; Hawkins, Caitlin; Ramirez, Magaly; Guterman, Jeffrey; Gross-Schulman, Sandra; Sklaroff, Laura Myerchin; Ell, Kathleen

2014-01-01

Depression is a significant challenge for ambulatory care because it worsens health status and outcomes, increases health care utilizations and costs, and elevates suicide risk. An automatic telephonic assessment (ATA) system that links with tasks and alerts to providers may improve quality of depression care and increase provider productivity. We used ATA system in a trial to assess and monitor depressive symptoms of 444 safety-net primary care patients with diabetes. We assessed system properties, evaluated preliminary clinical outcomes, and estimated cost savings. The ATA system is feasible, reliable, valid, safe, and likely cost-effective for depression screening and monitoring for low-income primary care population.

Food for Thought ... Mechanistic Validation

PubMed Central

Hartung, Thomas; Hoffmann, Sebastian; Stephens, Martin

2013-01-01

Summary Validation of new approaches in regulatory toxicology is commonly defined as the independent assessment of the reproducibility and relevance (the scientific basis and predictive capacity) of a test for a particular purpose. In large ring trials, the emphasis to date has been mainly on reproducibility and predictive capacity (comparison to the traditional test) with less attention given to the scientific or mechanistic basis. Assessing predictive capacity is difficult for novel approaches (which are based on mechanism), such as pathways of toxicity or the complex networks within the organism (systems toxicology). This is highly relevant for implementing Toxicology for the 21st Century, either by high-throughput testing in the ToxCast/ Tox21 project or omics-based testing in the Human Toxome Project. This article explores the mostly neglected assessment of a test's scientific basis, which moves mechanism and causality to the foreground when validating/qualifying tests. Such mechanistic validation faces the problem of establishing causality in complex systems. However, pragmatic adaptations of the Bradford Hill criteria, as well as bioinformatic tools, are emerging. As critical infrastructures of the organism are perturbed by a toxic mechanism we argue that by focusing on the target of toxicity and its vulnerability, in addition to the way it is perturbed, we can anchor the identification of the mechanism and its verification. PMID:23665802

Measuring interdependence in ambulatory care.

PubMed

Katerndahl, David; Wood, Robert; Jaen, Carlos R

2017-04-01

Complex systems differ from complicated systems in that they are nonlinear, unpredictable and lacking clear cause-and-effect relationships, largely due to the interdependence of their components (effects of interconnectedness on system behaviour and consequences). The purpose of this study was to demonstrate the potential for network density to serve as a measure of interdependence, assess its concurrent validity and test whether the use of valued or binary ties yields better results. This secondary analysis used the 2010 National Ambulatory Care Medical Survey to assess interdependence of 'top 20' diagnoses seen and medications prescribed for 14 specialties. The degree of interdependence was measured as the level of association between diagnoses and drug interactions among medications. Both valued and binary network densities were computed for each specialty. To assess concurrent validity, these measures were correlated with previously-derived valid measures of complexity of care using the same database, adjusting for diagnosis and medication diversity. Partial correlations between diagnosis density, and both diagnosis and total input complexity, were significant, as were those between medication density and both medication and total output complexity; for both diagnosis and medication densities, adjusted correlations were higher for binary rather than valued densities. This study demonstrated the feasibility and validity of using network density as a measure of interdependence. When adjusted for measure diversity, density-complexity correlations were significant and higher for binary than valued density. This approach complements other methods of estimating complexity of care and may be applicable to unique settings. © 2015 John Wiley & Sons, Ltd.

Validity and Reliability of Wii Fit Balance Board for the Assessment of Balance of Healthy Young Adults and the Elderly

PubMed Central

Chang, Wen-Dien; Chang, Wan-Yi; Lee, Chia-Lun; Feng, Chi-Yen

2013-01-01

[Purpose] Balance is an integral part of human ability. The smart balance master system (SBM) is a balance test instrument with good reliability and validity, but it is expensive. Therefore, we modified a Wii Fit balance board, which is a convenient balance assessment tool, and analyzed its reliability and validity. [Subjects and Methods] We recruited 20 healthy young adults and 20 elderly people, and administered 3 balance tests. The correlation coefficient and intraclass correlation of both instruments were analyzed. [Results] There were no statistically significant differences in the 3 tests between the Wii Fit balance board and the SBM. The Wii Fit balance board had a good intraclass correlation (0.86–0.99) for the elderly people and positive correlations (r = 0.58–0.86) with the SBM. [Conclusions] The Wii Fit balance board is a balance assessment tool with good reliability and high validity for elderly people, and we recommend it as an alternative tool for assessing balance ability. PMID:24259769

Commercially available gaming systems as clinical assessment tools to improve value in the orthopaedic setting: a systematic review.

PubMed

Ruff, Jessica; Wang, Tiffany L; Quatman-Yates, Catherine C; Phieffer, Laura S; Quatman, Carmen E

2015-02-01

Commercially available gaming systems (CAGS) such as the Wii Balance Board (WBB) and Microsoft Xbox with Kinect (Xbox Kinect) are increasingly used as balance training and rehabilitation tools. The purpose of this review was to answer the question, "Are commercially available gaming systems valid and reliable instruments for use as clinical diagnostic and functional assessment tools in orthopaedic settings?" and provide a summary of relevant studies, identify their strengths and weaknesses, and generate conclusions regarding general validity/reliability of WBB and Xbox Kinect in orthopaedics. A systematic search was performed using MEDLINE (1996-2013) and Scopus (1996-2013). Inclusion criteria were minimum of 5 subjects, full manuscript provided in English or translated, and studies incorporating investigation of CAG measurement properties. Exclusion criteria included reviews, systematic reviews, summary/clinical commentaries, or case studies; conference proceedings/presentations; cadaveric studies; studies of non-reversible, non-orthopaedic-related musculoskeletal disease; non-human trials; and therapeutic studies not reporting comparative evaluation to already established functional assessment criteria. All studies meeting inclusion and exclusion criteria were appraised for quality by two independent reviewers. Evidence levels (I-V) were assigned to each study based on established methodological criteria. 3 Level II, 7 level III, and 1 Level IV studies met inclusion criteria and provided information related to the use of the WBB and Xbox Kinect as clinical assessment tools in the field of orthopaedics. Studies have used the WBB in a variety of clinical applications, including the measurement of center of pressure (COP), measurement of medial-to-lateral (M/L) or anterior-to-posterior (A/P) symmetry, assessment anatomic landmark positioning, and assessment of fall risk. However, no uniform protocols or outcomes were used to evaluate the quality of the WBB as a clinical assessment tool; therefore a wide range of sensitivities, specificities, accuracies, and validities were reported. Currently it is not possible to make a universal generalization about the clinical utility of CAGS in the field of orthopaedics. However, there is evidence to support using the WBB and the Xbox Kinect as tools to obtain reliable and valid COP measurements. The Wii Fit Game may specifically provide reliable and valid measurements for predicting fall risk. Copyright © 2014 Elsevier Ltd. All rights reserved.

Development of a clinical feeding assessment scale for very young infants in South Africa

PubMed Central

2016-01-01

Background There is a need for validated neonatal feeding assessment instruments in South Africa. A locally developed instrument may contribute to standardised evaluation procedures of high-risk neonates and address needs in resource constrained developing settings. Objective The aim of the study was to develop and validate the content of a clinical feeding assessment scale to diagnose oropharyngeal dysphagia (OPD) in neonates. Method The Neonatal Feeding Assessment Scale (NFAS) was developed using the Delphi method. Five international and South African speech-language therapists (SLTs) formed the expert panel, participating in two rounds of electronic questionnaires to develop and validate the content of the NFAS. Results All participants agreed on the need for the development of a valid clinical feeding assessment instrument to use with the neonatal population. The initial NFAS consisted of 240 items across 8 sections, and after the Delphi process was implemented, the final format was reduced to 211 items across 6 sections. The final format of the NFAS is scored using a binary scoring system guiding the clinician to diagnose the presence or absence of OPD. All members agreed on the format, the scoring system and the feeding constructs addressed in the revised final format of the NFAS. Conclusion The Delphi method and the diverse clinical and research experience of participants could be integrated to develop the NFAS which may be used in clinical practice in South Africa or similar developing contexts. Because of demographically different work settings marked by developed versus developing contexts, participants did not have the same expectations of a clinical dysphagia assessment. The international participants contributed to evidence-based content development. Local participants considered the contextual challenges of South African SLTs entering the field with basic competencies in neonatal dysphagia management, thereby justifying a comprehensive clinical instrument. The NFAS is aimed at clinicians working in Neonatal Intensive Care Units where they manage large caseloads of high-risk neonates. Further validation of the NFAS is recommended to determine its criterion validity in comparison with a widely accepted standard such as the modified barium swallow study. PMID:27796101

TAMDAR Sensor Validation in 2003 AIRS II

NASA Technical Reports Server (NTRS)

Daniels, Taumi S.; Murray, John J.; Anderson, Mark V.; Mulally, Daniel J.; Jensen, Kristopher R.; Grainger, Cedric A.; Delene, David J.

2005-01-01

This study entails an assessment of TAMDAR in situ temperature, relative humidity and winds sensor data from seven flights of the UND Citation II. These data are undergoing rigorous assessment to determine their viability to significantly augment domestic Meteorological Data Communications Reporting System (MDCRS) and the international Aircraft Meteorological Data Reporting (AMDAR) system observational databases to improve the performance of regional and global numerical weather prediction models. NASA Langley Research Center participated in the Second Alliance Icing Research Study from November 17 to December 17, 2003. TAMDAR data taken during this period is compared with validation data from the UND Citation. The data indicate acceptable performance of the TAMDAR sensor when compared to measurements from the UND Citation research instruments.

Towards a Consolidated Approach for the Assessment of Evaluation Models of Nuclear Power Reactors

DOE PAGES

Epiney, A.; Canepa, S.; Zerkak, O.; ...

2016-11-02

The STARS project at the Paul Scherrer Institut (PSI) has adopted the TRACE thermal-hydraulic (T-H) code for best-estimate system transient simulations of the Swiss Light Water Reactors (LWRs). For analyses involving interactions between system and core, a coupling of TRACE with the SIMULATE-3K (S3K) LWR core simulator has also been developed. In this configuration, the TRACE code and associated nuclear power reactor simulation models play a central role to achieve a comprehensive safety analysis capability. Thus, efforts have now been undertaken to consolidate the validation strategy by implementing a more rigorous and structured assessment approach for TRACE applications involving eithermore » only system T-H evaluations or requiring interfaces to e.g. detailed core or fuel behavior models. The first part of this paper presents the preliminary concepts of this validation strategy. The principle is to systematically track the evolution of a given set of predicted physical Quantities of Interest (QoIs) over a multidimensional parametric space where each of the dimensions represent the evolution of specific analysis aspects, including e.g. code version, transient specific simulation methodology and model "nodalisation". If properly set up, such environment should provide code developers and code users with persistent (less affected by user effect) and quantified information (sensitivity of QoIs) on the applicability of a simulation scheme (codes, input models, methodology) for steady state and transient analysis of full LWR systems. Through this, for each given transient/accident, critical paths of the validation process can be identified that could then translate into defining reference schemes to be applied for downstream predictive simulations. In order to illustrate this approach, the second part of this paper presents a first application of this validation strategy to an inadvertent blowdown event that occurred in a Swiss BWR/6. The transient was initiated by the spurious actuation of the Automatic Depressurization System (ADS). The validation approach progresses through a number of dimensions here: First, the same BWR system simulation model is assessed for different versions of the TRACE code, up to the most recent one. The second dimension is the "nodalisation" dimension, where changes to the input model are assessed. The third dimension is the "methodology" dimension. In this case imposed power and an updated TRACE core model are investigated. For each step in each validation dimension, a common set of QoIs are investigated. For the steady-state results, these include fuel temperatures distributions. For the transient part of the present study, the evaluated QoIs include the system pressure evolution and water carry-over into the steam line.« less

Validation of Automated Scoring of Oral Reading

ERIC Educational Resources Information Center

Balogh, Jennifer; Bernstein, Jared; Cheng, Jian; Van Moere, Alistair; Townshend, Brent; Suzuki, Masanori

2012-01-01

A two-part experiment is presented that validates a new measurement tool for scoring oral reading ability. Data collected by the U.S. government in a large-scale literacy assessment of adults were analyzed by a system called VersaReader that uses automatic speech recognition and speech processing technologies to score oral reading fluency. In the…

Analytical methodology for safety validation of computer controlled subsystems. Volume 1 : state-of-the-art and assessment of safety verification/validation methodologies

DOT National Transportation Integrated Search

1995-09-01

This report describes the development of a methodology designed to assure that a sufficiently high level of safety is achieved and maintained in computer-based systems which perform safety critical functions in high-speed rail or magnetic levitation ...

Quantitative model validation of manipulative robot systems

NASA Astrophysics Data System (ADS)

Kartowisastro, Iman Herwidiana

This thesis is concerned with applying the distortion quantitative validation technique to a robot manipulative system with revolute joints. Using the distortion technique to validate a model quantitatively, the model parameter uncertainties are taken into account in assessing the faithfulness of the model and this approach is relatively more objective than the commonly visual comparison method. The industrial robot is represented by the TQ MA2000 robot arm. Details of the mathematical derivation of the distortion technique are given which explains the required distortion of the constant parameters within the model and the assessment of model adequacy. Due to the complexity of a robot model, only the first three degrees of freedom are considered where all links are assumed rigid. The modelling involves the Newton-Euler approach to obtain the dynamics model, and the Denavit-Hartenberg convention is used throughout the work. The conventional feedback control system is used in developing the model. The system behavior to parameter changes is investigated as some parameters are redundant. This work is important so that the most important parameters to be distorted can be selected and this leads to a new term called the fundamental parameters. The transfer function approach has been chosen to validate an industrial robot quantitatively against the measured data due to its practicality. Initially, the assessment of the model fidelity criterion indicated that the model was not capable of explaining the transient record in term of the model parameter uncertainties. Further investigations led to significant improvements of the model and better understanding of the model properties. After several improvements in the model, the fidelity criterion obtained was almost satisfied. Although the fidelity criterion is slightly less than unity, it has been shown that the distortion technique can be applied in a robot manipulative system. Using the validated model, the importance of friction terms in the model was highlighted with the aid of the partition control technique. It was also shown that the conventional feedback control scheme was insufficient for a robot manipulative system due to high nonlinearity which was inherent in the robot manipulator.

Intrarater and interrater reliability and validity in the assessment of the mechanism of injury and integrity of the posterior ligamentous complex: a novel injury severity scoring system for thoracolumbar injuries. Invited submission from the Joint Section Meeting On Disorders of the Spine and Peripheral Nerves, March 2005.

PubMed

Harrop, James S; Vaccaro, Alexander R; Hurlbert, R John; Wilsey, Jared T; Baron, Eli M; Shaffrey, Christopher I; Fisher, Charles G; Dvorak, Marcel F; Oner, F C; Wood, Kirkham B; Anand, Neel; Anderson, D Greg; Lim, Moe R; Lee, Joon Y; Bono, Christopher M; Arnold, Paul M; Rampersaud, Y Raja; Fehlings, Michael G

2006-02-01

A new classification and treatment algorithm for thoracolumbar injuries was recently introduced by Vaccaro and colleagues in 2005. A thoracolumbar injury severity scale (TLISS) was proposed for grading and guiding treatment for these injuries. The scale is based on the following: 1) the mechanism of injury; 2) the integrity of the posterior ligamentous complex (PLC); and 3) the patient's neurological status. The reliability and validity of assessing injury mechanism and the integrity of the PLC was assessed. Forty-eight spine surgeons, consisting of neurosurgeons and orthopedic surgeons, reviewed 56 clinical thoracolumbar injury case histories. Each was classified and scored to determine treatment recommendations according to a novel classification system. After 3 months the case histories were reordered and the physicians repeated the exercise. Validity of this classification was good among reviewers; the vast majority (> 90%) agreed with the system's treatment recommendations. Surgeons were unclear as to a cogent description of PLC disruption and fracture mechanism. The TLISS demonstrated acceptable reliability in terms of intra- and interobserver agreement on the algorithm's treatment recommendations. Replacing injury mechanism with a description of injury morphology and better definition of PLC injury will improve inter- and intraobserver reliability of this injury classification system.

Development of the American College of Rheumatology’s Rheumatoid Arthritis Electronic Clinical Quality Measures

PubMed Central

Yazdany, Jinoos; Robbins, Mark; Schmajuk, Gabriela; Desai, Sonali; Lacaille, Diane; Neogi, Tuhina; Singh, Jasvinder A.; Genovese, Mark; Myslinski, Rachel; Fisk, Natalie; Francisco, Melissa; Newman, Eric

2017-01-01

Background Electronic clinical quality measures (eCQMs) rely on computer algorithms to extract data from electronic health records (EHRs). On behalf of the American College of Rheumatology (ACR), we sought to develop and test eCQMs for rheumatoid arthritis (RA). Methods Drawing from published ACR guidelines, a working group developed candidate RA process measures and subsequently assessed face validity through an interdisciplinary panel of health care stakeholders. A public comment period followed. Measures that passed these levels of review were electronically specified using the Quality Data Model, which provides standard nomenclature for data elements (category, datatype, value sets) obtained through an EHR. For each eCQM, 3 clinical sites using different EHR systems tested the scientific feasibility and validity of measures. Measures appropriate for accountability were presented for national endorsement. Results Expert panel validity ratings were high for all measures (median 8–9 out of 9). Health system performance on the eCQMs was 53.6% for RA disease activity assessment, 69.1% for functional status assessment, 93.1% for disease modifying drug (DMARD) use and 72.8% for tuberculosis screening. Kappa statistics, evaluating whether the eCQM validly captured data obtained from manual EHR chart review, demonstrated moderate to substantial agreement (0.54 for functional status assessment, 0.73 for tuberculosis screening, 0.84 for disease activity, and 0.85 for DMARD use). Conclusion Four eCQMs for RA have achieved national endorsement and are recommended for use in federal quality reporting programs. Implementation and further refinement of these measures is ongoing in the ACR’s registry, the Rheumatology Informatics System for Effectiveness (RISE). PMID:27564778

Validation of selected analytical methods using accuracy profiles to assess the impact of a Tobacco Heating System on indoor air quality.

PubMed

Mottier, Nicolas; Tharin, Manuel; Cluse, Camille; Crudo, Jean-René; Lueso, María Gómez; Goujon-Ginglinger, Catherine G; Jaquier, Anne; Mitova, Maya I; Rouget, Emmanuel G R; Schaller, Mathieu; Solioz, Jennifer

2016-09-01

Studies in environmentally controlled rooms have been used over the years to assess the impact of environmental tobacco smoke on indoor air quality. As new tobacco products are developed, it is important to determine their impact on air quality when used indoors. Before such an assessment can take place it is essential that the analytical methods used to assess indoor air quality are validated and shown to be fit for their intended purpose. Consequently, for this assessment, an environmentally controlled room was built and seven analytical methods, representing eighteen analytes, were validated. The validations were carried out with smoking machines using a matrix-based approach applying the accuracy profile procedure. The performances of the methods were compared for all three matrices under investigation: background air samples, the environmental aerosol of Tobacco Heating System THS 2.2, a heat-not-burn tobacco product developed by Philip Morris International, and the environmental tobacco smoke of a cigarette. The environmental aerosol generated by the THS 2.2 device did not have any appreciable impact on the performances of the methods. The comparison between the background and THS 2.2 environmental aerosol samples generated by smoking machines showed that only five compounds were higher when THS 2.2 was used in the environmentally controlled room. Regarding environmental tobacco smoke from cigarettes, the yields of all analytes were clearly above those obtained with the other two air sample types. Copyright © 2016 The Authors. Published by Elsevier B.V. All rights reserved.

Functional gait assessment and balance evaluation system test: reliability, validity, sensitivity, and specificity for identifying individuals with Parkinson disease who fall.

PubMed

Leddy, Abigail L; Crowner, Beth E; Earhart, Gammon M

2011-01-01

Gait impairments, balance impairments, and falls are prevalent in individuals with Parkinson disease (PD). Although the Berg Balance Scale (BBS) can be considered the reference standard for the determination of fall risk, it has a noted ceiling effect. Development of ceiling-free measures that can assess balance and are good at discriminating "fallers" from "nonfallers" is needed. The purpose of this study was to compare the Functional Gait Assessment (FGA) and the Balance Evaluation Systems Test (BESTest) with the BBS among individuals with PD and evaluate the tests' reliability, validity, and discriminatory sensitivity and specificity for fallers versus nonfallers. This was an observational study of community-dwelling individuals with idiopathic PD. The BBS, FGA, and BESTest were administered to 80 individuals with PD. Interrater reliability (n=15) was assessed by 3 raters. Test-retest reliability was based on 2 tests of participants (n=24), 2 weeks apart. Intraclass correlation coefficients (2,1) were used to calculate reliability, and Spearman correlation coefficients were used to assess validity. Cutoff points, sensitivity, and specificity were based on receiver operating characteristic plots. Test-retest reliability was .80 for the BBS, .91 for the FGA, and .88 for the BESTest. Interrater reliability was greater than .93 for all 3 tests. The FGA and BESTest were correlated with the BBS (r=.78 and r=.87, respectively). Cutoff scores to identify fallers were 47/56 for the BBS, 15/30 for the FGA, and 69% for the BESTest. The overall accuracy (area under the curve) for the BBS, FGA, and BESTest was .79, .80, and .85, respectively. Fall reports were retrospective. Both the FGA and the BESTest have reliability and validity for assessing balance in individuals with PD. The BESTest is most sensitive for identifying fallers.

Face, content and concurrent validity of the Mimic® dV-Trainer for robot-assisted endoscopic surgery: a prospective study.

PubMed

Egi, H; Hattori, M; Tokunaga, M; Suzuki, T; Kawaguchi, K; Sawada, H; Ohdan, H

2013-01-01

The aim of this study was to determine whether any correlation exists between the performance of the Mimic® dV-Trainer (Mimic Technologies, Seattle, Wash., USA) and the da Vinci Surgical System (Intuitive Surgical, Sunnyvale, Calif., USA). Twelve participants were recruited, ranging from residents to consultants. We used four training tasks, consisting of 'Pick and Place', 'Peg Board', 'Thread the Rings' and 'Suture Sponge', from the software program of the Mimic dV-Trainer. The performance of the participants was recorded and measured. Additionally, we prepared the same tasks for the da Vinci Surgical System. All participants completed the tasks using the da Vinci Surgical System and were assessed according to time, the Objective Structured Assessment of Technical Skill checklist and the global rating score for endoscopic suturing assessed by two independent blinded observers. After performing these tasks, the participants completed a questionnaire that evaluated the Mimic dV-Trainer's face and content validity. The final results for each participant for the Mimic dV-Trainer and the da Vinci Surgical System were compared. All participants ranked the Mimic dV-Trainer as a realistic training platform that is useful for residency training. There was a significant relationship between the Mimic dV-Trainer and the da Vinci Surgical System in all four tasks. We verified the reliability of the assessment of the checklist and the global rating scores for endoscopic suturing assessed by the two blinded observers using Cronbach's alpha test (r = 0.803, 0.891). We evaluated the concurrent validity of the Mimic dV-Trainer and the da Vinci Surgical System. Our results suggest the possibility that training using the Mimic dV-Trainer may therefore be able to improve the operator's performance during live robot-assisted surgery. © 2013 S. Karger AG, Basel.

Strategic Defense Initiative Demonstration/Validation Program Environmental Assessment. Battle Management/Command and Control, and Communications (BM/C3),

DTIC Science & Technology

1987-08-01

POR A ENVIOMNE. STRATEGIC DEFENSE INITRIEDNU T IATI V EI 1 0193 O RTEI ONE..()SRTGC DEFENSE INITIATV E S RATION V EO ORORNIZATION WASINZGTON DC...facilities where Demonstration/Validation activities are planned.- e Ten areas of environmental consideration are addressed: (1) air quality; (2) . water...air quality, rater quality, and hazardous vaste (63). 2.2 ELCTRONIC SYSTEMS DIVISION The Electronic Systems Division administrative offices are located

Verification and Validation of NASA-Supported Enhancements to the Near Real Time Harmful Algal Blooms Observing System (HABSOS)

NASA Technical Reports Server (NTRS)

Spruce, Joseph P.; Hall, Calllie; McPherson, Terry; Spiering, Bruce; Brown, Richard; Estep, Lee; Lunde, Bruce; Guest, DeNeice; Navard, Andy; Pagnutti, Mary;

Use of an automated learning management system to validate nursing competencies.

PubMed

Dumpe, Michelle L; Kanyok, Nancy; Hill, Kristin

2007-01-01

Maintaining nurse competencies in a dynamic environment is not an easy task and requires the use of resources already strained. An online learning management system was created, and 24 annual competencies were redesigned for online validation. As a result of this initiative, competencies have been standardized across many disciplines and are completed in a more timely manner, nurses and managers are more satisfied with this method of annual assessments, and cost savings have been realized.

The surface drifter program for real time and off-line validation of ocean forecasts and reanalyses

NASA Astrophysics Data System (ADS)

Hernandez, Fabrice; Regnier, Charly; Drévillon, Marie

2017-04-01

As part of the Global Ocean Observing System, the Global Drifter Program (GDP) is comprised of an array of about 1250 drifting buoys spread over the global ocean, that provide operational, near-real time surface velocity, sea surface temperature (SST) and sea level pressure observations. This information is used mainly used for numerical weather forecasting, research, and in-situ calibration/verification of satellite observations. Since 2013 the drifting buoy SST measurements are used for near real time assessment of global forecasting systems from Canada, France, UK, USA, Australia in the frame of the GODAE OceanView Intercomparison and Validation Task. For most of these operational systems, these data are not used for assimilation, and offer an independent observation assessment. This approach mimics the validation performed for SST satellite products. More recently, validation procedures have been proposed in order to assess the surface dynamics of Mercator Océan global and regional forecast and reanalyses. Velocities deduced from drifter trajectories are used in two ways. First, the Eulerian approach where buoy and ocean model velocity values are compared at the position of drifters. Then, from discrepancies, statistics are computed and provide an evaluation of the ocean model's surface dynamics reliability. Second, the Lagrangian approach, where drifting trajectories are simulated at each location of the real drifter trajectory using the ocean model velocity fields. Then, on daily basis, real and simulated drifter trajectories are compared by analyzing the spread after one day, two days etc…. The cumulated statistics on specific geographical boxes are evaluated in term of dispersion properties of the "real ocean" as captured by drifters, and those properties in the ocean model. This approach allows to better evaluate forecasting score for surface dispersion applications, like Search and Rescue, oil spill forecast, drift of other objects or contaminant, larvae dispersion etc… These Eulerian and Lagrangian validation approach can be applied for real time or offline assessment of ocean velocity products. In real time, the main limitation is our capability to detect drifter drogue's loss, causing erroneous assessment. Several methods, by comparison to wind entrainment effect or other velocity estimates like from satellite altimetry, are used. These Eulerian and Lagrangian surface velocity validation methods are planned to be adopted by the GODAE OceanView operational community in order to offer independent verification of surface current forecast.

Soil properties differently influence estimates of soil CO2 efflux from three chamber-based measurement systems

Treesearch

John R. Butnor; Kurt H. Johnsen; Chris A. Maier

2005-01-01

Soil C02 efflux is a major component of net ecosystem productivity (NEP) of forest systems. Combining data from multiple researchers for larger-scale modeling and assessment will only be valid if their methodologies provide directly comparable results. We conducted a series of laboratory and field tests to assess the presence and magnitude of...

Assessing Online Textual Feedback to Support Student Intrinsic Motivation Using a Collaborative Text-Based Dialogue System: A Qualitative Study

ERIC Educational Resources Information Center

Shroff, Ronnie H.; Deneen, Christopher

2011-01-01

This paper assesses textual feedback to support student intrinsic motivation using a collaborative text-based dialogue system. A research model is presented based on research into intrinsic motivation, and the specific construct of feedback provides a framework for the model. A qualitative research methodology is used to validate the model.…

Operational calibration and validation of landsat data continuity mission (LDCM) sensors using the image assessment system (IAS)

USGS Publications Warehouse

Micijevic, Esad; Morfitt, Ron

2010-01-01

Systematic characterization and calibration of the Landsat sensors and the assessment of image data quality are performed using the Image Assessment System (IAS). The IAS was first introduced as an element of the Landsat 7 (L7) Enhanced Thematic Mapper Plus (ETM+) ground segment and recently extended to Landsat 4 (L4) and 5 (L5) Thematic Mappers (TM) and Multispectral Sensors (MSS) on-board the Landsat 1-5 satellites. In preparation for the Landsat Data Continuity Mission (LDCM), the IAS was developed for the Earth Observer 1 (EO-1) Advanced Land Imager (ALI) with a capability to assess pushbroom sensors. This paper describes the LDCM version of the IAS and how it relates to unique calibration and validation attributes of its on-board imaging sensors. The LDCM IAS system will have to handle a significantly larger number of detectors and the associated database than the previous IAS versions. An additional challenge is that the LDCM IAS must handle data from two sensors, as the LDCM products will combine the Operational Land Imager (OLI) and Thermal Infrared Sensor (TIRS) spectral bands.

Ecological validity and clinical utility of Patient-Reported Outcomes Measurement Information System (PROMIS®) instruments for detecting premenstrual symptoms of depression, anger, and fatigue.

PubMed

Junghaenel, Doerte U; Schneider, Stefan; Stone, Arthur A; Christodoulou, Christopher; Broderick, Joan E

2014-04-01

This study examined the ecological validity and clinical utility of NIH Patient Reported-Outcomes Measurement Information System (PROMIS®) instruments for anger, depression, and fatigue in women with premenstrual symptoms. One-hundred women completed daily diaries and weekly PROMIS assessments over 4weeks. Weekly assessments were administered through Computerized Adaptive Testing (CAT). Weekly CATs and corresponding daily scores were compared to evaluate ecological validity. To test clinical utility, we examined if CATs could detect changes in symptom levels, if these changes mirrored those obtained from daily scores, and if CATs could identify clinically meaningful premenstrual symptom change. PROMIS CAT scores were higher in the pre-menstrual than the baseline (ps<.0001) and post-menstrual (ps<.0001) weeks. The correlations between CATs and aggregated daily scores ranged from .73 to .88 supporting ecological validity. Mean CAT scores showed systematic changes in accordance with the menstrual cycle and the magnitudes of the changes were similar to those obtained from the daily scores. Finally, Receiver Operating Characteristic (ROC) analyses demonstrated the ability of the CATs to discriminate between women with and without clinically meaningful premenstrual symptom change. PROMIS CAT instruments for anger, depression, and fatigue demonstrated validity and utility in premenstrual symptom assessment. The results provide encouraging initial evidence of the utility of PROMIS instruments for the measurement of affective premenstrual symptoms. Copyright © 2014 Elsevier Inc. All rights reserved.

[Support of the nursing process through electronic nursing documentation systems (UEPD) – Initial validation of an instrument].

PubMed

Hediger, Hannele; Müller-Staub, Maria; Petry, Heidi

2016-01-01

Electronic nursing documentation systems, with standardized nursing terminology, are IT-based systems for recording the nursing processes. These systems have the potential to improve the documentation of the nursing process and to support nurses in care delivery. This article describes the development and initial validation of an instrument (known by its German acronym UEPD) to measure the subjectively-perceived benefits of an electronic nursing documentation system in care delivery. The validity of the UEPD was examined by means of an evaluation study carried out in an acute care hospital (n = 94 nurses) in German-speaking Switzerland. Construct validity was analyzed by principal components analysis. Initial references of validity of the UEPD could be verified. The analysis showed a stable four factor model (FS = 0.89) scoring in 25 items. All factors loaded ≥ 0.50 and the scales demonstrated high internal consistency (Cronbach's α = 0.73 – 0.90). Principal component analysis revealed four dimensions of support: establishing nursing diagnosis and goals; recording a case history/an assessment and documenting the nursing process; implementation and evaluation as well as information exchange. Further testing with larger control samples and with different electronic documentation systems are needed. Another potential direction would be to employ the UEPD in a comparison of various electronic documentation systems.

Development and validation of a disease-specific health-related quality of life measure, the LupusQol, for adults with systemic lupus erythematosus.

PubMed

McElhone, Kathleen; Abbott, Janice; Shelmerdine, Joanna; Bruce, Ian N; Ahmad, Yasmeen; Gordon, Caroline; Peers, Kate; Isenberg, David; Ferenkeh-Koroma, Ada; Griffiths, Bridget; Akil, Mohamed; Maddison, Peter; Teh, Lee-Suan

2007-08-15

To develop and validate a disease-specific health-related quality of life (HRQOL) instrument for adults with systemic lupus erythematosus (SLE). The work consisted of 6 stages. Stage 1 included item generation for questionnaire content from semistructured interviews with SLE patients. In stage 2 item selection for the draft questionnaire was performed by thematic analysis of the patient interview transcripts and expert panel agreement. In stage 3 the content validity of the draft questionnaire was assessed by patients completing the questionnaire and providing critical feedback. In stages 4 and 5 construct validity and internal reliability of the 3 versions of the LupusQoL were evaluated using principal component analysis with varimax rotation and Cronbach's alpha coefficients, respectively. In stage 6 discriminatory validity, concurrent validity, and test-retest reliability were evaluated. Stages 1, 2, and 3 resulted in a preliminary instrument containing 63 items. In stage 4, 8 domains were identified. This factor structure, accounting for 82% of the variance, was confirmed in stage 5. The domains and Cronbach's alpha coefficients were physical health (0.94), emotional health (0.94), body image (0.89), pain (0.92), planning (0.93), fatigue (0.88), intimate relationships (0.96), and burden to others (0.94). Discriminant validity was demonstrated for different levels of disease activity (British Isles Lupus Assessment Group Index) and damage (Systemic Lupus International Collaborating Clinics/American College of Rheumatology Damage Index). High correlations (r = 0.71-0.79) between comparable domains of the Short Form 36 and the LupusQoL assured acceptable concurrent validity. Good test-retest reliability (r = 0.72-0.93) was demonstrated. The LupusQoL is a validated SLE-specific HRQOL instrument with 34 items across 8 domains defined by patients as being important.

Diagnosis and treatment of posterior sacroiliac complex pain: a systematic review with comprehensive analysis of the published data.

PubMed

King, Wade; Ahmed, Shihab U; Baisden, Jamie; Patel, Nileshkumar; Kennedy, David J; Duszynski, Belinda; MacVicar, John

2015-02-01

To assess the evidence on the validity of sacral lateral branch blocks and the effectiveness of sacral lateral branch thermal radiofrequency neurotomy in managing sacroiliac complex pain. Systematic review with comprehensive analysis of all published data. Six reviewers searched the literature on sacral lateral branch interventions. Each assessed the methodologies of studies found and the quality of the evidence presented. The outcomes assessed were diagnostic validity and effectiveness of treatment for sacroiliac complex pain. The evidence found was appraised in accordance with the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) system of evaluating scientific evidence. The searches yielded two primary publications on sacral lateral branch blocks and 15 studies of the effectiveness of sacral lateral branch thermal radiofrequency neurotomy. One study showed multisite, multidepth sacral lateral branch blocks can anesthetize the posterior sacroiliac ligaments. Therapeutic studies show sacral lateral branch thermal radiofrequency neurotomy can relieve sacroiliac complex pain to some extent. The evidence of the validity of these blocks and the effectiveness of this treatment were rated as moderate in accordance with the GRADE system. The literature on sacral lateral branch interventions is sparse. One study demonstrates the face validity of multisite, multidepth sacral lateral branch blocks for diagnosis of posterior sacroiliac complex pain. Some evidence of moderate quality exists on therapeutic procedures, but it is insufficient to determine the indications and effectiveness of sacral lateral branch thermal radiofrequency neurotomy, and more research is required. Wiley Periodicals, Inc.

Instrument Motion Metrics for Laparoscopic Skills Assessment in Virtual Reality and Augmented Reality.

PubMed

Fransson, Boel A; Chen, Chi-Ya; Noyes, Julie A; Ragle, Claude A

2016-11-01

To determine the construct and concurrent validity of instrument motion metrics for laparoscopic skills assessment in virtual reality and augmented reality simulators. Evaluation study. Veterinarian students (novice, n = 14) and veterinarians (experienced, n = 11) with no or variable laparoscopic experience. Participants' minimally invasive surgery (MIS) experience was determined by hospital records of MIS procedures performed in the Teaching Hospital. Basic laparoscopic skills were assessed by 5 tasks using a physical box trainer. Each participant completed 2 tasks for assessments in each type of simulator (virtual reality: bowel handling and cutting; augmented reality: object positioning and a pericardial window model). Motion metrics such as instrument path length, angle or drift, and economy of motion of each simulator were recorded. None of the motion metrics in a virtual reality simulator showed correlation with experience, or to the basic laparoscopic skills score. All metrics in augmented reality were significantly correlated with experience (time, instrument path, and economy of movement), except for the hand dominance metric. The basic laparoscopic skills score was correlated to all performance metrics in augmented reality. The augmented reality motion metrics differed between American College of Veterinary Surgeons diplomates and residents, whereas basic laparoscopic skills score and virtual reality metrics did not. Our results provide construct validity and concurrent validity for motion analysis metrics for an augmented reality system, whereas a virtual reality system was validated only for the time score. © Copyright 2016 by The American College of Veterinary Surgeons.

Assessing validity of observational intervention studies - the Benchmarking Controlled Trials.

PubMed

Malmivaara, Antti

2016-09-01

Benchmarking Controlled Trial (BCT) is a concept which covers all observational studies aiming to assess impact of interventions or health care system features to patients and populations. To create and pilot test a checklist for appraising methodological validity of a BCT. The checklist was created by extracting the most essential elements from the comprehensive set of criteria in the previous paper on BCTs. Also checklists and scientific papers on observational studies and respective systematic reviews were utilized. Ten BCTs published in the Lancet and in the New England Journal of Medicine were used to assess feasibility of the created checklist. The appraised studies seem to have several methodological limitations, some of which could be avoided in planning, conducting and reporting phases of the studies. The checklist can be used for planning, conducting, reporting, reviewing, and critical reading of observational intervention studies. However, the piloted checklist should be validated in further studies. Key messages Benchmarking Controlled Trial (BCT) is a concept which covers all observational studies aiming to assess impact of interventions or health care system features to patients and populations. This paper presents a checklist for appraising methodological validity of BCTs and pilot-tests the checklist with ten BCTs published in leading medical journals. The appraised studies seem to have several methodological limitations, some of which could be avoided in planning, conducting and reporting phases of the studies. The checklist can be used for planning, conducting, reporting, reviewing, and critical reading of observational intervention studies.

Measurement in Sensory Modulation: The Sensory Processing Scale Assessment

PubMed Central

Miller, Lucy J.; Sullivan, Jillian C.

2014-01-01

OBJECTIVE. Sensory modulation issues have a significant impact on participation in daily life. Moreover, understanding phenotypic variation in sensory modulation dysfunction is crucial for research related to defining homogeneous groups and for clinical work in guiding treatment planning. We thus evaluated the new Sensory Processing Scale (SPS) Assessment. METHOD. Research included item development, behavioral scoring system development, test administration, and item analyses to evaluate reliability and validity across sensory domains. RESULTS. Items with adequate reliability (internal reliability >.4) and discriminant validity (p < .01) were retained. Feedback from the expert panel also contributed to decisions about retaining items in the scale. CONCLUSION. The SPS Assessment appears to be a reliable and valid measure of sensory modulation (scale reliability >.90; discrimination between group effect sizes >1.00). This scale has the potential to aid in differential diagnosis of sensory modulation issues. PMID:25184464

Continuous Monitoring of Essential Tremor Using a Portable System Based on Smartwatch.

PubMed

Zheng, Xiaochen; Vieira Campos, Alba; Ordieres-Meré, Joaquín; Balseiro, Jose; Labrador Marcos, Sergio; Aladro, Yolanda

2017-01-01

Essential tremor (ET) shows amplitude fluctuations throughout the day, presenting challenges in both clinical and treatment monitoring. Tremor severity is currently evaluated by validated rating scales, which only provide a timely and subjective assessment during a clinical visit. Motor sensors have shown favorable performances in quantifying tremor objectively. A new highly portable system was used to monitor tremor continuously during daily lives. It consists of a smartwatch with a triaxial accelerometer, a smartphone, and a remote server. An experiment was conducted involving eight ET patients. The average effective data collection time per patient was 26 (±6.05) hours. Fahn-Tolosa-Marin Tremor Rating Scale (FTMTRS) was adopted as the gold standard to classify tremor and to validate the performance of the system. Quantitative analysis of tremor severity on different time scales is validated. Significant correlations were observed between neurologist's FTMTRS and patient's FTMTRS auto-assessment scores ( r  = 0.84; p  = 0.009), between the device quantitative measures and the scores from the standardized assessments of neurologists ( r  = 0.80; p  = 0.005) and patient's auto-evaluation ( r  = 0.97; p  = 0.032), and between patient's FTMTRS auto-assessment scores day-to-day ( r  = 0.87; p  < 0.001). A graphical representation of four patients with different degrees of tremor was presented, and a representative system is proposed to summarize the tremor scoring at different time scales. This study demonstrates the feasibility of prolonged and continuous monitoring of tremor severity during daily activities by a highly portable non-restrictive system, a useful tool to analyze efficacy and effectiveness of treatment.

Evaluation of the Validity and Response Burden of Patient Self-Report Measures of the Pain Assessment Screening Tool and Outcomes Registry (PASTOR).

PubMed

Cook, Karon F; Kallen, Michael A; Buckenmaier, Chester; Flynn, Diane M; Hanling, Steven R; Collins, Teresa S; Joltes, Kristin; Kwon, Kyung; Medina-Torne, Sheila; Nahavandi, Parisa; Suen, Joshua; Gershon, Richard

2017-07-01

In 2009, the Army Pain Management Task Force was chartered. On the basis of their findings, the Department of Defense recommended a comprehensive pain management strategy that included development of a standardized pain assessment system that would collect patient-reported outcomes data to inform the patient-provider clinical encounter. The result was the Pain Assessment Screening Tool and Outcomes Registry (PASTOR). The purpose of this study was to assess the validity and response burden of the patient-reported outcome measures in PASTOR. Data for analyses were collected from 681 individuals who completed PASTOR at baseline and follow-up as part of their routine clinical care. The survey tool included self-report measures of pain severity and pain interference (measured using the National Institutes of Health Patient-Reported Outcome Measurement Information System [PROMIS] and the Defense and Veterans Pain Rating scale). PROMIS measures of pain correlates also were administered. Validation analyses included estimation of score associations among measures, comparison of scores of known groups, responsiveness, ceiling and floor effects, and response burden. Results of psychometric testing provided substantial evidence for the validity of PASTOR self-report measures in this population. Expected associations among scores largely supported the concurrent validity of the measures. Scores effectively distinguished among respondents on the basis of their self-reported impressions of general health. PROMIS measures were administered using computer adaptive testing and each, on average, required less than 1 minute to administer. Statistical and graphical analyses demonstrated the responsiveness of PASTOR measures over time. Reprint & Copyright © 2017 Association of Military Surgeons of the U.S.

Long-term predictions using natural analogues

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ewing, R.C.

1995-09-01

One of the unique and scientifically most challenging aspects of nuclear waste isolation is the extrapolation of short-term laboratory data (hours to years) to the long time periods (10{sup 3}-10{sup 5} years) required by regulatory agencies for performance assessment. The direct validation of these extrapolations is not possible, but methods must be developed to demonstrate compliance with government regulations and to satisfy the lay public that there is a demonstrable and reasonable basis for accepting the long-term extrapolations. Natural systems (e.g., {open_quotes}natural analogues{close_quotes}) provide perhaps the only means of partial {open_quotes}validation,{close_quotes} as well as data that may be used directlymore » in the models that are used in the extrapolation. Natural systems provide data on very large spatial (nm to km) and temporal (10{sup 3}-10{sup 8} years) scales and in highly complex terranes in which unknown synergisms may affect radionuclide migration. This paper reviews the application (and most importantly, the limitations) of data from natural analogue systems to the {open_quotes}validation{close_quotes} of performance assessments.« less

Standardization of the Functional Assessment and Intervention Program (FAIP) with Children Who Have Externalizing Behaviors

ERIC Educational Resources Information Center

Hartwig, Laurie; Heathfield, Lora Tuesday; Jenson, William R.

2004-01-01

The purpose of this study was to develop standardization data for the Functional Assessment Intervention Program (FAIP; University of Utah, Utah State University, & Utah State Office of Education, 1999), a computerized, functional behavioral assessment expert system. Reliability, validity, and utility analyses were conducted with students serving…

Assessing Scientific and Technological Enquiry Skills at Age 11 Using the E-Scape System

ERIC Educational Resources Information Center

Davies, Dan; Collier, Chris; Howe, Alan

2012-01-01

This article reports on the outcomes from the "e-scape Primary Scientific and Technological Understanding Assessment Project" (2009-2010), which aimed to support primary teachers in developing valid portfolio-based tasks to assess pupils' scientific and technological enquiry skills at age 11. This was part of the wider…

A novel augmented reality simulator for skills assessment in minimal invasive surgery.

PubMed

Lahanas, Vasileios; Loukas, Constantinos; Smailis, Nikolaos; Georgiou, Evangelos

2015-08-01

Over the past decade, simulation-based training has come to the foreground as an efficient method for training and assessment of surgical skills in minimal invasive surgery. Box-trainers and virtual reality (VR) simulators have been introduced in the teaching curricula and have substituted to some extent the traditional model of training based on animals or cadavers. Augmented reality (AR) is a new technology that allows blending of VR elements and real objects within a real-world scene. In this paper, we present a novel AR simulator for assessment of basic laparoscopic skills. The components of the proposed system include: a box-trainer, a camera and a set of laparoscopic tools equipped with custom-made sensors that allow interaction with VR training elements. Three AR tasks were developed, focusing on basic skills such as perception of depth of field, hand-eye coordination and bimanual operation. The construct validity of the system was evaluated via a comparison between two experience groups: novices with no experience in laparoscopic surgery and experienced surgeons. The observed metrics included task execution time, tool pathlength and two task-specific errors. The study also included a feedback questionnaire requiring participants to evaluate the face-validity of the system. Between-group comparison demonstrated highly significant differences (<0.01) in all performance metrics and tasks denoting the simulator's construct validity. Qualitative analysis on the instruments' trajectories highlighted differences between novices and experts regarding smoothness and economy of motion. Subjects' ratings on the feedback questionnaire highlighted the face-validity of the training system. The results highlight the potential of the proposed simulator to discriminate groups with different expertise providing a proof of concept for the potential use of AR as a core technology for laparoscopic simulation training.

Validity and reliability of balance assessment software using the Nintendo Wii balance board: usability and validation

PubMed Central

2014-01-01

Background A balance test provides important information such as the standard to judge an individual’s functional recovery or make the prediction of falls. The development of a tool for a balance test that is inexpensive and widely available is needed, especially in clinical settings. The Wii Balance Board (WBB) is designed to test balance, but there is little software used in balance tests, and there are few studies on reliability and validity. Thus, we developed a balance assessment software using the Nintendo Wii Balance Board, investigated its reliability and validity, and compared it with a laboratory-grade force platform. Methods Twenty healthy adults participated in our study. The participants participated in the test for inter-rater reliability, intra-rater reliability, and concurrent validity. The tests were performed with balance assessment software using the Nintendo Wii balance board and a laboratory-grade force platform. Data such as Center of Pressure (COP) path length and COP velocity were acquired from the assessment systems. The inter-rater reliability, the intra-rater reliability, and concurrent validity were analyzed by an intraclass correlation coefficient (ICC) value and a standard error of measurement (SEM). Results The inter-rater reliability (ICC: 0.89-0.79, SEM in path length: 7.14-1.90, SEM in velocity: 0.74-0.07), intra-rater reliability (ICC: 0.92-0.70, SEM in path length: 7.59-2.04, SEM in velocity: 0.80-0.07), and concurrent validity (ICC: 0.87-0.73, SEM in path length: 5.94-0.32, SEM in velocity: 0.62-0.08) were high in terms of COP path length and COP velocity. Conclusion The balance assessment software incorporating the Nintendo Wii balance board was used in our study and was found to be a reliable assessment device. In clinical settings, the device can be remarkably inexpensive, portable, and convenient for the balance assessment. PMID:24912769

Validity and reliability of balance assessment software using the Nintendo Wii balance board: usability and validation.

PubMed

Park, Dae-Sung; Lee, GyuChang

2014-06-10

A balance test provides important information such as the standard to judge an individual's functional recovery or make the prediction of falls. The development of a tool for a balance test that is inexpensive and widely available is needed, especially in clinical settings. The Wii Balance Board (WBB) is designed to test balance, but there is little software used in balance tests, and there are few studies on reliability and validity. Thus, we developed a balance assessment software using the Nintendo Wii Balance Board, investigated its reliability and validity, and compared it with a laboratory-grade force platform. Twenty healthy adults participated in our study. The participants participated in the test for inter-rater reliability, intra-rater reliability, and concurrent validity. The tests were performed with balance assessment software using the Nintendo Wii balance board and a laboratory-grade force platform. Data such as Center of Pressure (COP) path length and COP velocity were acquired from the assessment systems. The inter-rater reliability, the intra-rater reliability, and concurrent validity were analyzed by an intraclass correlation coefficient (ICC) value and a standard error of measurement (SEM). The inter-rater reliability (ICC: 0.89-0.79, SEM in path length: 7.14-1.90, SEM in velocity: 0.74-0.07), intra-rater reliability (ICC: 0.92-0.70, SEM in path length: 7.59-2.04, SEM in velocity: 0.80-0.07), and concurrent validity (ICC: 0.87-0.73, SEM in path length: 5.94-0.32, SEM in velocity: 0.62-0.08) were high in terms of COP path length and COP velocity. The balance assessment software incorporating the Nintendo Wii balance board was used in our study and was found to be a reliable assessment device. In clinical settings, the device can be remarkably inexpensive, portable, and convenient for the balance assessment.

Risk Factors for Venous Thromboembolism in Pediatric Trauma Patients and Validation of a Novel Scoring System: The Risk of Clots in Kids with Trauma (ROCKIT score)

PubMed Central

Yen, Jennifer; Van Arendonk, Kyle J.; Streiff, Michael B.; McNamara, LeAnn; Stewart, F. Dylan; Conner G, Kim G; Thompson, Richard E.; Haut, Elliott R.; Takemoto, Clifford M.

2017-01-01

OBJECTIVES Identify risk factors for venous thromboembolism (VTE) and develop a VTE risk assessment model for pediatric trauma patients. DESIGN, SETTING, AND PATIENTS We performed a retrospective review of patients 21 years and younger who were hospitalized following traumatic injuries at the John Hopkins level 1 adult and pediatric trauma center (1987-2011). The clinical characteristics of patients with and without VTE were compared, and multivariable logistic regression analysis was used to identify independent risk factors for VTE. Weighted risk assessment scoring systems were developed based on these and previously identified factors from patients in the National Trauma Data Bank (NTDB 2008-2010); the scoring systems were validated in this cohort from Johns Hopkins as well as a cohort of pediatric admissions from the NTDB (2011-2012). MAIN RESULTS Forty-nine of 17,366 pediatric trauma patients (0.28%) were diagnosed with VTE after admission to our trauma center. After adjusting for potential confounders, VTE was independently associated with older age, surgery, blood transfusion, higher Injury Severity Score (ISS), and lower Glasgow Coma Scale (GCS) score. These and additional factors were identified in 402,329 pediatric patients from the NTDB from 2008-2010; independent risk factors from the logistic regression analysis of this NTDB cohort were selected and incorporated into weighted risk assessment scoring systems. Two models were developed and were cross-validated in 2 separate pediatric trauma cohorts: 1) 282,535 patients in the NTDB from 2011 to 2012 2) 17,366 patients from Johns Hopkins. The receiver operator curve using these models in the validation cohorts had area under the curves that ranged 90% to 94%. CONCLUSIONS VTE is infrequent after trauma in pediatric patients. We developed weighted scoring systems to stratify pediatric trauma patients at risk for VTE. These systems may have potential to guide risk-appropriate VTE prophylaxis in children after trauma. PMID:26963757

Digital avionics systems - Overview of FAA/NASA/industry-wide briefing

NASA Technical Reports Server (NTRS)

Larsen, William E.; Carro, Anthony

1986-01-01

The effects of incorporating digital technology into the design of aircraft on the airworthiness criteria and certification procedures for aircraft are investigated. FAA research programs aimed at providing data for the functional assessment of aircraft which use digital systems for avionics and flight control functions are discussed. The need to establish testing, assurance assessment, and configuration management technologies to insure the reliability of digital systems is discussed; consideration is given to design verification, system performance/robustness, and validation technology.

Validation of risk assessment scoring systems for an audit of elective surgery for gastrointestinal cancer in elderly patients: an audit.

PubMed

Wakabayashi, Hisao; Sano, Takanori; Yachida, Shinichi; Okano, Keiichi; Izuishi, Kunihiko; Suzuki, Yasuyuki

2007-10-01

The goal of this study was to validate the usefulness of risk assessment scoring systems for a surgical audit in elective digestive surgery for elderly patients. The validated scoring systems used were the Physiological and Operative Severity Score for enUmeration of Mortality and morbidity (POSSUM) and the Portsmouth predictor equation for mortality (P-POSSUM). This study involved 153 consecutive patients aged 75 years and older who underwent elective gastric or colorectal surgery between July 2004 and June 2006. A retrospective analysis was performed on data collected prior to each surgery. The predicted mortality and morbidity risks were calculated using each of the scoring systems and were used to obtain the observed/predicted (O/E) mortality and morbidity ratios. New logistic regression equations for morbidity and mortality were then calculated using the scores from the POSSUM system and applied retrospectively. The O/E ratio for morbidity obtained from POSSUM score was 0.23. The O/E ratios for mortality from the POSSUM score and the P-POSSUM were 0.15 and 0.38, respectively. Utilizing the new equations using scores from the POSSUM, the O/E ratio increased to 0.88. Both the POSSUM and P-POSSUM over-predicted the morbidity and mortality in elective gastrointestinal surgery for malignant tumors in elderly patients. However, if a surgical unit makes appropriate calculations using its own patient series and updates these equations, the POSSUM system can be useful in the risk assessment for surgery in elderly patients.

The Reliability and Validity of the Thoracolumbar Injury Classification System in Pediatric Spine Trauma.

PubMed

Savage, Jason W; Moore, Timothy A; Arnold, Paul M; Thakur, Nikhil; Hsu, Wellington K; Patel, Alpesh A; McCarthy, Kathryn; Schroeder, Gregory D; Vaccaro, Alexander R; Dimar, John R; Anderson, Paul A

2015-09-15

The thoracolumbar injury classification system (TLICS) was evaluated in 20 consecutive pediatric spine trauma cases. The purpose of this study was to determine the reliability and validity of the TLICS in pediatric spine trauma. The TLICS was developed to improve the categorization and management of thoracolumbar trauma. TLICS has been shown to have good reliability and validity in the adult population. The clinical and radiographical findings of 20 pediatric thoracolumbar fractures were prospectively presented to 20 surgeons with disparate levels of training and experience with spinal trauma. These injuries were consecutively scored using the TLICS. Cohen unweighted κ coefficients and Spearman rank order correlation values were calculated for the key parameters (injury morphology, status of posterior ligamentous complex, neurological status, TLICS total score, and proposed management) to assess the inter-rater reliabilities. Five surgeons scored the same cases 3 months later to assess the intra-rater reliability. The actual management of each case was then compared with the treatment recommended by the TLICS algorithm to assess validity. The inter-rater κ statistics of all subgroups (injury morphology, status of the posterior ligamentous complex, neurological status, TLICS total score, and proposed treatment) were within the range of moderate to substantial reproducibility (0.524-0.958). All subgroups had excellent intra-rater reliability (0.748-1.000). The various indices for validity were calculated (80.3% correct, 0.836 sensitivity, 0.785 specificity, 0.676 positive predictive value, 0.899 negative predictive value). Overall, TLICS demonstrated good validity. The TLICS has good reliability and validity when used in the pediatric population. The inter-rater reliability of predicting management and indices for validity are lower than those in adults with thoracolumbar fractures, which is likely due to differences in the way children are treated for certain types of injuries. TLICS can be used to reliably categorize thoracolumbar injuries in the pediatric population; however, modifications may be needed to better guide treatment in this specific patient population. 4.

Development and validation of self-reported line drawings of the modified Beighton score for the assessment of generalised joint hypermobility.

PubMed

Cooper, Dale J; Scammell, Brigitte E; Batt, Mark E; Palmer, Debbie

2018-01-17

The impracticalities and comparative expense of carrying out a clinical assessment is an obstacle in many large epidemiological studies. The purpose of this study was to develop and validate a series of electronic self-reported line drawing instruments based on the modified Beighton scoring system for the assessment of self-reported generalised joint hypermobility. Five sets of line drawings were created to depict the 9-point Beighton score criteria. Each instrument consisted of an explanatory question whereby participants were asked to select the line drawing which best represented their joints. Fifty participants completed the self-report online instrument on two occasions, before attending a clinical assessment. A blinded expert clinical observer then assessed participants' on two occasions, using a standardised goniometry measurement protocol. Validity of the instrument was assessed by participant-observer agreement and reliability by participant repeatability and observer repeatability using unweighted Cohen's kappa (k). Validity and reliability were assessed for each item in the self-reported instrument separately, and for the sum of the total scores. An aggregate score for generalised joint hypermobility was determined based on a Beighton score of 4 or more out of 9. Observer-repeatability between the two clinical assessments demonstrated perfect agreement (k 1.00; 95% CI 1.00, 1.00). Self-reported participant-repeatability was lower but it was still excellent (k 0.91; 95% CI 0.74, 1.00). The participant-observer agreement was excellent (k 0.96; 95% CI 0.87, 1.00). Validity was excellent for the self-report instrument, with a good sensitivity of 0.87 (95% CI 0.81, 0.91) and excellent specificity of 0.99 (95% CI 0.98, 1.00). The self-reported instrument provides a valid and reliable assessment of the presence of generalised joint hypermobility and may have practical use in epidemiological studies.

Development and preliminary validation of an interactive remote physical therapy system.

PubMed

Mishra, Anup K; Skubic, Marjorie; Abbott, Carmen

2015-01-01

In this paper, we present an interactive physical therapy system (IPTS) for remote quantitative assessment of clients in the home. The system consists of two different interactive interfaces connected through a network, for a real-time low latency video conference using audio, video, skeletal, and depth data streams from a Microsoft Kinect. To test the potential of IPTS, experiments were conducted with 5 independent living senior subjects in Kansas City, MO. Also, experiments were conducted in the lab to validate the real-time biomechanical measures calculated using the skeletal data from the Microsoft Xbox 360 Kinect and Microsoft Xbox One Kinect, with ground truth data from a Vicon motion capture system. Good agreements were found in the validation tests. The results show potential capabilities of the IPTS system to provide remote physical therapy to clients, especially older adults, who may find it difficult to visit the clinic.

LADAR Performance Simulations with a High Spectral Resolution Atmospheric Transmittance and Radiance Model-LEEDR

DTIC Science & Technology

2012-03-01

such as FASCODE is accomplished. The assessment is limited by the correctness of the models used; validating the models is beyond the scope of this...comparisons with other models and validation against data sets (Snell et al. 2000). 2.3.2 Previous Research Several LADAR simulations have been produced...performance models would better capture the atmosphere physics and climatological effects on these systems. Also, further validation needs to be performed

Milestone-compatible neurology resident assessments: A role for observable practice activities.

PubMed

Jones, Lyell K; Dimberg, Elliot L; Boes, Christopher J; Eggers, Scott D Z; Dodick, David W; Cutsforth-Gregory, Jeremy K; Leep Hunderfund, Andrea N; Capobianco, David J

2015-06-02

Beginning in 2014, US neurology residency programs were required to report each trainee's educational progression within 29 neurology Milestone competency domains. Trainee assessment systems will need to be adapted to inform these requirements. The primary aims of this study were to validate neurology resident assessment content using observable practice activities (OPAs) and to develop assessment formats easily translated to the Neurology Milestones. A modified Delphi technique was used to establish consensus perceptions of importance of 73 neurology OPAs among neurology educators and trainees at 3 neurology residency programs. A content validity score (CVS) was derived for each neurology OPA, with scores ≥4.0 determined in advance to indicate sufficient content validity. The mean CVS for all OPAs was 4.4 (range 3.5-5.0). Fifty-seven (78%) OPAs had a CVS ≥4.0, leaving 16 (22%) below the pre-established threshold for content validity. Trainees assigned a higher importance to individual OPAs (mean CVS 4.6) compared to faculty (mean 4.4, p = 0.016), but the effect size was small (η(2) = 0.10). There was no demonstrated effect of length of education experience on perceived importance of neurology OPAs (p = 0.23). Two sample resident assessment formats were developed, one using neurology OPAs alone and another using a combination of neurology OPAs and the Neurology Milestones. This study provides neurology training programs with content validity evidence for items to include in resident assessments, and sample assessment formats that directly translate to the Neurology Milestones. Length of education experience has little effect on perceptions of neurology OPA importance. © 2015 American Academy of Neurology.

Development of Flight-Test Performance Estimation Techniques for Small Unmanned Aerial Systems

NASA Astrophysics Data System (ADS)

McCrink, Matthew Henry

This dissertation provides a flight-testing framework for assessing the performance of fixed-wing, small-scale unmanned aerial systems (sUAS) by leveraging sub-system models of components unique to these vehicles. The development of the sub-system models, and their links to broader impacts on sUAS performance, is the key contribution of this work. The sub-system modeling and analysis focuses on the vehicle's propulsion, navigation and guidance, and airframe components. Quantification of the uncertainty in the vehicle's power available and control states is essential for assessing the validity of both the methods and results obtained from flight-tests. Therefore, detailed propulsion and navigation system analyses are presented to validate the flight testing methodology. Propulsion system analysis required the development of an analytic model of the propeller in order to predict the power available over a range of flight conditions. The model is based on the blade element momentum (BEM) method. Additional corrections are added to the basic model in order to capture the Reynolds-dependent scale effects unique to sUAS. The model was experimentally validated using a ground based testing apparatus. The BEM predictions and experimental analysis allow for a parameterized model relating the electrical power, measurable during flight, to the power available required for vehicle performance analysis. Navigation system details are presented with a specific focus on the sensors used for state estimation, and the resulting uncertainty in vehicle state. Uncertainty quantification is provided by detailed calibration techniques validated using quasi-static and hardware-in-the-loop (HIL) ground based testing. The HIL methods introduced use a soft real-time flight simulator to provide inertial quality data for assessing overall system performance. Using this tool, the uncertainty in vehicle state estimation based on a range of sensors, and vehicle operational environments is presented. The propulsion and navigation system models are used to evaluate flight-testing methods for evaluating fixed-wing sUAS performance. A brief airframe analysis is presented to provide a foundation for assessing the efficacy of the flight-test methods. The flight-testing presented in this work is focused on validating the aircraft drag polar, zero-lift drag coefficient, and span efficiency factor. Three methods are detailed and evaluated for estimating these design parameters. Specific focus is placed on the influence of propulsion and navigation system uncertainty on the resulting performance data. Performance estimates are used in conjunction with the propulsion model to estimate the impact sensor and measurement uncertainty on the endurance and range of a fixed-wing sUAS. Endurance and range results for a simplistic power available model are compared to the Reynolds-dependent model presented in this work. Additional parameter sensitivity analysis related to state estimation uncertainties encountered in flight-testing are presented. Results from these analyses indicate that the sub-system models introduced in this work are of first-order importance, on the order of 5-10% change in range and endurance, in assessing the performance of a fixed-wing sUAS.

Validation of Sea levels from coastal altimetry waveform retracking expert system: a case study around the Prince William Sound in Alaska

NASA Astrophysics Data System (ADS)

Idris, N. H.; Deng, X.; Idris, N. H.

2017-05-01

This paper presents the validation of Coastal Altimetry Waveform Retracking Expert System (CAWRES), a novel method to optimize the Jason satellite altimetric sea levels from multiple retracking solutions. The validation is conducted over the region of Prince William Sound in Alaska, USA, where altimetric waveforms are perturbed by emerged land and sea states. Validation is performed in twofold. First, comparison with existing retrackers (i.e. MLE4 and Ice) from the Sensor Geophysical Data Records (SGDR), and second, comparison with in-situ tide gauge data. From the first validation assessment, in general, CAWRES outperforms the MLE4 and Ice retrackers. In 4 out of 6 cases, the value of improvement percentage (standard deviation of difference) is higher (lower) than those of the SGDR retrackers. CAWRES also presents the best performance in producing valid observations, and has the lowest noise when compared to the SGDR retrackers. From the second assessment with tide gauge, CAWRES retracked sea level anomalies (SLAs) are consistent with those of the tide gauge. The accuracy of CAWRES retracked SLAs is slightly better than those of the MLE4. However, the performance of Ice retracker is better than those of CAWRES and MLE4, suggesting the empirical-based retracker is more effective. The results demonstrate that the CAWRES would have potential to be applied to coastal regions elsewhere.

OneDep: Unified wwPDB System for Deposition, Biocuration, and Validation of Macromolecular Structures in the PDB Archive.

PubMed

Young, Jasmine Y; Westbrook, John D; Feng, Zukang; Sala, Raul; Peisach, Ezra; Oldfield, Thomas J; Sen, Sanchayita; Gutmanas, Aleksandras; Armstrong, David R; Berrisford, John M; Chen, Li; Chen, Minyu; Di Costanzo, Luigi; Dimitropoulos, Dimitris; Gao, Guanghua; Ghosh, Sutapa; Gore, Swanand; Guranovic, Vladimir; Hendrickx, Pieter M S; Hudson, Brian P; Igarashi, Reiko; Ikegawa, Yasuyo; Kobayashi, Naohiro; Lawson, Catherine L; Liang, Yuhe; Mading, Steve; Mak, Lora; Mir, M Saqib; Mukhopadhyay, Abhik; Patwardhan, Ardan; Persikova, Irina; Rinaldi, Luana; Sanz-Garcia, Eduardo; Sekharan, Monica R; Shao, Chenghua; Swaminathan, G Jawahar; Tan, Lihua; Ulrich, Eldon L; van Ginkel, Glen; Yamashita, Reiko; Yang, Huanwang; Zhuravleva, Marina A; Quesada, Martha; Kleywegt, Gerard J; Berman, Helen M; Markley, John L; Nakamura, Haruki; Velankar, Sameer; Burley, Stephen K

2017-03-07

OneDep, a unified system for deposition, biocuration, and validation of experimentally determined structures of biological macromolecules to the PDB archive, has been developed as a global collaboration by the worldwide PDB (wwPDB) partners. This new system was designed to ensure that the wwPDB could meet the evolving archiving requirements of the scientific community over the coming decades. OneDep unifies deposition, biocuration, and validation pipelines across all wwPDB, EMDB, and BMRB deposition sites with improved focus on data quality and completeness in these archives, while supporting growth in the number of depositions and increases in their average size and complexity. In this paper, we describe the design, functional operation, and supporting infrastructure of the OneDep system, and provide initial performance assessments. Published by Elsevier Ltd.

Feasibility and validity of animal-based indicators for on-farm welfare assessment of thermal stress in dairy goats

NASA Astrophysics Data System (ADS)

Battini, Monica; Barbieri, Sara; Fioni, Luna; Mattiello, Silvana

2016-02-01

This investigation tested the feasibility and validity of indicators of cold and heat stress in dairy goats for on-farm welfare assessment protocols. The study was performed on two intensive dairy farms in Italy. Two different 3-point scale (0-2) scoring systems were applied to assess cold and heat stress. Cold and heat stress scores were visually assessed from outside the pen in the morning, afternoon and evening in January-February, April-May and July 2013 for a total of nine sessions of observations/farm. Temperature (°C), relative humidity (%) and wind speed (km/h) were recorded and Thermal Heat Index (THI) was calculated. The sessions were allocated to three climatic seasons, depending on THI ranges: cold (<50), neutral (50-65) and hot (>65). Score 2 was rarely assessed; therefore, scores 1 and 2 were aggregated for statistical analysis. The amount of goats suffering from cold stress was significantly higher in the cold season than in neutral ( P < 0.01) and hot ( P < 0.001) seasons. Signs of heat stress were recorded only in the hot season ( P < 0.001). The visual assessment from outside the pen confirms the on-farm feasibility of both indicators: No constraint was found and time required was less than 10 min. Our results show that cold and heat stress scores are valid indicators to detect thermal stress in intensively managed dairy goats. The use of a binary scoring system (presence/absence), merging scores 1 and 2, may be a further refinement to improve the feasibility. This study also allows the prediction of optimal ranges of THI for dairy goat breeds in intensive husbandry systems, setting a comfort zone included into 55 and 70.

A knowledge-based patient assessment system: conceptual and technical design.

PubMed Central

Reilly, C. A.; Zielstorff, R. D.; Fox, R. L.; O'Connell, E. M.; Carroll, D. L.; Conley, K. A.; Fitzgerald, P.; Eng, T. K.; Martin, A.; Zidik, C. M.; Segal, M.

2000-01-01

This paper describes the design of an inpatient patient assessment application that captures nursing assessment data using a wireless laptop computer. The primary aim of this system is to capture structured information for facilitating decision support and quality monitoring. The system also aims to improve efficiency of recording patient assessments, reduce costs, and improve discharge planning and early identification of patient learning needs. Object-oriented methods were used to elicit functional requirements and to model the proposed system. A tools-based development approach is being used to facilitate rapid development and easy modification of assessment items and rules for decision support. Criteria for evaluation include perceived utility by clinician users, validity of decision support rules, time spent recording assessments, and perceived utility of aggregate reports for quality monitoring. PMID:11079970

A knowledge-based patient assessment system: conceptual and technical design.

PubMed

Reilly, C A; Zielstorff, R D; Fox, R L; O'Connell, E M; Carroll, D L; Conley, K A; Fitzgerald, P; Eng, T K; Martin, A; Zidik, C M; Segal, M

2000-01-01

This paper describes the design of an inpatient patient assessment application that captures nursing assessment data using a wireless laptop computer. The primary aim of this system is to capture structured information for facilitating decision support and quality monitoring. The system also aims to improve efficiency of recording patient assessments, reduce costs, and improve discharge planning and early identification of patient learning needs. Object-oriented methods were used to elicit functional requirements and to model the proposed system. A tools-based development approach is being used to facilitate rapid development and easy modification of assessment items and rules for decision support. Criteria for evaluation include perceived utility by clinician users, validity of decision support rules, time spent recording assessments, and perceived utility of aggregate reports for quality monitoring.

Patient-Centered Technological Assessment and Monitoring of Depression for Low-Income Patients

PubMed Central

Wu, Shinyi; Vidyanti, Irene; Liu, Pai; Hawkins, Caitlin; Ramirez, Magaly; Guterman, Jeffrey; Gross-Schulman, Sandra; Sklaroff, Laura Myerchin; Ell, Kathleen

2014-01-01

Depression is a significant challenge for ambulatory care because it worsens health status and outcomes, increases health care utilizations and costs, and elevates suicide risk. An automatic telephonic assessment (ATA) system that links with tasks and alerts to providers may improve quality of depression care and increase provider productivity. We used ATA system in a trial to assess and monitor depressive symptoms of 444 safety-net primary care patients with diabetes. We assessed system properties, evaluated preliminary clinical outcomes, and estimated cost savings. The ATA system is feasible, reliable, valid, safe, and likely cost-effective for depression screening and monitoring for low-income primary care population. PMID:24525531

German National Proficiency Scales in Biology: Internal Structure, Relations to General Cognitive Abilities and Verbal Skills

ERIC Educational Resources Information Center

Kampa, Nele; Köller, Olaf

2016-01-01

National and international large-scale assessments (LSA) have a major impact on educational systems, which raises fundamental questions about the validity of the measures regarding their internal structure and their relations to relevant covariates. Given its importance, research on the validity of instruments specifically developed for LSA is…

The Use of Variants of the Trail Making Test in Serial Assessment: A Construct Validity Study

ERIC Educational Resources Information Center

Atkinson, Thomas M.; Ryan, Jeanne P.

2008-01-01

The construct validity of three variants of the Trail Making Test was investigated using 162 undergraduate psychology students. During a 3-week period, the Trail Making Test of the Delis-Kaplan Executive Function System, Comprehensive Trail Making Test, and Connections Task were administered in six possible orders. Using confirmatory factor…

Validation of the Seating and Mobility Script Concordance Test

ERIC Educational Resources Information Center

Cohen, Laura J.; Fitzgerald, Shirley G.; Lane, Suzanne; Boninger, Michael L.; Minkel, Jean; McCue, Michael

2009-01-01

The purpose of this study was to develop the scoring system for the Seating and Mobility Script Concordance Test (SMSCT), obtain and appraise internal and external structure evidence, and assess the validity of the SMSCT. The SMSCT purpose is to provide a method for testing knowledge of seating and mobility prescription. A sample of 106 therapists…

Validation of Unsupervised Computer-Based Screening for Reading Disability in Greek Elementary Grades 3 and 4

ERIC Educational Resources Information Center

Protopapas, Athanassios; Skaloumbakas, Christos; Bali, Persefoni

2008-01-01

After reviewing past efforts related to computer-based reading disability (RD) assessment, we present a fully automated screening battery that evaluates critical skills relevant for RD diagnosis designed for unsupervised application in the Greek educational system. Psychometric validation in 301 children, 8-10 years old (grades 3 and 4; including…

Measuring Life Stress: A Comparison of the Predictive Validity of Different Scoring Systems for the Social Readjustment Rating Scale.

ERIC Educational Resources Information Center

McGrath, Robert E. V.; Burkhart, Barry R.

1983-01-01

Assessed whether accounting for variables in the scoring of the Social Readjustment Rating Scale (SRRS) would improve the predictive validity of the inventory. Results from 107 sets of questionnaires showed that income and level of education are significant predictors of the capacity to cope with stress. (JAC)

An empirical study of the predictive validity of number grades in medical school using 3 decades of longitudinal data: implications for a grading system.

PubMed

Gonnella, Joseph S; Erdmann, James B; Hojat, Mohammadreza

2004-04-01

Context It is important to establish the predictive validity of medical school grades. The strength of predictive validity and the ability to identify at-risk students in medical schools depends upon assessment systems such as number grades, pass/fail (P/F) or honours/pass/fail (H/P/F) systems. Objective This study was designed to examine the predictive validity of number grades in medical school, and to determine whether any important information is lost in a shift from number to P/F and H/P/F grading systems. Subjects The participants in this prospective, longitudinal study were 6656 medical students who studied at Jefferson Medical College over 3 decades. They were grouped into 10 deciles based on their number grades in Year 1 of medical school. Methods Participants were compared on academic accomplishments in Years 2 and 3 of medical school, medical school class rank, delayed graduation and attrition, performance on medical licensing examinations and clinical competence ratings in the first postgraduate year. Results Results supported the short- and longterm predictive validity of the number grades. Ratings of clinical competence beyond medical school were predicted by number grades in medical school. We demonstrated that small differences in number grades are statistically meaningful, and that important information for identifying students in need of remedial education is lost when students who narrowly meet faculty's expectations are included with the rest of the class in a broad 'pass' category. Conclusions The findings refute the argument that knowledge of sciences basic to medicine is not critical to subsequent performance in medical school and beyond if an appropriate evaluation system is used. Furthermore, the results of this study raise questions about abandoning number grades in favour of a pass/fail system. Consideration of these findings in policy decisions regarding assessment systems of medical students is recommended.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Epiney, A.; Canepa, S.; Zerkak, O.

The STARS project at the Paul Scherrer Institut (PSI) has adopted the TRACE thermal-hydraulic (T-H) code for best-estimate system transient simulations of the Swiss Light Water Reactors (LWRs). For analyses involving interactions between system and core, a coupling of TRACE with the SIMULATE-3K (S3K) LWR core simulator has also been developed. In this configuration, the TRACE code and associated nuclear power reactor simulation models play a central role to achieve a comprehensive safety analysis capability. Thus, efforts have now been undertaken to consolidate the validation strategy by implementing a more rigorous and structured assessment approach for TRACE applications involving eithermore » only system T-H evaluations or requiring interfaces to e.g. detailed core or fuel behavior models. The first part of this paper presents the preliminary concepts of this validation strategy. The principle is to systematically track the evolution of a given set of predicted physical Quantities of Interest (QoIs) over a multidimensional parametric space where each of the dimensions represent the evolution of specific analysis aspects, including e.g. code version, transient specific simulation methodology and model "nodalisation". If properly set up, such environment should provide code developers and code users with persistent (less affected by user effect) and quantified information (sensitivity of QoIs) on the applicability of a simulation scheme (codes, input models, methodology) for steady state and transient analysis of full LWR systems. Through this, for each given transient/accident, critical paths of the validation process can be identified that could then translate into defining reference schemes to be applied for downstream predictive simulations. In order to illustrate this approach, the second part of this paper presents a first application of this validation strategy to an inadvertent blowdown event that occurred in a Swiss BWR/6. The transient was initiated by the spurious actuation of the Automatic Depressurization System (ADS). The validation approach progresses through a number of dimensions here: First, the same BWR system simulation model is assessed for different versions of the TRACE code, up to the most recent one. The second dimension is the "nodalisation" dimension, where changes to the input model are assessed. The third dimension is the "methodology" dimension. In this case imposed power and an updated TRACE core model are investigated. For each step in each validation dimension, a common set of QoIs are investigated. For the steady-state results, these include fuel temperatures distributions. For the transient part of the present study, the evaluated QoIs include the system pressure evolution and water carry-over into the steam line.« less

Latent Factor Structure of the Das-Naglieri Cognitive Assessment System: A Confirmatory Factor Analysis in a Chinese Setting

ERIC Educational Resources Information Center

Deng, Ci-ping; Liu, Ming; Wei, Wei; Chan, Raymond C. K.; Das, J. P.

2011-01-01

This study aims to measure the psychometric properties of the Das-Naglieri Cognitive Assessment System (D-N CAS) and to determine its clinical utility in a Chinese context. Confirmatory factor analysis (CFA) was conducted to examine the construct validity of the Chinese version of the D-N CAS among a group of 567, normally developed children.…

Effective data validation of high-frequency data: time-point-, time-interval-, and trend-based methods.

PubMed

Horn, W; Miksch, S; Egghart, G; Popow, C; Paky, F

1997-09-01

Real-time systems for monitoring and therapy planning, which receive their data from on-line monitoring equipment and computer-based patient records, require reliable data. Data validation has to utilize and combine a set of fast methods to detect, eliminate, and repair faulty data, which may lead to life-threatening conclusions. The strength of data validation results from the combination of numerical and knowledge-based methods applied to both continuously-assessed high-frequency data and discontinuously-assessed data. Dealing with high-frequency data, examining single measurements is not sufficient. It is essential to take into account the behavior of parameters over time. We present time-point-, time-interval-, and trend-based methods for validation and repair. These are complemented by time-independent methods for determining an overall reliability of measurements. The data validation benefits from the temporal data-abstraction process, which provides automatically derived qualitative values and patterns. The temporal abstraction is oriented on a context-sensitive and expectation-guided principle. Additional knowledge derived from domain experts forms an essential part for all of these methods. The methods are applied in the field of artificial ventilation of newborn infants. Examples from the real-time monitoring and therapy-planning system VIE-VENT illustrate the usefulness and effectiveness of the methods.

Quantifying Soiling Loss Directly From PV Yield

DOE PAGES

Deceglie, Michael G.; Micheli, Leonardo; Muller, Matthew

2018-01-23

Soiling of photovoltaic (PV) panels is typically quantified through the use of specialized sensors. Here, we describe and validate a method for estimating soiling loss experienced by PV systems directly from system yield without the need for precipitation data. The method, termed the stochastic rate and recovery (SRR) method, automatically detects soiling intervals in a dataset, then stochastically generates a sample of possible soiling profiles based on the observed characteristics of each interval. In this paper, we describe the method, validate it against soiling station measurements, and compare it with other PV-yield-based soiling estimation methods. The broader application of themore » SRR method will enable the fleet scale assessment of soiling loss to facilitate mitigation planning and risk assessment.« less

Quantifying Soiling Loss Directly From PV Yield

DOE Office of Scientific and Technical Information (OSTI.GOV)

Deceglie, Michael G.; Micheli, Leonardo; Muller, Matthew

Soiling of photovoltaic (PV) panels is typically quantified through the use of specialized sensors. Here, we describe and validate a method for estimating soiling loss experienced by PV systems directly from system yield without the need for precipitation data. The method, termed the stochastic rate and recovery (SRR) method, automatically detects soiling intervals in a dataset, then stochastically generates a sample of possible soiling profiles based on the observed characteristics of each interval. In this paper, we describe the method, validate it against soiling station measurements, and compare it with other PV-yield-based soiling estimation methods. The broader application of themore » SRR method will enable the fleet scale assessment of soiling loss to facilitate mitigation planning and risk assessment.« less

The Systemic Sclerosis Questionnaire (SySQ): Validation of the translation of the original German version into Spanish and its relationship to the disease and to quality of life.

PubMed

Cruz-Domínguez, Maria Pilar; Casarrubias-Ramírez, Moisés; Gasca-Martínez, Victor; Maldonado García, Cindy; Carranza-Muleiro, Rosa Angélica; Medina, Gabriela; García-Collinot, Grettel; Montes-Cortes, Daniel Hector

2017-12-11

Translation, transculturation and validity of the self-administered questionnaire for functionality (Systemic Sclerosis Questionnaires [SySQ]) for use in Spanish patients with systemic sclerosis and its relationship to the severity of the disease and to quality of life. We conducted an observational analytical study to perform a cross-cultural validation of the self-administered questionnaire on functionality in scleroderma. The validity of the form and content was evaluated by an expert panel. The method included: a) adaptation into Spanish of the construct for translation and back translation, and transculturation; b) internal consistency with the SySQ (Cronbach's alpha), and c) reproducibility was assessed taking into account all occasions in which the test was performed with Cohen's kappa. Additionally, we calculated the Spearman correlation coefficient with the Medsger severity scale, Health Assessment Questionnaire score and SF-36 score. We included 70 patients with systemic sclerosis: age 17-78 (51±12) years, 65 (93%) were women, diffuse/limited subtype 64/36%, disease duration of 0.5-40 years. Optimal internal consistency for all categories of the final version of SySQ (Cronbach's α of 0.961) and intraobserver reliability in 2 tests over a 2-week interval (Cohen's kappa coefficient 0.618) and optimal interobserver reliability in 2 tests on the same day (Cohen's kappa coefficient 0.911). Moderate correlation between functionality by SySQ and by Health Assessment Questionnaire (r=0.573, P<.0001). Inverse correlation between SySQ and quality of life mental health domain SF-36 (r=-0.435, P<.001) and physical domain SF-36 (r=-0.638, P<.001). Medsger severity scale (tendon, heart, lung, vascular) also showed significant correlation with SySQ. SySQ in this validated Spanish version is a suitable instrument to measure functional status in patients with systemic sclerosis. Reduced functionality is related to greater tendon and peripheral vascular involvement and to a poorer quality of life. Copyright © 2017 Elsevier España, S.L.U. and Sociedad Española de Reumatología y Colegio Mexicano de Reumatología. All rights reserved.

Absolute fracture risk assessment using lumbar spine and femoral neck bone density measurements: derivation and validation of a hybrid system.

PubMed

Leslie, William D; Lix, Lisa M

2011-03-01

The World Health Organization (WHO) Fracture Risk Assessment Tool (FRAX) computes 10-year probability of major osteoporotic fracture from multiple risk factors, including femoral neck (FN) T-scores. Lumbar spine (LS) measurements are not currently part of the FRAX formulation but are used widely in clinical practice, and this creates confusion when there is spine-hip discordance. Our objective was to develop a hybrid 10-year absolute fracture risk assessment system in which nonvertebral (NV) fracture risk was assessed from the FN and clinical vertebral (V) fracture risk was assessed from the LS. We identified 37,032 women age 45 years and older undergoing baseline FN and LS dual-energy X-ray absorptiometry (DXA; 1990-2005) from a population database that contains all clinical DXA results for the Province of Manitoba, Canada. Results were linked to longitudinal health service records for physician billings and hospitalizations to identify nontrauma vertebral and nonvertebral fracture codes after bone mineral density (BMD) testing. The population was randomly divided into equal-sized derivation and validation cohorts. Using the derivation cohort, three fracture risk prediction systems were created from Cox proportional hazards models (adjusted for age and multiple FRAX risk factors): FN to predict combined all fractures, FN to predict nonvertebral fractures, and LS to predict vertebral (without nonvertebral) fractures. The hybrid system was the sum of nonvertebral risk from the FN model and vertebral risk from the LS model. The FN and hybrid systems were both strongly predictive of overall fracture risk (p < .001). In the validation cohort, ROC analysis showed marginally better performance of the hybrid system versus the FN system for overall fracture prediction (p = .24) and significantly better performance for vertebral fracture prediction (p < .001). In a discordance subgroup with FN and LS T-score differences greater than 1 SD, there was a significant improvement in overall fracture prediction with the hybrid method (p = .025). Risk reclassification under the hybrid system showed better alignment with observed fracture risk, with 6.4% of the women reclassified to a different risk category. In conclusion, a hybrid 10-year absolute fracture risk assessment system based on combining FN and LS information is feasible. The improvement in fracture risk prediction is small but supports clinical interest in a system that integrates LS in fracture risk assessment. Copyright © 2011 American Society for Bone and Mineral Research.

A new approach to assess movements and isometric postures of spine and trunk at the workplace.

PubMed

Wunderlich, Max; Rüther, Thomas; Essfeld, Dieter; Erren, Thomas C; Piekarski, Claus; Leyk, Dieter

2011-08-01

Low back pain is regarded as the primary cause of occupational disability in many countries worldwide. However, there is a lack of valid assessment of kinematic spine and trunk parameters to provide further insight into occupational spine loads. A new 3-dimensional mobile measurement system (3D-SpineMoveGuard) was developed and evaluated by means of repeated dynamic and isometric trunk positions by 10 male and 10 female volunteers. The interclass correlation coefficient indicates high test-retest reliability (r = 0.975-0.999) of the 3D-SpineMoveGuard. Moreover, analysis of validity revealed almost identical results for the new measurement system. The evaluation study indicates a good scientific quality for the use in occupational task analyses. The objective assessment of indirectly measured spine and trunk kinematics will give further insight to predict and prevent job-related spine loads.

Assessment of Preschool Hyperactivity: Combining Rating Scale and Objective Observation Measures.

ERIC Educational Resources Information Center

Mayes, Susan Dickerson

1987-01-01

Advantages and disadvantages of behavior rating scales and observation systems are presented, followed by preliminary validity data for the Mayes Hyperactivity Observation System, a clinically feasible system to identify preschool children with both Attention Deficit Disorder and Hyperactivity. Hyperactive and normal children were identified with…

PERCLOS: A Valid Psychophysiological Measure of Alertness As Assessed by Psychomotor Vigilance

DOT National Transportation Integrated Search

2002-04-01

The Logical Architecture is based on a Computer Aided Systems Engineering (CASE) model of the requirements for the flow of data and control through the various functions included in Intelligent Transportation Systems (ITS). Process Specifications pro...

Calibration validation for the new generation runway visual range system

DOT National Transportation Integrated Search

2000-07-01

A forward scattermeter, consisting of transmitter and receiver heads mounted on a fork, is used in the New Genreration Runway Visual Range (NGRVR) System to assess the clarity of the atmosphere. The scattermeter is calibrated by comparison with refer...

Translation, Cross-cultural Adaptation and Psychometric Validation of the Korean-Language Cardiac Rehabilitation Barriers Scale (CRBS-K).

PubMed

Baek, Sora; Park, Hee-Won; Lee, Yookyung; Grace, Sherry L; Kim, Won-Seok

2017-10-01

To perform a translation and cross-cultural adaptation of the Cardiac Rehabilitation Barriers Scale (CRBS) for use in Korea, followed by psychometric validation. The CRBS was developed to assess patients' perception of the degree to which patient, provider and health system-level barriers affect their cardiac rehabilitation (CR) participation. The CRBS consists of 21 items (barriers to adherence) rated on a 5-point Likert scale. The first phase was to translate and cross-culturally adapt the CRBS to the Korean language. After back-translation, both versions were reviewed by a committee. The face validity was assessed in a sample of Korean patients (n=53) with history of acute myocardial infarction that did not participate in CR through semi-structured interviews. The second phase was to assess the construct and criterion validity of the Korean translation as well as internal reliability, through administration of the translated version in 104 patients, principle component analysis with varimax rotation and cross-referencing against CR use, respectively. The length, readability, and clarity of the questionnaire were rated well, demonstrating face validity. Analysis revealed a six-factor solution, demonstrating construct validity. Cronbach's alpha was greater than 0.65. Barriers rated highest included not knowing about CR and not being contacted by a program. The mean CRBS score was significantly higher among non-attendees (2.71±0.26) than CR attendees (2.51±0.18) (p<0.01). The Korean version of CRBS has demonstrated face, content and criterion validity, suggesting it may be useful for assessing barriers to CR utilization in Korea.

Construct and face validity of the American College of Surgeons/Association of Program Directors in Surgery laparoscopic troubleshooting team training exercise.

PubMed

Arain, Nabeel A; Hogg, Deborah C; Gala, Rajiv B; Bhoja, Ravi; Tesfay, Seifu T; Webb, Erin M; Scott, Daniel J

2012-01-01

Our aim was to develop an objective scoring system and evaluate construct and face validity for a laparoscopic troubleshooting team training exercise. Surgery and gynecology novices (n = 14) and experts (n = 10) participated. Assessments included the following: time-out, scenario decision making (SDM) score (based on essential treatments rendered and completion time), operating room communication assessment (investigator developed), line operations safety audits (teamwork), and National Aeronautics and Space Administration-Task Load Index (workload). Significant differences were detected for SDM scores for scenarios 1 (192 vs 278; P = .01) and 3 (129 vs 225; P = .004), operating room communication assessment (67 vs 91; P = .002), and line operations safety audits (58 vs 87; P = .001), but not for time-out (46 vs 51) or scenario 2 SDM score (301 vs 322). Workload was similar for both groups and face validity (8.8 on a 10-point scale) was strongly supported. Objective decision-making scoring for 2 of 3 scenarios and communication and teamwork ratings showed construct validity. Face validity and participant feedback were excellent. Copyright © 2012 Elsevier Inc. All rights reserved.

Assessing the Culture of Residency Using the C - Change Resident Survey: Validity Evidence in 34 U.S. Residency Programs.

PubMed

Pololi, Linda H; Evans, Arthur T; Civian, Janet T; Shea, Sandy; Brennan, Robert T

2017-07-01

A practical instrument is needed to reliably measure the clinical learning environment and professionalism for residents. To develop and present evidence of validity of an instrument to assess the culture of residency programs and the clinical learning environment. During 2014-2015, we surveyed residents using the C - Change Resident Survey to assess residents' perceptions of the culture in their programs. Residents in all years of training in 34 programs in internal medicine, pediatrics, and general surgery in 14 geographically diverse public and private academic health systems. The C - Change Resident Survey assessed residents' perceptions of 13 dimensions of the culture: Vitality, Self-Efficacy, Institutional Support, Relationships/Inclusion, Values Alignment, Ethical/Moral Distress, Respect, Mentoring, Work-Life Integration, Gender Equity, Racial/Ethnic Minority Equity, and self-assessed Competencies. We measured the internal reliability of each of the 13 dimensions and evaluated response process, content validity, and construct-related evidence validity by assessing relationships predicted by our conceptual model and prior research. We also assessed whether the measurements were sensitive to differences in specialty and across institutions. A total of 1708 residents completed the survey [internal medicine: n = 956, pediatrics: n = 411, general surgery: n = 311 (51% women; 16% underrepresented in medicine minority)], with a response rate of 70% (range across programs, 51-87%). Internal consistency of each dimension was high (Cronbach α: 0.73-0.90). The instrument was able to detect significant differences in the learning environment across programs and sites. Evidence of validity was supported by a good response process and the demonstration of several relationships predicted by our conceptual model. The C - Change Resident Survey assesses the clinical learning environment for residents, and we encourage further study of validity in different contexts. Results could be used to facilitate and monitor improvements in the clinical learning environment and resident well-being.

AIRS Retrieval Validation During the EAQUATE

NASA Technical Reports Server (NTRS)

Zhou, Daniel K.; Smith, William L.; Cuomo, Vincenzo; Taylor, Jonathan P.; Barnet, Christopher D.; DiGirolamo, Paolo; Pappalardo, Gelsomina; Larar, Allen M.; Liu, Xu; Newman, Stuart M.

2006-01-01

Atmospheric and surface thermodynamic parameters retrieved with advanced hyperspectral remote sensors of Earth observing satellites are critical for weather prediction and scientific research. The retrieval algorithms and retrieved parameters from satellite sounders must be validated to demonstrate the capability and accuracy of both observation and data processing systems. The European AQUA Thermodynamic Experiment (EAQUATE) was conducted mainly for validation of the Atmospheric InfraRed Sounder (AIRS) on the AQUA satellite, but also for assessment of validation systems of both ground-based and aircraft-based instruments which will be used for other satellite systems such as the Infrared Atmospheric Sounding Interferometer (IASI) on the European MetOp satellite, the Cross-track Infrared Sounder (CrIS) from the NPOESS Preparatory Project and the following NPOESS series of satellites. Detailed inter-comparisons were conducted and presented using different retrieval methodologies: measurements from airborne ultraspectral Fourier transform spectrometers, aircraft in-situ instruments, dedicated dropsondes and radiosondes, and ground based Raman Lidar, as well as from the European Center for Medium range Weather Forecasting (ECMWF) modeled thermal structures. The results of this study not only illustrate the quality of the measurements and retrieval products but also demonstrate the capability of these validation systems which are put in place to validate current and future hyperspectral sounding instruments and their scientific products.

Group Peer Assessment for Summative Evaluation in a Graduate-Level Statistics Course for Ecologists

ERIC Educational Resources Information Center

ArchMiller, Althea; Fieberg, John; Walker, J.D.; Holm, Noah

2017-01-01

Peer assessment is often used for formative learning, but few studies have examined the validity of group-based peer assessment for the summative evaluation of course assignments. The present study contributes to the literature by using online technology (the course management system Moodle™) to implement structured, summative peer review based on…

Designing Assessments for Instruction and Accountability: An Application of Validity Theory to Assessing Scientific Inquiry

ERIC Educational Resources Information Center

Frederiksen, John R.; White, Barbara Y.

2004-01-01

This chapter is concerned with how assessments of students' work in classrooms, although primarily intended to promote learning, can also become an important source of information for evaluating a school's effectiveness within an accountability system. The authors begin their discussion by considering issues of fairness, to schools and to their…

Applying Systems Design and Item Response Theory to the Problem of Measuring Information Literacy Skills.

ERIC Educational Resources Information Center

O'Connor, Lisa G.; Radcliff, Carolyn J.; Gedeon, Julie A.

2002-01-01

Reports on the development of the Standardized Assessment of Information Literacy Skills (SAILS) at Kent State University (Ohio) for programmatic-level assessment of information literacy skills. Once validated, the instrument will be used to assess entry skills upon admission and longitudinally to ascertain whether there is significant change in…

M-Readiness Assessment Model Development and Validation: Investigation of Readiness Index and Factors Affecting Readiness

ERIC Educational Resources Information Center

Bakhsh, Muhammad; Mahmood, Amjad; Sangi, Nazir Ahmed

2018-01-01

It is important for distance learning institutions to be well prepared before designing and implementing any new technology based learning system to justify the investment and minimize failure risk. It can be achieved by systematically assessing the readiness of all stakeholders. This paper first proposes an m-readiness assessment process and…

Bispectral Index Monitoring: validity and utility in pediatric dentistry.

PubMed

Goyal, Ashima; Mittal, Neeti; Mittal, Parteek; Gauba, K

2014-01-01

Reliable and safe provision of sedation and general anesthesia is dependent on continuous vigilance of patient's sedation depth. Failure to do so may result in unintended oversedation or undersedation. It is a common practice to observe sedation depth by applying subjective sedation scales and in case of general anesthesia, practitioner is dependent on vital sign assessment. The Bispectral Index System (BIS) is a recently introduced objective, quantitative, easy to use, and free from observer bias, and clinically useful tool to assess sedation depth and it precludes the need to stimulate the patient to assess his sedation level. The present article is an attempt to orient the readers towards utility and validity of BIS for sedation and general anesthesia in pediatric dentistry. In this article, we attempt to make the readers understand the principle of BIS, its variation across sedation continuum, its validity across different age groups and for a variety of sedative drugs.

Long Island Sound Coastal Observatory: Assessment of Above-Water Radiometric Measurement Uncertainties Using Collocated Multi and Hyper-Spectral Systems

DTIC Science & Technology

2011-10-14

Chi]. These as- sumptions are usually not valid in coastal waters. This can create significant errors in BRDF estima- tions in coastal zones [38,39...collection of information if it does not display a currently valid OMB control number. PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ORGANIZATION. 1. REPORT...platform (LISCO) near Northport, New York, has been recently established to support validation of ocean color radiometry (OCR) satellite data. LISCO

Radioactive waste isolation in salt: special advisory report on the status of the Office of Nuclear Waste Isolation's plans for repository performance assessment

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ditmars, J.D.; Walbridge, E.W.; Rote, D.M.

1983-10-01

Repository performance assessment is analysis that identifies events and processes that might affect a repository system for isolation of radioactive waste, examines their effects on barriers to waste migration, and estimates the probabilities of their occurrence and their consequences. In 1983 Battelle Memorial Institute's Office of Nuclear Waste Isolation (ONWI) prepared two plans - one for performance assessment for a waste repository in salt and one for verification and validation of performance assessment technology. At the request of the US Department of Energy's Salt Repository Project Office (SRPO), Argonne National Laboratory reviewed those plans and prepared this report to advisemore » SRPO of specific areas where ONWI's plans for performance assessment might be improved. This report presents a framework for repository performance assessment that clearly identifies the relationships among the disposal problems, the processes underlying the problems, the tools for assessment (computer codes), and the data. In particular, the relationships among important processes and 26 model codes available to ONWI are indicated. A common suggestion for computer code verification and validation is the need for specific and unambiguous documentation of the results of performance assessment activities. A major portion of this report consists of status summaries of 27 model codes indicated as potentially useful by ONWI. The code summaries focus on three main areas: (1) the code's purpose, capabilities, and limitations; (2) status of the elements of documentation and review essential for code verification and validation; and (3) proposed application of the code for performance assessment of salt repository systems. 15 references, 6 figures, 4 tables.« less

Development and Calibration of an Item Bank for PE Metrics Assessments: Standard 1

ERIC Educational Resources Information Center

Zhu, Weimo; Fox, Connie; Park, Youngsik; Fisette, Jennifer L.; Dyson, Ben; Graber, Kim C.; Avery, Marybell; Franck, Marian; Placek, Judith H.; Rink, Judy; Raynes, De

2011-01-01

The purpose of this study was to develop and calibrate an assessment system, or bank, using the latest measurement theories and methods to promote valid and reliable student assessment in physical education. Using an anchor-test equating design, a total of 30 items or assessments were administered to 5,021 (2,568 boys and 2,453 girls) students in…

Assessing the validity of discourse analysis: transdisciplinary convergence

NASA Astrophysics Data System (ADS)

Jaipal-Jamani, Kamini

2014-12-01

Research studies using discourse analysis approaches make claims about phenomena or issues based on interpretation of written or spoken text, which includes images and gestures. How are findings/interpretations from discourse analysis validated? This paper proposes transdisciplinary convergence as a way to validate discourse analysis approaches to research. The argument is made that discourse analysis explicitly grounded in semiotics, systemic functional linguistics, and critical theory, offers a credible research methodology. The underlying assumptions, constructs, and techniques of analysis of these three theoretical disciplines can be drawn on to show convergence of data at multiple levels, validating interpretations from text analysis.

Baseline Assessment and Prioritization Framework for IVHM Integrity Assurance Enabling Capabilities

NASA Technical Reports Server (NTRS)

Cooper, Eric G.; DiVito, Benedetto L.; Jacklin, Stephen A.; Miner, Paul S.

2009-01-01

Fundamental to vehicle health management is the deployment of systems incorporating advanced technologies for predicting and detecting anomalous conditions in highly complex and integrated environments. Integrated structural integrity health monitoring, statistical algorithms for detection, estimation, prediction, and fusion, and diagnosis supporting adaptive control are examples of advanced technologies that present considerable verification and validation challenges. These systems necessitate interactions between physical and software-based systems that are highly networked with sensing and actuation subsystems, and incorporate technologies that are, in many respects, different from those employed in civil aviation today. A formidable barrier to deploying these advanced technologies in civil aviation is the lack of enabling verification and validation tools, methods, and technologies. The development of new verification and validation capabilities will not only enable the fielding of advanced vehicle health management systems, but will also provide new assurance capabilities for verification and validation of current generation aviation software which has been implicated in anomalous in-flight behavior. This paper describes the research focused on enabling capabilities for verification and validation underway within NASA s Integrated Vehicle Health Management project, discusses the state of the art of these capabilities, and includes a framework for prioritizing activities.

Developing and validating an instrument for measuring mobile computing self-efficacy.

PubMed

Wang, Yi-Shun; Wang, Hsiu-Yuan

2008-08-01

IT-related self-efficacy has been found to have a critical influence on system use. However, traditional measures of computer self-efficacy and Internet-related self-efficacy are perceived to be inapplicable in the context of mobile computing and commerce because they are targeted primarily at either desktop computer or wire-based technology contexts. Based on previous research, this study develops and validates a multidimensional instrument for measuring mobile computing self-efficacy (MCSE). This empirically validated instrument will be useful to researchers in developing and testing the theories of mobile user behavior, and to practitioners in assessing the mobile computing self-efficacy of users and promoting the use of mobile commerce systems.

Topological characterization versus synchronization for assessing (or not) dynamical equivalence

NASA Astrophysics Data System (ADS)

Letellier, Christophe; Mangiarotti, Sylvain; Sendiña-Nadal, Irene; Rössler, Otto E.

2018-04-01

Model validation from experimental data is an important and not trivial topic which is too often reduced to a simple visual inspection of the state portrait spanned by the variables of the system. Synchronization was suggested as a possible technique for model validation. By means of a topological analysis, we revisited this concept with the help of an abstract chemical reaction system and data from two electrodissolution experiments conducted by Jack Hudson's group. The fact that it was possible to synchronize topologically different global models led us to conclude that synchronization is not a recommendable technique for model validation. A short historical preamble evokes Jack Hudson's early career in interaction with Otto E. Rössler.

Face, content, and construct validity of human placenta as a haptic training tool in neurointerventional surgery.

PubMed

Ribeiro de Oliveira, Marcelo Magaldi; Nicolato, Arthur; Santos, Marcilea; Godinho, Joao Victor; Brito, Rafael; Alvarenga, Alexandre; Martins, Ana Luiza Valle; Prosdocimi, André; Trivelato, Felipe Padovani; Sabbagh, Abdulrahman J; Reis, Augusto Barbosa; Maestro, Rolando Del

2016-05-01

OBJECT The development of neurointerventional treatments of central nervous system disorders has resulted in the need for adequate training environments for novice interventionalists. Virtual simulators offer anatomical definition but lack adequate tactile feedback. Animal models, which provide more lifelike training, require an appropriate infrastructure base. The authors describe a training model for neurointerventional procedures using the human placenta (HP), which affords haptic training with significantly fewer resource requirements, and discuss its validation. METHODS Twelve HPs were prepared for simulated endovascular procedures. Training exercises performed by interventional neuroradiologists and novice fellows were placental angiography, stent placement, aneurysm coiling, and intravascular liquid embolic agent injection. RESULTS The endovascular training exercises proposed can be easily reproduced in the HP. Face, content, and construct validity were assessed by 6 neurointerventional radiologists and 6 novice fellows in interventional radiology. CONCLUSIONS The use of HP provides an inexpensive training model for the training of neurointerventionalists. Preliminary validation results show that this simulation model has face and content validity and has demonstrated construct validity for the interventions assessed in this study.

Multi-institutional validation of a web-based core competency assessment system.

PubMed

Tabuenca, Arnold; Welling, Richard; Sachdeva, Ajit K; Blair, Patrice G; Horvath, Karen; Tarpley, John; Savino, John A; Gray, Richard; Gulley, Julie; Arnold, Teresa; Wolfe, Kevin; Risucci, Donald A

2007-01-01

The Association of Program Directors in Surgery and the Division of Education of the American College of Surgeons developed and implemented a web-based system for end-of-rotation faculty assessment of ACGME core competencies of residents. This study assesses its reliability and validity across multiple programs. Each assessment included ratings (1-5 scale) on 23 items reflecting the 6 core competencies. A total of 4241 end-of-rotation assessments were completed for 332 general surgery residents (> or =5 evaluations each) at 5 sites during the 2004-2005 and 2005-2006 academic years. The mean rating for each resident on each item was computed for each academic year. The mean rating of items representing each competency was computed for each resident. Additional data included USMLE and ABSITE scores, PGY, and status in program (categorical, designated preliminary, and undesignated preliminary). Coefficient alpha was greater than 0.90 for each competency score. Mean ratings for each competency increased significantly (p < 0.01) as a function of PGY. Mean ratings for professionalism and interpersonal/communication skills (IPC) were significantly higher than all other competencies at all PGY levels. Competency ratings of PGY 1 residents correlated significantly with USMLE Step I, ranging from (r = 0.26, p < 0.01) for Professionalism to (r = 0.41, p < 0.001) for Systems-Based Practice. Ratings of Knowledge (r = 0.31, p < 0.01), Practice-Based Learning & Improvement (PBLI; r = 0.22, p < 0.05), and Systems-Based Practice (r = 0.20, p < 0.05) correlated significantly with 2005 ABSITE Total Percentile. Ratings of all competencies correlated significantly with the 2006 ABSITE Total Percentile Score (range: r = 0.20, p < 0.05 for professionalism to r = 0.35, p < 0.001 for knowledge). Categorical and designated preliminary residents received significantly higher ratings (p < 0.05) than nondesignated preliminaries for knowledge, patient care, PBLI, and systems-based practice only. Faculty ratings of core competencies are internally consistent. The pattern of statistically significant correlations between competency ratings and USMLE and ABSITE scores supports the postdictive and concurrent validity, respectively, of faculty perceptions of resident knowledge. The pattern of increased ratings as a function of PGY supports the construct validity of faculty ratings of resident core competencies.

Development and preliminary evidence for the validity of an instrument assessing implementation of human-factors principles in medication-related decision-support systems—I-MeDeSA

PubMed Central

Zachariah, Marianne; Seidling, Hanna M; Neri, Pamela M; Cresswell, Kathrin M; Duke, Jon; Bloomrosen, Meryl; Volk, Lynn A; Bates, David W

2011-01-01

Background Medication-related decision support can reduce the frequency of preventable adverse drug events. However, the design of current medication alerts often results in alert fatigue and high over-ride rates, thus reducing any potential benefits. Methods The authors previously reviewed human-factors principles for relevance to medication-related decision support alerts. In this study, instrument items were developed for assessing the appropriate implementation of these human-factors principles in drug–drug interaction (DDI) alerts. User feedback regarding nine electronic medical records was considered during the development process. Content validity, construct validity through correlation analysis, and inter-rater reliability were assessed. Results The final version of the instrument included 26 items associated with nine human-factors principles. Content validation on three systems resulted in the addition of one principle (Corrective Actions) to the instrument and the elimination of eight items. Additionally, the wording of eight items was altered. Correlation analysis suggests a direct relationship between system age and performance of DDI alerts (p=0.0016). Inter-rater reliability indicated substantial agreement between raters (κ=0.764). Conclusion The authors developed and gathered preliminary evidence for the validity of an instrument that measures the appropriate use of human-factors principles in the design and display of DDI alerts. Designers of DDI alerts may use the instrument to improve usability and increase user acceptance of medication alerts, and organizations selecting an electronic medical record may find the instrument helpful in meeting their clinicians' usability needs. PMID:21946241

Components of Standing Postural Control Evaluated in Pediatric Balance Measures: A Scoping Review.

PubMed

Sibley, Kathryn M; Beauchamp, Marla K; Van Ooteghem, Karen; Paterson, Marie; Wittmeier, Kristy D

2017-10-01

To identify measures of standing balance validated in pediatric populations, and to determine the components of postural control captured in each tool. Electronic searches of MEDLINE, Embase, and CINAHL databases using key word combinations of postural balance/equilibrium, psychometrics/reproducibility of results/predictive value of tests, and child/pediatrics; gray literature; and hand searches. Inclusion criteria were measures with a stated objective to assess balance, with pediatric (≤18y) populations, with at least 1 psychometric evaluation, with at least 1 standing task, with a standardized protocol and evaluation criteria, and published in English. Two reviewers independently identified studies for inclusion. There were 21 measures included. Two reviewers extracted descriptive characteristics, and 2 investigators independently coded components of balance in each measure using a systems perspective for postural control, an established framework for balance in pediatric populations. Components of balance evaluated in measures were underlying motor systems (100% of measures), anticipatory postural control (72%), static stability (62%), sensory integration (52%), dynamic stability (48%), functional stability limits (24%), cognitive influences (24%), verticality (9%), and reactive postural control (0%). Assessing children's balance with valid and comprehensive measures is important for ensuring development of safe mobility and independence with functional tasks. Balance measures validated in pediatric populations to date do not comprehensively assess standing postural control and omit some key components for safe mobility and independence. Existing balance measures, that have been validated in adult populations and address some of the existing gaps in pediatric measures, warrant consideration for validation in children. Copyright © 2017 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.

The Chelsea critical care physical assessment tool (CPAx): validation of an innovative new tool to measure physical morbidity in the general adult critical care population; an observational proof-of-concept pilot study.

PubMed

Corner, E J; Wood, H; Englebretsen, C; Thomas, A; Grant, R L; Nikoletou, D; Soni, N

2013-03-01

To develop a scoring system to measure physical morbidity in critical care - the Chelsea Critical Care Physical Assessment Tool (CPAx). The development process was iterative involving content validity indices (CVI), a focus group and an observational study of 33 patients to test construct validity against the Medical Research Council score for muscle strength, peak cough flow, Australian Therapy Outcome Measures score, Glasgow Coma Scale score, Bloomsbury sedation score, Sequential Organ Failure Assessment score, Short Form 36 (SF-36) score, days of mechanical ventilation and inter-rater reliability. Trauma and general critical care patients from two London teaching hospitals. Users of the CPAx felt that it possessed content validity, giving a final CVI of 1.00 (P<0.05). Construct validation data showed moderate to strong significant correlations between the CPAx score and all secondary measures, apart from the mental component of the SF-36 which demonstrated weak correlation with the CPAx score (r=0.024, P=0.720). Reliability testing showed internal consistency of α=0.798 and inter-rater reliability of κ=0.988 (95% confidence interval 0.791 to 1.000) between five raters. This pilot work supports proof of concept of the CPAx as a measure of physical morbidity in the critical care population, and is a cogent argument for further investigation of the scoring system. Copyright © 2012 Chartered Society of Physiotherapy. Published by Elsevier Ltd. All rights reserved.

Development, Validation, and Verification of a Self-Assessment Tool to Estimate Agnibala (Digestive Strength).

PubMed

Singh, Aparna; Singh, Girish; Patwardhan, Kishor; Gehlot, Sangeeta

2017-01-01

According to Ayurveda, the traditional system of healthcare of Indian origin, Agni is the factor responsible for digestion and metabolism. Four functional states (Agnibala) of Agni have been recognized: regular, irregular, intense, and weak. The objective of the present study was to develop and validate a self-assessment tool to estimate Agnibala The developed tool was evaluated for its reliability and validity by administering it to 300 healthy volunteers of either gender belonging to 18 to 40-year age group. Besides confirming the statistical validity and reliability, the practical utility of the newly developed tool was also evaluated by recording serum lipid parameters of all the volunteers. The results show that the lipid parameters vary significantly according to the status of Agni The tool, therefore, may be used to screen normal population to look for possible susceptibility to certain health conditions. © The Author(s) 2016.

Comparing health system performance assessment and management approaches in the Netherlands and Ontario, Canada

PubMed Central

Tawfik-Shukor, Ali R; Klazinga, Niek S; Arah, Onyebuchi A

2007-01-01

Background Given the proliferation and the growing complexity of performance measurement initiatives in many health systems, the Netherlands and Ontario, Canada expressed interests in cross-national comparisons in an effort to promote knowledge transfer and best practise. To support this cross-national learning, a study was undertaken to compare health system performance approaches in The Netherlands with Ontario, Canada. Methods We explored the performance assessment framework and system of each constituency, the embeddedness of performance data in management and policy processes, and the interrelationships between the frameworks. Methods used included analysing governmental strategic planning and policy documents, literature and internet searches, comparative descriptive tables, and schematics. Data collection and analysis took place in Ontario and The Netherlands. A workshop to validate and discuss the findings was conducted in Toronto, adding important insights to the study. Results Both Ontario and The Netherlands conceive health system performance within supportive frameworks. However they differ in their assessment approaches. Ontario's Scorecard links performance measurement with strategy, aimed at health system integration. The Dutch Health Care Performance Report (Zorgbalans) does not explicitly link performance with strategy, and focuses on the technical quality of healthcare by measuring dimensions of quality, access, and cost against healthcare needs. A backbone 'five diamond' framework maps both frameworks and articulates the interrelations and overlap between their goals, themes, dimensions and indicators. The workshop yielded more contextual insights and further validated the comparative values of each constituency's performance assessment system. Conclusion To compare the health system performance approaches between The Netherlands and Ontario, Canada, several important conceptual and contextual issues must be addressed, before even attempting any future content comparisons and benchmarking. Such issues would lend relevant interpretational credibility to international comparative assessments of the two health systems. PMID:17319947

Revising the Rorschach Ego Impairment Index to Accommodate Recent Recommendations about Improving Rorschach Validity

ERIC Educational Resources Information Center

Viglione, Donald J.; Perry, William; Giromini, Luciano; Meyer, Gregory J.

2011-01-01

We used multiple regression to calculate a new Ego Impairment Index (EII-3). The aim was to incorporate changes in the component variables and distribution of the number of responses as found in the new Rorschach Performance Assessment System, while sustaining the validity and reliability of previous EIIs. The EII-3 formula was derived from a…

Concurrent validation of an inertial measurement system to quantify kicking biomechanics in four football codes.

PubMed

Blair, Stephanie; Duthie, Grant; Robertson, Sam; Hopkins, William; Ball, Kevin

2018-05-17

Wearable inertial measurement systems (IMS) allow for three-dimensional analysis of human movements in a sport-specific setting. This study examined the concurrent validity of a IMS (Xsens MVN system) for measuring lower extremity and pelvis kinematics in comparison to a Vicon motion analysis system (MAS) during kicking. Thirty footballers from Australian football (n = 10), soccer (n = 10), rugby league and rugby union (n = 10) clubs completed 20 kicks across four conditions. Concurrent validity was assessed using a linear mixed-modelling approach, which allowed the partition of between and within-subject variance from the device measurement error. Results were expressed in raw and standardised units for assessments of differences in means and measurement error, and interpreted via non-clinical magnitude-based inferences. Trivial to small differences were found in linear velocities (foot and pelvis), angular velocities (knee, shank and thigh), sagittal joint (knee and hip) and segment angle (shank and pelvis) means (mean difference: 0.2-5.8%) between the IMS and MAS in Australian football, soccer and the rugby codes. Trivial to small measurement errors (from 0.1 to 5.8%) were found between the IMS and MAS in all kinematic parameters. The IMS demonstrated acceptable levels of concurrent validity compared to a MAS when measuring kicking biomechanics across the four football codes. Wearable IMS offers various benefits over MAS, such as, out-of-laboratory testing, larger measurement range and quick data output, to help improve the ecological validity of biomechanical testing and the timing of feedback. The results advocate the use of IMS to quantify biomechanics of high-velocity movements in sport-specific settings. Copyright © 2018 Elsevier Ltd. All rights reserved.

The validity and reliability of Systemic Lupus Erythematosus Quality of Life Questionnaire (L-QoL) in a Turkish population.

PubMed

Duruöz, M T; Unal, C; Toprak, C Sanal; Sezer, I; Yilmaz, F; Ulutatar, F; Atagündüz, P; Baklacioglu, H S

2017-12-01

Background Systemic lupus erythematosus (SLE) may have a profound impact on quality of life. There is increasing interest in measuring quality of life in lupus patients. The purpose of this study was to investigate the validity and reliability of SLE Quality of Life Questionnaire (L-QoL) in Turkish SLE patients. Methods SLE according to 2012 Systemic Lupus International Collaborating Clinics Classification Criteria were recruited into the study. Demographic data, clinical parameters and disease activity measured with the Systemic Lupus Erythematosus Disease Activity Index-2000 (SLEDAI-2K); were noted. Nottingham Health Profile and Health Assessment Questionnaire were filled out in addition to the Turkish L-QoL (LQoL-TR). Internal consistency, test-retest reliability, and convergent and discriminant validity were evaluated. Results The mean age of participants was 43.55 ± 14.33 years and the mean disease duration was 89.8 ± 92.1 months. The patients filled out LQoL-TR in 2.5 min. Strong correlation of LQoL-TR with all subgroups of the Nottingham Health Profile and the Health Assessment Questionnaire were established showing the convergent validity. The highest correlation was demonstrated with emotional reactions (rho = 0.72) and sleep component (rho = 0.65) of the Nottingham Health Profile scale ( p < 0.0001). Its poor and not significant correlation with nonfunctional parameters (age, disease duration, perceived general health, SLEDAI-2K) showed its discriminative properties. LQoL-TR demonstrated good internal reliability with a Cronbach's α of 0.93 and test-retest reliability with intraclass correlation coefficient of 0.87. Conclusion The LQoL-TR is a practical and useful tool which demonstrates good validity and reliability.

Reliability of the Balance Evaluation Systems Test (BESTest) and BESTest sections for adults with hemiparesis

PubMed Central

Rodrigues, Letícia C.; Marques, Aline P.; Barros, Paula B.; Michaelsen, Stella M.

2014-01-01

BACKGROUND: The Balance Evaluation Systems Test (BESTest) was recently created to allow the development of treatments according to the specific balance system affected in each patient. The Brazilian version of the BESTest has not been specifically tested after stroke. OBJECTIVE: To evaluate the intra- and inter-rater reliability and concurrent and convergent validity of the total score of the BESTest and BESTest sections for adults with hemiparesis after stroke. METHOD: The study included 16 subjects (61.1±7.5 years) with chronic hemiparesis (54.5±43.5 months after stroke). The BESTest was administered by two raters in the same week and one of the raters repeated the test after a one-week interval. Intraclass correlation coefficient (ICC) was calculated to assess intra- and interrater reliability. Concurrent validity with the Berg Balance Scale (BBS) and convergent validity with the Activities-specific Balance Confidence scale (ABC-Brazil) were assessed using Pearson's correlation coefficient. RESULTS: Both the BESTest total score (ICC=0.98) and the BESTest sections (ICC between 0.85 and 0.96) have excellent intrarater reliability. Interrater reliability for the total score was excellent (ICC=0.93) and, for the sections, it ranged between 0.71 and 0.94. The correlation coefficient between the BESTest and the BBS and ABC-Brazil were 0.78 and 0.59, respectively. CONCLUSIONS: The Brazilian version of the BESTest demonstrated adequate reliability when measured by sections and could identify what balance system was affected in patients after stroke. Concurrent validity was excellent with the BBS total score and good to excellent with the sections. The total scores but not the sections present adequate convergent validity with the ABC-Brazil. However, other psychometric properties should be further investigated. PMID:25003281

Assessing validity of observational intervention studies – the Benchmarking Controlled Trials

PubMed Central

Malmivaara, Antti

2016-01-01

Abstract Background: Benchmarking Controlled Trial (BCT) is a concept which covers all observational studies aiming to assess impact of interventions or health care system features to patients and populations. Aims: To create and pilot test a checklist for appraising methodological validity of a BCT. Methods: The checklist was created by extracting the most essential elements from the comprehensive set of criteria in the previous paper on BCTs. Also checklists and scientific papers on observational studies and respective systematic reviews were utilized. Ten BCTs published in the Lancet and in the New England Journal of Medicine were used to assess feasibility of the created checklist. Results: The appraised studies seem to have several methodological limitations, some of which could be avoided in planning, conducting and reporting phases of the studies. Conclusions: The checklist can be used for planning, conducting, reporting, reviewing, and critical reading of observational intervention studies. However, the piloted checklist should be validated in further studies.Key messagesBenchmarking Controlled Trial (BCT) is a concept which covers all observational studies aiming to assess impact of interventions or health care system features to patients and populations.This paper presents a checklist for appraising methodological validity of BCTs and pilot-tests the checklist with ten BCTs published in leading medical journals. The appraised studies seem to have several methodological limitations, some of which could be avoided in planning, conducting and reporting phases of the studies.The checklist can be used for planning, conducting, reporting, reviewing, and critical reading of observational intervention studies. PMID:27238631

OneDep: Unified wwPDB System for Deposition, Biocuration, and Validation of Macromolecular Structures in the PDB Archive

PubMed Central

Young, Jasmine Y.; Westbrook, John D.; Feng, Zukang; Sala, Raul; Peisach, Ezra; Oldfield, Thomas J.; Sen, Sanchayita; Gutmanas, Aleksandras; Armstrong, David R.; Berrisford, John M.; Chen, Li; Chen, Minyu; Di Costanzo, Luigi; Dimitropoulos, Dimitris; Gao, Guanghua; Ghosh, Sutapa; Gore, Swanand; Guranovic, Vladimir; Hendrickx, Pieter MS; Hudson, Brian P.; Igarashi, Reiko; Ikegawa, Yasuyo; Kobayashi, Naohiro; Lawson, Catherine L.; Liang, Yuhe; Mading, Steve; Mak, Lora; Mir, M. Saqib; Mukhopadhyay, Abhik; Patwardhan, Ardan; Persikova, Irina; Rinaldi, Luana; Sanz-Garcia, Eduardo; Sekharan, Monica R.; Shao, Chenghua; Swaminathan, G. Jawahar; Tan, Lihua; Ulrich, Eldon L.; van Ginkel, Glen; Yamashita, Reiko; Yang, Huanwang; Zhuravleva, Marina A.; Quesada, Martha; Kleywegt, Gerard J.; Berman, Helen M.; Markley, John L.; Nakamura, Haruki; Velankar, Sameer; Burley, Stephen K.

2017-01-01

SUMMARY OneDep, a unified system for deposition, biocuration, and validation of experimentally determined structures of biological macromolecules to the Protein Data Bank (PDB) archive, has been developed as a global collaboration by the Worldwide Protein Data Bank (wwPDB) partners. This new system was designed to ensure that the wwPDB could meet the evolving archiving requirements of the scientific community over the coming decades. OneDep unifies deposition, biocuration, and validation pipelines across all wwPDB, EMDB, and BMRB deposition sites with improved focus on data quality and completeness in these archives, while supporting growth in the number of depositions and increases in their average size and complexity. In this paper, we describe the design, functional operation, and supporting infrastructure of the OneDep system, and provide initial performance assessments. PMID:28190782

Validity of the Child Facial Coding System for the Assessment of Acute Pain in Children With Cerebral Palsy.

PubMed

Hadden, Kellie L; LeFort, Sandra; O'Brien, Michelle; Coyte, Peter C; Guerriere, Denise N

2016-04-01

The purpose of the current study was to examine the concurrent and discriminant validity of the Child Facial Coding System for children with cerebral palsy. Eighty-five children (mean = 8.35 years, SD = 4.72 years) were videotaped during a passive joint stretch with their physiotherapist and during 3 time segments: baseline, passive joint stretch, and recovery. Children's pain responses were rated from videotape using the Numerical Rating Scale and Child Facial Coding System. Results indicated that Child Facial Coding System scores during the passive joint stretch significantly correlated with Numerical Rating Scale scores (r = .72, P < .01). Child Facial Coding System scores were also significantly higher during the passive joint stretch than the baseline and recovery segments (P < .001). Facial activity was not significantly correlated with the developmental measures. These findings suggest that the Child Facial Coding System is a valid method of identifying pain in children with cerebral palsy. © The Author(s) 2015.

Polyp morphology: an interobserver evaluation for the Paris classification among international experts.

PubMed

van Doorn, Sascha C; Hazewinkel, Y; East, James E; van Leerdam, Monique E; Rastogi, Amit; Pellisé, Maria; Sanduleanu-Dascalescu, Silvia; Bastiaansen, Barbara A J; Fockens, Paul; Dekker, Evelien

2015-01-01

The Paris classification is an international classification system for describing polyp morphology. Thus far, the validity and reproducibility of this classification have not been assessed. We aimed to determine the interobserver agreement for the Paris classification among seven Western expert endoscopists. A total of 85 short endoscopic video clips depicting polyps were created and assessed by seven expert endoscopists according to the Paris classification. After a digital training module, the same 85 polyps were assessed again. We calculated the interobserver agreement with a Fleiss kappa and as the proportion of pairwise agreement. The interobserver agreement of the Paris classification among seven experts was moderate with a Fleiss kappa of 0.42 and a mean pairwise agreement of 67%. The proportion of lesions assessed as "flat" by the experts ranged between 13 and 40% (P<0.001). After the digital training, the interobserver agreement did not change (kappa 0.38, pairwise agreement 60%). Our study is the first to validate the Paris classification for polyp morphology. We demonstrated only a moderate interobserver agreement among international Western experts for this classification system. Our data suggest that, in its current version, the use of this classification system in daily practice is questionable and it is unsuitable for comparative endoscopic research. We therefore suggest introduction of a simplification of the classification system.

Image Quality Modeling and Characterization of Nyquist Sampled Framing Systems with Operational Considerations for Remote Sensing

NASA Astrophysics Data System (ADS)

Garma, Rey Jan D.

The trade between detector and optics performance is often conveyed through the Q metric, which is defined as the ratio of detector sampling frequency and optical cutoff frequency. Historically sensors have operated at Q ≈ 1, which introduces aliasing but increases the system modulation transfer function (MTF) and signal-to-noise ratio (SNR). Though mathematically suboptimal, such designs have been operationally ideal when considering system parameters such as pointing stability and detector performance. Substantial advances in read noise and quantum efficiency of modern detectors may compensate for the negative aspects associated with balancing detector/optics performance, presenting an opportunity to revisit the potential for implementing Nyquist-sampled (Q ≈ 2) sensors. A digital image chain simulation is developed and validated against a laboratory testbed using objective and subjective assessments. Objective assessments are accomplished by comparison of the modeled MTF and measurements from slant-edge photographs. Subjective assessments are carried out by performing a psychophysical study where subjects are asked to rate simulation and testbed imagery against a DeltaNIIRS scale with the aid of a marker set. Using the validated model, additional test cases are simulated to study the effects of increased detector sampling on image quality with operational considerations. First, a factorial experiment using Q-sampling, pointing stability, integration time, and detector performance is conducted to measure the main effects and interactions of each on the response variable, DeltaNIIRS. To assess the fidelity of current models, variants of the General Image Quality Equation (GIQE) are evaluated against subject-provided ratings and two modified GIQE versions are proposed. Finally, using the validated simulation and modified IQE, trades are conducted to ascertain the feasibility of implementing Q ≈ 2 designs in future systems.

Development of the Systems Thinking Scale for Adolescent Behavior Change.

PubMed

Moore, Shirley M; Komton, Vilailert; Adegbite-Adeniyi, Clara; Dolansky, Mary A; Hardin, Heather K; Borawski, Elaine A

2018-03-01

This report describes the development and psychometric testing of the Systems Thinking Scale for Adolescent Behavior Change (STS-AB). Following item development, initial assessments of understandability and stability of the STS-AB were conducted in a sample of nine adolescents enrolled in a weight management program. Exploratory factor analysis of the 16-item STS-AB and internal consistency assessments were then done with 359 adolescents enrolled in a weight management program. Test-retest reliability of the STS-AB was .71, p = .03; internal consistency reliability was .87. Factor analysis of the 16-item STS-AB indicated a one-factor solution with good factor loadings, ranging from .40 to .67. Evidence of construct validity was supported by significant correlations with established measures of variables associated with health behavior change. We provide beginning evidence of the reliability and validity of the STS-AB to measure systems thinking for health behavior change in young adolescents.

Development of the Systems Thinking Scale for Adolescent Behavior Change

PubMed Central

Moore, Shirley M.; Komton, Vilailert; Adegbite-Adeniyi, Clara; Dolansky, Mary A.; Hardin, Heather K.; Borawski, Elaine A.

2017-01-01

This report describes the development and psychometric testing of the Systems Thinking Scale for Adolescent Behavior Change (STS-AB). Following item development, initial assessments of understandability and stability of the STS-AB were conducted in a sample of nine adolescents enrolled in a weight management program. Exploratory factor analysis of the 16-item STS-AB and internal consistency assessments were then done with 359 adolescents enrolled in a weight management program. Test–retest reliability of the STS-AB was .71, p = .03; internal consistency reliability was .87. Factor analysis of the 16-item STS-AB indicated a one-factor solution with good factor loadings, ranging from .40 to .67. Evidence of construct validity was supported by significant correlations with established measures of variables associated with health behavior change. We provide beginning evidence of the reliability and validity of the STS-AB to measure systems thinking for health behavior change in young adolescents. PMID:28303755

Rater reliability and concurrent validity of the Keyboard Personal Computer Style instrument (K-PeCS).

PubMed

Baker, Nancy A; Cook, James R; Redfern, Mark S

2009-01-01

This paper describes the inter-rater and intra-rater reliability, and the concurrent validity of an observational instrument, the Keyboard Personal Computer Style instrument (K-PeCS), which assesses stereotypical postures and movements associated with computer keyboard use. Three trained raters independently rated the video clips of 45 computer keyboard users to ascertain inter-rater reliability, and then re-rated a sub-sample of 15 video clips to ascertain intra-rater reliability. Concurrent validity was assessed by comparing the ratings obtained using the K-PeCS to scores developed from a 3D motion analysis system. The overall K-PeCS had excellent reliability [inter-rater: intra-class correlation coefficients (ICC)=.90; intra-rater: ICC=.92]. Most individual items on the K-PeCS had from good to excellent reliability, although six items fell below ICC=.75. Those K-PeCS items that were assessed for concurrent validity compared favorably to the motion analysis data for all but two items. These results suggest that most items on the K-PeCS can be used to reliably document computer keyboarding style.

Establishing best practices for the validation of atmospheric composition measurements from satellites

NASA Astrophysics Data System (ADS)

Lambert, Jean-Christopher

As a contribution to the implementation of the Global Earth Observation System of Systems (GEOSS), the Committee on Earth Observation Satellites (CEOS) is developing a data quality strategy for satellite measurements. To achieve GEOSS requirements of consistency and interoperability (e.g. for comparison and for integrated interpretation) of the measurements and their derived data products, proper uncertainty assessment is essential and needs to be continuously monitored and traceable to standards. Therefore, CEOS has undertaken the task to establish a set of best practices and guidelines for satellite validation, starting with current practices that could be improved with time. Best practices are not intended to be imposed as firm requirements, but rather to be suggested as a baseline for comparing against, which could be used by the widest community and provide guidance to newcomers. The present paper reviews the current development of best practices and guidelines for the validation of atmospheric composition satellites. Terminologies and general principles of validation are reminded. Going beyond elementary definitions of validation like the assessment of uncertainties, the specific GEOSS context calls also for validation of individual service components and against user requirements. This paper insists on two important aspects. First one, the question of the "collocation". Validation generally involves comparisons with "reference" measurements of the same quantities, and the question of what constitutes a valid comparison is not the least of the challenges faced. We present a tentative scheme for defining the validity of a comparison and of the necessary "collocation" criteria. Second focus of this paper: the information content of the data product. Validation against user requirements, or the verification of the "fitness for purpose" of both the data products and their validation, needs to identify what information, in the final product, is contributed really by the measurement, as opposed to what is contributed by a priori constraints imposed by the retrieval.

The Serbian version of the Juvenile Arthritis Multidimensional Assessment Report (JAMAR).

PubMed

Susic, Gordana; Vojinovic, Jelena; Vijatov-Djuric, Gordana; Stevanovic, Dejan; Lazarevic, Dragana; Djurovic, Nada; Novakovic, Dusica; Consolaro, Alessandro; Bovis, Francesca; Ruperto, Nicolino

2018-04-01

The Juvenile Arthritis Multidimensional Assessment Report (JAMAR) is a new parent/patient-reported outcome measure that enables a thorough assessment of the disease status in children with juvenile idiopathic arthritis (JIA). We report the results of the cross-cultural adaptation and validation of the parent and patient versions of the JAMAR in the Serbian language. The reading comprehension of the questionnaire was tested in 10 JIA parents and patients. Each participating centre was asked to collect demographic, clinical data and the JAMAR in 100 consecutive JIA patients or all consecutive patients seen in a 6-month period and to administer the JAMAR to 100 healthy children and their parents. The statistical validation phase explored descriptive statistics and the psychometric issues of the JAMAR: the three Likert assumptions, floor/ceiling effects, internal consistency, Cronbach's alpha, interscale correlations, test-retest reliability, and construct validity (convergent and discriminant validity). A total of 248 JIA patients (5.2% systemic, 44.3% oligoarticular, 23.8% RF-negative polyarthritis, 26.7% other categories) and 100 healthy children were enrolled in three centres. The JAMAR components discriminated healthy subjects from JIA patients. All JAMAR components revealed good psychometric performances. In conclusion, the Serbian version of the JAMAR is a valid tool for the assessment of children with JIA and is suitable for use both in routine clinical practice and clinical research.

Setting performance standards for medical practice: a theoretical framework.

PubMed

Southgate, L; Hays, R B; Norcini, J; Mulholland, H; Ayers, B; Woolliscroft, J; Cusimano, M; McAvoy, P; Ainsworth, M; Haist, S; Campbell, M

2001-05-01

The assessment of performance in the real world of medical practice is now widely accepted as the goal of assessment at the postgraduate level. This is largely a validity issue, as it is recognised that tests of knowledge and in clinical simulations cannot on their own really measure how medical practitioners function in the broader health care system. However, the development of standards for performance-based assessment is not as well understood as in competency assessment, where simulations can more readily reflect narrower issues of knowledge and skills. This paper proposes a theoretical framework for the development of standards that reflect the more complex world in which experienced medical practitioners work. The paper reflects the combined experiences of a group of education researchers and the results of literature searches that included identifying current health system data sources that might contribute information to the measurement of standards. Standards that reflect the complexity of medical practice may best be developed through an "expert systems" analysis of clinical conditions for which desired health care outcomes reflect the contribution of several health professionals within a complex, three-dimensional, contextual model. Examples of the model are provided, but further work is needed to test validity and measurability.

Applying Resource Utilization Groups (RUG-III) in Hong Kong nursing homes.

PubMed

Chou, Kee-Lee; Chi, Iris; Leung, Joe C B

2008-01-01

Resource Utilization Groups III (RUG-III) is a case-mix system developed in the United States for categorization of nursing home residents and the financing of residential care services. In Hong Kong, RUG-III is based on several board groups of residents. The aim of this study was to examine the reliability and validity of the RUG-III in Hong Kong nursing homes. A cross-sectional survey was conducted in seven residential facilities operated by one agency. Residents ( N = 1,127) were assessed by the Minimum Data Set (MDS) and nursing as well as auxiliary staff care times were recorded within 2 weeks before or after the completion of MDS assessment. Forty-five out 1,127 residents were re-interviewed by an independent assessor to assess the inter-rater reliability. The inter-rater reliability of MDS assessment was excellent (kappa = 0.76) and the original RUG-III accounted for about 30 per cent of nursing staff time. Results provide preliminary evidence to support that RUG-III is a reliable and valid case-mix system for Hong Kong nursing homes, but future studies must be explored to reduce the variance of resource use explained by this case-mix system.

The German version of the Juvenile Arthritis Multidimensional Assessment Report (JAMAR).

PubMed

Holzinger, Dirk; Foell, Dirk; Horneff, Gerd; Foeldvari, Ivan; Tzaribachev, Nikolay; Tzaribachev, Catrin; Minden, Kirsten; Kallinich, Tilmann; Ganser, Gerd; Clara, Lucia; Haas, Johannes-Peter; Hügle, Boris; Huppertz, Hans-Iko; Weller, Frank; Consolaro, Alessandro; Bovis, Francesca; Ruperto, Nicolino

2018-04-01

The Juvenile Arthritis Multidimensional Assessment Report (JAMAR) is a new parent/patient reported outcome measure that enables a thorough assessment of the disease status in children with juvenile idiopathic arthritis (JIA). We report the results of the cross-cultural adaptation and validation of the parent and patient versions of the JAMAR in the German language. The reading comprehension of the questionnaire was tested in 10 JIA parents and patients. The participating centres were asked to collect demographic and clinical data along the JAMAR questionnaire in 100 consecutive JIA patients or all consecutive patients seen in a 6-month period and to administer the JAMAR to 100 healthy children and their parents. The statistical validation phase explored descriptive statistics and the psychometric issues of the JAMAR: the three Likert assumptions, floor/ceiling effects, internal consistency, Cronbach's alpha, interscale correlations, test-retest reliability, and construct validity (convergent and discriminant validity). A total of 319 JIA patients (2.8% systemic, 36.7% oligoarticular, 23.5% RF negative polyarthritis, and 37% other categories) and 100 healthy children were enrolled in eight centres. The JAMAR components discriminated well healthy subjects from JIA patients. All JAMAR components revealed good psychometric performances. In conclusion, the German version of the JAMAR is a valid tool for the assessment of children with JIA and is suitable for use both in routine clinical practice and in clinical research.

AQUATOX Model Validation Reports

EPA Pesticide Factsheets

AQUATOX has a myriad of potential applications to water management issues and programs, including water quality criteria and standards, TMDLs (Total Maximum Daily Loads), and ecological risk assessments of aquatic systems.

Reliability and validity of a novel Kinect-based software program for measuring posture, balance and side-bending.

PubMed

Grooten, Wilhelmus Johannes Andreas; Sandberg, Lisa; Ressman, John; Diamantoglou, Nicolas; Johansson, Elin; Rasmussen-Barr, Eva

2018-01-08

Clinical examinations are subjective and often show a low validity and reliability. Objective and highly reliable quantitative assessments are available in laboratory settings using 3D motion analysis, but these systems are too expensive to use for simple clinical examinations. Qinematic™ is an interactive movement analyses system based on the Kinect camera and is an easy-to-use clinical measurement system for assessing posture, balance and side-bending. The aim of the study was to test the test-retest the reliability and construct validity of Qinematic™ in a healthy population, and to calculate the minimal clinical differences for the variables of interest. A further aim was to identify the discriminative validity of Qinematic™ in people with low-back pain (LBP). We performed a test-retest reliability study (n = 37) with around 1 week between the occasions, a construct validity study (n = 30) in which Qinematic™ was tested against a 3D motion capture system, and a discriminative validity study, in which a group of people with LBP (n = 20) was compared to healthy controls (n = 17). We tested a large range of psychometric properties of 18 variables in three sections: posture (head and pelvic position, weight distribution), balance (sway area and velocity in single- and double-leg stance), and side-bending. The majority of the variables in the posture and balance sections, showed poor/fair reliability (ICC < 0.4) and poor/fair validity (Spearman <0.4), with significant differences between occasions, between Qinematic™ and the 3D-motion capture system. In the clinical study, Qinematic™ did not differ between people with LPB and healthy for these variables. For one variable, side-bending to the left, there was excellent reliability (ICC =0.898), excellent validity (r = 0.943), and Qinematic™ could differentiate between LPB and healthy individuals (p = 0.012). This paper shows that a novel software program (Qinematic™) based on the Kinect camera for measuring balance, posture and side-bending has poor psychometric properties, indicating that the variables on balance and posture should not be used for monitoring individual changes over time or in research. Future research on the dynamic tasks of Qinematic™ is warranted.

Extinguishing agent for magnesium fire, phases 5 and 6

NASA Astrophysics Data System (ADS)

Beeson, H. D.; Tapscott, R. E.; Mason, B. E.

1987-07-01

This report documents the validation testing of the extinguishing system for metal fires developed as part of Phases 1 to 4. The results of this validation testing form the basis of information from which draft military specifications necessary to procure the agent and the agent delivery system may be developed. The developed system was tested against a variety of large-scale metal fire scenarios and the capabilities of the system were assessed. In addition the response of the system to storage and to changes in ambient conditions was tested. Results of this testing revealed that the developed system represented a reliable metal fire extinguishing system that could control and extinguish very large metal fires. The specifications developed for the agent and for the delivery system are discussed in detail.

Image-Based Medical Expert Teleconsultation in Acute Care of Injuries. A Systematic Review of Effects on Information Accuracy, Diagnostic Validity, Clinical Outcome, and User Satisfaction

PubMed Central

Hasselberg, Marie; Beer, Netta; Blom, Lisa; Wallis, Lee A.; Laflamme, Lucie

2014-01-01

Objective To systematically review the literature on image-based telemedicine for medical expert consultation in acute care of injuries, considering system, user, and clinical aspects. Design Systematic review of peer-reviewed journal articles. Data sources Searches of five databases and in eligible articles, relevant reviews, and specialized peer-reviewed journals. Eligibility criteria Studies were included that covered teleconsultation systems based on image capture and transfer with the objective of seeking medical expertise for the diagnostic and treatment of acute injury care and that presented the evaluation of one or several aspects of the system based on empirical data. Studies of systems not under routine practice or including real-time interactive video conferencing were excluded. Method The procedures used in this review followed the PRISMA Statement. Predefined criteria were used for the assessment of the risk of bias. The DeLone and McLean Information System Success Model was used as a framework to synthesise the results according to system quality, user satisfaction, information quality and net benefits. All data extractions were done by at least two reviewers independently. Results Out of 331 articles, 24 were found eligible. Diagnostic validity and management outcomes were often studied; fewer studies focused on system quality and user satisfaction. Most systems were evaluated at a feasibility stage or during small-scale pilot testing. Although the results of the evaluations were generally positive, biases in the methodology of evaluation were concerning selection, performance and exclusion. Gold standards and statistical tests were not always used when assessing diagnostic validity and patient management. Conclusions Image-based telemedicine systems for injury emergency care tend to support valid diagnosis and influence patient management. The evidence relates to a few clinical fields, and has substantial methodological shortcomings. As in the case of telemedicine in general, user and system quality aspects are poorly documented, both of which affect scale up of such programs. PMID:24887257

Documentation of pharmaceutical care: Validation of an intervention oriented classification system.

PubMed

Maes, Karen A; Studer, Helene; Berger, Jérôme; Hersberger, Kurt E; Lampert, Markus L

2017-12-01

During the dispensing process, pharmacists may come across technical and clinical issues requiring a pharmaceutical intervention (PI). An intervention-oriented classification system is a helpful tool to document these PIs in a structured manner. Therefore, we developed the PharmDISC classification system (Pharmacists' Documentation of Interventions in Seamless Care). The aim of this study was to evaluate the PharmDISC system in the daily practice environment (in terms of interrater reliability, appropriateness, interpretability, acceptability, feasibility, and validity); to assess its user satisfaction, the descriptive manual, and the online training; and to explore first implementation aspects. Twenty-one pharmacists from different community pharmacies each classified 30 prescriptions requiring a PI with the PharmDISC system on 5 selected days within 5 weeks. Interrater reliability was determined using model PIs and Fleiss's kappa coefficients (κ) were calculated. User satisfaction was assessed by questionnaire with a 4-point Likert scale. The main outcome measures were interrater reliability (κ); appropriateness, interpretability, validity (ratio of completely classified PIs/all PIs); feasibility, and acceptability (user satisfaction and suggestions). The PharmDISC system reached an average substantial agreement (κ = 0.66). Of documented 519 PIs, 430 (82.9%) were completely classified. Most users found the system comprehensive (median user agreement 3 [2/3.25 quartiles]) and practical (3[2.75/3]). The PharmDISC system raised the awareness regarding drug-related problems for most users (n = 16). To facilitate its implementation, an electronic version that automatically connects to the prescription together with a task manager for PIs needing follow-up was suggested. Barriers could be time expenditure and lack of understanding the benefits. Substantial interrater reliability and acceptable user satisfaction indicate that the PharmDISC system is a valid system to document PIs in daily community pharmacy practice. © 2017 John Wiley & Sons, Ltd.

Simple Scoring System and Artificial Neural Network for Knee Osteoarthritis Risk Prediction: A Cross-Sectional Study

PubMed Central

Yoo, Tae Keun; Kim, Deok Won; Choi, Soo Beom; Oh, Ein; Park, Jee Soo

2016-01-01

Background Knee osteoarthritis (OA) is the most common joint disease of adults worldwide. Since the treatments for advanced radiographic knee OA are limited, clinicians face a significant challenge of identifying patients who are at high risk of OA in a timely and appropriate way. Therefore, we developed a simple self-assessment scoring system and an improved artificial neural network (ANN) model for knee OA. Methods The Fifth Korea National Health and Nutrition Examination Surveys (KNHANES V-1) data were used to develop a scoring system and ANN for radiographic knee OA. A logistic regression analysis was used to determine the predictors of the scoring system. The ANN was constructed using 1777 participants and validated internally on 888 participants in the KNHANES V-1. The predictors of the scoring system were selected as the inputs of the ANN. External validation was performed using 4731 participants in the Osteoarthritis Initiative (OAI). Area under the curve (AUC) of the receiver operating characteristic was calculated to compare the prediction models. Results The scoring system and ANN were built using the independent predictors including sex, age, body mass index, educational status, hypertension, moderate physical activity, and knee pain. In the internal validation, both scoring system and ANN predicted radiographic knee OA (AUC 0.73 versus 0.81, p<0.001) and symptomatic knee OA (AUC 0.88 versus 0.94, p<0.001) with good discriminative ability. In the external validation, both scoring system and ANN showed lower discriminative ability in predicting radiographic knee OA (AUC 0.62 versus 0.67, p<0.001) and symptomatic knee OA (AUC 0.70 versus 0.76, p<0.001). Conclusions The self-assessment scoring system may be useful for identifying the adults at high risk for knee OA. The performance of the scoring system is improved significantly by the ANN. We provided an ANN calculator to simply predict the knee OA risk. PMID:26859664

The Optimal Screening for Prediction of Referral and Outcome (OSPRO) in patients with musculoskeletal pain conditions: a longitudinal validation cohort from the USA

PubMed Central

George, Steven Z; Beneciuk, Jason M; Lentz, Trevor A; Wu, Samuel S

2017-01-01

Purpose There is an increased need for determining which patients with musculoskeletal pain benefit from additional diagnostic testing or psychologically informed intervention. The Optimal Screening for Prediction of Referral and Outcome (OSPRO) cohort studies were designed to develop and validate standard assessment tools for review of systems and yellow flags. This cohort profile paper provides a description of and future plans for the validation cohort. Participants Patients (n=440) with primary complaint of spine, shoulder or knee pain were recruited into the OSPRO validation cohort via a national Orthopaedic Physical Therapy-Investigative Network. Patients were followed up at 4 weeks, 6 months and 12 months for pain, functional status and quality of life outcomes. Healthcare utilisation outcomes were also collected at 6 and 12 months. Findings to date There are no longitudinal findings reported to date from the ongoing OSPRO validation cohort. The previously completed cross-sectional OSPRO development cohort yielded two assessment tools that were investigated in the validation cohort. Future plans Follow-up data collection was completed in January 2017. Primary analyses will investigate how accurately the OSPRO review of systems and yellow flag tools predict 12-month pain, functional status, quality of life and healthcare utilisation outcomes. Planned secondary analyses include prediction of pain interference and/or development of chronic pain, investigation of treatment expectation on patient outcomes and analysis of patient satisfaction following an episode of physical therapy. Trial registration number The OSPRO validation cohort was not registered. PMID:28600371

DIFAS: Differential Item Functioning Analysis System. Computer Program Exchange

ERIC Educational Resources Information Center

Penfield, Randall D.

2005-01-01

Differential item functioning (DIF) is an important consideration in assessing the validity of test scores (Camilli & Shepard, 1994). A variety of statistical procedures have been developed to assess DIF in tests of dichotomous (Hills, 1989; Millsap & Everson, 1993) and polytomous (Penfield & Lam, 2000; Potenza & Dorans, 1995) items. Some of these…

Vocational Rehabilitation Counselor Training Needs Assessment and Competence Measure: An Exploratory Factor Analysis

ERIC Educational Resources Information Center

Kundu, Madan M.; Dutta, Alo; Chan, Fong; Torres, Viviana; Fleming, Kayla

2011-01-01

Purpose: To validate an 80-item self-report measure, A Systems Approach to Placement: Self-Assessment for Students and Counselors (SAP-SASC), designed to identify critical areas of knowledge, skills, and competencies possessed by rehabilitation counselors in state vocational rehabilitation (VR) agency settings. Participants: 275 rehabilitation…

Quality Assessment Parameters for Student Support at Higher Education Institutions

ERIC Educational Resources Information Center

Sajiene, Laima; Tamuliene, Rasa

2012-01-01

The research presented in this article aims to validate quality assessment parameters for student support at higher education institutions. Student support is discussed as the system of services provided by a higher education institution which helps to develop student-centred curriculum and fulfils students' emotional, academic, social needs, and…

Assessing Computer Literacy: A Validated Instrument and Empirical Results.

ERIC Educational Resources Information Center

Gabriel, Roy M.

1985-01-01

Describes development of a comprehensive computer literacy assessment battery for K-12 curriculum based on objectives of a curriculum implemented in the Worldwide Department of Defense Dependents Schools system. Test development and field test data are discussed and a correlational analysis which assists in interpretation of test results is…

Assessing Student Learning: A Collection of Evaluation Tools

ERIC Educational Resources Information Center

Gottfried, Gail M.; Johnson, Kathy E.; Vosmik, Jordan R.

2009-01-01

Whereas grading systems based on tacit knowledge may be the norm in practice, the recent trend toward educational accountability--from granting organizations, accreditation boards, journals on the teaching of psychology, and even tenure/promotion committees--suggests a real need for reliable, validated assessment measures that can be used to…

Assessing Peer Entry and Play in Preschoolers at Risk for Maladjustment

ERIC Educational Resources Information Center

Brotman, Laurie Miller; Gouley, Kathleen Kiely; Chesir-Teran, Daniel

2005-01-01

This study evaluated the psychometric properties of an observational rating system for assessing preschoolers' peer entry and play skills: Observed Peer Play in Unfamiliar Settings (OPPUS). Participants were 84 preschoolers at risk for psychopathology. Reliability and concurrent validity are reported. The 30-min paradigm yielded reliable indexes…

An Advanced Bio-Inspired PhotoPlethysmoGraphy (PPG) and ECG Pattern Recognition System for Medical Assessment

PubMed Central

Rundo, Francesco; Ortis, Alessandro

2018-01-01

Physiological signals are widely used to perform medical assessment for monitoring an extensive range of pathologies, usually related to cardio-vascular diseases. Among these, both PhotoPlethysmoGraphy (PPG) and Electrocardiography (ECG) signals are those more employed. PPG signals are an emerging non-invasive measurement technique used to study blood volume pulsations through the detection and analysis of the back-scattered optical radiation coming from the skin. ECG is the process of recording the electrical activity of the heart over a period of time using electrodes placed on the skin. In the present paper we propose a physiological ECG/PPG “combo” pipeline using an innovative bio-inspired nonlinear system based on a reaction-diffusion mathematical model, implemented by means of the Cellular Neural Network (CNN) methodology, to filter PPG signal by assigning a recognition score to the waveforms in the time series. The resulting “clean” PPG signal exempts from distortion and artifacts is used to validate for diagnostic purpose an EGC signal simultaneously detected for a same patient. The multisite combo PPG-ECG system proposed in this work overpasses the limitations of the state of the art in this field providing a reliable system for assessing the above-mentioned physiological parameters and their monitoring over time for robust medical assessment. The proposed system has been validated and the results confirmed the robustness of the proposed approach. PMID:29385774

An Advanced Bio-Inspired PhotoPlethysmoGraphy (PPG) and ECG Pattern Recognition System for Medical Assessment.

PubMed

Rundo, Francesco; Conoci, Sabrina; Ortis, Alessandro; Battiato, Sebastiano

2018-01-30

Physiological signals are widely used to perform medical assessment for monitoring an extensive range of pathologies, usually related to cardio-vascular diseases. Among these, both PhotoPlethysmoGraphy (PPG) and Electrocardiography (ECG) signals are those more employed. PPG signals are an emerging non-invasive measurement technique used to study blood volume pulsations through the detection and analysis of the back-scattered optical radiation coming from the skin. ECG is the process of recording the electrical activity of the heart over a period of time using electrodes placed on the skin. In the present paper we propose a physiological ECG/PPG "combo" pipeline using an innovative bio-inspired nonlinear system based on a reaction-diffusion mathematical model, implemented by means of the Cellular Neural Network (CNN) methodology, to filter PPG signal by assigning a recognition score to the waveforms in the time series. The resulting "clean" PPG signal exempts from distortion and artifacts is used to validate for diagnostic purpose an EGC signal simultaneously detected for a same patient. The multisite combo PPG-ECG system proposed in this work overpasses the limitations of the state of the art in this field providing a reliable system for assessing the above-mentioned physiological parameters and their monitoring over time for robust medical assessment. The proposed system has been validated and the results confirmed the robustness of the proposed approach.

Mammographic image quality in relation to positioning of the breast: A multicentre international evaluation of the assessment systems currently used, to provide an evidence base for establishing a standardised method of assessment.

PubMed

Taylor, K; Parashar, D; Bouverat, G; Poulos, A; Gullien, R; Stewart, E; Aarre, R; Crystal, P; Wallis, M

2017-11-01

Optimum mammography positioning technique is necessary to maximise cancer detection. Current criteria for mammography appraisal lack reliability and validity with a need to develop a more objective system. We aimed to establish current international practice in assessing image quality (IQ), of screening mammograms then develop and validate a reproducible assessment tool. A questionnaire sent to centres in countries undertaking population screening identified practice, participants for an expert panel (EP) of radiologists/radiographers and a testing panel (TP) of radiographers. The EP developed category criteria and descriptors using a modified Delphi process to agree definitions. The EP scored 12 screening mammograms to test agreement then a main set of 178 cases. Weighted scores were derived for each descriptor enabling calculation of numerical parameters for each new category. The TP then scored the main set. Statistical analysis included ANOVA, t-tests and Kendall's coefficient. 11 centres in 8 countries responded forming an EP of 7 members and TP of 44 members. The EP showed moderate agreement when the scoring the mini test set W = 0.50 p < 0.001 and the main set W = 0.55 p < 0.001, 'posterior nipple line' being the most difficult descriptor. The weighted total scores differentiated the 4 new categories Perfect, Good, Adequate and Inadequate (p < 0.001). We have developed an assessment tool by Delphi consensus and weighted consensus criteria. We have successfully tabulated a range of numerical scores for each new category providing the first validated and reproducible mammography IQ scoring system. Copyright © 2017 The College of Radiographers. Published by Elsevier Ltd. All rights reserved.

Development and Validation of a Symptom-Based Activity Index for Adults with Eosinophilic Esophagitis

PubMed Central

Schoepfer, Alain M.; Straumann, Alex; Panczak, Radoslaw; Coslovsky, Michael; Kuehni, Claudia E.; Maurer, Elisabeth; Haas, Nadine A.; Romero, Yvonne; Hirano, Ikuo; Alexander, Jeffrey A.; Gonsalves, Nirmala; Furuta, Glenn T.; Dellon, Evan S.; Leung, John; Collins, Margaret H.; Bussmann, Christian; Netzer, Peter; Gupta, Sandeep K.; Aceves, Seema S.; Chehade, Mirna; Moawad, Fouad J.; Enders, Felicity T.; Yost, Kathleen J.; Taft, Tiffany H.; Kern, Emily; Zwahlen, Marcel; Safroneeva, Ekaterina

2015-01-01

BACKGROUND & AIMS Standardized instruments are needed to assess the activity of eosinophilic esophagitis (EoE), to provide endpoints for clinical trials and observational studies. We aimed to develop and validate a patient-reported outcome (PRO) instrument and score, based on items that could account for variations in patients’ assessments of disease severity. We also evaluated relationships between patients’ assessment of disease severity and EoE-associated endoscopic, histologic, and laboratory findings. METHODS We collected information from 186 patients with EoE in Switzerland and the US (69.4% male; median age, 43 years) via surveys (n = 135), focus groups (n = 27), and semi-structured interviews (n = 24). Items were generated for the instruments to assess biologic activity based on physician input. Linear regression was used to quantify the extent to which variations in patient-reported disease characteristics could account for variations in patients’ assessment of EoE severity. The PRO instrument was prospectively used in 153 adult patients with EoE (72.5% male; median age, 38 years), and validated in an independent group of 120 patients with EoE (60.8% male; median age, 40.5 years). RESULTS Seven PRO factors that are used to assess characteristics of dysphagia, behavioral adaptations to living with dysphagia, and pain while swallowing accounted for 67% of the variation in patients’ assessment of disease severity. Based on statistical consideration and patient input, a 7-day recall period was selected. Highly active EoE, based on endoscopic and histologic findings, was associated with an increase in patient-assessed disease severity. In the validation study, the mean difference between patient assessment of EoE severity and PRO score was 0.13 (on a scale from 0 to 10). CONCLUSIONS We developed and validated an EoE scoring system based on 7 PRO items that assesses symptoms over a 7-day recall period. Clinicaltrials.gov number: NCT00939263. PMID:25160980

Quality Management and System Change in Three Suburban Public School Districts.

ERIC Educational Resources Information Center

Obisesan, Anthonia A.

This report examines the potential of Quality Management (QM) to enhance system change by analyzing its implementation in three suburban public school districts. The paper assessed the capacity of QM to increase the efficiency and productivity of the school districts, validated the potential to sustain systemic change in a school organization, and…

On Selecting Commercial Information Systems

PubMed Central

Möhr, J.R.; Sawinski, R.; Kluge, A.; Alle, W.

1984-01-01

As more commercial information systems become available, the methodology for their selection gains importance. An instances where the method employed for the selection of laboratory information systems was multilevel assessment. The method is described and the experience gained in the project is summarized and discussed. Evidence is provided that the employed method is comprehensive, reproducible, valid and economic.

Factor structure and psychometric properties of a French and German shortened version of the Behavioural Inhibition System/Behavioural Activation System scales.

PubMed

Studer, Joseph; Baggio, Stéphanie; Mohler-Kuo, Meichun; Daeppen, Jean-Bernard; Gmel, Gerhard

2016-03-01

The Behavioural Inhibition System/Behavioural Activation System scales (BIS/BAS scales) constitute one of the most prominent questionnaires to assess individual differences in sensitivity to punishment and reward. However, some studies questioned its validity, especially that of the French and German translations. The aim of the present study was to re-evaluate the psychometric characteristics of the BIS/BAS scales in a large sample of French- and German-speaking young Swiss men (N = 5872). Results showed that factor structures previously found in the literature did not meet the standards of fit. Nine items had to be removed to achieve adequate fit statistics in confirmatory factor analysis, yielding a shortened version with four factors: one BIS factor comprising five items and three BAS factors, namely Reward Reactivity, Drive and Fun Seeking, each comprising two items. Convergent validity and group invariance analyses suggest that the shortened BIS/BAS scales constitute a valid and reliable instrument. Researchers interested in assessing individual differences in BIS and BAS reactivity in French- and German-speaking individuals should avoid using the BIS/BAS scales as originally specified. The shortened version may be a sound alternative at least in samples of young adults. Its shorter format may be particularly suited for surveys with constraints on questionnaire length.

Development, initial reliability and validity testing of an observational tool for assessing technical skills of operating room nurses.

PubMed

Sevdalis, Nick; Undre, Shabnam; Henry, Janet; Sydney, Elaine; Koutantji, Mary; Darzi, Ara; Vincent, Charles A

2009-09-01

The recent emergence of the Systems Approach to the safety and quality of surgical care has triggered individual and team skills training modules for surgeons and anaesthetists and relevant observational assessment tools have been developed. To develop an observational tool that captures operating room (OR) nurses' technical skill and can be used for assessment and training. The Imperial College Assessment of Technical Skills for Nurses (ICATS-N) assesses (i) gowning and gloving, (ii) setting up instrumentation, (iii) draping, and (iv) maintaining sterility. Three to five observable behaviours have been identified for each skill and are rated on 1-6 scales. Feasibility and aspects of reliability and validity were assessed in 20 simulation-based crisis management training modules for trainee nurses and doctors, carried out in a Simulated Operating Room. The tool was feasible to use in the context of simulation-based training. Satisfactory reliability (Cronbach alpha) was obtained across trainers' and trainees' scores (analysed jointly and separately). Moreover, trainer nurse's ratings of the four skills correlated positively, thus indicating adequate content validity. Trainer's and trainees' ratings did not correlate. Assessment of OR nurses' technical skill is becoming a training priority. The present evidence suggests that the ICATS-N could be considered for use as an assessment/training tool for junior OR nurses.

Reliability and validity of a smartphone pulse rate application for the assessment of resting and elevated pulse rate.

PubMed

Mitchell, Katy; Graff, Megan; Hedt, Corbin; Simmons, James

2016-08-01

Purpose/hypothesis: This study was designed to investigate the test-retest reliability, concurrent validity, and the standard error of measurement (SEm) of a pulse rate assessment application (Azumio®'s Instant Heart Rate) on both Android® and iOS® (iphone operating system) smartphones as compared to a FT7 Polar® Heart Rate monitor. Number of subjects: 111. Resting (sitting) pulse rate was assessed twice and then the participants were asked to complete a 1-min standing step test and then immediately re-assessed. The smartphone assessors were blinded to their measurements. Test-retest reliability (intraclass correlation coefficient [ICC 2,1] and 95% confidence interval) for the three tools at rest (time 1/time 2): iOS® (0.76 [0.67-0.83]); Polar® (0.84 [0.78-0.89]); and Android® (0.82 [0.75-0.88]). Concurrent validity at rest time 2 (ICC 2,1) with the Polar® device: IOS® (0.92 [0.88-0.94]) and Android® (0.95 [0.92-0.96]). Concurrent validity post-exercise (time 3) (ICC) with the Polar® device: iOS® (0.90 [0.86-0.93]) and Android® (0.94 [0.91-0.96]). The SEm values for the three devices at rest: iOS® (5.77 beats per minute [BPM]), Polar® (4.56 BPM) and Android® (4.96 BPM). The Android®, iOS®, and Polar® devices showed acceptable test-retest reliability at rest and post-exercise. Both the smartphone platforms demonstrated concurrent validity with the Polar® at rest and post-exercise. The Azumio® Instant Heart Rate application when used by either platform appears to be a reliable and valid tool to assess pulse rate in healthy individuals.

Automated evaluation of electronic discharge notes to assess quality of care for cardiovascular diseases using Medical Language Extraction and Encoding System (MedLEE)

PubMed Central

Lin, Jou-Wei; Yang, Chen-Wei

2010-01-01

The objective of this study was to develop and validate an automated acquisition system to assess quality of care (QC) measures for cardiovascular diseases. This system combining searching and retrieval algorithms was designed to extract QC measures from electronic discharge notes and to estimate the attainment rates to the current standards of care. It was developed on the patients with ST-segment elevation myocardial infarction and tested on the patients with unstable angina/non-ST-segment elevation myocardial infarction, both diseases sharing almost the same QC measures. The system was able to reach a reasonable agreement (κ value) with medical experts from 0.65 (early reperfusion rate) to 0.97 (β-blockers and lipid-lowering agents before discharge) for different QC measures in the test set, and then applied to evaluate QC in the patients who underwent coronary artery bypass grafting surgery. The result has validated a new tool to reliably extract QC measures for cardiovascular diseases. PMID:20442141

Modeling Terrorism Risk to the Air Transportation System: An Independent Assessment of TSA’s Risk Management Analysis Tool and Associated Methods

DTIC Science & Technology

2012-01-01

our own work for this discussion. DoD Instruction 5000.61 defines model validation as “the pro - cess of determining the degree to which a model and its... determined that RMAT is highly con - crete code, potentially leading to redundancies in the code itself and making RMAT more difficult to maintain...system con - ceptual models valid, and are the data used to support them adequate? (Chapters Two and Three) 2. Are the sources and methods for populating

Flight control optimization from design to assessment application on the Cessna Citation X business aircraft =

NASA Astrophysics Data System (ADS)

Boughari, Yamina

New methodologies have been developed to optimize the integration, testing and certification of flight control systems, an expensive process in the aerospace industry. This thesis investigates the stability of the Cessna Citation X aircraft without control, and then optimizes two different flight controllers from design to validation. The aircraft's model was obtained from the data provided by the Research Aircraft Flight Simulator (RAFS) of the Cessna Citation business aircraft. To increase the stability and control of aircraft systems, optimizations of two different flight control designs were performed: 1) the Linear Quadratic Regulation and the Proportional Integral controllers were optimized using the Differential Evolution algorithm and the level 1 handling qualities as the objective function. The results were validated for the linear and nonlinear aircraft models, and some of the clearance criteria were investigated; and 2) the Hinfinity control method was applied on the stability and control augmentation systems. To minimize the time required for flight control design and its validation, an optimization of the controllers design was performed using the Differential Evolution (DE), and the Genetic algorithms (GA). The DE algorithm proved to be more efficient than the GA. New tools for visualization of the linear validation process were also developed to reduce the time required for the flight controller assessment. Matlab software was used to validate the different optimization algorithms' results. Research platforms of the aircraft's linear and nonlinear models were developed, and compared with the results of flight tests performed on the Research Aircraft Flight Simulator. Some of the clearance criteria of the optimized H-infinity flight controller were evaluated, including its linear stability, eigenvalues, and handling qualities criteria. Nonlinear simulations of the maneuvers criteria were also investigated during this research to assess the Cessna Citation X's flight controller clearance, and therefore, for its anticipated certification.

Flight Test 4 Preliminary Results: NASA Ames SSI

NASA Technical Reports Server (NTRS)

Isaacson, Doug; Gong, Chester; Reardon, Scott; Santiago, Confesor

2016-01-01

Realization of the expected proliferation of Unmanned Aircraft System (UAS) operations in the National Airspace System (NAS) depends on the development and validation of performance standards for UAS Detect and Avoid (DAA) Systems. The RTCA Special Committee 228 is charged with leading the development of draft Minimum Operational Performance Standards (MOPS) for UAS DAA Systems. NASA, as a participating member of RTCA SC-228 is committed to supporting the development and validation of draft requirements as well as the safety substantiation and end-to-end assessment of DAA system performance. The Unmanned Aircraft System (UAS) Integration into the National Airspace System (NAS) Project conducted flight test program, referred to as Flight Test 4, at Armstrong Flight Research Center from April -June 2016. Part of the test flights were dedicated to the NASA Ames-developed Detect and Avoid (DAA) System referred to as JADEM (Java Architecture for DAA Extensibility and Modeling). The encounter scenarios, which involved NASA's Ikhana UAS and a manned intruder aircraft, were designed to collect data on DAA system performance in real-world conditions and uncertainties with four different surveillance sensor systems. Flight test 4 has four objectives: (1) validate DAA requirements in stressing cases that drive MOPS requirements, including: high-speed cooperative intruder, low-speed non-cooperative intruder, high vertical closure rate encounter, and Mode CS-only intruder (i.e. without ADS-B), (2) validate TCASDAA alerting and guidance interoperability concept in the presence of realistic sensor, tracking and navigational errors and in multiple-intruder encounters against both cooperative and non-cooperative intruders, (3) validate Well Clear Recovery guidance in the presence of realistic sensor, tracking and navigational errors, and (4) validate DAA alerting and guidance requirements in the presence of realistic sensor, tracking and navigational errors. The results will be presented at RTCA Special Committee 228 in support of final verification and validation of the DAA MOPS.

When Significant Others Suffer: German Validation of the Burden Assessment Scale (BAS)

PubMed Central

Hunger, Christina; Krause, Lena; Hilzinger, Rebecca; Ditzen, Beate; Schweitzer, Jochen

2016-01-01

There is a need of an economical, reliable, and valid instrument in the German-speaking countries to measure the burden of relatives who care for mentally ill persons. We translated the Burden Assessment Scale (BAS) and conducted a study investigating factor structure, psychometric quality and predictive validity. We used confirmative factor analyses (CFA, maximum-likelihood method) to examine the dimensionality of the German BAS in a sample of 215 relatives (72% women; M = 32 years, SD = 14, range: 18 to 77; 39% employed) of mentally ill persons (50% (ex-)partner or (best) friend; M = 32 years, SD = 13, range 8 to 64; main complaints were depression and/or anxiety). Cronbach’s α determined the internal consistency. We examined predictive validity using regression analyses including the BAS and validated scales of social systems functioning (Experience In Social Systems Questionnaire, EXIS.pers, EXIS.org) and psychopathology (Brief Symptom Inventory, BSI). Variables that might have influenced the dependent variables (e.g. age, gender, education, employment and civil status) were controlled by their introduction in the first step, and the BAS in the second step of the regression analyses. A model with four correlated factors (Disrupted Activities, Personal Distress, Time Perspective, Guilt) showed the best fit. With respect to the number of items included, the internal consistency was very good. The modified German BAS predicted relatives’ social systems functioning and psychopathology. The economical design makes the 19-item BAS promising for practice-oriented research, and for studies under time constraints. Strength, limitations and future directions are discussed. PMID:27764109

Web-based application on employee performance assessment using exponential comparison method

NASA Astrophysics Data System (ADS)

Maryana, S.; Kurnia, E.; Ruyani, A.

2017-02-01

Employee performance assessment is also called a performance review, performance evaluation, or assessment of employees, is an effort to assess the achievements of staffing performance with the aim to increase productivity of employees and companies. This application helps in the assessment of employee performance using five criteria: Presence, Quality of Work, Quantity of Work, Discipline, and Teamwork. The system uses the Exponential Comparative Method and Weighting Eckenrode. Calculation results using graphs were provided to see the assessment of each employee. Programming language used in this system is written in Notepad++ and MySQL database. The testing result on the system can be concluded that this application is correspond with the design and running properly. The test conducted is structural test, functional test, and validation, sensitivity analysis, and SUMI testing.

A model for technology assessment as applied to closed loop infusion systems. Technology Assessment Task Force of the Society of Critical Care Medicine.

PubMed

Jastremski, M; Jastremski, C; Shepherd, M; Friedman, V; Porembka, D; Smith, R; Gonzales, E; Swedlow, D; Belzberg, H; Crass, R

1995-10-01

To test a model for the assessment of critical care technology on closed loop infusion control, a technology that is in its early stages of development and testing on human subjects. A computer-assisted search of the English language literature and reviews of the gathered data by experts in the field of closed loop infusion control systems. Studies relating to closed loop infusion control that addressed one or more of the questions contained in our technology assessment template were analyzed. Study design was not a factor in article selection. However, the lack of well-designed clinical outcome studies was an important factor in determining our conclusions. A focus person summarized the data from the selected studies that related to each of the assessment questions. The preliminary data summary developed by the focus person was further analyzed and refined by the task force. Experts in closed loop systems were then added to the group to review the summary provided by the task force. These experts' comments were considered by the task force and this final consensus report was developed. Closed loop system control is a technological concept that may be applicable to several aspects of critical care practice. This is a technology in the early stages of evolution and much more research and data are needed before its introduction into usual clinical practice. Furthermore, each specific application and each device for each application (e.g., nitroprusside infusion, ventilator adjustment), although based on the same technological concept, are sufficiently different in terms of hardware and computer algorithms to require independent validation studies. Closed loop infusion systems may have a role in critical care practice. However, for most applications, further development is required to move this technology from the innovation phase to the point where it can be evaluated so that its role in critical car practice can be defined. Each application of closed loop infusion systems must be independently validated by appropriately designed research studies. Users should be provided with the clinical parameters driving each closed loop system so that they can ensure that it agrees with their opinion of acceptable medical practice. Clinical researchers and leaders in industry should collaborate to perform the scientifically valid, outcome-based research that is necessary to evaluate the effect of this new technology. The original model we developed for technology assessment required the addition of several more questions to produce a complete analysis of an emerging technology. An emerging technology should be systematically assessed (using a model such as the model developed by the Society of Critical Care Medicine), before its introduction into clinical practice in order to provide a focus for human outcome validation trials and to minimize the possibility of widespread use of an unproven technology.

Computerized Hammer Sounding Interpretation for Concrete Assessment with Online Machine Learning.

PubMed

Ye, Jiaxing; Kobayashi, Takumi; Iwata, Masaya; Tsuda, Hiroshi; Murakawa, Masahiro

2018-03-09

Developing efficient Artificial Intelligence (AI)-enabled systems to substitute the human role in non-destructive testing is an emerging topic of considerable interest. In this study, we propose a novel hammering response analysis system using online machine learning, which aims at achieving near-human performance in assessment of concrete structures. Current computerized hammer sounding systems commonly employ lab-scale data to validate the models. In practice, however, the response signal patterns can be far more complicated due to varying geometric shapes and materials of structures. To deal with a large variety of unseen data, we propose a sequential treatment for response characterization. More specifically, the proposed system can adaptively update itself to approach human performance in hammering sounding data interpretation. To this end, a two-stage framework has been introduced, including feature extraction and the model updating scheme. Various state-of-the-art online learning algorithms have been reviewed and evaluated for the task. To conduct experimental validation, we collected 10,940 response instances from multiple inspection sites; each sample was annotated by human experts with healthy/defective condition labels. The results demonstrated that the proposed scheme achieved favorable assessment accuracy with high efficiency and low computation load.

WaferOptics® mass volume production and reliability

NASA Astrophysics Data System (ADS)

Wolterink, E.; Demeyer, K.

2010-05-01

The Anteryon WaferOptics® Technology platform contains imaging optics designs, materials, metrologies and combined with wafer level based Semicon & MEMS production methods. WaferOptics® first required complete new system engineering. This system closes the loop between application requirement specifications, Anteryon product specification, Monte Carlo Analysis, process windows, process controls and supply reject criteria. Regarding the Anteryon product Integrated Lens Stack (ILS), new design rules, test methods and control systems were assessed, implemented, validated and customer released for mass production. This includes novel reflowable materials, mastering process, replication, bonding, dicing, assembly, metrology, reliability programs and quality assurance systems. Many of Design of Experiments were performed to assess correlations between optical performance parameters and machine settings of all process steps. Lens metrologies such as FFL, BFL, and MTF were adapted for wafer level production and wafer mapping was introduced for yield management. Test methods for screening and validating suitable optical materials were designed. Critical failure modes such as delamination and popcorning were assessed and modeled with FEM. Anteryon successfully managed to integrate the different technologies starting from single prototypes to high yield mass volume production These parallel efforts resulted in a steep yield increase from 30% to over 90% in a 8 months period.

Enhanced Requirements for Assessment in a Competency-Based, Time-Variable Medical Education System.

PubMed

Gruppen, Larry D; Ten Cate, Olle; Lingard, Lorelei A; Teunissen, Pim W; Kogan, Jennifer R

2018-03-01

Competency-based, time-variable medical education has reshaped the perceptions and practices of teachers, curriculum designers, faculty developers, clinician educators, and program administrators. This increasingly popular approach highlights the fact that learning among different individuals varies in duration, foundation, and goal. Time variability places particular demands on the assessment data that are so necessary for making decisions about learner progress. These decisions may be formative (e.g., feedback for improvement) or summative (e.g., decisions about advancing a student). This article identifies challenges to collecting assessment data and to making assessment decisions in a time-variable system. These challenges include managing assessment data, defining and making valid assessment decisions, innovating in assessment, and modeling the considerable complexity of assessment in real-world settings and richly interconnected social systems. There are hopeful signs of creativity in assessment both from researchers and practitioners, but the transition from a traditional to a competency-based medical education system will likely continue to create much controversy and offer opportunities for originality and innovation in assessment.

Reliability and Validity of the Footprint Assessment Method Using Photoshop CS5 Software in Young People with Down Syndrome.

PubMed

Gutiérrez-Vilahú, Lourdes; Massó-Ortigosa, Núria; Rey-Abella, Ferran; Costa-Tutusaus, Lluís; Guerra-Balic, Myriam

2016-05-01

People with Down syndrome present skeletal abnormalities in their feet that can be analyzed by commonly used gold standard indices (the Hernández-Corvo index, the Chippaux-Smirak index, the Staheli arch index, and the Clarke angle) based on footprint measurements. The use of Photoshop CS5 software (Adobe Systems Software Ireland Ltd, Dublin, Ireland) to measure footprints has been validated in the general population. The present study aimed to assess the reliability and validity of this footprint assessment technique in the population with Down syndrome. Using optical podography and photography, 44 footprints from 22 patients with Down syndrome (11 men [mean ± SD age, 23.82 ± 3.12 years] and 11 women [mean ± SD age, 24.82 ± 6.81 years]) were recorded in a static bipedal standing position. A blinded observer performed the measurements using a validated manual method three times during the 4-month study, with 2 months between measurements. Test-retest was used to check the reliability of the Photoshop CS5 software measurements. Validity and reliability were obtained by intraclass correlation coefficient (ICC). The reliability test for all of the indices showed very good values for the Photoshop CS5 method (ICC, 0.982-0.995). Validity testing also found no differences between the techniques (ICC, 0.988-0.999). The Photoshop CS5 software method is reliable and valid for the study of footprints in young people with Down syndrome.

Validity of Various Methods for Determining Velocity, Force, and Power in the Back Squat.

PubMed

Banyard, Harry G; Nosaka, Ken; Sato, Kimitake; Haff, G Gregory

2017-10-01

To examine the validity of 2 kinematic systems for assessing mean velocity (MV), peak velocity (PV), mean force (MF), peak force (PF), mean power (MP), and peak power (PP) during the full-depth free-weight back squat performed with maximal concentric effort. Ten strength-trained men (26.1 ± 3.0 y, 1.81 ± 0.07 m, 82.0 ± 10.6 kg) performed three 1-repetition-maximum (1RM) trials on 3 separate days, encompassing lifts performed at 6 relative intensities including 20%, 40%, 60%, 80%, 90%, and 100% of 1RM. Each repetition was simultaneously recorded by a PUSH band and commercial linear position transducer (LPT) (GymAware [GYM]) and compared with measurements collected by a laboratory-based testing device consisting of 4 LPTs and a force plate. Trials 2 and 3 were used for validity analyses. Combining all 120 repetitions indicated that the GYM was highly valid for assessing all criterion variables while the PUSH was only highly valid for estimations of PF (r = .94, CV = 5.4%, ES = 0.28, SEE = 135.5 N). At each relative intensity, the GYM was highly valid for assessing all criterion variables except for PP at 20% (ES = 0.81) and 40% (ES = 0.67) of 1RM. Moreover, the PUSH was only able to accurately estimate PF across all relative intensities (r = .92-.98, CV = 4.0-8.3%, ES = 0.04-0.26, SEE = 79.8-213.1 N). PUSH accuracy for determining MV, PV, MF, MP, and PP across all 6 relative intensities was questionable for the back squat, yet the GYM was highly valid at assessing all criterion variables, with some caution given to estimations of MP and PP performed at lighter loads.

Translation, Cross-cultural Adaptation and Psychometric Validation of the Korean-Language Cardiac Rehabilitation Barriers Scale (CRBS-K)

PubMed Central

2017-01-01

Objective To perform a translation and cross-cultural adaptation of the Cardiac Rehabilitation Barriers Scale (CRBS) for use in Korea, followed by psychometric validation. The CRBS was developed to assess patients' perception of the degree to which patient, provider and health system-level barriers affect their cardiac rehabilitation (CR) participation. Methods The CRBS consists of 21 items (barriers to adherence) rated on a 5-point Likert scale. The first phase was to translate and cross-culturally adapt the CRBS to the Korean language. After back-translation, both versions were reviewed by a committee. The face validity was assessed in a sample of Korean patients (n=53) with history of acute myocardial infarction that did not participate in CR through semi-structured interviews. The second phase was to assess the construct and criterion validity of the Korean translation as well as internal reliability, through administration of the translated version in 104 patients, principle component analysis with varimax rotation and cross-referencing against CR use, respectively. Results The length, readability, and clarity of the questionnaire were rated well, demonstrating face validity. Analysis revealed a six-factor solution, demonstrating construct validity. Cronbach's alpha was greater than 0.65. Barriers rated highest included not knowing about CR and not being contacted by a program. The mean CRBS score was significantly higher among non-attendees (2.71±0.26) than CR attendees (2.51±0.18) (p<0.01). Conclusion The Korean version of CRBS has demonstrated face, content and criterion validity, suggesting it may be useful for assessing barriers to CR utilization in Korea. PMID:29201826

Cross-Cultural Aspect of Behavior Assessment System for Children-2, Parent Rating Scale-Child: Standardization in Korean Children

PubMed Central

Song, Jungeun; Leventhal, Bennett L.; Koh, Yun-Joo; Cheon, Keun-Ah; Hong, Hyun Ju; Kim, Young-Key; Cho, Kyungjin; Lim, Eun-Chung; Park, Jee In

2017-01-01

Purpose Our study aimed to examine psychometric properties and cross-cultural utility of the Behavior Assessment System for Children-2, Parent Rating Scale-Child (BASC-2 PRS-C) in Korean children. Materials and Methods Two study populations were recruited: a general population sample (n=2115) of 1st to 6th graders from 16 elementary schools and a clinical population (n=219) of 6–12 years old from 5 child psychiatric clinics and an epidemiological sample of autism spectrum disorder. We assessed the validity and reliability of the Korean version of BASC-2 PRS-C (K-BASC-2 PRS-C) and compared subscales with those used for US populations. Results Our results indicate that the K-BASC-2 PRS-C is a valuable instrument with reliability and validity for measuring developmental psychopathology that is comparable to those in Western population. However, there were some differences noted in the mean scores of BASC-2 PRS-C between Korean and US populations. Conclusion K-BASC-2 PRS-C is an effective and useful instrument with psychometric properties that permits measurement of general developmental psychopathology. Observed Korean-US differences in patterns of parental reports of children's behaviors indicate the importance of the validation, standardization and cultural adaptation for tools assessing psychopathology especially when used in populations different from those for which the instrument was originally created. PMID:28120577

Cross-Cultural Aspect of Behavior Assessment System for Children-2, Parent Rating Scale-Child: Standardization in Korean Children.

PubMed

Song, Jungeun; Leventhal, Bennett L; Koh, Yun Joo; Cheon, Keun Ah; Hong, Hyun Ju; Kim, Young Key; Cho, Kyungjin; Lim, Eun Chung; Park, Jee In; Kim, Young Shin

2017-03-01

Our study aimed to examine psychometric properties and cross-cultural utility of the Behavior Assessment System for Children-2, Parent Rating Scale-Child (BASC-2 PRS-C) in Korean children. Two study populations were recruited: a general population sample (n=2115) of 1st to 6th graders from 16 elementary schools and a clinical population (n=219) of 6-12 years old from 5 child psychiatric clinics and an epidemiological sample of autism spectrum disorder. We assessed the validity and reliability of the Korean version of BASC-2 PRS-C (K-BASC-2 PRS-C) and compared subscales with those used for US populations. Our results indicate that the K-BASC-2 PRS-C is a valuable instrument with reliability and validity for measuring developmental psychopathology that is comparable to those in Western population. However, there were some differences noted in the mean scores of BASC-2 PRS-C between Korean and US populations. K-BASC-2 PRS-C is an effective and useful instrument with psychometric properties that permits measurement of general developmental psychopathology. Observed Korean-US differences in patterns of parental reports of children's behaviors indicate the importance of the validation, standardization and cultural adaptation for tools assessing psychopathology especially when used in populations different from those for which the instrument was originally created.

48 CFR 1401.7001-4 - Acquisition performance measurement systems.

Code of Federal Regulations, 2010 CFR

2010-10-01

...-pronged approach that includes self assessment, statistical data for validation and flexible quality... regulations governing the acquisition process; and (3) Identify and implement changes necessary to improve the...

48 CFR 2901.603-72 - Administrative procurement management reviews.

Code of Federal Regulations, 2014 CFR

2014-10-01

... improvement of managerial controls and best practices. (b) The administrative procurement review system is a three-pronged approach that includes self-assessment, statistical data for validation, and flexible...

48 CFR 2901.603-72 - Administrative procurement management reviews.

Code of Federal Regulations, 2011 CFR

2011-10-01

... improvement of managerial controls and best practices. (b) The administrative procurement review system is a three-pronged approach that includes self-assessment, statistical data for validation, and flexible...

48 CFR 2901.603-72 - Administrative procurement management reviews.

Code of Federal Regulations, 2012 CFR

2012-10-01

... improvement of managerial controls and best practices. (b) The administrative procurement review system is a three-pronged approach that includes self-assessment, statistical data for validation, and flexible...

48 CFR 2901.603-72 - Administrative procurement management reviews.

Code of Federal Regulations, 2013 CFR

2013-10-01

... improvement of managerial controls and best practices. (b) The administrative procurement review system is a three-pronged approach that includes self-assessment, statistical data for validation, and flexible...

Issues in developing valid assessments of speech pathology students' performance in the workplace.

PubMed

McAllister, Sue; Lincoln, Michelle; Ferguson, Alison; McAllister, Lindy

2010-01-01

Workplace-based learning is a critical component of professional preparation in speech pathology. A validated assessment of this learning is seen to be 'the gold standard', but it is difficult to develop because of design and validation issues. These issues include the role and nature of judgement in assessment, challenges in measuring quality, and the relationship between assessment and learning. Valid assessment of workplace-based performance needs to capture the development of competence over time and account for both occupation specific and generic competencies. This paper reviews important conceptual issues in the design of valid and reliable workplace-based assessments of competence including assessment content, process, impact on learning, measurement issues, and validation strategies. It then goes on to share what has been learned about quality assessment and validation of a workplace-based performance assessment using competency-based ratings. The outcomes of a four-year national development and validation of an assessment tool are described. A literature review of issues in conceptualizing, designing, and validating workplace-based assessments was conducted. Key factors to consider in the design of a new tool were identified and built into the cycle of design, trialling, and data analysis in the validation stages of the development process. This paper provides an accessible overview of factors to consider in the design and validation of workplace-based assessment tools. It presents strategies used in the development and national validation of a tool COMPASS, used in an every speech pathology programme in Australia, New Zealand, and Singapore. The paper also describes Rasch analysis, a model-based statistical approach which is useful for establishing validity and reliability of assessment tools. Through careful attention to conceptual and design issues in the development and trialling of workplace-based assessments, it has been possible to develop the world's first valid and reliable national assessment tool for the assessment of performance in speech pathology.

Neuro-QoL health-related quality of life measurement system: Validation in Parkinson's disease.

PubMed

Nowinski, Cindy J; Siderowf, Andrew; Simuni, Tanya; Wortman, Catherine; Moy, Claudia; Cella, David

2016-05-01

Neuro-QoL is a multidimensional patient-reported outcome measurement system assessing aspects of physical, mental, and social health identified by neurology patients and caregivers as important. One of the first neurology-specific patient-reported outcome measure systems created using modern test development methods, Neuro-Qol enables brief, yet precise, assessment and the ability to conduct both PD-specific and cross-disease comparisons. We present results of Neuro-QoL clinical validation using a sample of PD patients. A total of 120 PD patients recruited from academic medical centers were assessed at baseline, 1 week, and 6 months. Assessments included Neuro-QoL and general and PD-specific validity measures. Participants were 62% male and 95% white (average age = 66); H & Y stages were 1 (16%), 2 (61%), 3 (18%), and 4 (5%). Internal consistency and test-retest reliability of Neuro-QoL ranged from Cronbach's alphas = 0.81 to 0.94 with intraclass correlation coefficients = 0.66 to 0.80. Pearson's correlations between Neuro-QoL and legacy measures were generally moderate and in expected directions. UPDRS Part 2 was moderately correlated with Neuro-QoL Upper Extremity and Mobility, respectively (r's = -0.44; -0.59). Parkinson's Disease Questionnaire-39 and Neuro-QoL measures of similar constructs showed strong-to-moderate correlations (r's = 0.70-0.44). Neuro-QoL measures of fatigue, mobility, positive emotion, and emotional/behavioral control showed responsiveness to self-reported change. Neuro-QoL is valid for use in PD clinical research. Reliability for all but two measures is sufficient for group comparisons, with some evidence supporting responsiveness to change. Neuro-QoL possesses characteristics, such as brevity, flexibility in administration, and suitability, for cross-disease comparisons that may be advantageous to users in a variety of settings. © 2016 Movement Disorder Society. © 2016 International Parkinson and Movement Disorder Society.

Application and Validation of Concept Maturity Assessment Framework

DTIC Science & Technology

2011-03-01

process. The following chapter will discuss a proposed methodology for validation of the concept maturity framwork and its Concept Evaluation and...of each contractor‟s conceptual solution and any gaps in information that may have been overlooked. The organization also commented that the... conceptual and does not have a specific system tied to it is often vulnerable to losing interest and potentially funding from decision makers. However

The Convergent and Divergent Validity of the Matson Evaluation of Drug Side-Effects (MEDS) and the Dyskinesia Identification System: Condensed User Scale (DISCUS)

ERIC Educational Resources Information Center

Matson, Johnny L.; Fodstad, Jill C.; Rivet, Tessa T.

2008-01-01

Background: Medication side-effects such as tardive dyskinesia (TD) are known to occur in individuals with a history of psychotropic drug use. This study aimed to contribute to the development of measures for assessing TD by examining the validity of the "Matson Evaluation of Drug Side-effects" (MEDS) with the "Dyskinesia…

Implementing the Customer Contact Center: An Opportunity to Create a Valid Measurement System for Assessing and Improving a Library's Telephone Services

ERIC Educational Resources Information Center

Murphy, Sarah Anne; Cerqua, Judith

2012-01-01

A customer contact center offers academic libraries the ability to consistently improve their telephone, e-mail, and IM services. This paper discusses the establishment of a contact center and the benefits of implementing the contact center model at this institution. It then introduces a practical methodology for developing a valid measurement…

The Gap Concept as a Quality of Life Measure: Validation Study of the Child Quality of Life Systemic Inventory

ERIC Educational Resources Information Center

Etienne, Anne-Marie; Dupuis, Gilles; Spitz, Elisabeth; Lemetayer, Fabienne; Missotten, Pierre

2011-01-01

The objective was to determine the interest and psychometric properties of a new QOL self-assessment questionnaire suitable for children 8-12 years old measuring alpha, beta and gamma changes: the "Inventaire Systemique de Qualite de vie pour Enfants" (ISQV-E[C]). This was a cross-sectional validation study. 288 children have completed…

Measuring Mobility Limitations in Children with Cerebral Palsy: Content and Construct Validity of a Mobility Questionnaire (MobQues)

ERIC Educational Resources Information Center

Van Ravesteyn, Nicolien T.; Scholtes, Vanessa A.; Becher, Jules G.; Roorda, Leo D.; Verschuren, Olaf; Dallmeijer, Annet J.

2010-01-01

Aim: The objective of this study was to assess the validity of a mobility questionnaire (MobQues) that was developed to measure parent-reported mobility limitations in children with cerebral palsy (CP). Method: The parents of 439 children with CP (256 males and 183 females; age range 2-18y; Gross Motor Function Classification System [GMFCS] levels…

What Does the Cognitive Assessment System (CAS) Measure? Joint Confirmatory Factor Analysis of the CAS and the Woodcock-Johnson Tests of Cognitive Ability (3rd Edition).

ERIC Educational Resources Information Center

Keith, Timothy Z.; Kranzler, John H.; Flanagan, Dawn P.

2001-01-01

Reports the results of the first joint confirmatory factor analysis (CFA) of the Cognitive Assessment System (CAS) and the Woodcock-Johnson Tests of Cognitive Abilities-3rd Edition (WJ III). Results of these analyses do not support the construct validity of the CAS as a measure of the PASS (planning, attention, simultaneous, and sequential)…

Validation of a new classification system for skin tears.

PubMed

LeBlanc, Kimberly; Baranoski, Sharon; Holloway, Samantha; Langemo, Diane

2013-06-01

The aim of this study was to validate and establish reliability of the International Skin Tear classification system. A consensus panel of 12 internationally recognized key opinion leaders convened in 2011 to establish consensus statements on the prevention, prediction, assessment, and treatment of skin tears. Subsequently, a new skin tear classification system was proposed. The system was then tested for interrater and intrarater reliability between the experts before being tested more widely on a sample of 327 individuals from the United States, Canada, and Europe. The results of the study indicated a substantial level of agreement for the expert panel (Fleiss κ = 0.619; 2-month follow-up = 0.653). Intrarater reliability was high (Cohen κ = 0.877). Interrater reliability was moderate (Fleiss κ = 0.555) for healthcare professionals (n = 303) and fair for non-health professionals (Fleiss κ = 0.338; n = 24). This international study established the reliability and validity of a new classification system for skin tears.

Antenna gain of actively compensated free-space optical communication systems under strong turbulence conditions.

PubMed

Juarez, Juan C; Brown, David M; Young, David W

2014-05-19

Current Strehl ratio models for actively compensated free-space optical communications terminals do not accurately predict system performance under strong turbulence conditions as they are based on weak turbulence theory. For evaluation of compensated systems, we present an approach for simulating the Strehl ratio with both low-order (tip/tilt) and higher-order (adaptive optics) correction. Our simulation results are then compared to the published models and their range of turbulence validity is assessed. Finally, we propose a new Strehl ratio model and antenna gain equation that are valid for general turbulence conditions independent of the degree of compensation.

myTREEHOUSE Self-Concept Assessment: preliminary psychometric analysis of a new self-concept assessment for children with cerebral palsy.

PubMed

Cheong, Sau Kuan; Lang, Cathryne P; Hemphill, Sheryl A; Johnston, Leanne M

2017-06-01

To evaluate the preliminary validity and reliability of the myTREEHOUSE Self-Concept Assessment for children with cerebral palsy (CP) aged 8 to 12 years. The myTREEHOUSE Self-Concept Assessment includes 26 items divided into eight domains, assessed across three Performance Perspectives (Personal, Social, and Perceived) and an additional Importance Rating. Face and content validity was assessed by semi-structured interviews with seven expert professionals regarding the assessment construct, content, and clinical utility. Reliability was assessed with 50 children aged 8 to 12 years with CP (29 males, 21 females; mean age 10y 2mo; Gross Motor Function Classification System [GMFCS] level I=35, II=8, III=5, IV=1; mean Wechsler Intelligence Scale for Children - Fourth Edition [WISC-IV]=104), whose data was used to calculate internal consistency of the scale, and a subset of 35 children (20 males, 15 females; mean age 10y 5mo; GMFCS level I=26, II=4, III=4, IV=1; mean WISC-IV=103) who participated in test-retest reliability within 14 to 28 days. Face and content validity was supported by positive expert feedback, with only minor adjustments suggested to clarify the wording of some items. After these amendments, strong internal consistency (Cronbach's α 0.84-0.91) and moderate to good test-retest reliability (intraclass correlation coefficient 0.64-0.75) was found for each component. The myTREEHOUSE Self-Concept Assessment is a valid and reliable assessment of self-concept for children with CP aged 8 to 12 years. © 2017 Mac Keith Press.

Comparability of the Social Skills Improvement System to the Social Skills Rating System: A Norwegian Study

ERIC Educational Resources Information Center

Gamst-Klaussen, Thor; Rasmussen, Lene-Mari P.; Svartdal, Frode; Strømgren, Børge

2016-01-01

The Social Skills Improvement System-Rating Scales (SSIS-RS) is a multi-informant instrument assessing social skills and problem behavior in children and adolescents. It is a revised version of the Social Skills Rating System (SSRS). A Norwegian translation of the SSRS has been validated, but this has not yet been done for the Norwegian…

Testing Physical Diagnosis Skills with Videotape

ERIC Educational Resources Information Center

Stillman, Paula L.; And Others

1977-01-01

An inexpensive videotape testing system has been developed at the Department of Pediatrics and Department of Medical TV-Cinematography at the University of Arizona College of Medicine. The development and validation of a test using this system to assess observational skills important for accurate physical diagnosis are described. (LBH)

Literature review of questionnaires assessing vertigo and dizziness, and their impact on patients' quality of life.

PubMed

Duracinsky, Martin; Mosnier, Isabelle; Bouccara, Didier; Sterkers, Olivier; Chassany, Olivier

2007-01-01

Vertigo and dizziness, which are major symptoms of diseases affecting the vestibular system, drastically impair patients' health-related quality of life (QoL). Patient's perspectives are thus essential to symptom assessment. We sought to make a critical review of published questionnaires measuring vertigo or dizziness, and/or their impact on QoL. Twenty-nine articles reporting the validation or use in clinical trials of vertigo- or dizziness-specific questionnaires were identified over the 1991-2004 period, and reviewed using a methodological and a Patient-Reported Outcomes specific checklist. Questionnaires were classified into three categories according to content: QoL (or handicap), mixed (assessing both symptoms and QoL), and symptom questionnaires. Four QoL, three mixed questionnaires, two symptoms, and one Meniere's disease-specific questionnaire were identified. QoL questionnaire validation was usually not complete. The structural validity of the Dizziness Handicap Inventory is not established, although this questionnaire is considered to be the reference questionnaire in the QoL domain. Moreover, QoL questionnaires were not very specific to vertigo or dizziness. Similarly, the Vertigo Handicap Questionnaire appeared to have the most pertinent content, but its validation remains to be completed. Mixed questionnaires have the same imperfections. The Vertigo, Dizziness, Imbalance (VDI) Questionnaire had the best validation score from the checklist, but its responsiveness appears to be weak. Regarding symptom questionnaires, the European Evaluation of Vertigo questionnaire evaluated the five major symptoms of vestibular syndrome satisfactorily. The present literature review failed to find any relevant and validated questionnaire assessing the impact of vertigo or dizziness on QoL.

An Approach to Comprehensive and Sustainable Solar Wind Model Validation

NASA Astrophysics Data System (ADS)

Rastaetter, L.; MacNeice, P. J.; Mays, M. L.; Boblitt, J. M.; Wiegand, C.

2017-12-01

The number of models of the corona and inner heliosphere and of their updates and upgrades grows steadily, as does the number and character of the model inputs. Maintaining up to date validation of these models, in the face of this constant model evolution, is a necessary but very labor intensive activity. In the last year alone, both NASA's LWS program and the CCMC's ongoing support of model forecasting activities at NOAA SWPC have sought model validation reports on the quality of all aspects of the community's coronal and heliospheric models, including both ambient and CME related wind solutions at L1. In this presentation I will give a brief review of the community's previous model validation results of L1 wind representation. I will discuss the semi-automated web based system we are constructing at the CCMC to present comparative visualizations of all interesting aspects of the solutions from competing models.This system is designed to be easily queried to provide the essential comprehensive inputs to repeat andupdate previous validation studies and support extensions to them. I will illustrate this by demonstrating how the system is being used to support the CCMC/LWS Model Assessment Forum teams focused on the ambient and time dependent corona and solar wind, including CME arrival time and IMF Bz.I will also discuss plans to extend the system to include results from the Forum teams addressing SEP model validation.

Assessing eGovernment Systems Success: A Validation of the DeLone and McLean Model of Information Systems Success

ERIC Educational Resources Information Center

Wang, Yi-Shun; Liao, Yi-Wen

2008-01-01

With the proliferation of the Internet and World Wide Web applications, people are increasingly interacting with government to citizen (G2C) eGovernment systems. It is therefore important to measure the success of G2C eGovernment systems from the citizen's perspective. While general information systems (IS) success models have received much…

Validation and Improvement of Reliability Methods for Air Force Building Systems

DTIC Science & Technology

focusing primarily on HVAC systems . This research used contingency analysis to assess the performance of each model for HVAC systems at six Air Force...probabilistic model produced inflated reliability calculations for HVAC systems . In light of these findings, this research employed a stochastic method, a...Nonhomogeneous Poisson Process (NHPP), in an attempt to produce accurate HVAC system reliability calculations. This effort ultimately concluded that

Undergraduate nursing students' perspectives on clinical assessment at transition to practice.

PubMed

Wu, Xi Vivien; Wang, Wenru; Pua, Lay Hoon; Heng, Doreen Gek Noi; Enskär, Karin

2015-01-01

Assessment of clinical competence requires explicitly defined standards meeting the national standards of the nursing profession. This is a complex process because of the diverse nature of nursing practice. To explore the perceptions of final-year undergraduate nursing students regarding clinical assessment at transition to practice. An exploratory qualitative approach was adopted. Twenty-four students participated in three focus group discussions. Thematic analysis was conducted. Five themes emerged: the need for a valid and reliable clinical assessment tool, the need for a flexible style of reflection and specific feedback, the dynamic clinical learning environment, students' efforts in learning and assessment, and the unclear support system for preceptors. Workload, time, resource availability, adequate preparation of preceptors, and the provision of valid and reliable clinical assessment tools were deemed to influence the quality of students' clinical learning and assessment. Nursing leadership in hospitals and educational institutions has a joint responsibility in shaping the clinical learning environment and providing clinical assessments for the students.

Systematic review of methods for quantifying teamwork in the operating theatre

PubMed Central

Marshall, D.; Sykes, M.; McCulloch, P.; Shalhoub, J.; Maruthappu, M.

2018-01-01

Background Teamwork in the operating theatre is becoming increasingly recognized as a major factor in clinical outcomes. Many tools have been developed to measure teamwork. Most fall into two categories: self‐assessment by theatre staff and assessment by observers. A critical and comparative analysis of the validity and reliability of these tools is lacking. Methods MEDLINE and Embase databases were searched following PRISMA guidelines. Content validity was assessed using measurements of inter‐rater agreement, predictive validity and multisite reliability, and interobserver reliability using statistical measures of inter‐rater agreement and reliability. Quantitative meta‐analysis was deemed unsuitable. Results Forty‐eight articles were selected for final inclusion; self‐assessment tools were used in 18 and observational tools in 28, and there were two qualitative studies. Self‐assessment of teamwork by profession varied with the profession of the assessor. The most robust self‐assessment tool was the Safety Attitudes Questionnaire (SAQ), although this failed to demonstrate multisite reliability. The most robust observational tool was the Non‐Technical Skills (NOTECHS) system, which demonstrated both test–retest reliability (P > 0·09) and interobserver reliability (Rwg = 0·96). Conclusion Self‐assessment of teamwork by the theatre team was influenced by professional differences. Observational tools, when used by trained observers, circumvented this.

Empirical evaluation of decision support systems: Needs, definitions, potential methods, and an example pertaining to waterfowl management

USGS Publications Warehouse

Sojda, R.S.

2007-01-01

Decision support systems are often not empirically evaluated, especially the underlying modelling components. This can be attributed to such systems necessarily being designed to handle complex and poorly structured problems and decision making. Nonetheless, evaluation is critical and should be focused on empirical testing whenever possible. Verification and validation, in combination, comprise such evaluation. Verification is ensuring that the system is internally complete, coherent, and logical from a modelling and programming perspective. Validation is examining whether the system is realistic and useful to the user or decision maker, and should answer the question: “Was the system successful at addressing its intended purpose?” A rich literature exists on verification and validation of expert systems and other artificial intelligence methods; however, no single evaluation methodology has emerged as preeminent. At least five approaches to validation are feasible. First, under some conditions, decision support system performance can be tested against a preselected gold standard. Second, real-time and historic data sets can be used for comparison with simulated output. Third, panels of experts can be judiciously used, but often are not an option in some ecological domains. Fourth, sensitivity analysis of system outputs in relation to inputs can be informative. Fifth, when validation of a complete system is impossible, examining major components can be substituted, recognizing the potential pitfalls. I provide an example of evaluation of a decision support system for trumpeter swan (Cygnus buccinator) management that I developed using interacting intelligent agents, expert systems, and a queuing system. Predicted swan distributions over a 13-year period were assessed against observed numbers. Population survey numbers and banding (ringing) studies may provide long term data useful in empirical evaluation of decision support.

The German Version of the Manchester Triage System and Its Quality Criteria – First Assessment of Validity and Reliability

PubMed Central

Gräff, Ingo; Goldschmidt, Bernd; Glien, Procula; Bogdanow, Manuela; Fimmers, Rolf; Hoeft, Andreas; Kim, Se-Chan; Grigutsch, Daniel

2014-01-01

Background The German Version of the Manchester Triage System (MTS) has found widespread use in EDs across German-speaking Europe. Studies about the quality criteria validity and reliability of the MTS currently only exist for the English-language version. Most importantly, the content of the German version differs from the English version with respect to presentation diagrams and change indicators, which have a significant impact on the category assigned. This investigation offers a preliminary assessment in terms of validity and inter-rater reliability of the German MTS. Methods Construct validity of assigned MTS level was assessed based on comparisons to hospitalization (general / intensive care), mortality, ED and hospital length of stay, level of prehospital care and number of invasive diagnostics. A sample of 45,469 patients was used. Inter-rater agreement between an expert and triage nurses (reliability) was calculated separately for a subset group of 167 emergency patients. Results For general hospital admission the area under the curve (AUC) of the receiver operating characteristic was 0.749; for admission to ICU it was 0.871. An examination of MTS-level and number of deceased patients showed that the higher the priority derived from MTS, the higher the number of deaths (p<0.0001 / χ2 Test). There was a substantial difference in the 30-day survival among the 5 MTS categories (p<0.0001 / log-rank test).The AUC for the predict 30-day mortality was 0.613. Categories orange and red had the highest numbers of heart catheter and endoscopy. Category red and orange were mostly accompanied by an emergency physician, whereas categories blue and green were walk-in patients. Inter-rater agreement between expert triage nurses was almost perfect (κ = 0.954). Conclusion The German version of the MTS is a reliable and valid instrument for a first assessment of emergency patients in the emergency department. PMID:24586477

Preliminary validation of 2 magnetic resonance image scoring systems for osteoarthritis of the hip according to the OMERACT filter.

PubMed

Maksymowych, Walter P; Cibere, Jolanda; Loeuille, Damien; Weber, Ulrich; Zubler, Veronika; Roemer, Frank W; Jaremko, Jacob L; Sayre, Eric C; Lambert, Robert G W

2014-02-01

Development of a validated magnetic resonance image (MRI) scoring system is essential in hip OA because radiographs are insensitive to change. We assessed the feasibility and reliability of 2 previously developed scoring methods: (1) the Hip Inflammation MRI Scoring System (HIMRISS) and (2) the Hip Osteoarthritis MRI Scoring System (HOAMS). Six readers (3 radiologists, 3 rheumatologists) participated in 2 reading exercises. In Reading Exercise 1, MRI of the hip of 20 subjects were read at a single time point followed by further standardization of methodology. In Reading Exercise 2, MRI of the hip of 18 subjects from a randomized controlled trial, assessed at 2 timepoints, and 27 subjects from a cross-sectional study were read for HIMRISS and HOAMS bone marrow lesions (BML) and synovitis. Reliability was assessed using intraclass correlation coefficient (ICC) and kappa statistics. Both methods were considered feasible. For Reading 1, HIMRISS ICC were 0.52, 0.61, 0.70, and 0.58 for femoral BML, acetabular BML, effusion, and total scores, respectively; and for HOAMS, summed BML and synovitis ICC were 0.52 and 0.46, respectively. For Reading 2, HIMRISS and HOAMS ICC for BML and synovitis-effusion improved substantially. Interobserver reliability for change scores was 0.81 and 0.71 for HIMRISS femoral and HOAMS summed BML, respectively. Responsiveness and discrimination was moderate to high for synovitis-effusion. Significant associations were noted between BML or synovitis scores and Western Ontario and McMaster Universities Osteoarthritis Index pain scores for baseline values (p ≤ 0.001). The BML and synovitis-effusion components of both HIMRISS and HOAMS scoring systems are feasible and reliable, and should be validated further.

Assessment of Intelligent Tutoring Systems Technologies and Opportunities (Evaluation et opportunites des technologies des systemes de tutorat intelligents)

DTIC Science & Technology

2018-01-01

His research designs adaptive systems for online content, by integrating research in psychology and education, human- ANNEX A − INTELLIGENT TUTORING...related scientific activities that include systems engineering, operational research and analysis, synthesis, integration and validation of knowledge...System Analysis and Studies Panel • SCI Systems Concepts and Integration Panel • SET Sensors and Electronics Technology Panel These Panels and Group

What are we assessing when we measure food security? A compendium and review of current metrics.

PubMed

Jones, Andrew D; Ngure, Francis M; Pelto, Gretel; Young, Sera L

2013-09-01

The appropriate measurement of food security is critical for targeting food and economic aid; supporting early famine warning and global monitoring systems; evaluating nutrition, health, and development programs; and informing government policy across many sectors. This important work is complicated by the multiple approaches and tools for assessing food security. In response, we have prepared a compendium and review of food security assessment tools in which we review issues of terminology, measurement, and validation. We begin by describing the evolving definition of food security and use this discussion to frame a review of the current landscape of measurement tools available for assessing food security. We critically assess the purpose/s of these tools, the domains of food security assessed by each, the conceptualizations of food security that underpin each metric, as well as the approaches that have been used to validate these metrics. Specifically, we describe measurement tools that 1) provide national-level estimates of food security, 2) inform global monitoring and early warning systems, 3) assess household food access and acquisition, and 4) measure food consumption and utilization. After describing a number of outstanding measurement challenges that might be addressed in future research, we conclude by offering suggestions to guide the selection of appropriate food security metrics.

What Are We Assessing When We Measure Food Security? A Compendium and Review of Current Metrics12

PubMed Central

Jones, Andrew D.; Ngure, Francis M.; Pelto, Gretel; Young, Sera L.

2013-01-01

The appropriate measurement of food security is critical for targeting food and economic aid; supporting early famine warning and global monitoring systems; evaluating nutrition, health, and development programs; and informing government policy across many sectors. This important work is complicated by the multiple approaches and tools for assessing food security. In response, we have prepared a compendium and review of food security assessment tools in which we review issues of terminology, measurement, and validation. We begin by describing the evolving definition of food security and use this discussion to frame a review of the current landscape of measurement tools available for assessing food security. We critically assess the purpose/s of these tools, the domains of food security assessed by each, the conceptualizations of food security that underpin each metric, as well as the approaches that have been used to validate these metrics. Specifically, we describe measurement tools that 1) provide national-level estimates of food security, 2) inform global monitoring and early warning systems, 3) assess household food access and acquisition, and 4) measure food consumption and utilization. After describing a number of outstanding measurement challenges that might be addressed in future research, we conclude by offering suggestions to guide the selection of appropriate food security metrics. PMID:24038241

Optimizing and Validating a Brief Assessment for Identifying Children of Service Members at Risk for Psychological Health Problems Following Parent Deployment

DTIC Science & Technology

2014-07-01

item measure as a measure of children’s interpersonal competence and social adjustment in the classroom . Preschool Social Competence Scale...Assessment System for Children (BASC) ADHD Monitor. Circle Pines, MN: American Guidance Service. Keane, T., Fairbank, J., Caddell, J., Zimering, R

Introducing the TAPS Pyramid Model

ERIC Educational Resources Information Center

Earle, Sarah

2015-01-01

The Teacher Assessment in Primary Science (TAPS) project is a three-year project based at Bath Spa University and funded by the Primary Science Teaching Trust (PSTT). It aims to develop support for a valid, reliable and manageable system of science assessment that will have a positive impact on children's learning. In this article, the author…

An Observational Assessment of Physical Activity Levels and Social Behaviour during Elementary School Recess

ERIC Educational Resources Information Center

Roberts, Simon J.; Fairclough, Stuart J.; Ridgers, Nicola D.; Porteous, Conor

2013-01-01

Objective: The purpose of the present study was to assess children's physical activity, social play behaviour, activity type and social interactions during elementary school recess using a pre-validated systematic observation system. Design: Cross-sectional. Setting: Two elementary schools located in Merseyside, England. Method: Fifty-six…

Finding One's Voice: The Pacesetter Model for More Equitable Assessment.

ERIC Educational Resources Information Center

Badger, Elizabeth

1996-01-01

Describes the College Board's Pacesetter Program, high school courses developed using principles of ongoing performance testing and portfolios, standards, and curriculum. The model is illustrated in a description of the Voices of Modern Culture language arts course. Argues that this assessment process has systemic validity and is more relevant to…

CRiSP: An Instrument for Assessing Student Perceptions of Classroom Response Systems

ERIC Educational Resources Information Center

Richardson, Alice M.; Dunn, Peter K.; McDonald, Christine; Oprescu, Florin

2015-01-01

This paper describes the development and validation of an instrument for evaluating classroom response systems (CRS). While a number of studies evaluating CRS have been published to date, no standardised instrument exists as a means of evaluating the impact of using the CRS. This means that comparing the different systems, or evaluating the…

Development of a Student Health Assessment System: Health Knowledge, Attitudes, and Behaviors in Middle-School Students. Research Report. ETS RR-10-04

ERIC Educational Resources Information Center

MacCann, Carolyn; Roberts, Richard D.

2010-01-01

Newly developed assessments of nutrition and exercise knowledge, attitudes, and behavior were administered to 383 eighth-graders. Evidence for the validity of assessment scores was evaluated with five findings. First, parent- and self-reported behaviors were similar and congruent for healthy eating and exercising but not for sedentary behaviors or…

Evidence of the Validity of "Teaching Strategies GOLD[R]" Assessment Tool for English Language Learners and Children with Disabilities

ERIC Educational Resources Information Center

Kim, Do-Hong; Lambert, Richard G.; Burts, Diane C.

2013-01-01

Research Findings: This study examined the measurement equivalence of the "Teaching Strategies GOLD[R]" assessment system across subgroups of children based on their primary language and disability status. This study is based on teacher-collected assessment data for 3-, 4-, and 5-year-old children for the fall of 2010, winter of 2010, and spring…

Safety Assurance Factors for Electronic Health Record Resilience (SAFER): study protocol

PubMed Central

2013-01-01

Background Implementation and use of electronic health records (EHRs) could lead to potential improvements in quality of care. However, the use of EHRs also introduces unique and often unexpected patient safety risks. Proactive assessment of risks and vulnerabilities can help address potential EHR-related safety hazards before harm occurs; however, current risk assessment methods are underdeveloped. The overall objective of this project is to develop and validate proactive assessment tools to ensure that EHR-enabled clinical work systems are safe and effective. Methods/Design This work is conceptually grounded in an 8-dimension model of safe and effective health information technology use. Our first aim is to develop self-assessment guides that can be used by health care institutions to evaluate certain high-risk components of their EHR-enabled clinical work systems. We will solicit input from subject matter experts and relevant stakeholders to develop guides focused on 9 specific risk areas and will subsequently pilot test the guides with individuals representative of likely users. The second aim will be to examine the utility of the self-assessment guides by beta testing the guides at selected facilities and conducting on-site evaluations. Our multidisciplinary team will use a variety of methods to assess the content validity and perceived usefulness of the guides, including interviews, naturalistic observations, and document analysis. The anticipated output of this work will be a series of self-administered EHR safety assessment guides with clear, actionable, checklist-type items. Discussion Proactive assessment of patient safety risks increases the resiliency of health care organizations to unanticipated hazards of EHR use. The resulting products and lessons learned from the development of the assessment guides are expected to be helpful to organizations that are beginning the EHR selection and implementation process as well as those that have already implemented EHRs. Findings from our project, currently underway, will inform future efforts to validate and implement tools that can be used by health care organizations to improve the safety of EHR-enabled clinical work systems. PMID:23587208

Validating workplace performance assessments in health sciences students: a case study from speech pathology.

PubMed

McAllister, Sue; Lincoln, Michelle; Ferguson, Allison; McAllister, Lindy

2013-01-01

Valid assessment of health science students' ability to perform in the real world of workplace practice is critical for promoting quality learning and ultimately certifying students as fit to enter the world of professional practice. Current practice in performance assessment in the health sciences field has been hampered by multiple issues regarding assessment content and process. Evidence for the validity of scores derived from assessment tools are usually evaluated against traditional validity categories with reliability evidence privileged over validity, resulting in the paradoxical effect of compromising the assessment validity and learning processes the assessments seek to promote. Furthermore, the dominant statistical approaches used to validate scores from these assessments fall under the umbrella of classical test theory approaches. This paper reports on the successful national development and validation of measures derived from an assessment of Australian speech pathology students' performance in the workplace. Validation of these measures considered each of Messick's interrelated validity evidence categories and included using evidence generated through Rasch analyses to support score interpretation and related action. This research demonstrated that it is possible to develop an assessment of real, complex, work based performance of speech pathology students, that generates valid measures without compromising the learning processes the assessment seeks to promote. The process described provides a model for other health professional education programs to trial.

Pediatric Heart Donor Assessment Tool (PH-DAT): A novel donor risk scoring system to predict 1-year mortality in pediatric heart transplantation.

PubMed

Zafar, Farhan; Jaquiss, Robert D; Almond, Christopher S; Lorts, Angela; Chin, Clifford; Rizwan, Raheel; Bryant, Roosevelt; Tweddell, James S; Morales, David L S

2018-03-01

In this study we sought to quantify hazards associated with various donor factors into a cumulative risk scoring system (the Pediatric Heart Donor Assessment Tool, or PH-DAT) to predict 1-year mortality after pediatric heart transplantation (PHT). PHT data with complete donor information (5,732) were randomly divided into a derivation cohort and a validation cohort (3:1). From the derivation cohort, donor-specific variables associated with 1-year mortality (exploratory p-value < 0.2) were incorporated into a multivariate logistic regression model. Scores were assigned to independent predictors (p < 0.05) based on relative odds ratios (ORs). The final model had an acceptable predictive value (c-statistic = 0.62). The significant 5 variables (ischemic time, stroke as the cause of death, donor-to-recipient height ratio, donor left ventricular ejection fraction, glomerular filtration rate) were used for the scoring system. The validation cohort demonstrated a strong correlation between the observed and expected rates of 1-year mortality (r = 0.87). The risk of 1-year mortality increases by 11% (OR 1.11 [1.08 to 1.14]; p < 0.001) in the derivation cohort and 9% (OR 1.09 [1.04 to 1.14]; p = 0.001) in the validation cohort with an increase of 1-point in score. Mortality risk increased 5 times from the lowest to the highest donor score in this cohort. Based on this model, a donor score range of 10 to 28 predicted 1-year recipient mortality of 11% to 31%. This novel pediatric-specific, donor risk scoring system appears capable of predicting post-transplant mortality. Although the PH-DAT may benefit organ allocation and assessment of recipient risk while controlling for donor risk, prospective validation of this model is warranted. Copyright © 2018 International Society for the Heart and Lung Transplantation. Published by Elsevier Inc. All rights reserved.

Definition and initial validation of a Lupus Low Disease Activity State (LLDAS).

PubMed

Franklyn, Kate; Lau, Chak Sing; Navarra, Sandra V; Louthrenoo, Worawit; Lateef, Aisha; Hamijoyo, Laniyati; Wahono, C Singgih; Chen, Shun Le; Jin, Ou; Morton, Susan; Hoi, Alberta; Huq, Molla; Nikpour, Mandana; Morand, Eric F

2016-09-01

Treating to low disease activity is routine in rheumatoid arthritis, but no comparable goal has been defined for systemic lupus erythematosus (SLE). We sought to define and validate a Lupus Low Disease Activity State (LLDAS). A consensus definition of LLDAS was generated using Delphi and nominal group techniques. Criterion validity was determined by measuring the ability of LLDAS attainment, in a single-centre SLE cohort, to predict non-accrual of irreversible organ damage, measured using the Systemic Lupus International Collaborating Clinics Damage Index (SDI). Consensus methodology led to the following definition of LLDAS: (1) SLE Disease Activity Index (SLEDAI)-2K ≤4, with no activity in major organ systems (renal, central nervous system (CNS), cardiopulmonary, vasculitis, fever) and no haemolytic anaemia or gastrointestinal activity; (2) no new lupus disease activity compared with the previous assessment; (3) a Safety of Estrogens in Lupus Erythematosus National Assessment (SELENA)-SLEDAI physician global assessment (scale 0-3) ≤1; (4) a current prednisolone (or equivalent) dose ≤7.5 mg daily; and (5) well tolerated standard maintenance doses of immunosuppressive drugs and approved biological agents. Achievement of LLDAS was determined in 191 patients followed for a mean of 3.9 years. Patients who spent greater than 50% of their observed time in LLDAS had significantly reduced organ damage accrual compared with patients who spent less than 50% of their time in LLDAS (p=0.0007) and were significantly less likely to have an increase in SDI of ≥1 (relative risk 0.47, 95% CI 0.28 to 0.79, p=0.005). A definition of LLDAS has been generated, and preliminary validation demonstrates its attainment to be associated with improved outcomes in SLE. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

Predicting the need for massive transfusion in trauma patients: the Traumatic Bleeding Severity Score.

PubMed

Ogura, Takayuki; Nakamura, Yoshihiko; Nakano, Minoru; Izawa, Yoshimitsu; Nakamura, Mitsunobu; Fujizuka, Kenji; Suzukawa, Masayuki; Lefor, Alan T

2014-05-01

The ability to easily predict the need for massive transfusion may improve the process of care, allowing early mobilization of resources. There are currently no clear criteria to activate massive transfusion in severely injured trauma patients. The aims of this study were to create a scoring system to predict the need for massive transfusion and then to validate this scoring system. We reviewed the records of 119 severely injured trauma patients and identified massive transfusion predictors using statistical methods. Each predictor was converted into a simple score based on the odds ratio in a multivariate logistic regression analysis. The Traumatic Bleeding Severity Score (TBSS) was defined as the sum of the component scores. The predictive value of the TBSS for massive transfusion was then validated, using data from 113 severely injured trauma patients. Receiver operating characteristic curve analysis was performed to compare the results of TBSS with the Trauma-Associated Severe Hemorrhage score and the Assessment of Blood Consumption score. In the development phase, five predictors of massive transfusion were identified, including age, systolic blood pressure, the Focused Assessment with Sonography for Trauma scan, severity of pelvic fracture, and lactate level. The maximum TBSS is 57 points. In the validation study, the average TBSS in patients who received massive transfusion was significantly greater (24.2 [6.7]) than the score of patients who did not (6.2 [4.7]) (p < 0.01). The area under the receiver operating characteristic curve, sensitivity, and specificity for a TBSS greater than 15 points was 0.985 (significantly higher than the other scoring systems evaluated at 0.892 and 0.813, respectively), 97.4%, and 96.2%, respectively. The TBSS is simple to calculate using an available iOS application and is accurate in predicting the need for massive transfusion. Additional multicenter studies are needed to further validate this scoring system and further assess its utility. Prognostic study, level III.

DOT&E

Science.gov Websites

of Defense on operational and live fire test and evaluation of Department of Defense weapon systems Guidance on the Validation of Models and Simulation used in Operational Test and Live Fire Assessments has

Probabilistic Approaches for Multi-Hazard Risk Assessment of Structures and Systems

NASA Astrophysics Data System (ADS)

Kwag, Shinyoung

Performance assessment of structures, systems, and components for multi-hazard scenarios has received significant attention in recent years. However, the concept of multi-hazard analysis is quite broad in nature and the focus of existing literature varies across a wide range of problems. In some cases, such studies focus on hazards that either occur simultaneously or are closely correlated with each other. For example, seismically induced flooding or seismically induced fires. In other cases, multi-hazard studies relate to hazards that are not dependent or correlated but have strong likelihood of occurrence at different times during the lifetime of a structure. The current approaches for risk assessment need enhancement to account for multi-hazard risks. It must be able to account for uncertainty propagation in a systems-level analysis, consider correlation among events or failure modes, and allow integration of newly available information from continually evolving simulation models, experimental observations, and field measurements. This dissertation presents a detailed study that proposes enhancements by incorporating Bayesian networks and Bayesian updating within a performance-based probabilistic framework. The performance-based framework allows propagation of risk as well as uncertainties in the risk estimates within a systems analysis. Unlike conventional risk assessment techniques such as a fault-tree analysis, a Bayesian network can account for statistical dependencies and correlations among events/hazards. The proposed approach is extended to develop a risk-informed framework for quantitative validation and verification of high fidelity system-level simulation tools. Validation of such simulations can be quite formidable within the context of a multi-hazard risk assessment in nuclear power plants. The efficiency of this approach lies in identification of critical events, components, and systems that contribute to the overall risk. Validation of any event or component on the critical path is relatively more important in a risk-informed environment. Significance of multi-hazard risk is also illustrated for uncorrelated hazards of earthquakes and high winds which may result in competing design objectives. It is also illustrated that the number of computationally intensive nonlinear simulations needed in performance-based risk assessment for external hazards can be significantly reduced by using the power of Bayesian updating in conjunction with the concept of equivalent limit-state.

Reliability and Validity Evidence of Multiple Balance Assessments in Athletes With a Concussion

PubMed Central

Murray, Nicholas; Salvatore, Anthony; Powell, Douglas; Reed-Jones, Rebecca

2014-01-01

Context: An estimated 300 000 sport-related concussion injuries occur in the United States annually. Approximately 30% of individuals with concussions experience balance disturbances. Common methods of balance assessment include the Clinical Test of Sensory Organization and Balance (CTSIB), the Sensory Organization Test (SOT), the Balance Error Scoring System (BESS), and the Romberg test; however, the National Collegiate Athletic Association recommended the Wii Fit as an alternative measure of balance in athletes with a concussion. A central concern regarding the implementation of the Wii Fit is whether it is reliable and valid for measuring balance disturbance in athletes with concussion. Objective: To examine the reliability and validity evidence for the CTSIB, SOT, BESS, Romberg test, and Wii Fit for detecting balance disturbance in athletes with a concussion. Data Sources: Literature considered for review included publications with reliability and validity data for the assessments of balance (CTSIB, SOT, BESS, Romberg test, and Wii Fit) from PubMed, PsycINFO, and CINAHL. Data Extraction: We identified 63 relevant articles for consideration in the review. Of the 63 articles, 28 were considered appropriate for inclusion and 35 were excluded. Data Synthesis: No current reliability or validity information supports the use of the CTSIB, SOT, Romberg test, or Wii Fit for balance assessment in athletes with a concussion. The BESS demonstrated moderate to high reliability (interclass correlation coefficient = 0.87) and low to moderate validity (sensitivity = 34%, specificity = 87%). However, the Romberg test and Wii Fit have been shown to be reliable tools in the assessment of balance in Parkinson patients. Conclusions: The BESS can evaluate balance problems after a concussion. However, it lacks the ability to detect balance problems after the third day of recovery. Further investigation is needed to establish the use of the CTSIB, SOT, Romberg test, and Wii Fit for assessing balance in athletes with concussions. PMID:24933431

The Effects of Differing Response Criteria on the Assessment of Writing Competence.

ERIC Educational Resources Information Center

Winters, Lynn

The purpose of this study was to investigate the relative validities of four essay scoring systems, reflecting alternative conceptualizations of the writing process, for identifying "competent" writers. Each rater was trained in two of the four scoring systems: General Impression Scoring (GI), Diederich Expository Scale (DES), CSE…

Simulation of Climate Change Impacts on Wheat-Fallow Cropping Systems

USDA-ARS?s Scientific Manuscript database

Agricultural system simulation models are predictive tools for assessing climate change impacts on crop production. In this study, RZWQM2 that contains the DSSAT 4.0-CERES model was evaluated for simulating climate change impacts on wheat growth. The model was calibrated and validated using data fro...

Assessing wheat yield, Biomass, and water productivity responses to growth stage based irrigation water allocation

USDA-ARS?s Scientific Manuscript database

Increasing irrigated wheat yields is important to the overall profitability of limited-irrigation cropping systems in western Kansas. A simulation study was conducted to (1) validate APSIM's (Agricultural Production Systems sIMulator) ability to simulate wheat growth and yield in Kansas, and (2) app...

Sensor validation and fusion for gas turbine vibration monitoring

NASA Astrophysics Data System (ADS)

Yan, Weizhong; Goebel, Kai F.

2003-08-01

Vibration monitoring is an important practice throughout regular operation of gas turbine power systems and, even more so, during characterization tests. Vibration monitoring relies on accurate and reliable sensor readings. To obtain accurate readings, sensors are placed such that the signal is maximized. In the case of characterization tests, strain gauges are placed at the location of vibration modes on blades inside the gas turbine. Due to the prevailing harsh environment, these sensors have a limited life and decaying accuracy, both of which impair vibration assessment. At the same time bandwidth limitations may restrict data transmission, which in turn limits the number of sensors that can be used for assessment. Knowing the sensor status (normal or faulty), and more importantly, knowing the true vibration level of the system all the time is essential for successful gas turbine vibration monitoring. This paper investigates a dynamic sensor validation and system health reasoning scheme that addresses the issues outlined above by considering only the information required to reliably assess system health status. In particular, if abnormal system health is suspected or if the primary sensor is determined to be faulted, information from available "sibling" sensors is dynamically integrated. A confidence expresses the complex interactions of sensor health and system health, their reliabilities, conflicting information, and what the health assessment is. Effectiveness of the scheme in achieving accurate and reliable vibration evaluation is then demonstrated using a combination of simulated data and a small sample of a real-world application data where the vibration of compressor blades during a real time characterization test of a new gas turbine power system is monitored.

Worldwide Protein Data Bank validation information: usage and trends.

PubMed

Smart, Oliver S; Horský, Vladimír; Gore, Swanand; Svobodová Vařeková, Radka; Bendová, Veronika; Kleywegt, Gerard J; Velankar, Sameer

2018-03-01

Realising the importance of assessing the quality of the biomolecular structures deposited in the Protein Data Bank (PDB), the Worldwide Protein Data Bank (wwPDB) partners established Validation Task Forces to obtain advice on the methods and standards to be used to validate structures determined by X-ray crystallography, nuclear magnetic resonance spectroscopy and three-dimensional electron cryo-microscopy. The resulting wwPDB validation pipeline is an integral part of the wwPDB OneDep deposition, biocuration and validation system. The wwPDB Validation Service webserver (https://validate.wwpdb.org) can be used to perform checks prior to deposition. Here, it is shown how validation metrics can be combined to produce an overall score that allows the ranking of macromolecular structures and domains in search results. The ValTrends DB database provides users with a convenient way to access and analyse validation information and other properties of X-ray crystal structures in the PDB, including investigating trends in and correlations between different structure properties and validation metrics.

Worldwide Protein Data Bank validation information: usage and trends

PubMed Central

Horský, Vladimír; Gore, Swanand; Svobodová Vařeková, Radka; Bendová, Veronika

2018-01-01

Realising the importance of assessing the quality of the biomolecular structures deposited in the Protein Data Bank (PDB), the Worldwide Protein Data Bank (wwPDB) partners established Validation Task Forces to obtain advice on the methods and standards to be used to validate structures determined by X-ray crystallography, nuclear magnetic resonance spectroscopy and three-dimensional electron cryo-microscopy. The resulting wwPDB validation pipeline is an integral part of the wwPDB OneDep deposition, biocuration and validation system. The wwPDB Validation Service webserver (https://validate.wwpdb.org) can be used to perform checks prior to deposition. Here, it is shown how validation metrics can be combined to produce an overall score that allows the ranking of macromolecular structures and domains in search results. The ValTrendsDB database provides users with a convenient way to access and analyse validation information and other properties of X-ray crystal structures in the PDB, including investigating trends in and correlations between different structure properties and validation metrics. PMID:29533231

Rapid stepping test towards virtual visual objects: Feasibility and convergent validity in older adults.

PubMed

Hutzler, Yeshayahu; Korsensky, Olga; Laufer, Yocheved

2017-01-01

Rapid voluntary stepping has been recognized as an important measure of balance control. The purpose of this study was to assess the feasibility and convergent validity of a Rapid Stepping Test protocol utilizing a virtual reality SeeMeTM system (VR-RST) in elderly ambulatory and independent individuals living in a community residential home. Associations between step execution times determined by the system and the Activities-specific Balance Confidence (ABC) Questionnaire, and clinical measures of balance performance in the MiniBESTest and Timed Up and Go (TUG) test, were established in 60 participants (mean age 88.2 ± 5.0 years). All participants completed the study. The correlations of the ABC questionnaire and the clinical tests with VR-RST forward and backward stepping were moderate (ρ rage 0.42-0.52), and weak to moderate with sideward stepping (ρ rage 0.32-0.52). Moderate to strong correlations were found across stepping directions (ρ rage 0.45-0.87). Findings support the test's feasibility and validity and confirm the utility of the VR-RST as an assessment tool in an elderly population.

The French version of the Juvenile Arthritis Multidimensional Assessment Report (JAMAR).

PubMed

Quartier, Pierre; Hofer, Michael; Wouters, Carine; Truong, Thi Thanh Thao; Duong, Ngoc-Phoi; Agbo-Kpati, Kokou-Placide; Uettwiller, Florence; Melki, Isabelle; Mouy, Richard; Bader-Meunier, Brigitte; Consolaro, Alessandro; Bovis, Francesca; Ruperto, Nicolino

2018-04-01

The Juvenile Arthritis Multidimensional Assessment Report (JAMAR) is a new parent/patient reported outcome measure that enables a thorough assessment of the disease status in children with juvenile idiopathic arthritis (JIA). We report the results of the cross-cultural adaptation and validation of the parent and patient versions of the JAMAR in the French language. The reading comprehension of the questionnaire was tested in 10 JIA parents and patients. Each participating centre was asked to collect demographic, clinical data and the JAMAR in 100 consecutive JIA patients or all consecutive patients seen in a 6-month period and to administer the JAMAR to 100 healthy children and their parents. The statistical validation phase explored descriptive statistics and the psychometric issues of the JAMAR: the three Likert assumptions, floor/ceiling effects, internal consistency, Cronbach's alpha, interscale correlations and construct validity (convergent and discriminant validity). A total of 100 JIA patients (23% systemic, 45% oligoarticular, 20% RF negative polyarthritis, 12% other categories) and 122 healthy children, were enrolled at the paediatric rheumatology centre of the Necker Children's Hospital in Paris. Notably, none of the enrolled JIA patients is affected with psoriatic arthritis. The JAMAR components discriminated well healthy subjects from JIA patients. All JAMAR components revealed good psychometric performances. In conclusion, the French version of the JAMAR is a valid tool for the assessment of children with JIA and is suitable for use both in routine clinical practice and clinical research.

The Italian version of the Juvenile Arthritis Multidimensional Assessment Report (JAMAR).

PubMed

Consolaro, Alessandro; Bovis, Francesca; Pistorio, Angela; Cimaz, Rolando; De Benedetti, Fabrizio; Miniaci, Angela; Corona, Fabrizia; Gerloni, Valeria; Martino, Silvana; Pastore, Serena; Barone, Patrizia; Pieropan, Sara; Cortis, Elisabetta; Podda, Rosa Anna; Gallizzi, Romina; Civino, Adele; Torre, Francesco La; Rigante, Donato; Consolini, Rita; Maggio, Maria Cristina; Magni-Manzoni, Silvia; Perfetti, Francesca; Filocamo, Giovanni; Toppino, Claudia; Licciardi, Francesco; Garrone, Marco; Scala, Silvia; Patrone, Elisa; Tonelli, Monica; Tani, Daniela; Ravelli, Angelo; Martini, Alberto; Ruperto, Nicolino

2018-04-01

The Juvenile Arthritis Multidimensional Assessment Report (JAMAR) is a new parent/patient reported outcome measure that enables a thorough assessment of the disease status in children with juvenile idiopathic arthritis (JIA). We report the results of the cross-cultural adaptation and validation of the parent and patient versions of the JAMAR in the Italian language.The reading comprehension of the questionnaire was tested in 10 JIA parents and patients. Each participating centre was asked to collect demographic, clinical data and the JAMAR in 100 consecutive JIA patients or all consecutive patients seen in a 6-month period and to administer the JAMAR to 100 healthy children and their parents.The statistical validation phase explored descriptive statistics and the psychometric issues of the JAMAR: the 3 Likert assumptions, floor/ceiling effects, internal consistency, Cronbach's alpha, interscale correlations, test-retest reliability, and construct validity (convergent and discriminant validity).A total of 1296 JIA patients (7.2% systemic, 59.5% oligoarticular, 21.4% RF negative polyarthritis, 11.9% other categories) and 100 healthy children, were enrolled in 18 centres. The JAMAR components discriminated well healthy subjects from JIA patients except for the Health Related Quality of Life (HRQoL) Psychosocial Health (PsH) subscales. All JAMAR components revealed good psychometric performances.In conclusion, the Italian version of the JAMAR is a valid tool for the assessment of children with JIA and is suitable for use both in routine clinical practice and clinical research.

The Paraguayan Spanish version of the Juvenile Arthritis Multidimensional Assessment Report (JAMAR).

PubMed

Morel Ayala, Zoilo; Burgos-Vargas, Ruben; Consolaro, Alessandro; Bovis, Francesca; Ruperto, Nicolino

2018-04-01

The Juvenile Arthritis Multidimensional Assessment Report (JAMAR) is a new parent/patient reported outcome measure that enables a thorough assessment of the disease status in children with juvenile idiopathic arthritis (JIA). We report the results of the cross-cultural adaptation and validation of the parent and patient versions of the JAMAR in the Paraguayan Spanish language. The reading comprehension of the questionnaire was tested in 10 JIA parents and patients. Each participating centre was asked to collect demographic, clinical data and the JAMAR in 100 consecutive JIA patients or all consecutive patients seen in a 6-month period and to administer the JAMAR to 100 healthy children and their parents. The statistical validation phase explored descriptive statistics and the psychometric issues of the JAMAR: the 3 Likert assumptions, floor/ceiling effects, internal consistency, Cronbach's alpha, interscale correlations, and construct validity (convergent and discriminant validity). A total of 51 JIA patients (2% systemic, 27.4% oligoarticular, 37.2% RF negative polyarthritis, 33.4% other categories) and 100 healthy children, were enrolled. The JAMAR components discriminated well healthy subjects from JIA patients. Notably, there was no significant difference between healthy subjects and their affected peers in the school-related problem variable. All JAMAR components revealed good psychometric performances. In conclusion, the Paraguayan Spanish version of the JAMAR is a valid tool for the assessment of children with JIA and is suitable for use both in routine clinical practice and clinical research.

Reliability and Validity of the Footprint Assessment Method Using Photoshop CS5 Software.

PubMed

Gutiérrez-Vilahú, Lourdes; Massó-Ortigosa, Núria; Costa-Tutusaus, Lluís; Guerra-Balic, Myriam

2015-05-01

Several sophisticated methods of footprint analysis currently exist. However, it is sometimes useful to apply standard measurement methods of recognized evidence with an easy and quick application. We sought to assess the reliability and validity of a new method of footprint assessment in a healthy population using Photoshop CS5 software (Adobe Systems Inc, San Jose, California). Forty-two footprints, corresponding to 21 healthy individuals (11 men with a mean ± SD age of 20.45 ± 2.16 years and 10 women with a mean ± SD age of 20.00 ± 1.70 years) were analyzed. Footprints were recorded in static bipedal standing position using optical podography and digital photography. Three trials for each participant were performed. The Hernández-Corvo, Chippaux-Smirak, and Staheli indices and the Clarke angle were calculated by manual method and by computerized method using Photoshop CS5 software. Test-retest was used to determine reliability. Validity was obtained by intraclass correlation coefficient (ICC). The reliability test for all of the indices showed high values (ICC, 0.98-0.99). Moreover, the validity test clearly showed no difference between techniques (ICC, 0.99-1). The reliability and validity of a method to measure, assess, and record the podometric indices using Photoshop CS5 software has been demonstrated. This provides a quick and accurate tool useful for the digital recording of morphostatic foot study parameters and their control.

Developing and validating a nutrition knowledge questionnaire: key methods and considerations.

PubMed

Trakman, Gina Louise; Forsyth, Adrienne; Hoye, Russell; Belski, Regina

2017-10-01

To outline key statistical considerations and detailed methodologies for the development and evaluation of a valid and reliable nutrition knowledge questionnaire. Literature on questionnaire development in a range of fields was reviewed and a set of evidence-based guidelines specific to the creation of a nutrition knowledge questionnaire have been developed. The recommendations describe key qualitative methods and statistical considerations, and include relevant examples from previous papers and existing nutrition knowledge questionnaires. Where details have been omitted for the sake of brevity, the reader has been directed to suitable references. We recommend an eight-step methodology for nutrition knowledge questionnaire development as follows: (i) definition of the construct and development of a test plan; (ii) generation of the item pool; (iii) choice of the scoring system and response format; (iv) assessment of content validity; (v) assessment of face validity; (vi) purification of the scale using item analysis, including item characteristics, difficulty and discrimination; (vii) evaluation of the scale including its factor structure and internal reliability, or Rasch analysis, including assessment of dimensionality and internal reliability; and (viii) gathering of data to re-examine the questionnaire's properties, assess temporal stability and confirm construct validity. Several of these methods have previously been overlooked. The measurement of nutrition knowledge is an important consideration for individuals working in the nutrition field. Improved methods in the development of nutrition knowledge questionnaires, such as the use of factor analysis or Rasch analysis, will enable more confidence in reported measures of nutrition knowledge.

Above, Beyond, and Over the Side rails: Evaluating the New Memorial Emergency Department Fall-Risk-Assessment Tool.

PubMed

Scott, Robin A; Oman, Kathleen S; Flarity, Kathleen; Comer, Jennifer L

2018-03-06

Patient falls are a significant issue in hospitalized patients and financially costly to hospitals. The Joint Commission requires that patients be assessed for fall risk and interventions in place to mitigate the risk of falls. It is imperative to have a patient population/setting specific fall risk assessment tool to identify patients at risk for falling. The purpose of this study was to evaluate the reliability and validity of the 2013 Memorial ED Fall Risk Assessment tool (MEDFRAT) specifically designed for the ED population. A two-phase prospective design was used for this study. Phase one determined the interrater reliability of the MEDFRAT. Phase two assessed the validity of the MEDFRAT in an emergency department (ED) within a 600-bed academic/teaching institution; Level II Trauma Center with >100,000 annual patient visits. The Memorial ED Fall Risk Assessment Tool was validated in this ED setting. The tool demonstrated positive interrater reliability (k=0.701) and when implemented with a falls prevention strategy and staff education demonstrated a 48% decrease in ED fall rate (0.57 falls/1000 patient visits) post implementation during the study period. The MEDFRAT, an evidenced based ED-specific fall risk tool was implemented on the basis of the risk factors consistently identified in the literature: prior fall history, impaired mobility, altered mental status, altered elimination, and the use of sedative medication. The Memorial ED Fall Risk Assessment Tool demonstrated to be a valid tool for this hospital system. Copyright © 2018 Emergency Nurses Association. Published by Elsevier Inc. All rights reserved.

Measurement properties of existing clinical assessment methods evaluating scapular positioning and function. A systematic review.

PubMed

Larsen, Camilla Marie; Juul-Kristensen, Birgit; Lund, Hans; Søgaard, Karen

2014-10-01

The aims were to compile a schematic overview of clinical scapular assessment methods and critically appraise the methodological quality of the involved studies. A systematic, computer-assisted literature search using Medline, CINAHL, SportDiscus and EMBASE was performed from inception to October 2013. Reference lists in articles were also screened for publications. From 50 articles, 54 method names were identified and categorized into three groups: (1) Static positioning assessment (n = 19); (2) Semi-dynamic (n = 13); and (3) Dynamic functional assessment (n = 22). Fifteen studies were excluded for evaluation due to no/few clinimetric results, leaving 35 studies for evaluation. Graded according to the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN checklist), the methodological quality in the reliability and validity domains was "fair" (57%) to "poor" (43%), with only one study rated as "good". The reliability domain was most often investigated. Few of the assessment methods in the included studies that had "fair" or "good" measurement property ratings demonstrated acceptable results for both reliability and validity. We found a substantially larger number of clinical scapular assessment methods than previously reported. Using the COSMIN checklist the methodological quality of the included measurement properties in the reliability and validity domains were in general "fair" to "poor". None were examined for all three domains: (1) reliability; (2) validity; and (3) responsiveness. Observational evaluation systems and assessment of scapular upward rotation seem suitably evidence-based for clinical use. Future studies should test and improve the clinimetric properties, and especially diagnostic accuracy and responsiveness, to increase utility for clinical practice.

Strategic Defense Initiative Demonstration/Validation Program Environmental Assessment. Exoatmospheric Reentry Vehicle Interception System (ERIS),

DTIC Science & Technology

1987-08-01

proceed to Demonstration/Validation for ERIS vould not preclude other technologies, nor vould it mandate the eventual Full-Scale Development or Production ...Full-Scale Development, and Production /Deployment. These four stages are separated by three major decision points (Milestones I, II, and III). Prior...percent facility population increase would require increased power plant gener- ating capacity. One concern is the nitrogen oxide emissions which is

easyCBM Beginning Reading Measures: Grades K-1 Alternate Form Reliability and Criterion Validity with the SAT-10. Technical Report #1403

ERIC Educational Resources Information Center

Wray, Kraig; Lai, Cheng-Fei; Sáez, Leilani; Alonzo, Julie; Tindal, Gerald

2013-01-01

We report the results of an alternate form reliability and criterion validity study of kindergarten and grade 1 (N = 84-199) reading measures from the easyCBM© assessment system and Stanford Early School Achievement Test/Stanford Achievement Test, 10th edition (SESAT/SAT-­10) across 5 time points. The alternate form reliabilities ranged from…

Validation and evaluation of the advanced aeronautical CFD system SAUNA: A method developer's view

NASA Astrophysics Data System (ADS)

Shaw, J. A.; Peace, A. J.; Georgala, J. M.; Childs, P. N.

1993-09-01

This paper is concerned with a detailed validation and evaluation of the SAUNA CFD system for complex aircraft configurations. The methodology of the complete system is described in brief, including its unique use of differing grid generation strategies (structured, unstructured or both) depending on the geometric complexity of the configuration. A wide range of configurations and flow conditions are chosen in the validation and evaluation exercise to demonstrate the scope of SAUNA. A detailed description of the results from the method is preceded by a discussion on the philosophy behind the strategy followed in the exercise, in terms of equality assessment and the differing roles of the code developer and the code user. It is considered that SAUNA has grown into a highly usable tool for the aircraft designer, in combining flexibility and accuracy in an efficient manner.

The Chinese version of monitoring and evaluation system strengthening tool for human immunodeficiency virus (HIV) capacity building: Development and evaluation.

PubMed

Zhao, Ran; Chen, Ren; Zhang, Bing; Ma, Ying; Qin, Xia; Hu, Zhi

2015-08-01

Monitoring and evaluation (M&E) for human immunodeficiency virus (HIV) capacity building has become a significant step for HIV prevention and control. The M&E system strengthening tool published by the United Nations Joint Programme on HIV/AIDS (UNAIDS) was intended to be the most authoritative assessment tool internationally. Facing the fact that the M&E system in China did not function at an optimum level, we considered taking the international standards for reference. By linguistic validating and different stages' discussions and revisions, we came up with the Chinese version of the capacity diagnosis tool with at least 12 components and tested its validity and reliability. The tool turned out to have a sufficiently linguistic validation and proved to be a scientific and feasible instrument which was suitable for China's national conditions.

Development and validation of the positive affect and well-being scale for the neurology quality of life (Neuro-QOL) measurement system.

PubMed

Salsman, John M; Victorson, David; Choi, Seung W; Peterman, Amy H; Heinemann, Allen W; Nowinski, Cindy; Cella, David

2013-11-01

To develop and validate an item-response theory-based patient-reported outcomes assessment tool of positive affect and well-being (PAW). This is part of a larger NINDS-funded study to develop a health-related quality of life measurement system across major neurological disorders, called Neuro-QOL. Informed by a literature review and qualitative input from clinicians and patients, item pools were created to assess PAW concepts. Items were administered to a general population sample (N = 513) and a group of individuals with a variety of neurologic conditions (N = 581) for calibration and validation purposes, respectively. A 23-item calibrated bank and a 9-item short form of PAW was developed, reflecting components of positive affect, life satisfaction, or an overall sense of purpose and meaning. The Neuro-QOL PAW measure demonstrated sufficient unidimensionality and displayed good internal consistency, test-retest reliability, model fit, convergent and discriminant validity, and responsiveness. The Neuro-QOL PAW measure was designed to aid clinicians and researchers to better evaluate and understand the potential role of positive health processes for individuals with chronic neurological conditions. Further psychometric testing within and between neurological conditions, as well as testing in non-neurologic chronic diseases, will help evaluate the generalizability of this new tool.

PRA (Probabilistic Risk Assessments) Participation versus Validation

NASA Technical Reports Server (NTRS)

DeMott, Diana; Banke, Richard

2013-01-01

Probabilistic Risk Assessments (PRAs) are performed for projects or programs where the consequences of failure are highly undesirable. PRAs primarily address the level of risk those projects or programs posed during operations. PRAs are often developed after the design has been completed. Design and operational details used to develop models include approved and accepted design information regarding equipment, components, systems and failure data. This methodology basically validates the risk parameters of the project or system design. For high risk or high dollar projects, using PRA methodologies during the design process provides new opportunities to influence the design early in the project life cycle to identify, eliminate or mitigate potential risks. Identifying risk drivers before the design has been set allows the design engineers to understand the inherent risk of their current design and consider potential risk mitigation changes. This can become an iterative process where the PRA model can be used to determine if the mitigation technique is effective in reducing risk. This can result in more efficient and cost effective design changes. PRA methodology can be used to assess the risk of design alternatives and can demonstrate how major design changes or program modifications impact the overall program or project risk. PRA has been used for the last two decades to validate risk predictions and acceptability. Providing risk information which can positively influence final system and equipment design the PRA tool can also participate in design development, providing a safe and cost effective product.

[Validation of the IBS-SSS].

PubMed

Betz, C; Mannsdörfer, K; Bischoff, S C

2013-10-01

Irritable bowel syndrome (IBS) is a functional gastrointestinal disorder characterised by abdominal pain, associated with stool abnormalities and changes in stool consistency. Diagnosis of IBS is based on characteristic symptoms and exclusion of other gastrointestinal diseases. A number of questionnaires exist to assist diagnosis and assessment of severity of the disease. One of these is the irritable bowel syndrome - severity scoring system (IBS-SSS). The IBS-SSS was validated 1997 in its English version. In the present study, the IBS-SSS has been validated in German language. To do this, a cohort of 60 patients with IBS according to the Rome III criteria, was compared with a control group of healthy individuals (n = 38). We studied sensitivity and reproducibility of the score, as well as the sensitivity to detect changes of symptom severity. The results of the German validation largely reflect the results of the English validation. The German version of the IBS-SSS is also a valid, meaningful and reproducible questionnaire with a high sensitivity to assess changes in symptom severity, especially in IBS patients with moderate symptoms. It is unclear if the IBS-SSS is also a valid questionnaire in IBS patients with severe symptoms because this group of patients was not studied. © Georg Thieme Verlag KG Stuttgart · New York.

Portable recording in the assessment of obstructive sleep apnea. ASDA standards of practice.

PubMed

Ferber, R; Millman, R; Coppola, M; Fleetham, J; Murray, C F; Iber, C; McCall, V; Nino-Murcia, G; Pressman, M; Sanders, M

1994-06-01

The objective assessment of patients with a presumptive diagnosis of obstructive sleep apnea (OSA) has primarily used attended polysomnographic study. Recent technologic advances and issues of availability, convenience and cost have led to a rapid increase in the use of portable recording devices. However, limited scientific information has been published regarding the evaluation of the efficacy, accuracy, validity, utility, cost effectiveness and limitations of this portable equipment. Attaining a clear assessment of the role of portable devices is complicated by the multiplicity of recording systems and the variability of clinical settings in which they have been analyzed. This paper reviews the current knowledge base regarding portable recording in the assessment of OSA, including technical considerations, validation studies, potential advantages and disadvantages, issues of safety, current clinical usage and areas most in need of further study.

Validation of Procedures for Monitoring Crewmember Immune Function

NASA Technical Reports Server (NTRS)

Crucian, Brian; Stowe, Raymond; Mehta, Satish; Uchakin, Peter; Quiriarte, Heather; Pierson, Duane; Sams, Clarence

2008-01-01

There is ample evidence to suggest that space flight leads to immune system dysregulation. This may be a result of microgravity, confinement, physiological stress, radiation, environment or other mission-associated factors. The clinical risk (if any) from prolonged immune dysregulation during exploration-class space flight has not yet been determined, but may include increased incidence of infection, allergy, hypersensitivity, hematological malignancy or altered wound healing. Each of the clinical events resulting from immune dysfunction has the potential to impact mission critical objectives during exploration-class missions. To date, precious little in-flight immune data has been generated to assess this phenomenon. The majority of recent flight immune studies have been post-flight assessments, which may not accurately reflect the in-flight status of immunity as it resolves over prolonged flight. There are no procedures currently in place to monitor immune function or its effect on crew health. The objective of this Supplemental Medical Objective (SMO) is to develop and validate an immune monitoring strategy consistent with operational flight requirements and constraints. This SMO will assess immunity, latent viral reactivation and physiological stress during both short and long duration flights. Upon completion, it is expected that any clinical risks resulting from the adverse effects of space flight on the human immune system will have been determined. In addition, a flight-compatible immune monitoring strategy will have been developed with which countermeasures validation could be performed. This study will determine, to the best level allowed by current technology, the in-flight status of crewmembers' immune systems. The in-flight samples will allow a distinction between legitimate in-flight alterations and the physiological stresses of landing and readaptation which are believed to alter R+0 assessments. The overall status of the immune system during flight (activation, deficiency, dysregulation) and the response of the immune system to specific latent virus reactivation (known to occur during space flight) will be thoroughly assessed. The first in-flight activity for integrated immunity very recently occurred during the STS-120 Space Shuttle mission. The protocols functioned well from a technical perspective, and accurate in-flight data was obtained from 1 Shuttle and 2 ISS crewmembers. Crew participation rates for the study continue to be robust.

Longitudinal evaluation of Patient Reported Outcomes Measurement Information Systems (PROMIS) measures in pediatric chronic pain

PubMed Central

Kashikar-Zuck, Susmita; Carle, Adam; Barnett, Kimberly; Goldschneider, Kenneth R.; Sherry, David D.; Mara, Constance A.; Cunningham, Natoshia; Farrell, Jennifer; Tress, Jenna; DeWitt, Esi Morgan

2015-01-01

The Patient Reported Outcomes Measurement Information System (PROMIS) initiative is a comprehensive strategy by the National Institutes of Health to support the development and validation of precise instruments to assess self-reported health domains across healthy and disease-specific populations. Much progress has been made in instrument development but there remains a gap in the validation of PROMIS measures for pediatric chronic pain. The purpose of this study was to investigate the construct validity and responsiveness to change of seven PROMIS domains for the assessment of children (ages 8-18) with chronic pain – Pain Interference, Fatigue, Anxiety, Depression, Mobility, Upper Extremity Function and Peer Relationships. PROMIS measures were administered at the initial visit and two follow-up visits at an outpatient chronic pain clinic (CPC; N=82) and at an intensive amplified pain day-treatment program (AMP; N= 63). Aim 1 examined construct validity of PROMIS measures by comparing them with corresponding “legacy” measures administered as part of usual care in the CPC sample. Aim 2 examined sensitivity to change in both CPC and AMP samples. Longitudinal growth models showed that PROMIS Pain Interference, Anxiety, Depression, Mobility, Upper Extremity and Peer Relationship measures and legacy instruments generally performed similarly with slightly steeper slopes of improvement in legacy measures. All seven PROMIS domains showed responsiveness to change. Results offered initial support for the validity of PROMIS measures in pediatric chronic pain. Further validation with larger and more diverse pediatric pain samples and additional legacy measures would broaden the scope of use of PROMIS in clinical research. PMID:26447704

Convergent and discriminant validity and reliability of the pediatric anxiety rating scale in youth with autism spectrum disorders.

PubMed

Storch, Eric A; Wood, Jeffrey J; Ehrenreich-May, Jill; Jones, Anna M; Park, Jennifer M; Lewin, Adam B; Murphy, Tanya K

2012-11-01

The psychometric properties of the Pediatric Anxiety Rating Scale (PARS), a clinician-administered measure for assessing severity of anxiety symptoms, were examined in 72 children and adolescents diagnosed with an autism spectrum disorder (ASD). The internal consistency of the PARS was 0.59, suggesting that the items were related but not repetitive. The PARS showed high 26-day test-retest (ICC = 0.83) and inter-rater reliability (ICC = 0.86). The PARS was strongly correlated with clinician-ratings of overall anxiety severity and parent-report anxiety measures, supporting convergent validity. Results for divergent validity were mixed. Although the PARS was not associated with the sum of the Social and Communication items on the Autism Diagnostic Observation System, it was moderately correlated with parent-reported inattention, aggression and externalizing behavior. Overall, these results suggest that the psychometric properties of the PARS are adequate for assessing anxiety symptoms in youth with ASD, although additional clarification of divergent validity is needed.

Attachment, assessment, and psychological intervention: a case study of anorexia.

PubMed

Lis, Adriana; Mazzeschi, Claudia; Di Riso, Daniela; Salcuni, Silvia

2011-01-01

Attachment patterns and personality dimensions have always been considered important to the development and adaptation of the individual. The first aim of this article was to address some basic questions about the place of attachment in a multimethod assessment when compiling a complete picture of the patient's personality functioning. The second aim was to present the Adult Attachment Projective Picture System (AAP; George & West, 2001) as a valid and productive assessment measure. Based on a single case study of an anorexic young woman, the article demonstrates how the AAP is integrated with the Rorschach Comprehensive System (Exner, 1991, 1993) and other assessment tools in both the assessment and in developing a treatment plan.

Implementing Lumberjacks and Black Swans Into Model-Based Tools to Support Human-Automation Interaction.

PubMed

Sebok, Angelia; Wickens, Christopher D

2017-03-01

The objectives were to (a) implement theoretical perspectives regarding human-automation interaction (HAI) into model-based tools to assist designers in developing systems that support effective performance and (b) conduct validations to assess the ability of the models to predict operator performance. Two key concepts in HAI, the lumberjack analogy and black swan events, have been studied extensively. The lumberjack analogy describes the effects of imperfect automation on operator performance. In routine operations, an increased degree of automation supports performance, but in failure conditions, increased automation results in more significantly impaired performance. Black swans are the rare and unexpected failures of imperfect automation. The lumberjack analogy and black swan concepts have been implemented into three model-based tools that predict operator performance in different systems. These tools include a flight management system, a remotely controlled robotic arm, and an environmental process control system. Each modeling effort included a corresponding validation. In one validation, the software tool was used to compare three flight management system designs, which were ranked in the same order as predicted by subject matter experts. The second validation compared model-predicted operator complacency with empirical performance in the same conditions. The third validation compared model-predicted and empirically determined time to detect and repair faults in four automation conditions. The three model-based tools offer useful ways to predict operator performance in complex systems. The three tools offer ways to predict the effects of different automation designs on operator performance.

Novel System for Real-Time Integration of 3-D Echocardiography and Fluoroscopy for Image-Guided Cardiac Interventions: Preclinical Validation and Clinical Feasibility Evaluation.

PubMed

Arujuna, Aruna V; Housden, R James; Ma, Yingliang; Rajani, Ronak; Gao, Gang; Nijhof, Niels; Cathier, Pascal; Bullens, Roland; Gijsbers, Geert; Parish, Victoria; Kapetanakis, Stamatis; Hancock, Jane; Rinaldi, C Aldo; Cooklin, Michael; Gill, Jaswinder; Thomas, Martyn; O'neill, Mark D; Razavi, Reza; Rhode, Kawal S

2014-01-01

Real-time imaging is required to guide minimally invasive catheter-based cardiac interventions. While transesophageal echocardiography allows for high-quality visualization of cardiac anatomy, X-ray fluoroscopy provides excellent visualization of devices. We have developed a novel image fusion system that allows real-time integration of 3-D echocardiography and the X-ray fluoroscopy. The system was validated in the following two stages: 1) preclinical to determine function and validate accuracy; and 2) in the clinical setting to assess clinical workflow feasibility and determine overall system accuracy. In the preclinical phase, the system was assessed using both phantom and porcine experimental studies. Median 2-D projection errors of 4.5 and 3.3 mm were found for the phantom and porcine studies, respectively. The clinical phase focused on extending the use of the system to interventions in patients undergoing either atrial fibrillation catheter ablation (CA) or transcatheter aortic valve implantation (TAVI). Eleven patients were studied with nine in the CA group and two in the TAVI group. Successful real-time view synchronization was achieved in all cases with a calculated median distance error of 2.2 mm in the CA group and 3.4 mm in the TAVI group. A standard clinical workflow was established using the image fusion system. These pilot data confirm the technical feasibility of accurate real-time echo-fluoroscopic image overlay in clinical practice, which may be a useful adjunct for real-time guidance during interventional cardiac procedures.

Fully automated system for the quantification of human osteoarthritic knee joint effusion volume using magnetic resonance imaging.

PubMed

Li, Wei; Abram, François; Pelletier, Jean-Pierre; Raynauld, Jean-Pierre; Dorais, Marc; d'Anjou, Marc-André; Martel-Pelletier, Johanne

2010-01-01

Joint effusion is frequently associated with osteoarthritis (OA) flare-up and is an important marker of therapeutic response. This study aimed at developing and validating a fully automated system based on magnetic resonance imaging (MRI) for the quantification of joint effusion volume in knee OA patients. MRI examinations consisted of two axial sequences: a T2-weighted true fast imaging with steady-state precession and a T1-weighted gradient echo. An automated joint effusion volume quantification system using MRI was developed and validated (a) with calibrated phantoms (cylinder and sphere) and effusion from knee OA patients; (b) with assessment by manual quantification; and (c) by direct aspiration. Twenty-five knee OA patients with joint effusion were included in the study. The automated joint effusion volume quantification was developed as a four stage sequencing process: bone segmentation, filtering of unrelated structures, segmentation of joint effusion, and subvoxel volume calculation. Validation experiments revealed excellent coefficients of variation with the calibrated cylinder (1.4%) and sphere (0.8%) phantoms. Comparison of the OA knee joint effusion volume assessed by the developed automated system and by manual quantification was also excellent (r = 0.98; P < 0.0001), as was the comparison with direct aspiration (r = 0.88; P = 0.0008). The newly developed fully automated MRI-based system provided precise quantification of OA knee joint effusion volume with excellent correlation with data from phantoms, a manual system, and joint aspiration. Such an automated system will be instrumental in improving the reproducibility/reliability of the evaluation of this marker in clinical application.

The Validation of a Case-Based, Cumulative Assessment and Progressions Examination

PubMed Central

Coker, Adeola O.; Copeland, Jeffrey T.; Gottlieb, Helmut B.; Horlen, Cheryl; Smith, Helen E.; Urteaga, Elizabeth M.; Ramsinghani, Sushma; Zertuche, Alejandra; Maize, David

2016-01-01

Objective. To assess content and criterion validity, as well as reliability of an internally developed, case-based, cumulative, high-stakes third-year Annual Student Assessment and Progression Examination (P3 ASAP Exam). Methods. Content validity was assessed through the writing-reviewing process. Criterion validity was assessed by comparing student scores on the P3 ASAP Exam with the nationally validated Pharmacy Curriculum Outcomes Assessment (PCOA). Reliability was assessed with psychometric analysis comparing student performance over four years. Results. The P3 ASAP Exam showed content validity through representation of didactic courses and professional outcomes. Similar scores on the P3 ASAP Exam and PCOA with Pearson correlation coefficient established criterion validity. Consistent student performance using Kuder-Richardson coefficient (KR-20) since 2012 reflected reliability of the examination. Conclusion. Pharmacy schools can implement internally developed, high-stakes, cumulative progression examinations that are valid and reliable using a robust writing-reviewing process and psychometric analyses. PMID:26941435

Evaluation of objectivity, reliability and criterion validity of the key indicator method for manual handling operations (KIM-MHO), draft 2007.

PubMed

Klußmann, André; Gebhardt, Hansjürgen; Rieger, Monika; Liebers, Falk; Steinberg, Ulf

2012-01-01

Upper extremity musculoskeletal symptoms and disorders are common in the working population. The economic and social impact of such disorders is considerable. Long-time, dynamic repetitive exposure of the hand-arm system during manual handling operations (MHO) alone or in combination with static and postural effort are recognised as causes of musculoskeletal symptoms and disorders. The assessment of these manual work tasks is crucial to estimate health risks of exposed employees. For these work tasks, a new method for the assessment of the working conditions was developed and a validation study was performed. The results suggest satisfying criterion validity and moderate objectivity of the KIM-MHO draft 2007. The method was modified and evaluated again. It is planned to release a new version of KIM-MHO in spring 2012.

Design of a Competency Evaluation Model for Clinical Nursing Practicum, Based on Standardized Language Systems: Psychometric Validation Study.

PubMed

Iglesias-Parra, Maria Rosa; García-Guerrero, Alfonso; García-Mayor, Silvia; Kaknani-Uttumchandani, Shakira; León-Campos, Álvaro; Morales-Asencio, José Miguel

2015-07-01

To develop an evaluation system of clinical competencies for the practicum of nursing students based on the Nursing Interventions Classification (NIC). Psychometric validation study: the first two phases addressed definition and content validation, and the third phase consisted of a cross-sectional study for analyzing reliability. The study population was undergraduate nursing students and clinical tutors. Through the Delphi technique, 26 competencies and 91 interventions were isolated. Cronbach's α was 0.96. Factor analysis yielded 18 factors that explained 68.82% of the variance. Overall inter-item correlation was 0.26, and total-item correlation ranged between 0.66 and 0.19. A competency system for the nursing practicum, structured on the NIC, is a reliable method for assessing and evaluating clinical competencies. Further evaluations in other contexts are needed. The availability of standardized language systems in the nursing discipline supposes an ideal framework to develop the nursing curricula. © 2015 Sigma Theta Tau International.

Development and psychometric validation of a self-administered questionnaire assessing the acceptance of influenza vaccination: the Vaccinees' Perception of Injection (VAPI©) questionnaire

PubMed Central

Chevat, Catherine; Viala-Danten, Muriel; Dias-Barbosa, Carla; Nguyen, Van Hung

2009-01-01

Background Influenza is among the most common infectious diseases. The main protection against influenza is vaccination. A self-administered questionnaire was developed and validated for use in clinical trials to assess subjects' perception and acceptance of influenza vaccination and its subsequent injection site reactions (ISR). Methods The VAPI questionnaire was developed based on interviews with vaccinees. The initial version was administered to subjects in international clinical trials comparing intradermal with intramuscular influenza vaccination. Item reduction and scale construction were carried out using principal component and multitrait analyses (n = 549). Psychometric validation of the final version was conducted per country (n = 5,543) and included construct and clinical validity and internal consistency reliability. All subjects gave their written informed consent before being interviewed or included in the clinical studies. Results The final questionnaire comprised 4 dimensions ("bother from ISR"; "arm movement"; "sleep"; "acceptability") grouping 16 items, and 5 individual items (anxiety before vaccination; bother from pain during vaccination; satisfaction with injection system; willingness to be vaccinated next year; anxiety about vaccination next year). Construct validity was confirmed for all scales in most of the countries. Internal consistency reliability was good for all versions (Cronbach's alpha ranging from 0.68 to 0.94), as was clinical validity: scores were positively correlated with the severity of ISR and pain. Conclusion The VAPI questionnaire is a valid and reliable tool, assessing the acceptance of vaccine injection and reactions following vaccination. Trial registration NCT00258934, NCT00383526, NCT00383539. PMID:19261173

Geographic Information Systems to Assess External Validity in Randomized Trials.

PubMed

Savoca, Margaret R; Ludwig, David A; Jones, Stedman T; Jason Clodfelter, K; Sloop, Joseph B; Bollhalter, Linda Y; Bertoni, Alain G

2017-08-01

To support claims that RCTs can reduce health disparities (i.e., are translational), it is imperative that methodologies exist to evaluate the tenability of external validity in RCTs when probabilistic sampling of participants is not employed. Typically, attempts at establishing post hoc external validity are limited to a few comparisons across convenience variables, which must be available in both sample and population. A Type 2 diabetes RCT was used as an example of a method that uses a geographic information system to assess external validity in the absence of a priori probabilistic community-wide diabetes risk sampling strategy. A geographic information system, 2009-2013 county death certificate records, and 2013-2014 electronic medical records were used to identify community-wide diabetes prevalence. Color-coded diabetes density maps provided visual representation of these densities. Chi-square goodness of fit statistic/analysis tested the degree to which distribution of RCT participants varied across density classes compared to what would be expected, given simple random sampling of the county population. Analyses were conducted in 2016. Diabetes prevalence areas as represented by death certificate and electronic medical records were distributed similarly. The simple random sample model was not a good fit for death certificate record (chi-square, 17.63; p=0.0001) and electronic medical record data (chi-square, 28.92; p<0.0001). Generally, RCT participants were oversampled in high-diabetes density areas. Location is a highly reliable "principal variable" associated with health disparities. It serves as a directly measurable proxy for high-risk underserved communities, thus offering an effective and practical approach for examining external validity of RCTs. Copyright © 2017 American Journal of Preventive Medicine. Published by Elsevier Inc. All rights reserved.

Psychometric validation of the PROQOL-HIV questionnaire, a new health-related quality of life instrument-specific to HIV disease.

PubMed

Duracinsky, Martin; Lalanne, Christophe; Le Coeur, Sophie; Herrmann, Susan; Berzins, Baiba; Armstrong, Andrew Richard; Lau, Joseph Tak Fai; Fournier, Isabelle; Chassany, Olivier

2012-04-15

This study reports the psychometric validation of a new HIV/AIDS-specific health-related quality of life (HRQL) questionnaire, the Patient Reported Outcomes Quality of Life-HIV. The instrument was developed simultaneously across Europe, North and South America, Africa, Asia, and Australia to assess multidimensional quality of life impairments in the era of highly active antiretroviral therapy. A cross-sectional study was performed in 8 countries. The pilot 70-item questionnaire was co-administered with the HIV symptoms index, the EQ-5D and Medical Outcomes Study-HIV questionnaires. Demographic and biomedical data were collected. After item analysis and reduction, convergent discriminant concurrent validity and known-group validity were examined. Internal consistency and reliability scores were assessed using Cronbach alpha and intraclass correlation. The final sample of 791 patients was composed of 64% males (median age: 41 years, HIV diagnosis = 5 years), 13.8% were treatment naive. Item reduction yielded a 43-item form surveying 8 dimensions and 1 global health item that showed good convergent and discriminant validity and reliability (98% scaling success; Cronbach alphas 0.77-0.89). Correlations with EQ-5D and Medical Outcomes Study-HIV complied with concurrent validity expectations; likewise, correlations against the number of self-reported symptoms and depression showed good support for criterion validity. A test-retest study on French patients (n = 34) showed temporal stability (intraclass correlation coefficient = 0.86). Significant and meaningful differences of HRQL scores between countries were found. The Patient Reported Outcomes Quality of Life-HIV questionnaire is a valid and reliable instrument for assessing HRQL specific to HIV disease in different cultures and healthcare systems.

Computer-based tools for assessing micro-longitudinal patterns of cognitive function in older adults.

PubMed

Brown, Laura J E; Adlam, Tim; Hwang, Faustina; Khadra, Hassan; Maclean, Linda M; Rudd, Bridey; Smith, Tom; Timon, Claire; Williams, Elizabeth A; Astell, Arlene J

2016-08-01

Patterns of cognitive change over micro-longitudinal timescales (i.e., ranging from hours to days) are associated with a wide range of age-related health and functional outcomes. However, practical issues of conducting high-frequency assessments make investigations of micro-longitudinal cognition costly and burdensome to run. One way of addressing this is to develop cognitive assessments that can be performed by older adults, in their own homes, without a researcher being present. Here, we address the question of whether reliable and valid cognitive data can be collected over micro-longitudinal timescales using unsupervised cognitive tests.In study 1, 48 older adults completed two touchscreen cognitive tests, on three occasions, in controlled conditions, alongside a battery of standard tests of cognitive functions. In study 2, 40 older adults completed the same two computerized tasks on multiple occasions, over three separate week-long periods, in their own homes, without a researcher present. Here, the tasks were incorporated into a wider touchscreen system (Novel Assessment of Nutrition and Ageing (NANA)) developed to assess multiple domains of health and behavior. Standard tests of cognitive function were also administered prior to participants using the NANA system.Performance on the two "NANA" cognitive tasks showed convergent validity with, and similar levels of reliability to, the standard cognitive battery in both studies. Completion and accuracy rates were also very high. These results show that reliable and valid cognitive data can be collected from older adults using unsupervised computerized tests, thus affording new opportunities for the investigation of cognitive.

Evaluation of passenger health risk assessment of sustainable indoor air quality monitoring in metro systems based on a non-Gaussian dynamic sensor validation method.

PubMed

Kim, MinJeong; Liu, Hongbin; Kim, Jeong Tai; Yoo, ChangKyoo

2014-08-15

Sensor faults in metro systems provide incorrect information to indoor air quality (IAQ) ventilation systems, resulting in the miss-operation of ventilation systems and adverse effects on passenger health. In this study, a new sensor validation method is proposed to (1) detect, identify and repair sensor faults and (2) evaluate the influence of sensor reliability on passenger health risk. To address the dynamic non-Gaussianity problem of IAQ data, dynamic independent component analysis (DICA) is used. To detect and identify sensor faults, the DICA-based squared prediction error and sensor validity index are used, respectively. To restore the faults to normal measurements, a DICA-based iterative reconstruction algorithm is proposed. The comprehensive indoor air-quality index (CIAI) that evaluates the influence of the current IAQ on passenger health is then compared using the faulty and reconstructed IAQ data sets. Experimental results from a metro station showed that the DICA-based method can produce an improved IAQ level in the metro station and reduce passenger health risk since it more accurately validates sensor faults than do conventional methods. Copyright © 2014 Elsevier B.V. All rights reserved.

New methodology to assess activity status of occlusal caries in primary teeth using laser fluorescence device.

PubMed

Braga, Mariana Minatel; de Benedetto, Monique Saveriano; Imparato, Jose Carlos Pettorossi; Mendes, Fausto Medeiros

2010-01-01

An in vivo study was conducted to verify the ability of laser fluorescence (LF) to assess the activity status of occlusal caries in primary teeth, using different air-drying times. Occlusal sites (707) were examined using LF (DIAGNOdent) after air-drying for 3 s and 15 s, and the difference between readings (DIF15 s-3 s) was calculated. For concurrent validation of LF, visual criteria-Nyvad (NY) and Lesion Activity Assessment associated with the International Caries Detection and Assessment System (LAA-ICDAS)-were the reference standards for lesion activity. Histological exam using a pH-indicator dye (0.1% methyl red) was performed in 46 exfoliated/extracted teeth for criterion validation. LF readings and DIF15 s-3 s were compared using Kruskall-Wallis and Mann-Whitney tests. Receiver operating characteristic analyses were performed and validity parameters calculated, considering the caries activity assessment. Using NY, active lesions (3 s: 30.0+/-29.3; 15 s: 34.2+/-30.6) presented higher LF readings than inactive lesions (3 s: 17.0+/-16.3; 15 s: 19.2+/-17.3; p<0.05), different from LAA-ICDAS. Active cavitated caries resulted in higher LF readings (3 s: 50.3+/-3.5; 15 s: 54.7+/-30.2) than inactive cavitated caries (3 s: 19.9+/-16.3; 15 s: 22.8+/-16.8). Therefore, LF can distinguish cavitated active and inactive lesions classified by NY, but not by LAA-ICDAS; however, this difference might be related to the visual system rather than to LF. The air-drying time could be an alternative to improve the caries activity assessment; however, longer air-drying time is suggested to be tested subsequently.

Back-and-Forth Methodology for Objective Voice Quality Assessment: From/to Expert Knowledge to/from Automatic Classification of Dysphonia

NASA Astrophysics Data System (ADS)

Fredouille, Corinne; Pouchoulin, Gilles; Ghio, Alain; Revis, Joana; Bonastre, Jean-François; Giovanni, Antoine

2009-12-01

This paper addresses voice disorder assessment. It proposes an original back-and-forth methodology involving an automatic classification system as well as knowledge of the human experts (machine learning experts, phoneticians, and pathologists). The goal of this methodology is to bring a better understanding of acoustic phenomena related to dysphonia. The automatic system was validated on a dysphonic corpus (80 female voices), rated according to the GRBAS perceptual scale by an expert jury. Firstly, focused on the frequency domain, the classification system showed the interest of 0-3000 Hz frequency band for the classification task based on the GRBAS scale. Later, an automatic phonemic analysis underlined the significance of consonants and more surprisingly of unvoiced consonants for the same classification task. Submitted to the human experts, these observations led to a manual analysis of unvoiced plosives, which highlighted a lengthening of VOT according to the dysphonia severity validated by a preliminary statistical analysis.

Building a Performance-Based Assessment System To Diagnose Strengths and Weaknesses in Reading Achievement.

ERIC Educational Resources Information Center

Hennings, Sara S.; Hughes, Kay E.

This paper provides a brief description of the development of the Diagnostic Assessments of Reading with Trial Teaching Strategies (DARTTS) program by F. G. Roswell and J. S. Chall. It also describes the editorial and statistical procedures that were used to validate the program for determining students' strengths and weaknesses in important areas…

An Assessment System for Competence Based Education: The Educational Development, Dissemination, and Evaluation Training Program.

ERIC Educational Resources Information Center

Hood, Paul D.; Blackwell, Laird

This manual provides a description of the development and a guide to the use of the assessment resources developed in connection with the Far West Development, Dissemination, and Evaluation (DD&E) Functional Competence Training Program. The document concentrates on a user-oriented description of the content, validation, and use of the final…

Live versus Video Observations: Comparing the Reliability and Validity of Two Methods of Assessing Classroom Quality

ERIC Educational Resources Information Center

Curby, Timothy W.; Johnson, Price; Mashburn, Andrew J.; Carlis, Lydia

2016-01-01

When conducting classroom observations, researchers are often confronted with the decision of whether to conduct observations live or by using pre-recorded video. The present study focuses on comparing and contrasting observations of live and video administrations of the Classroom Assessment Scoring System-PreK (CLASS-PreK). Associations between…

The NEO-FFI in Multiple Sclerosis: Internal Consistency, Factorial Validity, and Correspondence between Self and Informant Reports

ERIC Educational Resources Information Center

Schwartz, Eben S.; Chapman, Benjamin P.; Duberstein, Paul R.; Weinstock-Guttman, Bianca; Benedict, Ralph H. B.

2011-01-01

Personality assessment is a potentially important component of clinical and empirical work with neurological patients because (a) individual differences in personality may be associated with different neurological outcomes and (b) central nervous system changes may give rise to alteration in personality. For personality assessment to be useful to…

Device and Component Testing | Water Power | NREL

Science.gov Websites

actuators. Specialized component validation of blades may be accomplished by applying loads at the system's during this time has assessed hundreds of wind blades. The NWTC has pioneered the development of

48 CFR 1401.7001-4 - Acquisition performance measurement systems.

Code of Federal Regulations, 2013 CFR

2013-10-01

...-pronged approach that includes self assessment, statistical data for validation and flexible quality... regulations governing the acquisition process; and (3) Identify and implement changes necessary to improve the... through the review and oversight process. ...

48 CFR 1401.7001-4 - Acquisition performance measurement systems.

Code of Federal Regulations, 2014 CFR

2014-10-01

...-pronged approach that includes self assessment, statistical data for validation and flexible quality... regulations governing the acquisition process; and (3) Identify and implement changes necessary to improve the... through the review and oversight process. ...

48 CFR 1401.7001-4 - Acquisition performance measurement systems.

Code of Federal Regulations, 2011 CFR

2011-10-01

...-pronged approach that includes self assessment, statistical data for validation and flexible quality... regulations governing the acquisition process; and (3) Identify and implement changes necessary to improve the... through the review and oversight process. ...

48 CFR 1401.7001-4 - Acquisition performance measurement systems.

Code of Federal Regulations, 2012 CFR

2012-10-01

...-pronged approach that includes self assessment, statistical data for validation and flexible quality... regulations governing the acquisition process; and (3) Identify and implement changes necessary to improve the... through the review and oversight process. ...

[Emotional Intelligence Index: a tool for the routine assessment of mental health promotion programs in schools].

PubMed

Veltro, Franco; Ialenti, Valentina; Morales García, Manuel Alejandro; Gigantesco, Antonella

2016-01-01

After critical examination of several aspects relating to the evaluation of some dimensions of emotional intelligence through self-assessment tools, is described the procedure of construction and validation of an Index for its measurement, conceived only for the routine assessment of health promotion programs mental in schools that include among their objectives the improvement of emotional intelligence specifically "outcome-oriented". On the basis of the two most common international tools, are listed 27 items plus 6 of control, illustrated two Focus Group (FG) of students (face validity). The scale obtained by FG was administered to 300 students, and the results were submitted to factorial analysis (construct validity). It was also evaluated the internal consistency with Cronbach's Alpha and studied concurrent validity with the emotional quotient inventory, a scale of perceived self-efficacy and a stress test rating. From the analysis of FG all the original items were modified, deleted 4, and reduced the encoding system from 6 to 4 levels of Likert scale. Of the 23 items included in the analysis have emerged five factors (intra-psychic dimension, interpersonal, impulsivity, adaptive coping, sense of self-efficacy) for a total of 15 items. Very satisfactory were the results of the validation process of internal consistency (0.72) and the concurrent validity. The results are positive. It is obtained in fact the shortest routine assessment tool currently available in Italy which constitutes a real Index, for which compilation are required on average 3 minutes. Is emphasized the characteristic of an Index, and not of questionnaire or interview for clinical use, highlighting the only specific use for mental health promotion programs in schools.

A new approach to the characterization of subtle errors in everyday action: implications for mild cognitive impairment.

PubMed

Seligman, Sarah C; Giovannetti, Tania; Sestito, John; Libon, David J

2014-01-01

Mild functional difficulties have been associated with early cognitive decline in older adults and increased risk for conversion to dementia in mild cognitive impairment, but our understanding of this decline has been limited by a dearth of objective methods. This study evaluated the reliability and validity of a new system to code subtle errors on an established performance-based measure of everyday action and described preliminary findings within the context of a theoretical model of action disruption. Here 45 older adults completed the Naturalistic Action Test (NAT) and neuropsychological measures. NAT performance was coded for overt errors, and subtle action difficulties were scored using a novel coding system. An inter-rater reliability coefficient was calculated. Validity of the coding system was assessed using a repeated-measures ANOVA with NAT task (simple versus complex) and error type (overt versus subtle) as within-group factors. Correlation/regression analyses were conducted among overt NAT errors, subtle NAT errors, and neuropsychological variables. The coding of subtle action errors was reliable and valid, and episodic memory breakdown predicted subtle action disruption. Results suggest that the NAT can be useful in objectively assessing subtle functional decline. Treatments targeting episodic memory may be most effective in addressing early functional impairment in older age.

Validation of an MRI Brain Injury and Growth Scoring System in Very Preterm Infants Scanned at 29- to 35-Week Postmenstrual Age.

PubMed

George, J M; Fiori, S; Fripp, J; Pannek, K; Bursle, J; Moldrich, R X; Guzzetta, A; Coulthard, A; Ware, R S; Rose, S E; Colditz, P B; Boyd, R N

2017-07-01

The diagnostic and prognostic potential of brain MR imaging before term-equivalent age is limited until valid MR imaging scoring systems are available. This study aimed to validate an MR imaging scoring system of brain injury and impaired growth for use at 29 to 35 weeks postmenstrual age in infants born at <31 weeks gestational age. Eighty-three infants in a prospective cohort study underwent early 3T MR imaging between 29 and 35 weeks' postmenstrual age (mean, 32 +2 ± 1 +3 weeks; 49 males, born at median gestation of 28 +4 weeks; range, 23 +6 -30 +6 weeks; mean birthweight, 1068 ± 312 g). Seventy-seven infants had a second MR scan at term-equivalent age (mean, 40 +6 ± 1 +3 weeks). Structural images were scored using a modified scoring system which generated WM, cortical gray matter, deep gray matter, cerebellar, and global scores. Outcome at 12-months corrected age (mean, 12 months 4 days ± 1 +2 weeks) consisted of the Bayley Scales of Infant and Toddler Development, 3rd ed. (Bayley III), and the Neuro-Sensory Motor Developmental Assessment. Early MR imaging global, WM, and deep gray matter scores were negatively associated with Bayley III motor (regression coefficient for global score β = -1.31; 95% CI, -2.39 to -0.23; P = .02), cognitive (β = -1.52; 95% CI, -2.39 to -0.65; P < .01) and the Neuro-Sensory Motor Developmental Assessment outcomes (β = -1.73; 95% CI, -3.19 to -0.28; P = .02). Early MR imaging cerebellar scores were negatively associated with the Neuro-Sensory Motor Developmental Assessment (β = -5.99; 95% CI, -11.82 to -0.16; P = .04). Results were reconfirmed at term-equivalent-age MR imaging. This clinically accessible MR imaging scoring system is valid for use at 29 to 35 weeks postmenstrual age in infants born very preterm. It enables identification of infants at risk of adverse outcomes before the current standard of term-equivalent age. © 2017 by American Journal of Neuroradiology.

Soil Moisture Active Passive Mission L4_SM Data Product Assessment (Version 2 Validated Release)

NASA Technical Reports Server (NTRS)

Reichle, Rolf Helmut; De Lannoy, Gabrielle J. M.; Liu, Qing; Ardizzone, Joseph V.; Chen, Fan; Colliander, Andreas; Conaty, Austin; Crow, Wade; Jackson, Thomas; Kimball, John;

Construction, internal validation and implementation in a mobile application of a scoring system to predict nonadherence to proton pump inhibitors.

PubMed

Mares-García, Emma; Palazón-Bru, Antonio; Folgado-de la Rosa, David Manuel; Pereira-Expósito, Avelino; Martínez-Martín, Álvaro; Cortés-Castell, Ernesto; Gil-Guillén, Vicente Francisco

2017-01-01

Other studies have assessed nonadherence to proton pump inhibitors (PPIs), but none has developed a screening test for its detection. To construct and internally validate a predictive model for nonadherence to PPIs. This prospective observational study with a one-month follow-up was carried out in 2013 in Spain, and included 302 patients with a prescription for PPIs. The primary variable was nonadherence to PPIs (pill count). Secondary variables were gender, age, antidepressants, type of PPI, non-guideline-recommended prescription (NGRP) of PPIs, and total number of drugs. With the secondary variables, a binary logistic regression model to predict nonadherence was constructed and adapted to a points system. The ROC curve, with its area (AUC), was calculated and the optimal cut-off point was established. The points system was internally validated through 1,000 bootstrap samples and implemented in a mobile application (Android). The points system had three prognostic variables: total number of drugs, NGRP of PPIs, and antidepressants. The AUC was 0.87 (95% CI [0.83-0.91], p  < 0.001). The test yielded a sensitivity of 0.80 (95% CI [0.70-0.87]) and a specificity of 0.82 (95% CI [0.76-0.87]). The three parameters were very similar in the bootstrap validation. A points system to predict nonadherence to PPIs has been constructed, internally validated and implemented in a mobile application. Provided similar results are obtained in external validation studies, we will have a screening tool to detect nonadherence to PPIs.

Validity of a novel computerized screening test system for mild cognitive impairment.

PubMed

Park, Jin-Hyuck; Jung, Minye; Kim, Jongbae; Park, Hae Yean; Kim, Jung-Ran; Park, Ji-Hyuk

2018-06-20

ABSTRACTBackground:The mobile screening test system for screening mild cognitive impairment (mSTS-MCI) was developed for clinical use. However, the clinical usefulness of mSTS-MCI to detect elderly with MCI from those who are cognitively healthy has yet to be validated. Moreover, the comparability between this system and traditional screening tests for MCI has not been evaluated. The purpose of this study was to examine the validity and reliability of the mSTS-MCI and confirm the cut-off scores to detect MCI. The data were collected from 107 healthy elderly people and 74 elderly people with MCI. Concurrent validity was examined using the Korean version of Montreal Cognitive Assessment (MoCA-K) as a gold standard test, and test-retest reliability was investigated using 30 of the study participants at four-week intervals. The sensitivity, specificity, positive predictive value, and negative predictive value (NPV) were confirmed through Receiver Operating Characteristic (ROC) analysis, and the cut-off scores for elderly people with MCI were identified. Concurrent validity showed statistically significant correlations between the mSTS-MCI and MoCA-K and test-rests reliability indicated high correlation. As a result of screening predictability, the mSTS-MCI had a higher NPV than the MoCA-K. The mSTS-MCI was identified as a system with a high degree of validity and reliability. In addition, the mSTS-MCI showed high screening predictability, indicating it can be used in the clinical field as a screening test system for mild cognitive impairment.

Reliability and criterion validity of measurements using a smart phone-based measurement tool for the transverse rotation angle of the pelvis during single-leg lifting.

PubMed

Jung, Sung-Hoon; Kwon, Oh-Yun; Jeon, In-Cheol; Hwang, Ui-Jae; Weon, Jong-Hyuck

2018-01-01

The purposes of this study were to determine the intra-rater test-retest reliability of a smart phone-based measurement tool (SBMT) and a three-dimensional (3D) motion analysis system for measuring the transverse rotation angle of the pelvis during single-leg lifting (SLL) and the criterion validity of the transverse rotation angle of the pelvis measurement using SBMT compared with a 3D motion analysis system (3DMAS). Seventeen healthy volunteers performed SLL with their dominant leg without bending the knee until they reached a target placed 20 cm above the table. This study used a 3DMAS, considered the gold standard, to measure the transverse rotation angle of the pelvis to assess the criterion validity of the SBMT measurement. Intra-rater test-retest reliability was determined using the SBMT and 3DMAS using intra-class correlation coefficient (ICC) [3,1] values. The criterion validity of the SBMT was assessed with ICC [3,1] values. Both the 3DMAS (ICC = 0.77) and SBMT (ICC = 0.83) showed excellent intra-rater test-retest reliability in the measurement of the transverse rotation angle of the pelvis during SLL in a supine position. Moreover, the SBMT showed an excellent correlation with the 3DMAS (ICC = 0.99). Measurement of the transverse rotation angle of the pelvis using the SBMT showed excellent reliability and criterion validity compared with the 3DMAS.

A Short Measure of the Revised Reinforcement Sensitivity Theory - RSQ17.

PubMed

Čolović, Petar; Smederevac, Snežana; Oljača, Milan; Nikolašević, Željka; Mitrović, Dušanka

2018-04-03

The need for a research and practical tool, such as a short, reliable, and valid personality assessment test, suggests researchers to create shortened versions of original instruments. Reinforcement sensitivity questionnaire (RSQ) was created in line with some basic premises of revised Reinforcement sensitivity theory, which proposes three motivational and emotional systems: Behavioral inhibition system (BIS), responsible for scanning environment for potential threats, Behavioral activation system (BAS), responsible for aproaching behavior, and the Fight/Flight/Freeze system (FFFS), responsible for behavior in the present threat. RSQ comprises five scales: BIS, BAS, Fight, Flight, and Freeze. The aim of this study was to develop a short version of RSQ, which would be beneficial to both research and practical purposes. Item response theory analyses were used for item selection. The study comprised two samples of participants, whereby Sample 1 (N = 837, 34.6% male, aged 18 - 82, M = 31.63, SD = 13.54) served as the derivation sample, while Sample 2 (818 participants, 43.6% male, 18-75 years, M = 29.65, SD = 12.52) served as validation sample. Factorial validity of the short RSQ was examined on both Sample 1 and Sample 2. Convergent and divergent validity of the short RSQ was examined using RST-PQ, Jackson-5, BIS/BAS scales, and Big Five Inventory. The results point to satisfactory internal consistency, factorial validity, and construct validity of the short RSQ, suggesting that it is an adequate measure for research settings or other contexts which require the use of short personality questionnaires.

International Space Station Model Correlation Analysis

NASA Technical Reports Server (NTRS)

Laible, Michael R.; Fitzpatrick, Kristin; Hodge, Jennifer; Grygier, Michael

2018-01-01

This paper summarizes the on-orbit structural dynamic data and the related modal analysis, model validation and correlation performed for the International Space Station (ISS) configuration ISS Stage ULF7, 2015 Dedicated Thruster Firing (DTF). The objective of this analysis is to validate and correlate the analytical models used to calculate the ISS internal dynamic loads and compare the 2015 DTF with previous tests. During the ISS configurations under consideration, on-orbit dynamic measurements were collected using the three main ISS instrumentation systems; Internal Wireless Instrumentation System (IWIS), External Wireless Instrumentation System (EWIS) and the Structural Dynamic Measurement System (SDMS). The measurements were recorded during several nominal on-orbit DTF tests on August 18, 2015. Experimental modal analyses were performed on the measured data to extract modal parameters including frequency, damping, and mode shape information. Correlation and comparisons between test and analytical frequencies and mode shapes were performed to assess the accuracy of the analytical models for the configurations under consideration. These mode shapes were also compared to earlier tests. Based on the frequency comparisons, the accuracy of the mathematical models is assessed and model refinement recommendations are given. In particular, results of the first fundamental mode will be discussed, nonlinear results will be shown, and accelerometer placement will be assessed.

Neuroimaging supports behavioral personality assessment: Overlapping activations during reflective and impulsive risk taking.

PubMed

Pletzer, Belinda; M Ortner, Tuulia

2016-09-01

Personality assessment has been challenged by the fact that different assessment methods (implicit measures, behavioral measures and explicit rating scales) show little or no convergence in behavioral studies. In this neuroimaging study we address for the first time, whether different assessment methods rely on separate or overlapping neuronal systems. Fifty nine healthy adult participants completed two objective personality tests of risk propensity: the more implicit Balloon Analogue Risk Task (BART) and the more explicit Game of Dice Task (GDT). Significant differences in activation, as well as connectivity patterns between both tasks were observed. In both tasks, risky decisions yielded significantly stronger activations than safe decisions in the bilateral caudate, as well as the bilateral Insula. The finding of overlapping brain areas validates different assessment methods, despite their behavioral non-convergence. This suggests that neuroimaging can be an important tool of validation in the field of personality assessment. Copyright © 2016 Elsevier B.V. All rights reserved.

Lymphoedema Functioning, Disability and Health Questionnaire for Lower Limb Lymphoedema (Lymph-ICF-LL): reliability and validity.

PubMed

Devoogdt, Nele; De Groef, An; Hendrickx, Ad; Damstra, Robert; Christiaansen, Anke; Geraerts, Inge; Vervloesem, Nele; Vergote, Ignace; Van Kampen, Marijke

2014-05-01

Patients may develop primary (congenital) or secondary (acquired) lymphedema, causing significant physical and psychosocial problems. To plan treatment for lymphedema and monitor a patient's progress, swelling, and problems in functioning associated with lymphedema development should be assessed at baseline and follow-up. The purpose of this study was to investigate the reliability (test-retest, internal consistency, and measurement variability) and validity (content and construct) of data obtained with the Lymphoedema Functioning, Disability and Health Questionnaire for Lower Limb Lymphoedema (Lymph-ICF-LL). This was a multicenter, cross-sectional study. The Lymph-ICF-LL is a descriptive, evaluative tool containing 28 questions about impairments in function, activity limitations, and participation restrictions in patients with lower limb lymphedema. The questionnaire has 5 domains: physical function, mental function, general tasks/household activities, mobility activities, and life domains/social life. The reliability and validity of the Lymph-ICF-LL were examined in 30 participants with objective lower limb lymphedema. Intraclass correlation coefficients for test-retest reliability ranged from .69 to .94, and Cronbach alpha coefficients for internal consistency ranged from .82 to .97. Measurement variability was acceptable (standard error of measurement=5.9-12.6). Content validity was good because all questions were understandable for 93% of participants, the scoring system (visual analog scale) was clear, and the questionnaire was comprehensive for 90% of participants. Construct validity was good. All hypotheses for assessing convergent validity and divergent validity were accepted. The known-groups validity and responsiveness of the Dutch Lymph-ICF-LL and the cross-cultural validity of the English version of the Lymph-ICF-LL were not investigated. The Lymph-ICF-LL is a Dutch questionnaire with evidence of reliability and validity for assessing impairments in function, activity limitations, and participation restrictions in people with primary or secondary lower limb lymphedema.

Computer-assisted update of a consumer health vocabulary through mining of social network data.

PubMed

Doing-Harris, Kristina M; Zeng-Treitler, Qing

2011-05-17

Consumer health vocabularies (CHVs) have been developed to aid consumer health informatics applications. This purpose is best served if the vocabulary evolves with consumers' language. Our objective was to create a computer assisted update (CAU) system that works with live corpora to identify new candidate terms for inclusion in the open access and collaborative (OAC) CHV. The CAU system consisted of three main parts: a Web crawler and an HTML parser, a candidate term filter that utilizes natural language processing tools including term recognition methods, and a human review interface. In evaluation, the CAU system was applied to the health-related social network website PatientsLikeMe.com. The system's utility was assessed by comparing the candidate term list it generated to a list of valid terms hand extracted from the text of the crawled webpages. The CAU system identified 88,994 unique terms 1- to 7-grams ("n-grams" are n consecutive words within a sentence) in 300 crawled PatientsLikeMe.com webpages. The manual review of the crawled webpages identified 651 valid terms not yet included in the OAC CHV or the Unified Medical Language System (UMLS) Metathesaurus, a collection of vocabularies amalgamated to form an ontology of medical terms, (ie, 1 valid term per 136.7 candidate n-grams). The term filter selected 774 candidate terms, of which 237 were valid terms, that is, 1 valid term among every 3 or 4 candidates reviewed. The CAU system is effective for generating a list of candidate terms for human review during CHV development.

[CLIMATE CHANGE AND ALLERGIC AIRWAY DISEASE] OBSERVATIONAL,LABORATORY, AND MODELING STUDIES OF THE IMPACTS OF CLIMATE CHANGE ONALLERGIC AIRWAY DISEASE

EPA Science Inventory

Based on these data and preliminary studies, this proposal will be composed of a multiscale source-to-dose analysis approach for assessing the exposure interactions of environmental and biological systems. Once the entire modeling system is validated, it will run f...

Developing a Multi-Year Learning Progression for Carbon Cycling in Socio-Ecological Systems

ERIC Educational Resources Information Center

Mohan, Lindsey; Chen, Jing; Anderson, Charles W.

2009-01-01

This study reports on our steps toward achieving a conceptually coherent and empirically validated learning progression for carbon cycling in socio-ecological systems. It describes an iterative process of designing and analyzing assessment and interview data from students in upper elementary through high school. The product of our development…

If Language Is a Complex Adaptive System, What Is Language Assessment?

ERIC Educational Resources Information Center

Mislevy, Robert J.; Yin, Chengbin

2009-01-01

Individuals' use of language in contexts emerges from second-to-second processes of activating and integrating traces of past experiences--an interactionist view compatible with the study of language as a complex adaptive system but quite different from the trait-based framework through which measurement specialists investigate validity, establish…

ASSESSMENT OF CHEMICAL EFFECTS ON NEURONAL DIFFERENTIATION USING THE ARRAYSCAN HIGH CONTENT SCREENING SYSTEM

EPA Science Inventory

The development of alternative methods for toxicity testing is driven by the need for scientifically valid data that can be obtained in a rapid and cost-efficient manner. In vitro systems provide a model in which chemical effects on cellular events can be examined using technique...

Validation of a Hybrid Microwave-Optical Monitor to Investigate Thermal Provocation in the Microvasculature.

PubMed

Al-Armaghany, Allann; Tong, Kenneth; Highton, David; Leung, Terence S

2016-01-01

We have previously developed a hybrid microwave-optical system to monitor microvascular changes in response to thermal provocation in muscle. The hybrid probe is capable of inducing deep heat from the skin surface using mild microwaves (1-3 W) and raises the tissue temperature by a few degrees Celsius. This causes vasodilation and the subsequent increase in blood volume is detected by the hybrid probe using near infrared spectroscopy. The hybrid probe is also equipped with a skin cooling system which lowers the skin temperature while allowing microwaves to warm up deeper tissues. The hybrid system can be used to assess the condition of the vasculature in response to thermal stimulation. In this validation study, thermal imaging has been used to assess the temperature distribution on the surface of phantoms and human calf, following microwave warming. The results show that the hybrid system is capable of changing the skin temperature with a combination of microwave warming and skin cooling. It can also detect thermal responses in terms of changes of oxy/deoxy-hemoglobin concentrations.

Potentials of Optical Damage Assessment Techniques in Automotive Crash-Concepts composed of FRP-Steel Hybrid Material Systems

NASA Astrophysics Data System (ADS)

Dlugosch, M.; Spiegelhalter, B.; Soot, T.; Lukaszewicz, D.; Fritsch, J.; Hiermaier, S.

2017-05-01

With car manufacturers simultaneously facing increasing passive safety and efficiency requirements, FRP-metal hybrid material systems are one way to design lightweight and crashworthy vehicle structures. Generic automotive hybrid structural concepts have been tested under crash loading conditions. In order to assess the state of overall damage and structural integrity, and primarily to validate simulation data, several NDT techniques have been assessed regarding their potential to detect common damage mechanisms in such hybrid systems. Significant potentials were found particularly in combining 3D-topography laser scanning and X-Ray imaging results. Ultrasonic testing proved to be limited by the signal coupling quality on damaged or curved surfaces.

Rolling bearing fault diagnosis and health assessment using EEMD and the adjustment Mahalanobis-Taguchi system

NASA Astrophysics Data System (ADS)

Chen, Junxun; Cheng, Longsheng; Yu, Hui; Hu, Shaolin

2018-01-01