Assessment of the Validity of the Research Diagnostic Criteria for Temporomandibular Disorders: Overview and Methodology

PubMed Central

Schiffman, Eric L.; Truelove, Edmond L.; Ohrbach, Richard; Anderson, Gary C.; John, Mike T.; List, Thomas; Look, John O.

2011-01-01

AIMS The purpose of the Research Diagnostic Criteria for Temporomandibular Disorders (RDC/TMD) Validation Project was to assess the diagnostic validity of this examination protocol. An overview is presented, including Axis I and II methodology and descriptive statistics for the study participant sample. This paper details the development of reliable methods to establish the reference standards for assessing criterion validity of the Axis I RDC/TMD diagnoses. Validity testing for the Axis II biobehavioral instruments was based on previously validated reference standards. METHODS The Axis I reference standards were based on the consensus of 2 criterion examiners independently performing a comprehensive history, clinical examination, and evaluation of imaging. Intersite reliability was assessed annually for criterion examiners and radiologists. Criterion exam reliability was also assessed within study sites. RESULTS Study participant demographics were comparable to those of participants in previous studies using the RDC/TMD. Diagnostic agreement of the criterion examiners with each other and with the consensus-based reference standards was excellent with all kappas ≥ 0.81, except for osteoarthrosis (moderate agreement, k = 0.53). Intrasite criterion exam agreement with reference standards was excellent (k ≥ 0.95). Intersite reliability of the radiologists for detecting computed tomography-disclosed osteoarthrosis and magnetic resonance imaging-disclosed disc displacement was good to excellent (k = 0.71 and 0.84, respectively). CONCLUSION The Validation Project study population was appropriate for assessing the reliability and validity of the RDC/TMD Axis I and II. The reference standards used to assess the validity of Axis I TMD were based on reliable and clinically credible methods. PMID:20213028

Assessment of generalizability, applicability and predictability (GAP) for evaluating external validity in studies of universal family-based prevention of alcohol misuse in young people: systematic methodological review of randomized controlled trials.

PubMed

Fernandez-Hermida, Jose Ramon; Calafat, Amador; Becoña, Elisardo; Tsertsvadze, Alexander; Foxcroft, David R

2012-09-01

To assess external validity characteristics of studies from two Cochrane Systematic Reviews of the effectiveness of universal family-based prevention of alcohol misuse in young people. Two reviewers used an a priori developed external validity rating form and independently assessed three external validity dimensions of generalizability, applicability and predictability (GAP) in randomized controlled trials. The majority (69%) of the included 29 studies were rated 'unclear' on the reporting of sufficient information for judging generalizability from sample to study population. Ten studies (35%) were rated 'unclear' on the reporting of sufficient information for judging applicability to other populations and settings. No study provided an assessment of the validity of the trial end-point measures for subsequent mortality, morbidity, quality of life or other economic or social outcomes. Similarly, no study reported on the validity of surrogate measures using established criteria for assessing surrogate end-points. Studies evaluating the benefits of family-based prevention of alcohol misuse in young people are generally inadequate at reporting information relevant to generalizability of the findings or implications for health or social outcomes. Researchers, study authors, peer reviewers, journal editors and scientific societies should take steps to improve the reporting of information relevant to external validity in prevention trials. © 2012 The Authors. Addiction © 2012 Society for the Study of Addiction.

The measurement of threat orientations.

PubMed

Thompson, Suzanne C; Schlehofer, Michèle M; Bovin, Michelle J

2006-01-01

To develop measures of 3 threat orientations that affect responses to health behavior messages. In Study 1, college students (N = 47) completed items assessing threat orientations and health behaviors. In Study 2, college students and community adults (N = 110) completed the threat orientation items and measures of convergent and discriminant validity. In Study 1, the control-based, denial-based, and heightened-sensitivity-based threat orientation scales demonstrated good internal consistency and correlated with engagement in health behaviors. In Study 2, the convergent and discriminant validity of the 3 measures was established. The 3 scales have good internal reliability and construct validity.

Validating a Fidelity Scale to Understand Intervention Effects in Classroom-Based Studies

ERIC Educational Resources Information Center

Buckley, Pamela; Moore, Brooke; Boardman, Alison G.; Arya, Diana J.; Maul, Andrew

2017-01-01

K-12 intervention studies often include fidelity of implementation (FOI) as a mediating variable, though most do not report the validity of fidelity measures. This article discusses the critical need for validated FOI scales. To illustrate our point, we describe the development and validation of the Implementation Validity Checklist (IVC-R), an…

Latency-Based and Psychophysiological Measures of Sexual Interest Show Convergent and Concurrent Validity.

PubMed

Ó Ciardha, Caoilte; Attard-Johnson, Janice; Bindemann, Markus

2018-04-01

Latency-based measures of sexual interest require additional evidence of validity, as do newer pupil dilation approaches. A total of 102 community men completed six latency-based measures of sexual interest. Pupillary responses were recorded during three of these tasks and in an additional task where no participant response was required. For adult stimuli, there was a high degree of intercorrelation between measures, suggesting that tasks may be measuring the same underlying construct (convergent validity). In addition to being correlated with one another, measures also predicted participants' self-reported sexual interest, demonstrating concurrent validity (i.e., the ability of a task to predict a more validated, simultaneously recorded, measure). Latency-based and pupillometric approaches also showed preliminary evidence of concurrent validity in predicting both self-reported interest in child molestation and viewing pornographic material containing children. Taken together, the study findings build on the evidence base for the validity of latency-based and pupillometric measures of sexual interest.

Assessing impact of physical activity-based youth development programs: validation of the Life Skills Transfer Survey (LSTS).

PubMed

Weiss, Maureen R; Bolter, Nicole D; Kipp, Lindsay E

2014-09-01

A signature characteristic of positive youth development (PYD) programs is the opportunity to develop life skills, such as social, behavioral, and moral competencies, that can be generalized to domains beyond the immediate activity. Although context-specific instruments are available to assess developmental outcomes, a measure of life skills transfer would enable evaluation of PYD programs in successfully teaching skills that youth report using in other domains. The purpose of our studies was to develop and validate a measure of perceived life skills transfer, based on data collected with The First Tee, a physical activity-based PYD program. In 3 studies, we conducted a series of steps to provide content and construct validity and internal consistency reliability for the Life Skills Transfer Survey (LSTS), a measure of perceived life skills transfer. Study 1 provided content validity for the LSTS that included 8 life skills and 50 items. Study 2 revealed construct validity (structural validity) through a confirmatory factor analysis and convergent validity by correlating scores on the LSTS with scores on an assessment tool that measures a related construct. Study 3 offered additional construct validity by reassessing youth 1 year later and showing that scores during both time periods were invariant in factor pattern, loadings, and variances and covariances. Studies 2 and 3 demonstrated internal consistency reliability of the LSTS. RESULTS from 3 studies provide evidence of content and construct validity and internal consistency reliability for the LSTS, which can be used in evaluation research with youth development programs.

Gathering Validity Evidence for Surgical Simulation: A Systematic Review.

PubMed

Borgersen, Nanna Jo; Naur, Therese M H; Sørensen, Stine M D; Bjerrum, Flemming; Konge, Lars; Subhi, Yousif; Thomsen, Ann Sofia S

2018-06-01

To identify current trends in the use of validity frameworks in surgical simulation, to provide an overview of the evidence behind the assessment of technical skills in all surgical specialties, and to present recommendations and guidelines for future validity studies. Validity evidence for assessment tools used in the evaluation of surgical performance is of paramount importance to ensure valid and reliable assessment of skills. We systematically reviewed the literature by searching 5 databases (PubMed, EMBASE, Web of Science, PsycINFO, and the Cochrane Library) for studies published from January 1, 2008, to July 10, 2017. We included original studies evaluating simulation-based assessments of health professionals in surgical specialties and extracted data on surgical specialty, simulator modality, participant characteristics, and the validity framework used. Data were synthesized qualitatively. We identified 498 studies with a total of 18,312 participants. Publications involving validity assessments in surgical simulation more than doubled from 2008 to 2010 (∼30 studies/year) to 2014 to 2016 (∼70 to 90 studies/year). Only 6.6% of the studies used the recommended contemporary validity framework (Messick). The majority of studies used outdated frameworks such as face validity. Significant differences were identified across surgical specialties. The evaluated assessment tools were mostly inanimate or virtual reality simulation models. An increasing number of studies have gathered validity evidence for simulation-based assessments in surgical specialties, but the use of outdated frameworks remains common. To address the current practice, this paper presents guidelines on how to use the contemporary validity framework when designing validity studies.

Reliability and validity of play-based assessments of motor and cognitive skills for infants and young children: a systematic review.

PubMed

O'Grady, Michael G; Dusing, Stacey C

2015-01-01

Play is vital for development. Infants and children learn through play. Traditional standardized developmental tests measure whether a child performs individual skills within controlled environments. Play-based assessments can measure skill performance during natural, child-driven play. The purpose of this study was to systematically review reliability, validity, and responsiveness of all play-based assessments that quantify motor and cognitive skills in children from birth to 36 months of age. Studies were identified from a literature search using PubMed, ERIC, CINAHL, and PsycINFO databases and the reference lists of included papers. Included studies investigated reliability, validity, or responsiveness of play-based assessments that measured motor and cognitive skills for children to 36 months of age. Two reviewers independently screened 40 studies for eligibility and inclusion. The reviewers independently extracted reliability, validity, and responsiveness data. They examined measurement properties and methodological quality of the included studies. Four current play-based assessment tools were identified in 8 included studies. Each play-based assessment tool measured motor and cognitive skills in a different way during play. Interrater reliability correlations ranged from .86 to .98 for motor development and from .23 to .90 for cognitive development. Test-retest reliability correlations ranged from .88 to .95 for motor development and from .45 to .91 for cognitive development. Structural validity correlations ranged from .62 to .90 for motor development and from .42 to .93 for cognitive development. One study assessed responsiveness to change in motor development. Most studies had small and poorly described samples. Lack of transparency in data management and statistical analysis was common. Play-based assessments have potential to be reliable and valid tools to assess cognitive and motor skills, but higher-quality research is needed. Psychometric properties should be considered for each play-based assessment before it is used in clinical and research practice. © 2015 American Physical Therapy Association.

Validity And Practicality of Experiment Integrated Guided Inquiry-Based Module on Topic of Colloidal Chemistry for Senior High School Learning

NASA Astrophysics Data System (ADS)

Andromeda, A.; Lufri; Festiyed; Ellizar, E.; Iryani, I.; Guspatni, G.; Fitri, L.

2018-04-01

This Research & Development study aims to produce a valid and practical experiment integrated guided inquiry based module on topic of colloidal chemistry. 4D instructional design model was selected in this study. Limited trial of the product was conducted at SMAN 7 Padang. Instruments used were validity and practicality questionnaires. Validity and practicality data were analyzed using Kappa moment. Analysis of the data shows that Kappa moment for validity was 0.88 indicating a very high degree of validity. Kappa moments for the practicality from students and teachers were 0.89 and 0.95 respectively indicating high degree of practicality. Analysis on the module filled in by students shows that 91.37% students could correctly answer critical thinking, exercise, prelab, postlab and worksheet questions asked in the module. These findings indicate that the integrated guided inquiry based module on topic of colloidal chemistry was valid and practical for chemistry learning in senior high school.

Single-Item Screening for Agoraphobic Symptoms: Validation of a Web-Based Audiovisual Screening Instrument

PubMed Central

van Ballegooijen, Wouter; Riper, Heleen; Donker, Tara; Martin Abello, Katherina; Marks, Isaac; Cuijpers, Pim

2012-01-01

The advent of web-based treatments for anxiety disorders creates a need for quick and valid online screening instruments, suitable for a range of social groups. This study validates a single-item multimedia screening instrument for agoraphobia, part of the Visual Screener for Common Mental Disorders (VS-CMD), and compares it with the text-based agoraphobia items of the PDSS-SR. The study concerned 85 subjects in an RCT of the effects of web-based therapy for panic symptoms. The VS-CMD item and items 4 and 5 of the PDSS-SR were validated by comparing scores to the outcomes of the CIDI diagnostic interview. Screening for agoraphobia was found moderately valid for both the multimedia item (sensitivity.81, specificity.66, AUC.734) and the text-based items (AUC.607–.697). Single-item multimedia screening for anxiety disorders should be further developed and tested in the general population and in patient, illiterate and immigrant samples. PMID:22844391

The validity and reproducibility of food-frequency questionnaire-based total antioxidant capacity estimates in Swedish women.

PubMed

Rautiainen, Susanne; Serafini, Mauro; Morgenstern, Ralf; Prior, Ronald L; Wolk, Alicja

2008-05-01

Total antioxidant capacity (TAC) provides an assessment of antioxidant activity and synergistic interactions of redox molecules in foods and plasma. We investigated the validity and reproducibility of food-frequency questionnaire (FFQ)-based TAC estimates assessed by oxygen radical absorbance capacity (ORAC), total radical-trapping antioxidant parameters (TRAP), and ferric-reducing antioxidant power (FRAP) food values. Validity and reproducibility were evaluated in 2 random samples from the Swedish Mammography Cohort. Validity was studied by comparing FFQ-based TAC estimates with one measurement of plasma TAC in 108 women (54-73-y-old dietary supplement nonusers). Reproducibility was studied in 300 women (56-75 y old, 50.7% dietary supplement nonusers) who completed 2 FFQs 1 y apart. Fruit and vegetables (mainly apples, pears, oranges, and berries) were the major contributors to FFQ-based ORAC (56.5%), TRAP (41.7%), and FRAP (38.0%) estimates. In the validity study, whole plasma ORAC was correlated (Pearson) with FFQ-based ORAC (r = 0.35), TRAP (r = 0.31), and FRAP (r = 0.28) estimates from fruit and vegetables. Correlations between lipophilic plasma ORAC and FFQ-based ORAC, TRAP, and FRAP estimates from fruit and vegetables were 0.41, 0.31, and 0.28, and correlations with plasma TRAP estimates were 0.31, 0.30, and 0.28, respectively. Hydrophilic plasma ORAC and plasma FRAP values did not correlate with FFQ-based TAC estimates. Reproducibility, assessed by intraclass correlations, was 0.60, 0.61, and 0.61 for FFQ-based ORAC, TRAP, and FRAP estimates, respectively, from fruit and vegetables. FFQ-based TAC values represent valid and reproducible estimates that may be used in nutritional epidemiology to assess antioxidant intake from foods. Further studies in other populations to confirm these results are needed.

Clinical Validity of the ADI-R in a US-Based Latino Population

ERIC Educational Resources Information Center

Vanegas, Sandra B.; Magaña, Sandra; Morales, Miguel; McNamara, Ellyn

2016-01-01

The Autism Diagnostic Interview-Revised (ADI-R) has been validated as a tool to aid in the diagnosis of Autism; however, given the growing diversity in the United States, the ADI-R must be validated for different languages and cultures. This study evaluates the validity of the ADI-R in a US-based Latino, Spanish-speaking population of 50 children…

Cooperative Learning: Improving University Instruction by Basing Practice on Validated Theory

ERIC Educational Resources Information Center

Johnson, David W.; Johnson, Roger T.; Smith, Karl A.

2014-01-01

Cooperative learning is an example of how theory validated by research may be applied to instructional practice. The major theoretical base for cooperative learning is social interdependence theory. It provides clear definitions of cooperative, competitive, and individualistic learning. Hundreds of research studies have validated its basic…

Developing CORE model-based worksheet with recitation task to facilitate students’ mathematical communication skills in linear algebra course

NASA Astrophysics Data System (ADS)

Risnawati; Khairinnisa, S.; Darwis, A. H.

2018-01-01

The purpose of this study was to develop a CORE model-based worksheet with recitation task that were valid and practical and could facilitate students’ communication skills in Linear Algebra course. This study was conducted in mathematics education department of one public university in Riau, Indonesia. Participants of the study were media and subject matter experts as validators as well as students from mathematics education department. The objects of this study are students’ worksheet and students’ mathematical communication skills. The results of study showed that: (1) based on validation of the experts, the developed students’ worksheet was valid and could be applied for students in Linear Algebra courses; (2) based on the group trial, the practicality percentage was 92.14% in small group and 90.19% in large group, so the worksheet was very practical and could attract students to learn; and (3) based on the post test, the average percentage of ideals was 87.83%. In addition, the results showed that the students’ worksheet was able to facilitate students’ mathematical communication skills in linear algebra course.

Validity of Cognitive Load Measures in Simulation-Based Training: A Systematic Review.

PubMed

Naismith, Laura M; Cavalcanti, Rodrigo B

2015-11-01

Cognitive load theory (CLT) provides a rich framework to inform instructional design. Despite the applicability of CLT to simulation-based medical training, findings from multimedia learning have not been consistently replicated in this context. This lack of transferability may be related to issues in measuring cognitive load (CL) during simulation. The authors conducted a review of CLT studies across simulation training contexts to assess the validity evidence for different CL measures. PRISMA standards were followed. For 48 studies selected from a search of MEDLINE, EMBASE, PsycInfo, CINAHL, and ERIC databases, information was extracted about study aims, methods, validity evidence of measures, and findings. Studies were categorized on the basis of findings and prevalence of validity evidence collected, and statistical comparisons between measurement types and research domains were pursued. CL during simulation training has been measured in diverse populations including medical trainees, pilots, and university students. Most studies (71%; 34) used self-report measures; others included secondary task performance, physiological indices, and observer ratings. Correlations between CL and learning varied from positive to negative. Overall validity evidence for CL measures was low (mean score 1.55/5). Studies reporting greater validity evidence were more likely to report that high CL impaired learning. The authors found evidence that inconsistent correlations between CL and learning may be related to issues of validity in CL measures. Further research would benefit from rigorous documentation of validity and from triangulating measures of CL. This can better inform CLT instructional design for simulation-based medical training.

Current Status of Simulation-based Training Tools in Orthopedic Surgery: A Systematic Review.

PubMed

Morgan, Michael; Aydin, Abdullatif; Salih, Alan; Robati, Shibby; Ahmed, Kamran

To conduct a systematic review of orthopedic training and assessment simulators with reference to their level of evidence (LoE) and level of recommendation. Medline and EMBASE library databases were searched for English language articles published between 1980 and 2016, describing orthopedic simulators or validation studies of these models. All studies were assessed for LoE, and each model was subsequently awarded a level of recommendation using a modified Oxford Centre for Evidence-Based Medicine classification, adapted for education. A total of 76 articles describing orthopedic simulators met the inclusion criteria, 47 of which described at least 1 validation study. The most commonly identified models (n = 34) and validation studies (n = 26) were for knee arthroscopy. Construct validation was the most frequent validation study attempted by authors. In all, 62% (47 of 76) of the simulator studies described arthroscopy simulators, which also contained validation studies with the highest LoE. Orthopedic simulators are increasingly being subjected to validation studies, although the LoE of such studies generally remain low. There remains a lack of focus on nontechnical skills and on cost analyses of orthopedic simulators. Copyright © 2017 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Agent-Based vs. Equation-based Epidemiological Models:A Model Selection Case Study

DOE Office of Scientific and Technical Information (OSTI.GOV)

Sukumar, Sreenivas R; Nutaro, James J

This paper is motivated by the need to design model validation strategies for epidemiological disease-spread models. We consider both agent-based and equation-based models of pandemic disease spread and study the nuances and complexities one has to consider from the perspective of model validation. For this purpose, we instantiate an equation based model and an agent based model of the 1918 Spanish flu and we leverage data published in the literature for our case- study. We present our observations from the perspective of each implementation and discuss the application of model-selection criteria to compare the risk in choosing one modeling paradigmmore » to another. We conclude with a discussion of our experience and document future ideas for a model validation framework.« less

Development and validation of the Body and Appearance Self-Conscious Emotions Scale (BASES).

PubMed

Castonguay, Andrée L; Sabiston, Catherine M; Crocker, Peter R E; Mack, Diane E

2014-03-01

The purpose of these studies was to develop a psychometrically sound measure of shame, guilt, authentic pride, and hubristic pride for use in body and appearance contexts. In Study 1, 41 potential items were developed and assessed for item quality and comprehension. In Study 2, a panel of experts (N=8; M=11, SD=6.5 years of experience) reviewed the scale and items for evidence of content validity. Participants in Study 3 (n=135 males, n=300 females) completed the BASES and various body image, personality, and emotion scales. A separate sample (n=155; 35.5% male) in Study 3 completed the BASES twice using a two-week time interval. The BASES subscale scores demonstrated evidence for internal consistency, item-total correlations, concurrent, convergent, incremental, and discriminant validity, and 2-week test-retest reliability. The 4-factor solution was a good fit in confirmatory factor analysis, reflecting body-related shame, guilt, authentic and hubristic pride subscales of the BASES. The development and validation of the BASES may help advance body image and self-conscious emotion research by providing a foundation to examine the unique antecedents and outcomes of these specific emotional experiences. Copyright © 2014 Elsevier Ltd. All rights reserved.

Development and Validation of Extract the Base: An English Derivational Morphology Test for Third through Fifth Grade Monolingual Students and Spanish-Speaking English Language Learners

ERIC Educational Resources Information Center

Goodwin, Amanda P.; Huggins, A. Corinne; Carlo, Maria; Malabonga, Valerie; Kenyon, Dorry; Louguit, Mohammed; August, Diane

2012-01-01

This study describes the development and validation of the Extract the Base test (ETB), which assesses derivational morphological awareness. Scores on this test were validated for 580 monolingual students and 373 Spanish-speaking English language learners (ELLs) in third through fifth grade. As part of the validation of the internal structure,…

The Development and Validation of the School-Based Counseling Self-Efficacy Scale

ERIC Educational Resources Information Center

Boughfman, Erica M.

2010-01-01

The purpose of this study was to develop and validate the School-Based Counseling Self-Efficacy Scale (SB-SES). Two hundred sixty-five (N = 265) licensed mental health professionals participated in this study. Fifty-eight percent of the participants reported experience working as a school-based counselor with the remaining 42% reporting no…

Temporal validation for landsat-based volume estimation model

Treesearch

Renaldo J. Arroyo; Emily B. Schultz; Thomas G. Matney; David L. Evans; Zhaofei Fan

2015-01-01

Satellite imagery can potentially reduce the costs and time associated with ground-based forest inventories; however, for satellite imagery to provide reliable forest inventory data, it must produce consistent results from one time period to the next. The objective of this study was to temporally validate a Landsat-based volume estimation model in a four county study...

Convergent Validity Evidence regarding the Validity of the Chilean Standards-Based Teacher Evaluation System

ERIC Educational Resources Information Center

Santelices, Maria Veronica; Taut, Sandy

2011-01-01

This paper describes convergent validity evidence regarding the mandatory, standards-based Chilean national teacher evaluation system (NTES). The study examined whether NTES identifies--and thereby rewards or punishes--the "right" teachers as high- or low-performing. We collected in-depth teaching performance data on a sample of 58…

The predictive value of the sacral base pressure test in detecting specific types of sacroiliac dysfunction

PubMed Central

Mitchell, Travis D.; Urli, Kristina E.; Breitenbach, Jacques; Yelverton, Chris

2007-01-01

Abstract Objective This study aimed to evaluate the validity of the sacral base pressure test in diagnosing sacroiliac joint dysfunction. It also determined the predictive powers of the test in determining which type of sacroiliac joint dysfunction was present. Methods This was a double-blind experimental study with 62 participants. The results from the sacral base pressure test were compared against a cluster of previously validated tests of sacroiliac joint dysfunction to determine its validity and predictive powers. The external rotation of the feet, occurring during the sacral base pressure test, was measured using a digital inclinometer. Results There was no statistically significant difference in the results of the sacral base pressure test between the types of sacroiliac joint dysfunction. In terms of the results of validity, the sacral base pressure test was useful in identifying positive values of sacroiliac joint dysfunction. It was fairly helpful in correctly diagnosing patients with negative test results; however, it had only a “slight” agreement with the diagnosis for κ interpretation. Conclusions In this study, the sacral base pressure test was not a valid test for determining the presence of sacroiliac joint dysfunction or the type of dysfunction present. Further research comparing the agreement of the sacral base pressure test or other sacroiliac joint dysfunction tests with a criterion standard of diagnosis is necessary. PMID:19674694

Assessing Procedural Competence: Validity Considerations.

PubMed

Pugh, Debra M; Wood, Timothy J; Boulet, John R

2015-10-01

Simulation-based medical education (SBME) offers opportunities for trainees to learn how to perform procedures and to be assessed in a safe environment. However, SBME research studies often lack robust evidence to support the validity of the interpretation of the results obtained from tools used to assess trainees' skills. The purpose of this paper is to describe how a validity framework can be applied when reporting and interpreting the results of a simulation-based assessment of skills related to performing procedures. The authors discuss various sources of validity evidence because they relate to SBME. A case study is presented.

The Contribution of Rubrics to the Validity of Performance Assessment: A Study of the Conservation-Restoration and Design Undergraduate Degrees

ERIC Educational Resources Information Center

Menéndez-Varela, José-Luis; Gregori-Giralt, Eva

2016-01-01

Rubrics have attained considerable importance in the authentic and sustainable assessment paradigm; nevertheless, few studies have examined their contribution to validity, especially outside the domain of educational studies. This empirical study used a quantitative approach to analyse the validity of a rubrics-based performance assessment. Raters…

Validation of biomarkers of food intake-critical assessment of candidate biomarkers.

PubMed

Dragsted, L O; Gao, Q; Scalbert, A; Vergères, G; Kolehmainen, M; Manach, C; Brennan, L; Afman, L A; Wishart, D S; Andres Lacueva, C; Garcia-Aloy, M; Verhagen, H; Feskens, E J M; Praticò, G

2018-01-01

Biomarkers of food intake (BFIs) are a promising tool for limiting misclassification in nutrition research where more subjective dietary assessment instruments are used. They may also be used to assess compliance to dietary guidelines or to a dietary intervention. Biomarkers therefore hold promise for direct and objective measurement of food intake. However, the number of comprehensively validated biomarkers of food intake is limited to just a few. Many new candidate biomarkers emerge from metabolic profiling studies and from advances in food chemistry. Furthermore, candidate food intake biomarkers may also be identified based on extensive literature reviews such as described in the guidelines for Biomarker of Food Intake Reviews (BFIRev). To systematically and critically assess the validity of candidate biomarkers of food intake, it is necessary to outline and streamline an optimal and reproducible validation process. A consensus-based procedure was used to provide and evaluate a set of the most important criteria for systematic validation of BFIs. As a result, a validation procedure was developed including eight criteria, plausibility, dose-response, time-response, robustness, reliability, stability, analytical performance, and inter-laboratory reproducibility. The validation has a dual purpose: (1) to estimate the current level of validation of candidate biomarkers of food intake based on an objective and systematic approach and (2) to pinpoint which additional studies are needed to provide full validation of each candidate biomarker of food intake. This position paper on biomarker of food intake validation outlines the second step of the BFIRev procedure but may also be used as such for validation of new candidate biomarkers identified, e.g., in food metabolomic studies.

The Turkish Version of Web-Based Learning Platform Evaluation Scale: Reliability and Validity Study

ERIC Educational Resources Information Center

Dag, Funda

2016-01-01

The purpose of this study is to determine the language equivalence and the validity and reliability of the Turkish version of the "Web-Based Learning Platform Evaluation Scale" ("Web Tabanli Ögrenme Ortami Degerlendirme Ölçegi" [WTÖODÖ]) used in the selection and evaluation of web-based learning environments. Within this scope,…

Screening tool for oropharyngeal dysphagia in stroke - Part I: evidence of validity based on the content and response processes.

PubMed

Almeida, Tatiana Magalhães de; Cola, Paula Cristina; Pernambuco, Leandro de Araújo; Magalhães, Hipólito Virgílio; Magnoni, Carlos Daniel; Silva, Roberta Gonçalves da

2017-08-17

The aim of the present study was to identify the evidence of validity based on the content and response process of the Rastreamento de Disfagia Orofaríngea no Acidente Vascular Encefálico (RADAVE; "Screening Tool for Oropharyngeal Dysphagia in Stroke"). The criteria used to elaborate the questions were based on a literature review. A group of judges consisting of 19 different health professionals evaluated the relevance and representativeness of the questions, and the results were analyzed using the Content Validity Index. In order to evidence validity based on the response processes, 23 health professionals administered the screening tool and analyzed the questions using a structured scale and cognitive interview. The RADAVE structured to be applied in two stages. The first version consisted of 18 questions in stage I and 11 questions in stage II. Eight questions in stage I and four in stage II did not reach the minimum Content Validity Index, requiring reformulation by the authors. The cognitive interview demonstrated some misconceptions. New adjustments were made and the final version was produced with 12 questions in stage I and six questions in stage II. It was possible to develop a screening tool for dysphagia in stroke with adequate evidence of validity based on content and response processes. Both validity evidences obtained so far allowed to adjust the screening tool in relation to its construct. The next studies will analyze the other evidences of validity and the measures of accuracy.

Exploring Validity of Computer-Based Test Scores with Examinees' Response Behaviors and Response Times

ERIC Educational Resources Information Center

Sahin, Füsun

2017-01-01

Examining the testing processes, as well as the scores, is needed for a complete understanding of validity and fairness of computer-based assessments. Examinees' rapid-guessing and insufficient familiarity with computers have been found to be major issues that weaken the validity arguments of scores. This study has three goals: (a) improving…

Predictive Validity of Curriculum-Based Measures for English Learners at Varying English Proficiency Levels

ERIC Educational Resources Information Center

Kim, Jennifer Sun; Vanderwood, Michael L.; Lee, Catherine Y.

2016-01-01

This study examined the predictive validity of curriculum-based measures in reading for Spanish-speaking English learners (ELs) at various levels of English proficiency. Third-grade Spanish-speaking EL students were screened during the fall using DIBELS Oral Reading Fluency (DORF) and Daze. Predictive validity was examined in relation to spring…

Helping Students Evaluate the Validity of a Research Study.

ERIC Educational Resources Information Center

Morgan, George A.; Gliner, Jeffrey A.

Students often have difficulty in evaluating the validity of a study. A conceptually and linguistically meaningful framework for evaluating research studies is proposed that is based on the discussion of internal and external validity of T. D. Cook and D. T. Campbell (1979). The proposal includes six key dimensions, three related to internal…

A Framework for Text Mining in Scientometric Study: A Case Study in Biomedicine Publications

NASA Astrophysics Data System (ADS)

Silalahi, V. M. M.; Hardiyati, R.; Nadhiroh, I. M.; Handayani, T.; Rahmaida, R.; Amelia, M.

2018-04-01

The data of Indonesians research publications in the domain of biomedicine has been collected to be text mined for the purpose of a scientometric study. The goal is to build a predictive model that provides a classification of research publications on the potency for downstreaming. The model is based on the drug development processes adapted from the literatures. An effort is described to build the conceptual model and the development of a corpus on the research publications in the domain of Indonesian biomedicine. Then an investigation is conducted relating to the problems associated with building a corpus and validating the model. Based on our experience, a framework is proposed to manage the scientometric study based on text mining. Our method shows the effectiveness of conducting a scientometric study based on text mining in order to get a valid classification model. This valid model is mainly supported by the iterative and close interactions with the domain experts starting from identifying the issues, building a conceptual model, to the labelling, validation and results interpretation.

Do short courses in evidence based medicine improve knowledge and skills? Validation of Berlin questionnaire and before and after study of courses in evidence based medicine

PubMed Central

Fritsche, L; Greenhalgh, T; Falck-Ytter, Y; Neumayer, H-H; Kunz, R

2002-01-01

Objective To develop and validate an instrument for measuring knowledge and skills in evidence based medicine and to investigate whether short courses in evidence based medicine lead to a meaningful increase in knowledge and skills. Design Development and validation of an assessment instrument and before and after study. Setting Various postgraduate short courses in evidence based medicine in Germany. Participants The instrument was validated with experts in evidence based medicine, postgraduate doctors, and medical students. The effect of courses was assessed by postgraduate doctors from medical and surgical backgrounds. Intervention Intensive 3 day courses in evidence based medicine delivered through tutor facilitated small groups. Main outcome measure Increase in knowledge and skills. Results The questionnaire distinguished reliably between groups with different expertise in evidence based medicine. Experts attained a threefold higher average score than students. Postgraduates who had not attended a course performed better than students but significantly worse than experts. Knowledge and skills in evidence based medicine increased after the course by 57% (mean score before course 6.3 (SD 2.9) v 9.9 (SD 2.8), P<0.001). No difference was found among experts or students in absence of an intervention. Conclusions The instrument reliably assessed knowledge and skills in evidence based medicine. An intensive 3 day course in evidence based medicine led to a significant increase in knowledge and skills. What is already known on this topicNumerous observational studies have investigated the impact of teaching evidence based medicine to healthcare professionals, with conflicting resultsMost of the studies were of poor methodological qualityWhat this study addsAn instrument assessing basic knowledge and skills required for practising evidence based medicine was developed and validatedAn intensive 3 day course on evidence based medicine for doctors from various backgrounds and training level led to a clinically meaningful improvement of knowledge and skills PMID:12468485

The Virtual Care Climate Questionnaire: Development and Validation of a Questionnaire Measuring Perceived Support for Autonomy in a Virtual Care Setting.

PubMed

Smit, Eline Suzanne; Dima, Alexandra Lelia; Immerzeel, Stephanie Annette Maria; van den Putte, Bas; Williams, Geoffrey Colin

2017-05-08

Web-based health behavior change interventions may be more effective if they offer autonomy-supportive communication facilitating the internalization of motivation for health behavior change. Yet, at this moment no validated tools exist to assess user-perceived autonomy-support of such interventions. The aim of this study was to develop and validate the virtual climate care questionnaire (VCCQ), a measure of perceived autonomy-support in a virtual care setting. Items were developed based on existing questionnaires and expert consultation and were pretested among experts and target populations. The virtual climate care questionnaire was administered in relation to Web-based interventions aimed at reducing consumption of alcohol (Study 1; N=230) or cannabis (Study 2; N=228). Item properties, structural validity, and reliability were examined with item-response and classical test theory methods, and convergent and divergent validity via correlations with relevant concepts. In Study 1, 20 of 23 items formed a one-dimensional scale (alpha=.97; omega=.97; H=.66; mean 4.9 [SD 1.0]; range 1-7) that met the assumptions of monotonicity and invariant item ordering. In Study 2, 16 items fitted these criteria (alpha=.92; H=.45; omega=.93; mean 4.2 [SD 1.1]; range 1-7). Only 15 items remained in the questionnaire in both studies, thus we proceeded to the analyses of the questionnaire's reliability and construct validity with a 15-item version of the virtual climate care questionnaire. Convergent validity of the resulting 15-item virtual climate care questionnaire was confirmed by positive associations with autonomous motivation (Study 1: r=.66, P<.001; Study 2: r=.37, P<.001) and perceived competence for reducing alcohol intake (Study 1: r=.52, P<.001). Divergent validity could only be confirmed by the nonsignificant association with perceived competence for learning (Study 2: r=.05, P=.48). The virtual climate care questionnaire accurately assessed participants' perceived autonomy-support offered by two Web-based health behavior change interventions. Overall, the scale showed the expected properties and relationships with relevant concepts, and the studies presented suggest this first version of the virtual climate care questionnaire to be reasonably valid and reliable. As a result, the current version may cautiously be used in future research and practice to measure perceived support for autonomy within a virtual care climate. Future research efforts are required that focus on further investigating the virtual climate care questionnaire's divergent validity, on determining the virtual climate care questionnaire's validity and reliability when used in the context of Web-based interventions aimed at improving nonaddictive or other health behaviors, and on developing and validating a short form virtual climate care questionnaire. ©Eline Suzanne Smit, Alexandra Lelia Dima, Stephanie Annette Maria Immerzeel, Bas van den Putte, Geoffrey Colin Williams. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 08.05.2017.

A Field-Based Testing Protocol for Assessing Gross Motor Skills in Preschool Children: The Children's Activity and Movement in Preschool Study Motor Skills Protocol

ERIC Educational Resources Information Center

Williams, Harriet G.; Pfeiffer, Karin A.; Dowda, Marsha; Jeter, Chevy; Jones, Shaverra; Pate, Russell R.

2009-01-01

The purpose of this study was to develop a valid and reliable tool for use in assessing motor skills in preschool children in field-based settings. The development of the Children's Activity and Movement in Preschool Study Motor Skills Protocol included evidence of its reliability and validity for use in field-based environments as part of large…

Development and validation of the simulation-based learning evaluation scale.

PubMed

Hung, Chang-Chiao; Liu, Hsiu-Chen; Lin, Chun-Chih; Lee, Bih-O

2016-05-01

The instruments that evaluate a student's perception of receiving simulated training are English versions and have not been tested for reliability or validity. The aim of this study was to develop and validate a Chinese version Simulation-Based Learning Evaluation Scale (SBLES). Four stages were conducted to develop and validate the SBLES. First, specific desired competencies were identified according to the National League for Nursing and Taiwan Nursing Accreditation Council core competencies. Next, the initial item pool was comprised of 50 items related to simulation that were drawn from the literature of core competencies. Content validity was established by use of an expert panel. Finally, exploratory factor analysis and confirmatory factor analysis were conducted for construct validity, and Cronbach's coefficient alpha determined the scale's internal consistency reliability. Two hundred and fifty students who had experienced simulation-based learning were invited to participate in this study. Two hundred and twenty-five students completed and returned questionnaires (response rate=90%). Six items were deleted from the initial item pool and one was added after an expert panel review. Exploratory factor analysis with varimax rotation revealed 37 items remaining in five factors which accounted for 67% of the variance. The construct validity of SBLES was substantiated in a confirmatory factor analysis that revealed a good fit of the hypothesized factor structure. The findings tally with the criterion of convergent and discriminant validity. The range of internal consistency for five subscales was .90 to .93. Items were rated on a 5-point scale from 1 (strongly disagree) to 5 (strongly agree). The results of this study indicate that the SBLES is valid and reliable. The authors recommend that the scale could be applied in the nursing school to evaluate the effectiveness of simulation-based learning curricula. Copyright © 2016 Elsevier Ltd. All rights reserved.

Studies on mineral dust using airborne lidar, ground-based remote sensing, and in situ instrumentation

NASA Astrophysics Data System (ADS)

Marenco, Franco; Ryder, Claire; Estellés, Victor; Segura, Sara; Amiridis, Vassilis; Proestakis, Emmanouil; Marinou, Eleni; Tsekeri, Alexandra; Smith, Helen; Ulanowski, Zbigniew; O'Sullivan, Debbie; Brooke, Jennifer; Pradhan, Yaswant; Buxmann, Joelle

2018-04-01

In August 2015, the AER-D campaign made use of the FAAM research aircraft based in Cape Verde, and targeted mineral dust. First results will be shown here. The campaign had multiple objectives: (1) lidar dust mapping for the validation of satellite and model products; (2) validation of sunphotometer remote sensing with airborne measurements; (3) coordinated measurements with the CATS lidar on the ISS; (4) radiative closure studies; and (5) the validation of a new model of dustsonde.

Validating the Attitudes toward Sex Offenders Scale

ERIC Educational Resources Information Center

Whitehead, Charmeka

2009-01-01

This study investigated the reliability and validity of the Attitudes Toward Sex Offenders Scale (ATS) measurement by assessing attitudes of counselors in training towards juvenile sex offenders. The specific aims of this study were to determine (a) internal consistency of the ATS, (b) construct validity of the ATS based on exploratory factor…

Qualitative Research Findings: What Do We Do to Improve and Estimate Their Validity?

ERIC Educational Resources Information Center

Dawson, Judith A.

This paper is based on the premise that relatively little is known about how to improve validity in qualitative research and less is known about how to estimate validity in studies conducted by others. The purpose of the study was to describe the conceptualization of validity in qualitative inquiry to determine how it was used by the author of a…

The Research Diagnostic Criteria for Temporomandibular Disorders. I: overview and methodology for assessment of validity.

PubMed

Schiffman, Eric L; Truelove, Edmond L; Ohrbach, Richard; Anderson, Gary C; John, Mike T; List, Thomas; Look, John O

2010-01-01

The purpose of the Research Diagnostic Criteria for Temporomandibular Disorders (RDC/TMD) Validation Project was to assess the diagnostic validity of this examination protocol. The aim of this article is to provide an overview of the project's methodology, descriptive statistics, and data for the study participant sample. This article also details the development of reliable methods to establish the reference standards for assessing criterion validity of the Axis I RDC/TMD diagnoses. The Axis I reference standards were based on the consensus of two criterion examiners independently performing a comprehensive history, clinical examination, and evaluation of imaging. Intersite reliability was assessed annually for criterion examiners and radiologists. Criterion examination reliability was also assessed within study sites. Study participant demographics were comparable to those of participants in previous studies using the RDC/TMD. Diagnostic agreement of the criterion examiners with each other and with the consensus-based reference standards was excellent with all kappas > or = 0.81, except for osteoarthrosis (moderate agreement, k = 0.53). Intrasite criterion examiner agreement with reference standards was excellent (k > or = 0.95). Intersite reliability of the radiologists for detecting computed tomography-disclosed osteoarthrosis and magnetic resonance imaging-disclosed disc displacement was good to excellent (k = 0.71 and 0.84, respectively). The Validation Project study population was appropriate for assessing the reliability and validity of the RDC/TMD Axis I and II. The reference standards used to assess the validity of Axis I TMD were based on reliable and clinically credible methods.

Quality appraisal of generic self-reported instruments measuring health-related productivity changes: a systematic review

PubMed Central

2014-01-01

Background Health impairments can result in disability and changed work productivity imposing considerable costs for the employee, employer and society as a whole. A large number of instruments exist to measure health-related productivity changes; however their methodological quality remains unclear. This systematic review critically appraised the measurement properties in generic self-reported instruments that measure health-related productivity changes to recommend appropriate instruments for use in occupational and economic health practice. Methods PubMed, PsycINFO, Econlit and Embase were systematically searched for studies whereof: (i) instruments measured health-related productivity changes; (ii) the aim was to evaluate instrument measurement properties; (iii) instruments were generic; (iv) ratings were self-reported; (v) full-texts were available. Next, methodological quality appraisal was based on COSMIN elements: (i) internal consistency; (ii) reliability; (iii) measurement error; (iv) content validity; (v) structural validity; (vi) hypotheses testing; (vii) cross-cultural validity; (viii) criterion validity; and (ix) responsiveness. Recommendations are based on evidence syntheses. Results This review included 25 articles assessing the reliability, validity and responsiveness of 15 different generic self-reported instruments measuring health-related productivity changes. Most studies evaluated criterion validity, none evaluated cross-cultural validity and information on measurement error is lacking. The Work Limitation Questionnaire (WLQ) was most frequently evaluated with moderate respectively strong positive evidence for content and structural validity and negative evidence for reliability, hypothesis testing and responsiveness. Less frequently evaluated, the Stanford Presenteeism Scale (SPS) showed strong positive evidence for internal consistency and structural validity, and moderate positive evidence for hypotheses testing and criterion validity. The Productivity and Disease Questionnaire (PRODISQ) yielded strong positive evidence for content validity, evidence for other properties is lacking. The other instruments resulted in mostly fair-to-poor quality ratings with limited evidence. Conclusions Decisions based on the content of the instrument, usage purpose, target country and population, and available evidence are recommended. Until high-quality studies are in place to accurately assess the measurement properties of the currently available instruments, the WLQ and, in a Dutch context, the PRODISQ are cautiously preferred based on its strong positive evidence for content validity. Based on its strong positive evidence for internal consistency and structural validity, the SPS is cautiously recommended. PMID:24495301

The Development of the Functional Literacy Experience Scale Based upon Ecological Theory (FLESBUET) and Validity-Reliability Study

ERIC Educational Resources Information Center

Özenç, Emine Gül; Dogan, M. Cihangir

2014-01-01

This study aims to perform a validity-reliability test by developing the Functional Literacy Experience Scale based upon Ecological Theory (FLESBUET) for primary education students. The study group includes 209 fifth grade students at Sabri Taskin Primary School in the Kartal District of Istanbul, Turkey during the 2010-2011 academic year.…

Measuring Practitioner Attitudes toward Evidence-Based Treatments: A Validation Study

ERIC Educational Resources Information Center

Ashcraft, Rindee G. P.; Foster, Sharon L.; Lowery, Amy E.; Henggeler, Scott W.; Chapman, Jason E.; Rowland, Melisa D.

2011-01-01

A better understanding of clinicians' attitudes toward evidence-based treatments (EBT) will presumably enhance the transfer of EBTs for substance-abusing adolescents from research to clinical application. The reliability and validity of two measures of therapist attitudes toward EBT were examined: the Evidence-Based Practice Attitude Scale…

Can We Study Autonomous Driving Comfort in Moving-Base Driving Simulators? A Validation Study.

PubMed

Bellem, Hanna; Klüver, Malte; Schrauf, Michael; Schöner, Hans-Peter; Hecht, Heiko; Krems, Josef F

2017-05-01

To lay the basis of studying autonomous driving comfort using driving simulators, we assessed the behavioral validity of two moving-base simulator configurations by contrasting them with a test-track setting. With increasing level of automation, driving comfort becomes increasingly important. Simulators provide a safe environment to study perceived comfort in autonomous driving. To date, however, no studies were conducted in relation to comfort in autonomous driving to determine the extent to which results from simulator studies can be transferred to on-road driving conditions. Participants ( N = 72) experienced six differently parameterized lane-change and deceleration maneuvers and subsequently rated the comfort of each scenario. One group of participants experienced the maneuvers on a test-track setting, whereas two other groups experienced them in one of two moving-base simulator configurations. We could demonstrate relative and absolute validity for one of the two simulator configurations. Subsequent analyses revealed that the validity of the simulator highly depends on the parameterization of the motion system. Moving-base simulation can be a useful research tool to study driving comfort in autonomous vehicles. However, our results point at a preference for subunity scaling factors for both lateral and longitudinal motion cues, which might be explained by an underestimation of speed in virtual environments. In line with previous studies, we recommend lateral- and longitudinal-motion scaling factors of approximately 50% to 60% in order to obtain valid results for both active and passive driving tasks.

Linking simulation-based educational assessments and patient-related outcomes: a systematic review and meta-analysis.

PubMed

Brydges, Ryan; Hatala, Rose; Zendejas, Benjamin; Erwin, Patricia J; Cook, David A

2015-02-01

To examine the evidence supporting the use of simulation-based assessments as surrogates for patient-related outcomes assessed in the workplace. The authors systematically searched MEDLINE, EMBASE, Scopus, and key journals through February 26, 2013. They included original studies that assessed health professionals and trainees using simulation and then linked those scores with patient-related outcomes assessed in the workplace. Two reviewers independently extracted information on participants, tasks, validity evidence, study quality, patient-related and simulation-based outcomes, and magnitude of correlation. All correlations were pooled using random-effects meta-analysis. Of 11,628 potentially relevant articles, the 33 included studies enrolled 1,203 participants, including postgraduate physicians (n = 24 studies), practicing physicians (n = 8), medical students (n = 6), dentists (n = 2), and nurses (n = 1). The pooled correlation for provider behaviors was 0.51 (95% confidence interval [CI], 0.38 to 0.62; n = 27 studies); for time behaviors, 0.44 (95% CI, 0.15 to 0.66; n = 7); and for patient outcomes, 0.24 (95% CI, -0.02 to 0.47; n = 5). Most reported validity evidence was favorable, though studies often included only correlational evidence. Validity evidence of internal structure (n = 13 studies), content (n = 12), response process (n = 2), and consequences (n = 1) were reported less often. Three tools showed large pooled correlations and favorable (albeit incomplete) validity evidence. Simulation-based assessments often correlate positively with patient-related outcomes. Although these surrogates are imperfect, tools with established validity evidence may replace workplace-based assessments for evaluating select procedural skills.

Development of Internet-Based Tasks for the Executive Function Performance Test.

PubMed

Rand, Debbie; Lee Ben-Haim, Keren; Malka, Rachel; Portnoy, Sigal

The Executive Function Performance Test (EFPT) is a reliable and valid performance-based tool to assess executive functions (EFs). This study's objective was to develop and verify two Internet-based tasks for the EFPT. A cross-sectional study assessed the alternate-form reliability of the Internet-based bill-paying and telephone-use tasks in healthy adults and people with subacute stroke (Study 1). It also sought to establish the tasks' criterion reliability for assessing EF deficits by correlating performance with that on the Trail Making Test in five groups: healthy young adults, healthy older adults, people with subacute stroke, people with chronic stroke, and young adults with attention deficit hyperactivity disorder (Study 2). The alternative-form reliability and initial construct validity for the Internet-based bill-paying task were verified. Criterion validity was established for both tasks. The Internet-based tasks are comparable to the original EFPT tasks and can be used for assessment of EF deficits. Copyright © 2018 by the American Occupational Therapy Association, Inc.

Reaction time as an indicator of insufficient effort: Development and validation of an embedded performance validity parameter.

PubMed

Stevens, Andreas; Bahlo, Simone; Licha, Christina; Liske, Benjamin; Vossler-Thies, Elisabeth

2016-11-30

Subnormal performance in attention tasks may result from various sources including lack of effort. In this report, the derivation and validation of a performance validity parameter for reaction time is described, using a set of malingering-indices ("Slick-criteria"), and 3 independent samples of participants (total n =893). The Slick-criteria yield an estimate of the probability of malingering based on the presence of an external incentive, evidence from neuropsychological testing, from self-report and clinical data. In study (1) a validity parameter is derived using reaction time data of a sample, composed of inpatients with recent severe brain lesions not involved in litigation and of litigants with and without brain lesion. In study (2) the validity parameter is tested in an independent sample of litigants. In study (3) the parameter is applied to an independent sample comprising cooperative and non-cooperative testees. Logistic regression analysis led to a derived validity parameter based on median reaction time and standard deviation. It performed satisfactorily in studies (2) and (3) (study 2 sensitivity=0.94, specificity=1.00; study 3 sensitivity=0.79, specificity=0.87). The findings suggest that median reaction time and standard deviation may be used as indicators of negative response bias. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

Preliminary Development and Validation of the Mindful Student Questionnaire

ERIC Educational Resources Information Center

Renshaw, Tyler L.

2017-01-01

Research validating mindfulness-based interventions with youths and in schools is growing, yet research validating measures of youths' mindfulness in schools has received far less empirical attention. The present study makes the case for and reports on the preliminary development and validation of a new, 15-item, multidimensional, self-report…

Student-to-Student Confirmation in the College Classroom: The Development and Validation of the Student-to-Student Confirmation Scale

ERIC Educational Resources Information Center

LaBelle, Sara; Johnson, Zac D.

2018-01-01

Three studies were conducted to generate a valid and reliable instrument to measure student-to-student confirmation. Study One (N = 396) sought to establish a factor structure based on previous research. Study Two (N = 396) sought to confirm this factor structure and assess criterion-related validity. Study Three (N = 283) sought to assess…

Relative validity of a semi-quantitative, web-based FFQ used in the 'Snart Forældre' cohort - a Danish study of diet and fertility.

PubMed

Knudsen, Vibeke K; Hatch, Elizabeth E; Cueto, Heidi; Tucker, Katherine L; Wise, Lauren; Christensen, Tue; Mikkelsen, Ellen M

2016-04-01

To assess the relative validity of a semi-quantitative, web-based FFQ completed by female pregnancy planners in the Danish 'Snart Forældre' study. We validated a web-based FFQ based on the FFQ used in the Danish National Birth Cohort against a 4 d food diary (FD) and assessed the relative validity of intakes of foods and nutrients. We compared means and medians of intakes, and calculated Pearson correlation coefficients and de-attenuated coefficients to assess agreement between the two methods. We also calculated the proportion correctly classified based on the same or adjacent quintile of intake and the proportion of grossly misclassified (extreme quintiles). Participants (n 128) in the 'Snart Forældre' study who had completed the web-based FFQ were invited to participate in the validation study. Participants in the 'Snart Forældre' study, in total ninety-seven women aged 20-42 years. Reported intakes of dairy products, vegetables and potatoes were higher in the FFQ compared with the FD, whereas reported intakes of fruit, meat, sugar and beverages were lower in the FFQ than in the FD. Overall the de-attenuated correlation coefficients were acceptable, ranging from 0·33 for energy to 0·93 for vitamin D. The majority of the women were classified in the same or adjacent quintile and few women were misclassified (extreme quintiles). The web-based FFQ performs well for ranking women of reproductive age according to high or low intake of foods and nutrients and, thus, provides a solid basis for investigating associations between diet and fertility.

What Adherence Measures Should Be Used in Trials of Home-Based Rehabilitation Interventions? A Systematic Review of the Validity, Reliability, and Acceptability of Measures.

PubMed

Frost, Rachael; Levati, Sara; McClurg, Doreen; Brady, Marian; Williams, Brian

2017-06-01

To systematically review methods for measuring adherence used in home-based rehabilitation trials and to evaluate their validity, reliability, and acceptability. In phase 1 we searched the CENTRAL database, NHS Economic Evaluation Database, and Health Technology Assessment Database (January 2000 to April 2013) to identify adherence measures used in randomized controlled trials of allied health professional home-based rehabilitation interventions. In phase 2 we searched the databases of MEDLINE, Embase, CINAHL, Allied and Complementary Medicine Database, PsycINFO, CENTRAL, ProQuest Nursing and Allied Health, and Web of Science (inception to April 2015) for measurement property assessments for each measure. Studies assessing the validity, reliability, or acceptability of adherence measures. Two reviewers independently extracted data on participant and measure characteristics, measurement properties evaluated, evaluation methods, and outcome statistics and assessed study quality using the COnsensus-based Standards for the selection of health Measurement INstruments checklist. In phase 1 we included 8 adherence measures (56 trials). In phase 2, from the 222 measurement property assessments identified in 109 studies, 22 high-quality measurement property assessments were narratively synthesized. Low-quality studies were used as supporting data. StepWatch Activity Monitor validly and acceptably measured short-term step count adherence. The Problematic Experiences of Therapy Scale validly and reliably assessed adherence to vestibular rehabilitation exercises. Adherence diaries had moderately high validity and acceptability across limited populations. The Borg 6 to 20 scale, Bassett and Prapavessis scale, and Yamax CW series had insufficient validity. Low-quality evidence supported use of the Joint Protection Behaviour Assessment. Polar A1 series heart monitors were considered acceptable by 1 study. Current rehabilitation adherence measures are limited. Some possess promising validity and acceptability for certain parameters of adherence, situations, and populations and should be used in these situations. Rigorous evaluation of adherence measures in a broader range of populations is needed. Copyright © 2016 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.

Validation of Student and Parent Reported Data on the Basic Grant Application Form: Pre-Award Validation Analysis Study. Revised Final Report.

ERIC Educational Resources Information Center

Applied Management Sciences, Inc., Silver Spring, MD.

The 1978-1979 pre-award institution validation process for the Basic Educational Opportunity Grant (BEOG) program was studied, based on applicant and grant recipient files as of the end of February 1979. The objective was to assess the impact of the validation process on the proper award of BEOGs, and to determine whether the criteria for…

A Confirmatory Factor Analysis of the Student Evidence-Based Practice Questionnaire (S-EBPQ) in an Australian sample.

PubMed

Beccaria, Lisa; Beccaria, Gavin; McCosker, Catherine

2018-03-01

It is crucial that nursing students develop skills and confidence in using Evidence-Based Practice principles early in their education. This should be assessed with valid tools however, to date, few measures have been developed and applied to the student population. To examine the structural validity of the Student Evidence-Based Practice Questionnaire (S-EBPQ), with an Australian online nursing student cohort. A cross-sectional study for constructing validity. Three hundred and forty-five undergraduate nursing students from an Australian regional university were recruited across two semesters. Confirmatory Factor Analysis was used to examine the structural validity. Confirmatory Factor Analysis was applied which resulted in a good fitting model, based on a revised 20-item tool. The S-EBPQ tool remains a psychometrically robust measure of evidence-based practice use, attitudes, and knowledge and skills and can be applied in an online Australian student context. The findings of this study provided further evidence of the reliability and four factor structure of the S-EBPQ. Opportunities for further refinement of the tool may result in improvements in structural validity. Copyright © 2018 Elsevier Ltd. All rights reserved.

Automated Writing Evaluation for Formative Assessment of Second Language Writing: Investigating the Accuracy and Usefulness of Feedback as Part of Argument-Based Validation

ERIC Educational Resources Information Center

Ranalli, Jim; Link, Stephanie; Chukharev-Hudilainen, Evgeny

2017-01-01

An increasing number of studies on the use of tools for automated writing evaluation (AWE) in writing classrooms suggest growing interest in their potential for formative assessment. As with all assessments, these applications should be validated in terms of their intended interpretations and uses. A recent argument-based validation framework…

Validity of the Learning Portfolio: Analysis of a Portfolio Proposal for the University

ERIC Educational Resources Information Center

Gregori-Giralt, Eva; Menéndez-Varela, José Luis

2015-01-01

Validity is a central issue in portfolio-based assessment. This empirical study used a quantitative approach to analyse the validity of the inferences drawn from a disciplinary course work portfolio assessment comprising profession-specific and learning competencies. The study also examined the problems involved in the development of the…

Derivation and Cross-Validation of Cutoff Scores for Patients With Schizophrenia Spectrum Disorders on WAIS-IV Digit Span-Based Performance Validity Measures.

PubMed

Glassmire, David M; Toofanian Ross, Parnian; Kinney, Dominique I; Nitch, Stephen R

2016-06-01

Two studies were conducted to identify and cross-validate cutoff scores on the Wechsler Adult Intelligence Scale-Fourth Edition Digit Span-based embedded performance validity (PV) measures for individuals with schizophrenia spectrum disorders. In Study 1, normative scores were identified on Digit Span-embedded PV measures among a sample of patients (n = 84) with schizophrenia spectrum diagnoses who had no known incentive to perform poorly and who put forth valid effort on external PV tests. Previously identified cutoff scores resulted in unacceptable false positive rates and lower cutoff scores were adopted to maintain specificity levels ≥90%. In Study 2, the revised cutoff scores were cross-validated within a sample of schizophrenia spectrum patients (n = 96) committed as incompetent to stand trial. Performance on Digit Span PV measures was significantly related to Full Scale IQ in both studies, indicating the need to consider the intellectual functioning of examinees with psychotic spectrum disorders when interpreting scores on Digit Span PV measures. © The Author(s) 2015.

A nationwide register study of the characteristics, incidence and validity of diagnosed Tourette syndrome and other tic disorders.

PubMed

Leivonen, Susanna; Voutilainen, Arja; Hinkka-Yli-Salomäki, Susanna; Timonen-Soivio, Laura; Chudal, Roshan; Gissler, Mika; Huttunen, Jukka; Sourander, Andre

2014-09-01

The aim of this study was to describe the characteristics and incidence rates of diagnosed tic disorders in the Finnish Hospital Discharge Register, including changing incidence rates between 1991 and 2010. We also aimed to validate the diagnoses of Tourette's syndrome recorded in the register. Children born between January 1, 1991 and December 31, 2010, who were diagnosed with tic disorders, were identified from the Finnish Hospital Discharge Register (n = 3003). We studied the validity of the Tourette's syndrome diagnoses by reviewing the medical charts of 88 children born since 1997 and carrying out telephone interviews with 55 of their guardians. The incidence rates of all diagnosed tic disorders increased during the study period. A comorbid diagnosis of hyperkinetic disorder diagnosis was recorded in 28.2% of the children with Tourette's syndrome, and the validity of the register-based Tourette's syndrome diagnosis was approximately 95%. This is the first nationwide study to demonstrate the increasing incidence of all register-based tic disorder diagnoses. The validity of the Tourette's syndrome diagnoses in the Finnish Hospital Discharge Register was good, and the data provided are suitable for use in further register-based studies of tic disorders. ©2014 Foundation Acta Paediatrica. Published by John Wiley & Sons Ltd.

Update on simulation-based surgical training and assessment in ophthalmology: a systematic review.

PubMed

Thomsen, Ann Sofia S; Subhi, Yousif; Kiilgaard, Jens Folke; la Cour, Morten; Konge, Lars

2015-06-01

This study reviews the evidence behind simulation-based surgical training of ophthalmologists to determine (1) the validity of the reported models and (2) the ability to transfer skills to the operating room. Simulation-based training is established widely within ophthalmology, although it often lacks a scientific basis for implementation. We conducted a systematic review of trials involving simulation-based training or assessment of ophthalmic surgical skills among health professionals. The search included 5 databases (PubMed, EMBASE, PsycINFO, Cochrane Library, and Web of Science) and was completed on March 1, 2014. Overall, the included trials were divided into animal, cadaver, inanimate, and virtual-reality models. Risk of bias was assessed using the Cochrane Collaboration's tool. Validity evidence was evaluated using a modern validity framework (Messick's). We screened 1368 reports for eligibility and included 118 trials. The most common surgery simulated was cataract surgery. Most validity trials investigated only 1 or 2 of 5 sources of validity (87%). Only 2 trials (48 participants) investigated transfer of skills to the operating room; 4 trials (65 participants) evaluated the effect of simulation-based training on patient-related outcomes. Because of heterogeneity of the studies, it was not possible to conduct a quantitative analysis. The methodologic rigor of trials investigating simulation-based surgical training in ophthalmology is inadequate. To ensure effective implementation of training models, evidence-based knowledge of validity and efficacy is needed. We provide a useful tool for implementation and evaluation of research in simulation-based training. Copyright © 2015 American Academy of Ophthalmology. Published by Elsevier Inc. All rights reserved.

A Preliminary Validation of Attention, Relevance, Confidence and Satisfaction Model-Based Instructional Material Motivational Survey in a Computer-Based Tutorial Setting

ERIC Educational Resources Information Center

Huang, Wenhao; Huang, Wenyeh; Diefes-Dux, Heidi; Imbrie, Peter K.

2006-01-01

This paper describes a preliminary validation study of the Instructional Material Motivational Survey (IMMS) derived from the Attention, Relevance, Confidence and Satisfaction motivational design model. Previous studies related to the IMMS, however, suggest its practical application for motivational evaluation in various instructional settings…

A Rasch-Based Validation of the Vocabulary Size Test

ERIC Educational Resources Information Center

Beglar, David

2010-01-01

The primary purpose of this study was to provide preliminary validity evidence for a 140-item form of the Vocabulary Size Test, which is designed to measure written receptive knowledge of the first 14,000 words of English. Nineteen native speakers of English and 178 native speakers of Japanese participated in the study. Analyses based on the Rasch…

Examining Teacher Evaluation Validity and Leadership Decision Making within a Standards-Based Evaluation System

ERIC Educational Resources Information Center

Kimball, Steven M.; Milanowski, Anthony

2009-01-01

Purpose: The article reports on a study of school leader decision making that examined variation in the validity of teacher evaluation ratings in a school district that has implemented a standards-based teacher evaluation system. Research Methods: Applying mixed methods, the study used teacher evaluation ratings and value-added student achievement…

Hopes and Cautions for Instrument-Based Evaluation of Consent Capacity: Results of a Construct Validity Study of Three Instruments

PubMed Central

Moye, Jennifer; Azar, Annin R.; Karel, Michele J.; Gurrera, Ronald J.

2016-01-01

Does instrument based evaluation of consent capacity increase the precision and validity of competency assessment or does ostensible precision provide a false sense of confidence without in fact improving validity? In this paper we critically examine the evidence for construct validity of three instruments for measuring four functional abilities important in consent capacity: understanding, appreciation, reasoning, and expressing a choice. Instrument based assessment of these abilities is compared through investigation of a multi-trait multi-method matrix in 88 older adults with mild to moderate dementia. Results find variable support for validity. There appears to be strong evidence for good hetero-method validity for the measurement of understanding, mixed evidence for validity in the measurement of reasoning, and strong evidence for poor hetero-method validity for the concepts of appreciation and expressing a choice, although the latter is likely due to extreme range restrictions. The development of empirically based tools for use in capacity evaluation should ultimately enhance the reliability and validity of assessment, yet clearly more research is needed to define and measure the constructs of decisional capacity. We would also emphasize that instrument based assessment of capacity is only one part of a comprehensive evaluation of competency which includes consideration of diagnosis, psychiatric and/or cognitive symptomatology, risk involved in the situation, and individual and cultural differences. PMID:27330455

Development and Validation of a Multimedia-Based Assessment of Scientific Inquiry Abilities

ERIC Educational Resources Information Center

Kuo, Che-Yu; Wu, Hsin-Kai; Jen, Tsung-Hau; Hsu, Ying-Shao

2015-01-01

The potential of computer-based assessments for capturing complex learning outcomes has been discussed; however, relatively little is understood about how to leverage such potential for summative and accountability purposes. The aim of this study is to develop and validate a multimedia-based assessment of scientific inquiry abilities (MASIA) to…

Is Teacher Assessment Reliable or Valid for High School Students under a Web-Based Portfolio Environment?

ERIC Educational Resources Information Center

Chang, Chi-Cheng; Wu, Bing-Hong

2012-01-01

This study explored the reliability and validity of teacher assessment under a Web-based portfolio assessment environment (or Web-based teacher portfolio assessment). Participants were 72 eleventh graders taking the "Computer Application" course. The students perform portfolio creation, inspection, self- and peer-assessment using the Web-based…

Are validated outcome measures used in distal radial fractures truly valid?

PubMed Central

Nienhuis, R. W.; Bhandari, M.; Goslings, J. C.; Poolman, R. W.; Scholtes, V. A. B.

2016-01-01

Objectives Patient-reported outcome measures (PROMs) are often used to evaluate the outcome of treatment in patients with distal radial fractures. Which PROM to select is often based on assessment of measurement properties, such as validity and reliability. Measurement properties are assessed in clinimetric studies, and results are often reviewed without considering the methodological quality of these studies. Our aim was to systematically review the methodological quality of clinimetric studies that evaluated measurement properties of PROMs used in patients with distal radial fractures, and to make recommendations for the selection of PROMs based on the level of evidence of each individual measurement property. Methods A systematic literature search was performed in PubMed, EMbase, CINAHL and PsycINFO databases to identify relevant clinimetric studies. Two reviewers independently assessed the methodological quality of the studies on measurement properties, using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. Level of evidence (strong / moderate / limited / lacking) for each measurement property per PROM was determined by combining the methodological quality and the results of the different clinimetric studies. Results In all, 19 out of 1508 identified unique studies were included, in which 12 PROMs were rated. The Patient-rated wrist evaluation (PRWE) and the Disabilities of Arm, Shoulder and Hand questionnaire (DASH) were evaluated on most measurement properties. The evidence for the PRWE is moderate that its reliability, validity (content and hypothesis testing), and responsiveness are good. The evidence is limited that its internal consistency and cross-cultural validity are good, and its measurement error is acceptable. There is no evidence for its structural and criterion validity. The evidence for the DASH is moderate that its responsiveness is good. The evidence is limited that its reliability and the validity on hypothesis testing are good. There is no evidence for the other measurement properties. Conclusion According to this systematic review, there is, at best, moderate evidence that the responsiveness of the PRWE and DASH are good, as are the reliability and validity of the PRWE. We recommend these PROMs in clinical studies in patients with distal radial fractures; however, more clinimetric studies of higher methodological quality are needed to adequately determine the other measurement properties. Cite this article: Dr Y. V. Kleinlugtenbelt. Are validated outcome measures used in distal radial fractures truly valid?: A critical assessment using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. Bone Joint Res 2016;5:153–161. DOI: 10.1302/2046-3758.54.2000462. PMID:27132246

How Sharp is a Unicorn's Horn?

ERIC Educational Resources Information Center

Johnston, Peter H.; Allignton, Richard L.

1983-01-01

Criticizes a study of the reliability and validity of curriculum-based reading inventories by L. S. Fuchs, D. Fuchs, and S. L. Deno and raises questions regarding the study's internal and external validity. (AEA)

A systematic review of validated methods to capture acute bronchospasm using administrative or claims data.

PubMed

Sharifi, Mona; Krishanswami, Shanthi; McPheeters, Melissa L

2013-12-30

To identify and assess billing, procedural, or diagnosis code, or pharmacy claim-based algorithms used to identify acute bronchospasm in administrative and claims databases. We searched the MEDLINE database from 1991 to September 2012 using controlled vocabulary and key terms related to bronchospasm, wheeze and acute asthma. We also searched the reference lists of included studies. Two investigators independently assessed the full text of studies against pre-determined inclusion criteria. Two reviewers independently extracted data regarding participant and algorithm characteristics. Our searches identified 677 citations of which 38 met our inclusion criteria. In these 38 studies, the most commonly used ICD-9 code was 493.x. Only 3 studies reported any validation methods for the identification of bronchospasm, wheeze or acute asthma in administrative and claims databases; all were among pediatric populations and only 2 offered any validation statistics. Some of the outcome definitions utilized were heterogeneous and included other disease based diagnoses, such as bronchiolitis and pneumonia, which are typically of an infectious etiology. One study offered the validation of algorithms utilizing Emergency Department triage chief complaint codes to diagnose acute asthma exacerbations with ICD-9 786.07 (wheezing) revealing the highest sensitivity (56%), specificity (97%), PPV (93.5%) and NPV (76%). There is a paucity of studies reporting rigorous methods to validate algorithms for the identification of bronchospasm in administrative data. The scant validated data available are limited in their generalizability to broad-based populations. Copyright © 2013 Elsevier Ltd. All rights reserved.

Validating Computational Human Behavior Models: Consistency and Accuracy Issues

DTIC Science & Technology

2004-06-01

includes a discussion of SME demographics, content, and organization of the datasets . This research generalizes data from two pilot studies and two base...meet requirements for validating the varied and complex behavioral models. Through a series of empirical studies , this research identifies subject...meet requirements for validating the varied and complex behavioral models. Through a series of empirical studies , this research identifies subject

Construct Validity Evidence for Single-Response Items to Estimate Physical Activity Levels in Large Sample Studies

ERIC Educational Resources Information Center

Jackson, Allen W.; Morrow, James R., Jr.; Bowles, Heather R.; FitzGerald, Shannon J.; Blair, Steven N.

2007-01-01

Valid measurement of physical activity is important for studying the risks for morbidity and mortality. The purpose of this study was to examine evidence of construct validity of two similar single-response items assessing physical activity via self-report. Both items are based on the stages of change model. The sample was 687 participants (men =…

Test of Achievement in Quantitative Economics for Secondary Schools: Construction and Validation Using Item Response Theory

ERIC Educational Resources Information Center

Eleje, Lydia I.; Esomonu, Nkechi P. M.

2018-01-01

A Test to measure achievement in quantitative economics among secondary school students was developed and validated in this study. The test is made up 20 multiple choice test items constructed based on quantitative economics sub-skills. Six research questions guided the study. Preliminary validation was done by two experienced teachers in…

Empirical validation of an agent-based model of wood markets in Switzerland

PubMed Central

Hilty, Lorenz M.; Lemm, Renato; Thees, Oliver

2018-01-01

We present an agent-based model of wood markets and show our efforts to validate this model using empirical data from different sources, including interviews, workshops, experiments, and official statistics. Own surveys closed gaps where data was not available. Our approach to model validation used a variety of techniques, including the replication of historical production amounts, prices, and survey results, as well as a historical case study of a large sawmill entering the market and becoming insolvent only a few years later. Validating the model using this case provided additional insights, showing how the model can be used to simulate scenarios of resource availability and resource allocation. We conclude that the outcome of the rigorous validation qualifies the model to simulate scenarios concerning resource availability and allocation in our study region. PMID:29351300

Is Learner Self-Assessment Reliable and Valid in a Web-Based Portfolio Environment for High School Students?

ERIC Educational Resources Information Center

Chang, Chi-Cheng; Liang, Chaoyun; Chen, Yi-Hui

2013-01-01

This study explored the reliability and validity of Web-based portfolio self-assessment. Participants were 72 senior high school students enrolled in a computer application course. The students created learning portfolios, viewed peers' work, and performed self-assessment on the Web-based portfolio assessment system. The results indicated: 1)…

Assessing Adolescent Mindfulness: Validation of an Adapted Mindful Attention Awareness Scale in Adolescent Normative and Psychiatric Populations

ERIC Educational Resources Information Center

Brown, Kirk Warren; West, Angela Marie; Loverich, Tamara M.; Biegel, Gina M.

2011-01-01

Interest in mindfulness-based interventions for children and adolescents is burgeoning, bringing with it the need for validated instruments to assess mindfulness in youths. The present studies were designed to validate among adolescents a measure of mindfulness previously validated for adults (e.g., Brown & Ryan, 2003), which we herein call…

The Value of Qualitative Methods in Social Validity Research

ERIC Educational Resources Information Center

Leko, Melinda M.

2014-01-01

One quality indicator of intervention research is the extent to which the intervention has a high degree of social validity, or practicality. In this study, I drew on Wolf's framework for social validity and used qualitative methods to ascertain five middle schoolteachers' perceptions of the social validity of System 44®--a phonics-based reading…

Constructing a question bank based on script concordance approach as a novel assessment methodology in surgical education.

PubMed

Aldekhayel, Salah A; Alselaim, Nahar A; Magzoub, Mohi Eldin; Al-Qattan, Mohammad M; Al-Namlah, Abdullah M; Tamim, Hani; Al-Khayal, Abdullah; Al-Habdan, Sultan I; Zamakhshary, Mohammed F

2012-10-24

Script Concordance Test (SCT) is a new assessment tool that reliably assesses clinical reasoning skills. Previous descriptions of developing SCT-question banks were merely subjective. This study addresses two gaps in the literature: 1) conducting the first phase of a multistep validation process of SCT in Plastic Surgery, and 2) providing an objective methodology to construct a question bank based on SCT. After developing a test blueprint, 52 test items were written. Five validation questions were developed and a validation survey was established online. Seven reviewers were asked to answer this survey. They were recruited from two countries, Saudi Arabia and Canada, to improve the test's external validity. Their ratings were transformed into percentages. Analysis was performed to compare reviewers' ratings by looking at correlations, ranges, means, medians, and overall scores. Scores of reviewers' ratings were between 76% and 95% (mean 86% ± 5). We found poor correlations between reviewers (Pearson's: +0.38 to -0.22). Ratings of individual validation questions ranged between 0 and 4 (on a scale 1-5). Means and medians of these ranges were computed for each test item (mean: 0.8 to 2.4; median: 1 to 3). A subset of test items comprising 27 items was generated based on a set of inclusion and exclusion criteria. This study proposes an objective methodology for validation of SCT-question bank. Analysis of validation survey is done from all angles, i.e., reviewers, validation questions, and test items. Finally, a subset of test items is generated based on a set of criteria.

European consensus on a competency-based virtual reality training program for basic endoscopic surgical psychomotor skills.

PubMed

van Dongen, Koen W; Ahlberg, Gunnar; Bonavina, Luigi; Carter, Fiona J; Grantcharov, Teodor P; Hyltander, Anders; Schijven, Marlies P; Stefani, Alessandro; van der Zee, David C; Broeders, Ivo A M J

2011-01-01

Virtual reality (VR) simulators have been demonstrated to improve basic psychomotor skills in endoscopic surgery. The exercise configuration settings used for validation in studies published so far are default settings or are based on the personal choice of the tutors. The purpose of this study was to establish consensus on exercise configurations and on a validated training program for a virtual reality simulator, based on the experience of international experts to set criterion levels to construct a proficiency-based training program. A consensus meeting was held with eight European teams, all extensively experienced in using the VR simulator. Construct validity of the training program was tested by 20 experts and 60 novices. The data were analyzed by using the t test for equality of means. Consensus was achieved on training designs, exercise configuration, and examination. Almost all exercises (7/8) showed construct validity. In total, 50 of 94 parameters (53%) showed significant difference. A European, multicenter, validated, training program was constructed according to the general consensus of a large international team with extended experience in virtual reality simulation. Therefore, a proficiency-based training program can be offered to training centers that use this simulator for training in basic psychomotor skills in endoscopic surgery.

Model-based verification and validation of the SMAP uplink processes

NASA Astrophysics Data System (ADS)

Khan, M. O.; Dubos, G. F.; Tirona, J.; Standley, S.

Model-Based Systems Engineering (MBSE) is being used increasingly within the spacecraft design community because of its benefits when compared to document-based approaches. As the complexity of projects expands dramatically with continually increasing computational power and technology infusion, the time and effort needed for verification and validation (V& V) increases geometrically. Using simulation to perform design validation with system-level models earlier in the life cycle stands to bridge the gap between design of the system (based on system-level requirements) and verifying those requirements/validating the system as a whole. This case study stands as an example of how a project can validate a system-level design earlier in the project life cycle than traditional V& V processes by using simulation on a system model. Specifically, this paper describes how simulation was added to a system model of the Soil Moisture Active-Passive (SMAP) mission's uplink process. Also discussed are the advantages and disadvantages of the methods employed and the lessons learned; which are intended to benefit future model-based and simulation-based development efforts.

The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study.

PubMed

Mokkink, Lidwine B; Terwee, Caroline B; Patrick, Donald L; Alonso, Jordi; Stratford, Paul W; Knol, Dirk L; Bouter, Lex M; de Vet, Henrica C W

2010-05-01

Aim of the COSMIN study (COnsensus-based Standards for the selection of health status Measurement INstruments) was to develop a consensus-based checklist to evaluate the methodological quality of studies on measurement properties. We present the COSMIN checklist and the agreement of the panel on the items of the checklist. A four-round Delphi study was performed with international experts (psychologists, epidemiologists, statisticians and clinicians). Of the 91 invited experts, 57 agreed to participate (63%). Panel members were asked to rate their (dis)agreement with each proposal on a five-point scale. Consensus was considered to be reached when at least 67% of the panel members indicated 'agree' or 'strongly agree'. Consensus was reached on the inclusion of the following measurement properties: internal consistency, reliability, measurement error, content validity (including face validity), construct validity (including structural validity, hypotheses testing and cross-cultural validity), criterion validity, responsiveness, and interpretability. The latter was not considered a measurement property. The panel also reached consensus on how these properties should be assessed. The resulting COSMIN checklist could be useful when selecting a measurement instrument, peer-reviewing a manuscript, designing or reporting a study on measurement properties, or for educational purposes.

The JaCVAM international validation study on the in vivo comet assay: Selection of test chemicals.

PubMed

Morita, Takeshi; Uno, Yoshifumi; Honma, Masamitsu; Kojima, Hajime; Hayashi, Makoto; Tice, Raymond R; Corvi, Raffaella; Schechtman, Leonard

2015-07-01

The Japanese Center for the Validation of Alternative Methods (JaCVAM) sponsored an international prevalidation and validation study of the in vivo rat alkaline pH comet assay. The main objective of the study was to assess the sensitivity and specificity of the assay for correctly identifying genotoxic carcinogens, as compared with the traditional rat liver unscheduled DNA synthesis assay. Based on existing carcinogenicity and genotoxicity data and chemical class information, 90 chemicals were identified as primary candidates for use in the validation study. From these 90 chemicals, 46 secondary candidates and then 40 final chemicals were selected based on a sufficiency of carcinogenic and genotoxic data, differences in chemical class or genotoxic or carcinogenic mode of action (MOA), availability, price, and ease of handling. These 40 chemicals included 19 genotoxic carcinogens, 6 genotoxic non-carcinogens, 7 non-genotoxic carcinogens and 8 non-genotoxic non-carcinogens. "Genotoxicity" was defined as positive in the Ames mutagenicity test or in one of the standard in vivo genotoxicity tests (primarily the erythrocyte micronucleus assay). These chemicals covered various chemicals classes, MOAs, and genotoxicity profiles and were considered to be suitable for the purpose of the validation study. General principles of chemical selection for validation studies are discussed. Copyright © 2015 Elsevier B.V. All rights reserved.

Social Skills Questionnaire for Argentinean College Students (SSQ-U) Development and Validation.

PubMed

Morán, Valeria E; Olaz, Fabián O; Del Prette, Zilda A P

2015-11-27

In this paper we present a new instrument called Social Skills Questionnaire for Argentinean College Students (SSQ-U). Based on the adapted version of the Social Skills Inventory - Del Prette (SSI-Del Prette) (Olaz, Medrano, Greco, & Del Prette, 2009), we wrote new items for the scale, and carried out psychometric analysis to assess the validity and reliability of the instrument. In the first study, we collected evidence based on test content through expert judges who evaluated the quality and the relevance of the items. In the second and third studies, we provided validity evidence based on the internal structure of the instrument using exploratory (n = 1067) and confirmatory (n = 661) factor analysis. Results suggested a five-factor structure consistent with the dimensions of social skills, as proposed by Kelly (2002). The fit indexes corresponding to the obtained model were adequate, and composite reliability coefficients of each factor were excellent (above .75). Finally, in the fourth study, we provided evidence of convergent and discriminant validity. The obtained results allow us to conclude that the SSQ-U is the first valid and reliable instrument for measuring social skills in Argentinean college students.

Development and Validation of a Job Exposure Matrix for Physical Risk Factors in Low Back Pain

PubMed Central

Solovieva, Svetlana; Pehkonen, Irmeli; Kausto, Johanna; Miranda, Helena; Shiri, Rahman; Kauppinen, Timo; Heliövaara, Markku; Burdorf, Alex; Husgafvel-Pursiainen, Kirsti; Viikari-Juntura, Eira

2012-01-01

Objectives The aim was to construct and validate a gender-specific job exposure matrix (JEM) for physical exposures to be used in epidemiological studies of low back pain (LBP). Materials and Methods We utilized two large Finnish population surveys, one to construct the JEM and another to test matrix validity. The exposure axis of the matrix included exposures relevant to LBP (heavy physical work, heavy lifting, awkward trunk posture and whole body vibration) and exposures that increase the biomechanical load on the low back (arm elevation) or those that in combination with other known risk factors could be related to LBP (kneeling or squatting). Job titles with similar work tasks and exposures were grouped. Exposure information was based on face-to-face interviews. Validity of the matrix was explored by comparing the JEM (group-based) binary measures with individual-based measures. The predictive validity of the matrix against LBP was evaluated by comparing the associations of the group-based (JEM) exposures with those of individual-based exposures. Results The matrix includes 348 job titles, representing 81% of all Finnish job titles in the early 2000s. The specificity of the constructed matrix was good, especially in women. The validity measured with kappa-statistic ranged from good to poor, being fair for most exposures. In men, all group-based (JEM) exposures were statistically significantly associated with one-month prevalence of LBP. In women, four out of six group-based exposures showed an association with LBP. Conclusions The gender-specific JEM for physical exposures showed relatively high specificity without compromising sensitivity. The matrix can therefore be considered as a valid instrument for exposure assessment in large-scale epidemiological studies, when more precise but more labour-intensive methods are not feasible. Although the matrix was based on Finnish data we foresee that it could be applicable, with some modifications, in other countries with a similar level of technology. PMID:23152793

Validation of an Instructional Observation Instrument for Teaching English as a Foreign Language in Spain

ERIC Educational Resources Information Center

Gomez-Garcia, Maria

2011-01-01

The design and validation of a classroom observation instrument to provide formative feedback for teachers of EFL in Spain is the overarching purpose of this study. This study proposes that a valid and reliable classroom observation instrument, based on effective practice in teaching EFL, can be developed and used in Spain to enable teachers to…

Construct Validation of Analytic Rating Scales in a Speaking Assessment: Reporting a Score Profile and a Composite

ERIC Educational Resources Information Center

Sawaki, Yasuyo

2007-01-01

This is a construct validation study of a second language speaking assessment that reported a language profile based on analytic rating scales and a composite score. The study addressed three key issues: score dependability, convergent/discriminant validity of analytic rating scales and the weighting of analytic ratings in the composite score.…

Analytic Validation of Immunohistochemical Assays: A Comparison of Laboratory Practices Before and After Introduction of an Evidence-Based Guideline.

PubMed

Fitzgibbons, Patrick L; Goldsmith, Jeffrey D; Souers, Rhona J; Fatheree, Lisa A; Volmar, Keith E; Stuart, Lauren N; Nowak, Jan A; Astles, J Rex; Nakhleh, Raouf E

2017-09-01

- Laboratories must demonstrate analytic validity before any test can be used clinically, but studies have shown inconsistent practices in immunohistochemical assay validation. - To assess changes in immunohistochemistry analytic validation practices after publication of an evidence-based laboratory practice guideline. - A survey on current immunohistochemistry assay validation practices and on the awareness and adoption of a recently published guideline was sent to subscribers enrolled in one of 3 relevant College of American Pathologists proficiency testing programs and to additional nonsubscribing laboratories that perform immunohistochemical testing. The results were compared with an earlier survey of validation practices. - Analysis was based on responses from 1085 laboratories that perform immunohistochemical staining. Of 1057 responses, 65.4% (691) were aware of the guideline recommendations before this survey was sent and 79.9% (550 of 688) of those have already adopted some or all of the recommendations. Compared with the 2010 survey, a significant number of laboratories now have written validation procedures for both predictive and nonpredictive marker assays and specifications for the minimum numbers of cases needed for validation. There was also significant improvement in compliance with validation requirements, with 99% (100 of 102) having validated their most recently introduced predictive marker assay, compared with 74.9% (326 of 435) in 2010. The difficulty in finding validation cases for rare antigens and resource limitations were cited as the biggest challenges in implementing the guideline. - Dissemination of the 2014 evidence-based guideline validation practices had a positive impact on laboratory performance; some or all of the recommendations have been adopted by nearly 80% of respondents.

Construct Validation Theory Applied to the Study of Personality Dysfunction

PubMed Central

Zapolski, Tamika C. B.; Guller, Leila; Smith, Gregory T.

2013-01-01

The authors review theory validation and construct validation principles as related to the study of personality dysfunction. Historically, personality disorders have been understood to be syndromes of heterogeneous symptoms. The authors argue that the syndrome approach to description results in diagnoses of unclear meaning and constrained validity. The alternative approach of describing personality dysfunction in terms of homogeneous dimensions of functioning avoids the problems of the syndromal approach and has been shown to provide more valid description and diagnosis. The authors further argue that description based on homogeneous dimensions of personality function/dysfunction is more useful, because it provides direct connections to validated treatments. PMID:22321263

Validation of a scenario-based assessment of critical thinking using an externally validated tool.

PubMed

Buur, Jennifer L; Schmidt, Peggy; Smylie, Dean; Irizarry, Kris; Crocker, Carlos; Tyler, John; Barr, Margaret

2012-01-01

With medical education transitioning from knowledge-based curricula to competency-based curricula, critical thinking skills have emerged as a major competency. While there are validated external instruments for assessing critical thinking, many educators have created their own custom assessments of critical thinking. However, the face validity of these assessments has not been challenged. The purpose of this study was to compare results from a custom assessment of critical thinking with the results from a validated external instrument of critical thinking. Students from the College of Veterinary Medicine at Western University of Health Sciences were administered a custom assessment of critical thinking (ACT) examination and the externally validated instrument, California Critical Thinking Skills Test (CCTST), in the spring of 2011. Total scores and sub-scores from each exam were analyzed for significant correlations using Pearson correlation coefficients. Significant correlations between ACT Blooms 2 and deductive reasoning and total ACT score and deductive reasoning were demonstrated with correlation coefficients of 0.24 and 0.22, respectively. No other statistically significant correlations were found. The lack of significant correlation between the two examinations illustrates the need in medical education to externally validate internal custom assessments. Ultimately, the development and validation of custom assessments of non-knowledge-based competencies will produce higher quality medical professionals.

Patient-reported outcome instruments that evaluate adherence behaviours in adults with asthma: A systematic review of measurement properties.

PubMed

Gagné, Myriam; Boulet, Louis-Philippe; Pérez, Norma; Moisan, Jocelyne

2018-04-30

To systematically identify the measurement properties of patient-reported outcome instruments (PROs) that evaluate adherence to inhaled maintenance medication in adults with asthma. We conducted a systematic review of six databases. Two reviewers independently included studies on the measurement properties of PROs that evaluated adherence in asthmatic participants aged ≥18 years. Based on the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN), the reviewers (1) extracted data on internal consistency, reliability, measurement error, content validity, structural validity, hypotheses testing, cross-cultural validity, criterion validity, and responsiveness; (2) assessed the methodological quality of the included studies; (3) assessed the quality of the measurement properties (positive or negative); and (4) summarised the level of evidence (limited, moderate, or strong). We screened 6,068 records and included 15 studies (14 PROs). No studies evaluated measurement error or responsiveness. Based on methodological and measurement property quality assessments, we found limited positive evidence of: (a) internal consistency of the Adherence Questionnaire, Refined Medication Adherence Reason Scale (MAR-Scale), Medication Adherence Report Scale for Asthma (MARS-A), and Test of the Adherence to Inhalers (TAI); (b) reliability of the TAI; and (c) structural validity of the Adherence Questionnaire, MAR-Scale, MARS-A, and TAI. We also found limited negative evidence of: (d) hypotheses testing of Adherence Questionnaire; (e) reliability of the MARS-A; and (f) criterion validity of the MARS-A and TAI. Our results highlighted the need to conduct further high-quality studies that will positively evaluate the reliability, validity, and responsiveness of the available PROs. This article is protected by copyright. All rights reserved.

An Engineering Method of Civil Jet Requirements Validation Based on Requirements Project Principle

NASA Astrophysics Data System (ADS)

Wang, Yue; Gao, Dan; Mao, Xuming

2018-03-01

A method of requirements validation is developed and defined to meet the needs of civil jet requirements validation in product development. Based on requirements project principle, this method will not affect the conventional design elements, and can effectively connect the requirements with design. It realizes the modern civil jet development concept, which is “requirement is the origin, design is the basis”. So far, the method has been successfully applied in civil jet aircraft development in China. Taking takeoff field length as an example, the validation process and the validation method of the requirements are detailed introduced in the study, with the hope of providing the experiences to other civil jet product design.

Testing of the SEE and OEE post-hip fracture.

PubMed

Resnick, Barbara; Orwig, Denise; Zimmerman, Sheryl; Hawkes, William; Golden, Justine; Werner-Bronzert, Michelle; Magaziner, Jay

2006-08-01

The purpose of this study was to test the reliability and validity of the Self-Efficacy for Exercise (SEE) and the Outcome Expectations for Exercise (OEE) scales in a sample of 166 older women post-hip fracture. There was some evidence of validity of the SEE and OEE based on confirmatory factor analysis and Rasch model testing, criterion based and convergent validity, and evidence of internal consistency based on alpha coefficients and separation indices and reliability based on R2 estimates. Rasch model testing demonstrated that some items had high variability. Based on these findings suggestions are made for how items could be revised and the scales improved for future use.

Validating a Theory-Based Survey to Evaluate Teaching Effectiveness in Higher Education

ERIC Educational Resources Information Center

Amrein-Beardsley, A.; Haladyna, T.

2012-01-01

Surveys to evaluate instructor effectiveness are commonly used in higher education. Yet the survey items included are often drawn from other surveys without reference to a theory of adult learning. The authors present the results from a validation study of such a theory-based survey. They evidence that an evaluation survey based on a theory that…

Development and Validation of a Parent Report Measure for Assessing Social-Emotional Competencies of Children and Adolescents

ERIC Educational Resources Information Center

Merrell, Kenneth W.; Felver-Gant, Josh C.; Tom, Karalyn M.

2011-01-01

Based on the premises that strength-based assessment of children and adolescents is an important emerging area, and that additional tools for this purpose are needed, this study details development and validation efforts on a new strength-based assessment: the Social-Emotional Assets and Resilience Scale, parent form (SEARS-P). Following careful…

The Validation of a Classroom Observation Instrument Based on the Construct of Teacher Adaptive Practice

ERIC Educational Resources Information Center

Loughland, Tony; Vlies, Penny

2016-01-01

Teacher adaptability is a key disposition for teachers that has been linked to outcomes of interests to schools. The aim of this study was to examine how the broader disposition of teacher adaptability might be observable as classroom-based adaptive practices using an argument-based approach to validation. The findings from the initial phase of…

The Development and Validation of the Algebra Curriculum Based Measure: A Measure of Preschool Children's Sorting and Classifying Skills

ERIC Educational Resources Information Center

Maherally, Mohammad Iqbal

2014-01-01

The purpose of this study was to develop and validate an assessment tool entitled the Algebra Curriculum Based Measure (ACBM) with the intent of measuring preschool children's sorting and classifying skills based on one attribute (color, shape, and size) and two attributes (color and shape) simultaneously; and their ability to explain their…

Development of Learning Models Based on Problem Solving and Meaningful Learning Standards by Expert Validity for Animal Development Course

NASA Astrophysics Data System (ADS)

Lufri, L.; Fitri, R.; Yogica, R.

2018-04-01

The purpose of this study is to produce a learning model based on problem solving and meaningful learning standards by expert assessment or validation for the course of Animal Development. This research is a development research that produce the product in the form of learning model, which consist of sub product, namely: the syntax of learning model and student worksheets. All of these products are standardized through expert validation. The research data is the level of validity of all sub products obtained using questionnaire, filled by validators from various field of expertise (field of study, learning strategy, Bahasa). Data were analysed using descriptive statistics. The result of the research shows that the problem solving and meaningful learning model has been produced. Sub products declared appropriate by expert include the syntax of learning model and student worksheet.

Simulation-based assessment in anesthesiology: requirements for practical implementation.

PubMed

Boulet, John R; Murray, David J

2010-04-01

Simulations have taken a central role in the education and assessment of medical students, residents, and practicing physicians. The introduction of simulation-based assessments in anesthesiology, especially those used to establish various competencies, has demanded fairly rigorous studies concerning the psychometric properties of the scores. Most important, major efforts have been directed at identifying, and addressing, potential threats to the validity of simulation-based assessment scores. As a result, organizations that wish to incorporate simulation-based assessments into their evaluation practices can access information regarding effective test development practices, the selection of appropriate metrics, the minimization of measurement errors, and test score validation processes. The purpose of this article is to provide a broad overview of the use of simulation for measuring physician skills and competencies. For simulations used in anesthesiology, studies that describe advances in scenario development, the development of scoring rubrics, and the validation of assessment results are synthesized. Based on the summary of relevant research, psychometric requirements for practical implementation of simulation-based assessments in anesthesiology are forwarded. As technology expands, and simulation-based education and evaluation takes on a larger role in patient safety initiatives, the groundbreaking work conducted to date can serve as a model for those individuals and organizations that are responsible for developing, scoring, or validating simulation-based education and assessment programs in anesthesiology.

Development of Two-Tier Diagnostic Test Pictorial-Based for Identifying High School Students Misconceptions on the Mole Concept

NASA Astrophysics Data System (ADS)

Siswaningsih, W.; Firman, H.; Zackiyah; Khoirunnisa, A.

2017-02-01

The aim of this study was to develop the two-tier pictorial-based diagnostic test for identifying student misconceptions on mole concept. The method of this study is used development and validation. The development of the test Obtained through four phases, development of any items, validation, determination key, and application test. Test was developed in the form of pictorial consisting of two tier, the first tier Consist of four possible answers and the second tier Consist of four possible reasons. Based on the results of content validity of 20 items using the CVR (Content Validity Ratio), a number of 18 items declared valid. Based on the results of the reliability test using SPSS, Obtained 17 items with Cronbach’s Alpha value of 0703, the which means that items have accepted. A total of 10 items was conducted to 35 students of senior high school students who have studied the mole concept on one of the high schools in Cimahi. Based on the results of the application test, student misconceptions were identified in each label concept in mole concept with the percentage of misconceptions on the label concept of mole (60.15%), Avogadro’s number (34.28%), relative atomic mass (62, 84%), relative molecule mass (77.08%), molar mass (68.53%), molar volume of gas (57.11%), molarity (71.32%), chemical equation (82.77%), limiting reactants (91.40%), and molecular formula (77.13%).

Validation of new psychosocial factors questionnaires: a Colombian national study.

PubMed

Villalobos, Gloria H; Vargas, Angélica M; Rondón, Martin A; Felknor, Sarah A

2013-01-01

The study of workers' health problems possibly associated with stressful conditions requires valid and reliable tools for monitoring risk factors. The present study validates two questionnaires to assess psychosocial risk factors for stress-related illnesses within a sample of Colombian workers. The validation process was based on a representative sample survey of 2,360 Colombian employees, aged 18-70 years. Worker response rate was 90%; 46% of the responders were women. Internal consistency was calculated, construct validity was tested with factor analysis and concurrent validity was tested with Spearman correlations. The questionnaires demonstrated adequate reliability (0.88-0.95). Factor analysis confirmed the dimensions proposed in the measurement model. Concurrent validity resulted in significant correlations with stress and health symptoms. "Work and Non-work Psychosocial Factors Questionnaires" were found to be valid and reliable for the assessment of workers' psychosocial factors, and they provide information for research and intervention. Copyright © 2012 Wiley Periodicals, Inc.

Psychometric instrumentation: reliability and validity of instruments used for clinical practice, evidence-based practice projects and research studies.

PubMed

Mayo, Ann M

2015-01-01

It is important for CNSs and other APNs to consider the reliability and validity of instruments chosen for clinical practice, evidence-based practice projects, or research studies. Psychometric testing uses specific research methods to evaluate the amount of error associated with any particular instrument. Reliability estimates explain more about how well the instrument is designed, whereas validity estimates explain more about scores that are produced by the instrument. An instrument may be architecturally sound overall (reliable), but the same instrument may not be valid. For example, if a specific group does not understand certain well-constructed items, then the instrument does not produce valid scores when used with that group. Many instrument developers may conduct reliability testing only once, yet continue validity testing in different populations over many years. All CNSs should be advocating for the use of reliable instruments that produce valid results. Clinical nurse specialists may find themselves in situations where reliability and validity estimates for some instruments that are being utilized are unknown. In such cases, CNSs should engage key stakeholders to sponsor nursing researchers to pursue this most important work.

Implementing the Science Assessment Standards: Developing and validating a set of laboratory assessment tasks in high school biology

NASA Astrophysics Data System (ADS)

Saha, Gouranga Chandra

Very often a number of factors, especially time, space and money, deter many science educators from using inquiry-based, hands-on, laboratory practical tasks as alternative assessment instruments in science. A shortage of valid inquiry-based laboratory tasks for high school biology has been cited. Driven by this need, this study addressed the following three research questions: (1) How can laboratory-based performance tasks be designed and developed that are doable by students for whom they are designed/written? (2) Do student responses to the laboratory-based performance tasks validly represent at least some of the intended process skills that new biology learning goals want students to acquire? (3) Are the laboratory-based performance tasks psychometrically consistent as individual tasks and as a set? To answer these questions, three tasks were used from the six biology tasks initially designed and developed by an iterative process of trial testing. Analyses of data from 224 students showed that performance-based laboratory tasks that are doable by all students require careful and iterative process of development. Although the students demonstrated more skill in performing than planning and reasoning, their performances at the item level were very poor for some items. Possible reasons for the poor performances have been discussed and suggestions on how to remediate the deficiencies have been made. Empirical evidences for validity and reliability of the instrument have been presented both from the classical and the modern validity criteria point of view. Limitations of the study have been identified. Finally implications of the study and directions for further research have been discussed.

Development and validation of an instrument for evaluating inquiry-based tasks in science textbooks

NASA Astrophysics Data System (ADS)

Yang, Wenyuan; Liu, Enshan

2016-12-01

This article describes the development and validation of an instrument that can be used for content analysis of inquiry-based tasks. According to the theories of educational evaluation and qualities of inquiry, four essential functions that inquiry-based tasks should serve are defined: (1) assisting in the construction of understandings about scientific concepts, (2) providing students opportunities to use inquiry process skills, (3) being conducive to establishing understandings about scientific inquiry, and (4) giving students opportunities to develop higher order thinking skills. An instrument - the Inquiry-Based Tasks Analysis Inventory (ITAI) - was developed to judge whether inquiry-based tasks perform these functions well. To test the reliability and validity of the ITAI, 4 faculty members were invited to use the ITAI to collect data from 53 inquiry-based tasks in the 3 most widely adopted senior secondary biology textbooks in Mainland China. The results indicate that (1) the inter-rater reliability reached 87.7%, (2) the grading criteria have high discriminant validity, (3) the items possess high convergent validity, and (4) the Cronbach's alpha reliability coefficient reached 0.792. The study concludes that the ITAI is valid and reliable. Because of its solid foundations in theoretical and empirical argumentation, the ITAI is trustworthy.

Recommendations for standardizing validation procedures assessing physical activity of older persons by monitoring body postures and movements.

PubMed

Lindemann, Ulrich; Zijlstra, Wiebren; Aminian, Kamiar; Chastin, Sebastien F M; de Bruin, Eling D; Helbostad, Jorunn L; Bussmann, Johannes B J

2014-01-10

Physical activity is an important determinant of health and well-being in older persons and contributes to their social participation and quality of life. Hence, assessment tools are needed to study this physical activity in free-living conditions. Wearable motion sensing technology is used to assess physical activity. However, there is a lack of harmonisation of validation protocols and applied statistics, which make it hard to compare available and future studies. Therefore, the aim of this paper is to formulate recommendations for assessing the validity of sensor-based activity monitoring in older persons with focus on the measurement of body postures and movements. Validation studies of body-worn devices providing parameters on body postures and movements were identified and summarized and an extensive inter-active process between authors resulted in recommendations about: information on the assessed persons, the technical system, and the analysis of relevant parameters of physical activity, based on a standardized and semi-structured protocol. The recommended protocols can be regarded as a first attempt to standardize validity studies in the area of monitoring physical activity.

Simulators' validation study: Problem solution logic

NASA Technical Reports Server (NTRS)

Schoultz, M. B.

1974-01-01

A study was conducted to validate the ground based simulators used for aircraft environment in ride-quality research. The logic to the approach for solving this problem is developed. The overall problem solution flow chart is presented. The factors which could influence the human response to the environment on board the aircraft are analyzed. The mathematical models used in the study are explained. The steps which were followed in conducting the validation tests are outlined.

The Anomalous Sentences Repetition Test: Replication and Validation Study.

ERIC Educational Resources Information Center

Weeks, David J.

1986-01-01

Presents a brief clinical test, derived from earlier neuropsychological instruments, with evidence for its reliability, interscorer agreement, and validity. The latter is based upon correlations with both CAT scan measures of cortical atrophy and ventricular enlargement, as well as correlations with seven other previously validated cognitive…

"La Clave Profesional": Validation of a Vocational Guidance Instrument

ERIC Educational Resources Information Center

Mudarra, Maria J.; Lázaro Martínez, Ángel

2014-01-01

Introduction: The current study demonstrates empirical and cultural validity of "La Clave Profesional" (Spanish adaptation of Career Key, Jones's test based Holland's RIASEC model). The process of providing validity evidence also includes a reflection on personal and career development and examines the relationahsips between RIASEC…

Structural Validation of the Holistic Wellness Assessment

ERIC Educational Resources Information Center

Brown, Charlene; Applegate, E. Brooks; Yildiz, Mustafa

2015-01-01

The Holistic Wellness Assessment (HWA) is a relatively new assessment instrument based on an emergent transdisciplinary model of wellness. This study validated the factor structure identified via exploratory factor analysis (EFA), assessed test-retest reliability, and investigated concurrent validity of the HWA in three separate samples. The…

Assessing the Validity of Discourse Analysis: Transdisciplinary Convergence

ERIC Educational Resources Information Center

Jaipal-Jamani, Kamini

2014-01-01

Research studies using discourse analysis approaches make claims about phenomena or issues based on interpretation of written or spoken text, which includes images and gestures. How are findings/interpretations from discourse analysis validated? This paper proposes transdisciplinary convergence as a way to validate discourse analysis approaches to…

Validation of electronic medical record-based phenotyping algorithms: results and lessons learned from the eMERGE network.

PubMed

Newton, Katherine M; Peissig, Peggy L; Kho, Abel Ngo; Bielinski, Suzette J; Berg, Richard L; Choudhary, Vidhu; Basford, Melissa; Chute, Christopher G; Kullo, Iftikhar J; Li, Rongling; Pacheco, Jennifer A; Rasmussen, Luke V; Spangler, Leslie; Denny, Joshua C

2013-06-01

Genetic studies require precise phenotype definitions, but electronic medical record (EMR) phenotype data are recorded inconsistently and in a variety of formats. To present lessons learned about validation of EMR-based phenotypes from the Electronic Medical Records and Genomics (eMERGE) studies. The eMERGE network created and validated 13 EMR-derived phenotype algorithms. Network sites are Group Health, Marshfield Clinic, Mayo Clinic, Northwestern University, and Vanderbilt University. By validating EMR-derived phenotypes we learned that: (1) multisite validation improves phenotype algorithm accuracy; (2) targets for validation should be carefully considered and defined; (3) specifying time frames for review of variables eases validation time and improves accuracy; (4) using repeated measures requires defining the relevant time period and specifying the most meaningful value to be studied; (5) patient movement in and out of the health plan (transience) can result in incomplete or fragmented data; (6) the review scope should be defined carefully; (7) particular care is required in combining EMR and research data; (8) medication data can be assessed using claims, medications dispensed, or medications prescribed; (9) algorithm development and validation work best as an iterative process; and (10) validation by content experts or structured chart review can provide accurate results. Despite the diverse structure of the five EMRs of the eMERGE sites, we developed, validated, and successfully deployed 13 electronic phenotype algorithms. Validation is a worthwhile process that not only measures phenotype performance but also strengthens phenotype algorithm definitions and enhances their inter-institutional sharing.

Reproducibility and validity of the Dutch translation of the de Morton Mobility Index (DEMMI) used by physiotherapists in older patients with knee or hip osteoarthritis.

PubMed

Jans, Marielle P; Slootweg, Vera C; Boot, Cecile R; de Morton, Natalie A; van der Sluis, Geert; van Meeteren, Nico L

2011-11-01

To examine the reproducibility, construct validity, and unidimensionality of the Dutch translation of the de Morton Mobility Index (DEMMI), a performance-based measure of mobility for older patients. Cross-sectional study. Rehabilitation center (reproducibility study) and hospital (validity study). Patients (N=28; age >65y) after orthopedic surgery (reproducibility study) and patients (N=219; age >65y) waiting for total hip or total knee arthroplasty (validity study). Not applicable. Not applicable. The intraclass correlation coefficient for interrater reliability was high (.85; 95% confidence interval, 71-.93), and minimal detectable change with 90% confidence was 7 on the 100-point DEMMI scale. Rasch analysis identified that the Dutch translation of the DEMMI is a unidimensional measure of mobility in this population. DEMMI scores showed high correlations with scores on other performance-based measures of mobility (Timed Up and Go test, Spearman r=-.73; Chair Rise Time, r=-.69; walking test, r=.74). A lower correlation of .44 was identified with the self-report measure Western Ontario and McMaster Universities Osteoarthritis Index. The Dutch translation of the DEMMI is a reproducible and valid performance-based measure for assessing mobility in older patients with knee or hip osteoarthritis. Copyright © 2011 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.

Validation of a new method for finding the rotational axes of the knee using both marker-based roentgen stereophotogrammetric analysis and 3D video-based motion analysis for kinematic measurements.

PubMed

Roland, Michelle; Hull, M L; Howell, S M

2011-05-01

In a previous paper, we reported the virtual axis finder, which is a new method for finding the rotational axes of the knee. The virtual axis finder was validated through simulations that were subject to limitations. Hence, the objective of the present study was to perform a mechanical validation with two measurement modalities: 3D video-based motion analysis and marker-based roentgen stereophotogrammetric analysis (RSA). A two rotational axis mechanism was developed, which simulated internal-external (or longitudinal) and flexion-extension (FE) rotations. The actual axes of rotation were known with respect to motion analysis and RSA markers within ± 0.0006 deg and ± 0.036 mm and ± 0.0001 deg and ± 0.016 mm, respectively. The orientation and position root mean squared errors for identifying the longitudinal rotation (LR) and FE axes with video-based motion analysis (0.26 deg, 0.28 m, 0.36 deg, and 0.25 mm, respectively) were smaller than with RSA (1.04 deg, 0.84 mm, 0.82 deg, and 0.32 mm, respectively). The random error or precision in the orientation and position was significantly better (p=0.01 and p=0.02, respectively) in identifying the LR axis with video-based motion analysis (0.23 deg and 0.24 mm) than with RSA (0.95 deg and 0.76 mm). There was no significant difference in the bias errors between measurement modalities. In comparing the mechanical validations to virtual validations, the virtual validations produced comparable errors to those of the mechanical validation. The only significant difference between the errors of the mechanical and virtual validations was the precision in the position of the LR axis while simulating video-based motion analysis (0.24 mm and 0.78 mm, p=0.019). These results indicate that video-based motion analysis with the equipment used in this study is the superior measurement modality for use with the virtual axis finder but both measurement modalities produce satisfactory results. The lack of significant differences between validation techniques suggests that the virtual sensitivity analysis previously performed was appropriately modeled. Thus, the virtual axis finder can be applied with a thorough understanding of its errors in a variety of test conditions.

The Development and Validation of the Mood-based Indoor Tanning Scale.

PubMed

Carcioppolo, Nick; Chen, Yixin; John, Kevin K; Gonzalez, Andrea Martinez; King, Andy J; Morgan, Susan E; Hu, Shasa

2017-01-01

Research indicates that mood-based motivations may be an important predictor of indoor tanning bed use and may be related to indoor tanning dependence. Problematically, little research has been conducted to develop a psychometric measure of mood-based tanning motivations. The current study seeks to develop and validate the moodbased indoor tanning scale (MITS). Two studies were conducted to identify and verify the MITS factor structure as well as assess construct validity. Study 1 was conducted at 5 geographically diverse universities in the United States. Study 2 was conducted by using a national online sample in the United States. Results from study 1 specified the factor structure of the MITS. Results from study 2 suggest that a one-point increase in the MITS measure corresponds with using indoor tanning beds 11 more times in the past year. These findings demonstrate that moodbased tanning motivations are a strong predictor of indoor tanning intentions and behavior. Further, they suggest that health behavior researchers and healthcare practitioners can use the MITS to assess the extent to which mood-based motivations impact indoor tanning bed use.

Development and validation of instrument for ergonomic evaluation of tablet arm chairs

PubMed Central

Tirloni, Adriana Seára; dos Reis, Diogo Cunha; Bornia, Antonio Cezar; de Andrade, Dalton Francisco; Borgatto, Adriano Ferreti; Moro, Antônio Renato Pereira

2016-01-01

The purpose of this study was to develop and validate an evaluation instrument for tablet arm chairs based on ergonomic requirements, focused on user perceptions and using Item Response Theory (IRT). This exploratory study involved 1,633 participants (university students and professors) in four steps: a pilot study (n=26), semantic validation (n=430), content validation (n=11) and construct validation (n=1,166). Samejima's graded response model was applied to validate the instrument. The results showed that all the steps (theoretical and practical) of the instrument's development and validation processes were successful and that the group of remaining items (n=45) had a high consistency (0.95). This instrument can be used in the furniture industry by engineers and product designers and in the purchasing process of tablet arm chairs for schools, universities and auditoriums. PMID:28337099

Initial Validation Study for a Scale Used to Determine Service Intensity for Itinerant Teachers of Students with Visual Impairments

ERIC Educational Resources Information Center

Pogrund, Rona L.; Darst, Shannon; Munro, Michael P.

2015-01-01

Introduction: The purpose of this study was to begin validation of a scale that will be used by teachers of students with visual impairments to determine appropriate recommended type and frequency of services for their students based on identified student need. Methods: Validity and reliability of the Visual Impairment Scale of Service Intensity…

Developing Statistical Physics Course Handout on Distribution Function Materials Based on Science, Technology, Engineering, and Mathematics

NASA Astrophysics Data System (ADS)

Riandry, M. A.; Ismet, I.; Akhsan, H.

2017-09-01

This study aims to produce a valid and practical statistical physics course handout on distribution function materials based on STEM. Rowntree development model is used to produce this handout. The model consists of three stages: planning, development and evaluation stages. In this study, the evaluation stage used Tessmer formative evaluation. It consists of 5 stages: self-evaluation, expert review, one-to-one evaluation, small group evaluation and field test stages. However, the handout is limited to be tested on validity and practicality aspects, so the field test stage is not implemented. The data collection technique used walkthroughs and questionnaires. Subjects of this study are students of 6th and 8th semester of academic year 2016/2017 Physics Education Study Program of Sriwijaya University. The average result of expert review is 87.31% (very valid category). One-to-one evaluation obtained the average result is 89.42%. The result of small group evaluation is 85.92%. From one-to-one and small group evaluation stages, averagestudent response to this handout is 87,67% (very practical category). Based on the results of the study, it can be concluded that the handout is valid and practical.

Validity of Scientific Based Chemistry Android Module to Empower Science Process Skills (SPS) in Solubility Equilibrium

NASA Astrophysics Data System (ADS)

Antrakusuma, B.; Masykuri, M.; Ulfa, M.

2018-04-01

Evolution of Android technology can be applied to chemistry learning, one of the complex chemistry concept was solubility equilibrium. this concept required the science process skills (SPS). This study aims to: 1) Characteristic scientific based chemistry Android module to empowering SPS, and 2) Validity of the module based on content validity and feasibility test. This research uses a Research and Development approach (RnD). Research subjects were 135 s1tudents and three teachers at three high schools in Boyolali, Central of Java. Content validity of the module was tested by seven experts using Aiken’s V technique, and the module feasibility was tested to students and teachers in each school. Characteristics of chemistry module can be accessed using the Android device. The result of validation of the module contents got V = 0.89 (Valid), and the results of the feasibility test Obtained 81.63% (by the student) and 73.98% (by the teacher) indicates this module got good criteria.

Using Ground-Based Measurements and Retrievals to Validate Satellite Data

NASA Technical Reports Server (NTRS)

Dong, Xiquan

2002-01-01

The proposed research is to use the DOE ARM ground-based measurements and retrievals as the ground-truth references for validating satellite cloud results and retrieving algorithms. This validation effort includes four different ways: (1) cloud properties on different satellites, therefore different sensors, TRMM VIRS and TERRA MODIS; (2) cloud properties at different climatic regions, such as DOE ARM SGP, NSA, and TWP sites; (3) different cloud types, low and high level cloud properties; and (4) day and night retrieving algorithms. Validation of satellite-retrieved cloud properties is very difficult and a long-term effort because of significant spatial and temporal differences between the surface and satellite observing platforms. The ground-based measurements and retrievals, only carefully analyzed and validated, can provide a baseline for estimating errors in the satellite products. Even though the validation effort is so difficult, a significant progress has been made during the proposed study period, and the major accomplishments are summarized in the follow.

Adaptation and validation of the Evidence-Based Practice Belief and Implementation scales for French-speaking Swiss nurses and allied healthcare providers.

PubMed

Verloo, Henk; Desmedt, Mario; Morin, Diane

2017-09-01

To evaluate two psychometric properties of the French versions of the Evidence-Based Practice Beliefs and Evidence-Based Practice Implementation scales, namely their internal consistency and construct validity. The Evidence-Based Practice Beliefs and Evidence-Based Practice Implementation scales developed by Melnyk et al. are recognised as valid, reliable instruments in English. However, no psychometric validation for their French versions existed. Secondary analysis of a cross sectional survey. Source data came from a cross-sectional descriptive study sample of 382 nurses and other allied healthcare providers. Cronbach's alpha was used to evaluate internal consistency, and principal axis factor analysis and varimax rotation were computed to determine construct validity. The French Evidence-Based Practice Beliefs and Evidence-Based Practice Implementation scales showed excellent reliability, with Cronbach's alphas close to the scores established by Melnyk et al.'s original versions. Principal axis factor analysis showed medium-to-high factor loading scores without obtaining collinearity. Principal axis factor analysis with varimax rotation of the 16-item Evidence-Based Practice Beliefs scale resulted in a four-factor loading structure. Principal axis factor analysis with varimax rotation of the 17-item Evidence-Based Practice Implementation scale revealed a two-factor loading structure. Further research should attempt to understand why the French Evidence-Based Practice Implementation scale showed a two-factor loading structure but Melnyk et al.'s original has only one. The French versions of the Evidence-Based Practice Beliefs and Evidence-Based Practice Implementation scales can both be considered valid and reliable instruments for measuring Evidence-Based Practice beliefs and implementation. The results suggest that the French Evidence-Based Practice Beliefs and Evidence-Based Practice Implementation scales are valid and reliable and can therefore be used to evaluate the effectiveness of organisational strategies aimed at increasing professionals' confidence in Evidence-Based Practice, supporting its use and implementation. © 2017 John Wiley & Sons Ltd.

Internal Cluster Validation on Earthquake Data in the Province of Bengkulu

NASA Astrophysics Data System (ADS)

Rini, D. S.; Novianti, P.; Fransiska, H.

2018-04-01

K-means method is an algorithm for cluster n object based on attribute to k partition, where k < n. There is a deficiency of algorithms that is before the algorithm is executed, k points are initialized randomly so that the resulting data clustering can be different. If the random value for initialization is not good, the clustering becomes less optimum. Cluster validation is a technique to determine the optimum cluster without knowing prior information from data. There are two types of cluster validation, which are internal cluster validation and external cluster validation. This study aims to examine and apply some internal cluster validation, including the Calinski-Harabasz (CH) Index, Sillhouette (S) Index, Davies-Bouldin (DB) Index, Dunn Index (D), and S-Dbw Index on earthquake data in the Bengkulu Province. The calculation result of optimum cluster based on internal cluster validation is CH index, S index, and S-Dbw index yield k = 2, DB Index with k = 6 and Index D with k = 15. Optimum cluster (k = 6) based on DB Index gives good results for clustering earthquake in the Bengkulu Province.

Emotional and tangible social support in a German population-based sample: Development and validation of the Brief Social Support Scale (BS6).

PubMed

Beutel, Manfred E; Brähler, Elmar; Wiltink, Jörg; Michal, Matthias; Klein, Eva M; Jünger, Claus; Wild, Philipp S; Münzel, Thomas; Blettner, Maria; Lackner, Karl; Nickels, Stefan; Tibubos, Ana N

2017-01-01

Aim of the study was the development and validation of the psychometric properties of a six-item bi-factorial instrument for the assessment of social support (emotional and tangible support) with a population-based sample. A cross-sectional data set of N = 15,010 participants enrolled in the Gutenberg Health Study (GHS) in 2007-2012 was divided in two sub-samples. The GHS is a population-based, prospective, observational single-center cohort study in the Rhein-Main-Region in western Mid-Germany. The first sub-sample was used for scale development by performing an exploratory factor analysis. In order to test construct validity, confirmatory factor analyses were run to compare the extracted bi-factorial model with the one-factor solution. Reliability of the scales was indicated by calculating internal consistency. External validity was tested by investigating demographic characteristics health behavior, and distress using analysis of variance, Spearman and Pearson correlation analysis, and logistic regression analysis. Based on an exploratory factor analysis, a set of six items was extracted representing two independent factors. The two-factor structure of the Brief Social Support Scale (BS6) was confirmed by the results of the confirmatory factor analyses. Fit indices of the bi-factorial model were good and better compared to the one-factor solution. External validity was demonstrated for the BS6. The BS6 is a reliable and valid short scale that can be applied in social surveys due to its brevity to assess emotional and practical dimensions of social support.

Content validation of terms and definitions in a wound glossary.

PubMed

Milne, Catherine T; Paine, Tim; Sullivan, Valerie; Sawyer, Allen

2011-12-01

A common language and lexicon provide the easiest means of mutual understanding. Inconsistency in terminology makes effective information exchange difficult. Previous studies identified the need to determine standard, accepted definitions for the vocabulary frequently used in wound care. The objective of this study was to establish content validation for these terms and develop an evidence-based glossary for this specialty. Members of the Association for the Advancement of Wound Care Quality of Care Task Force reviewed literature to determine glossary content generation and the associated literature-based definitions. Thirty-nine wound care professionals from wound care stakeholder professional organizations in the United States and Canada participated in the content validation process. Participants were asked to quantify the degree of validity using a 367-item, 4-point Likert-type scale. On a scale of 1 to 4, the mean score of the entire instrument was 3.84. The instrument's overall scale content validity index was 0.96. Terms with an item content validity index of less than 0.70 were removed from the glossary, leaving 365 items with established content validity. Qualitative data analysis revealed themes suggesting that enhanced communication between providers improves patient outcomes. The need for ongoing updates of the glossary was also identified. The wound care glossary in its finalized form proved valid. An evidence-based glossary bridges the chasm of miscommunication and nonstandardization so that wound care, as an emerging specialized medical science field, can move forward to optimize both process and clinical outcomes.

[The Basel Screening Instrument for Psychosis (BSIP): development, structure, reliability and validity].

PubMed

Riecher-Rössler, A; Aston, J; Ventura, J; Merlo, M; Borgwardt, S; Gschwandtner, U; Stieglitz, R-D

2008-04-01

Early detection of psychosis is of growing clinical importance. So far there is, however, no screening instrument for detecting individuals with beginning psychosis in the atypical early stages of the disease with sufficient validity. We have therefore developed the Basel Screening Instrument for Psychosis (BSIP) and tested its feasibility, interrater-reliability and validity. Aim of this paper is to describe the development and structure of the instrument, as well as to report the results of the studies on reliability and validity. The instrument was developed based on a comprehensive search of literature on the most important risk factors and early signs of schizophrenic psychoses. The interraterreliability study was conducted on 24 psychiatric cases. Validity was tested based on 206 individuals referred to our early detection clinic from 3/1/2000 until 2/28/2003. We identified seven categories of relevance for early detection of psychosis and used them to construct a semistructured interview. Interrater-reliability for high risk individuals was high (Kappa .87). Predictive validity was comparable to other, more comprehensive instruments: 16 (32 %) of 50 individuals classified as being at risk for psychosis by the BSIP have in fact developed frank psychosis within an follow-up period of two to five years. The BSIP is the first screening instrument for the early detection of psychosis which has been validated based on transition to psychosis. The BSIP is easy to use by experienced psychiatrists and has a very good interrater-reliability and predictive validity.

Mathematical modeling in realistic mathematics education

NASA Astrophysics Data System (ADS)

Riyanto, B.; Zulkardi; Putri, R. I. I.; Darmawijoyo

2017-12-01

The purpose of this paper is to produce Mathematical modelling in Realistics Mathematics Education of Junior High School. This study used development research consisting of 3 stages, namely analysis, design and evaluation. The success criteria of this study were obtained in the form of local instruction theory for school mathematical modelling learning which was valid and practical for students. The data were analyzed using descriptive analysis method as follows: (1) walk through, analysis based on the expert comments in the expert review to get Hypothetical Learning Trajectory for valid mathematical modelling learning; (2) analyzing the results of the review in one to one and small group to gain practicality. Based on the expert validation and students’ opinion and answers, the obtained mathematical modeling problem in Realistics Mathematics Education was valid and practical.

Development and Construct Validation of the Mentor Behavior Scale

ERIC Educational Resources Information Center

Brodeur, Pascale; Larose, Simon; Tarabulsy, George; Feng, Bei; Forget-Dubois, Nadine

2015-01-01

Researchers suggest that certain supportive behaviors of mentors could increase the benefits of school-based mentoring for youth. However, the literature contains few validated instruments to measure these behaviors. In our present study, we aimed to construct and validate a tool to measure the supportive behaviors of mentors participating in…

The Students' Perceptions of School Success Promoting Strategies Inventory (SPSI): development and validity evidence based studies.

PubMed

Moreira, Paulo A S; Oliveira, João Tiago; Dias, Paulo; Vaz, Filipa Machado; Torres-Oliveira, Isabel

2014-08-04

Students' perceptions about school success promotion strategies are of great importance for schools, as they are an indicator of how students perceive the school success promotion strategies. The objective of this study was to develop and analyze the validity evidence based of The Students' Perceptions of School Success Promoting Strategies Inventory (SPSI), which assesses both individual students' perceptions of their school success promoting strategies, and dimensions of school quality. A structure of 7 related factors was found, which showed good adjustment indices in two additional different samples, suggesting that this is a well-fitting multi-group model (p < .001). All scales presented good reliability values. Schools with good academic results registered higher values in Career development, Active learning, Proximity, Educational Technologies and Extra-curricular activities (p < .05). SPSI showed to be adequate to measure within-schools (students within schools) dimensions of school success. In addition, there is preliminary evidence for its adequacy for measuring school success promotion dimensions between schools for 4 dimensions. This study supports the validity evidence based of the SPSI (validity evidence based on test content, on internal structure, on relations to other variables and on consequences of testing). Future studies should test for within- and between-level variance in a bigger sample of schools.

Developing workshop module of realistic mathematics education: Follow-up workshop

NASA Astrophysics Data System (ADS)

Palupi, E. L. W.; Khabibah, S.

2018-01-01

Realistic Mathematics Education (RME) is a learning approach which fits the aim of the curriculum. The success of RME in teaching mathematics concepts, triggering students’ interest in mathematics and teaching high order thinking skills to the students will make teachers start to learn RME. Hence, RME workshop is often offered and done. This study applied development model proposed by Plomp. Based on the study by RME team, there are three kinds of RME workshop: start-up workshop, follow-up workshop, and quality boost. However, there is no standardized or validated module which is used in that workshops. This study aims to develop a module of RME follow-up workshop which is valid and can be used. Plopm’s developmental model includes materials analysis, design, realization, implementation, and evaluation. Based on the validation, the developed module is valid. While field test shows that the module can be used effectively.

Validity and Reliability of Field-Based Measures for Assessing Movement Skill Competency in Lifelong Physical Activities: A Systematic Review.

PubMed

Hulteen, Ryan M; Lander, Natalie J; Morgan, Philip J; Barnett, Lisa M; Robertson, Samuel J; Lubans, David R

2015-10-01

It has been suggested that young people should develop competence in a variety of 'lifelong physical activities' to ensure that they can be active across the lifespan. The primary aim of this systematic review is to report the methodological properties, validity, reliability, and test duration of field-based measures that assess movement skill competency in lifelong physical activities. A secondary aim was to clearly define those characteristics unique to lifelong physical activities. A search of four electronic databases (Scopus, SPORTDiscus, ProQuest, and PubMed) was conducted between June 2014 and April 2015 with no date restrictions. Studies addressing the validity and/or reliability of lifelong physical activity tests were reviewed. Included articles were required to assess lifelong physical activities using process-oriented measures, as well as report either one type of validity or reliability. Assessment criteria for methodological quality were adapted from a checklist used in a previous review of sport skill outcome assessments. Movement skill assessments for eight different lifelong physical activities (badminton, cycling, dance, golf, racquetball, resistance training, swimming, and tennis) in 17 studies were identified for inclusion. Methodological quality, validity, reliability, and test duration (time to assess a single participant), for each article were assessed. Moderate to excellent reliability results were found in 16 of 17 studies, with 71% reporting inter-rater reliability and 41% reporting intra-rater reliability. Only four studies in this review reported test-retest reliability. Ten studies reported validity results; content validity was cited in 41% of these studies. Construct validity was reported in 24% of studies, while criterion validity was only reported in 12% of studies. Numerous assessments for lifelong physical activities may exist, yet only assessments for eight lifelong physical activities were included in this review. Generalizability of results may be more applicable if more heterogeneous samples are used in future research. Moderate to excellent levels of inter- and intra-rater reliability were reported in the majority of studies. However, future work should look to establish test-retest reliability. Validity was less commonly reported than reliability, and further types of validity other than content validity need to be established in future research. Specifically, predictive validity of 'lifelong physical activity' movement skill competency is needed to support the assertion that such activities provide the foundation for a lifetime of activity.

Bibliometrics for Social Validation.

PubMed

Hicks, Daniel J

2016-01-01

This paper introduces a bibliometric, citation network-based method for assessing the social validation of novel research, and applies this method to the development of high-throughput toxicology research at the US Environmental Protection Agency. Social validation refers to the acceptance of novel research methods by a relevant scientific community; it is formally independent of the technical validation of methods, and is frequently studied in history, philosophy, and social studies of science using qualitative methods. The quantitative methods introduced here find that high-throughput toxicology methods are spread throughout a large and well-connected research community, which suggests high social validation. Further assessment of social validation involving mixed qualitative and quantitative methods are discussed in the conclusion.

Bibliometrics for Social Validation

PubMed Central

2016-01-01

This paper introduces a bibliometric, citation network-based method for assessing the social validation of novel research, and applies this method to the development of high-throughput toxicology research at the US Environmental Protection Agency. Social validation refers to the acceptance of novel research methods by a relevant scientific community; it is formally independent of the technical validation of methods, and is frequently studied in history, philosophy, and social studies of science using qualitative methods. The quantitative methods introduced here find that high-throughput toxicology methods are spread throughout a large and well-connected research community, which suggests high social validation. Further assessment of social validation involving mixed qualitative and quantitative methods are discussed in the conclusion. PMID:28005974

Technical skills assessment toolbox: a review using the unitary framework of validity.

PubMed

Ghaderi, Iman; Manji, Farouq; Park, Yoon Soo; Juul, Dorthea; Ott, Michael; Harris, Ilene; Farrell, Timothy M

2015-02-01

The purpose of this study was to create a technical skills assessment toolbox for 35 basic and advanced skills/procedures that comprise the American College of Surgeons (ACS)/Association of Program Directors in Surgery (APDS) surgical skills curriculum and to provide a critical appraisal of the included tools, using contemporary framework of validity. Competency-based training has become the predominant model in surgical education and assessment of performance is an essential component. Assessment methods must produce valid results to accurately determine the level of competency. A search was performed, using PubMed and Google Scholar, to identify tools that have been developed for assessment of the targeted technical skills. A total of 23 assessment tools for the 35 ACS/APDS skills modules were identified. Some tools, such as Operative Performance Rating System (OSATS) and Objective Structured Assessment of Technical Skill (OPRS), have been tested for more than 1 procedure. Therefore, 30 modules had at least 1 assessment tool, with some common surgical procedures being addressed by several tools. Five modules had none. Only 3 studies used Messick's framework to design their validity studies. The remaining studies used an outdated framework on the basis of "types of validity." When analyzed using the contemporary framework, few of these studies demonstrated validity for content, internal structure, and relationship to other variables. This study provides an assessment toolbox for common surgical skills/procedures. Our review shows that few authors have used the contemporary unitary concept of validity for development of their assessment tools. As we progress toward competency-based training, future studies should provide evidence for various sources of validity using the contemporary framework.

Reliability and validity of the revised Gibson Test of Cognitive Skills, a computer-based test battery for assessing cognition across the lifespan.

PubMed

Moore, Amy Lawson; Miller, Terissa M

2018-01-01

The purpose of the current study is to evaluate the validity and reliability of the revised Gibson Test of Cognitive Skills, a computer-based battery of tests measuring short-term memory, long-term memory, processing speed, logic and reasoning, visual processing, as well as auditory processing and word attack skills. This study included 2,737 participants aged 5-85 years. A series of studies was conducted to examine the validity and reliability using the test performance of the entire norming group and several subgroups. The evaluation of the technical properties of the test battery included content validation by subject matter experts, item analysis and coefficient alpha, test-retest reliability, split-half reliability, and analysis of concurrent validity with the Woodcock Johnson III Tests of Cognitive Abilities and Tests of Achievement. Results indicated strong sources of evidence of validity and reliability for the test, including internal consistency reliability coefficients ranging from 0.87 to 0.98, test-retest reliability coefficients ranging from 0.69 to 0.91, split-half reliability coefficients ranging from 0.87 to 0.91, and concurrent validity coefficients ranging from 0.53 to 0.93. The Gibson Test of Cognitive Skills-2 is a reliable and valid tool for assessing cognition in the general population across the lifespan.

Reexamining the Writing Apprehension Measure

ERIC Educational Resources Information Center

Autman, Hamlet; Kelly, Stephanie

2017-01-01

This article contains two measurement development studies on writing apprehension. Study 1 reexamines the validity of the writing apprehension measure based on the finding from prior research that a second false factor was embedded. The findings from Study 1 support the validity of a reduced measure with 6 items versus the original 20-item…

A validated case definition for chronic rhinosinusitis in administrative data: a Canadian perspective.

PubMed

Rudmik, Luke; Xu, Yuan; Kukec, Edward; Liu, Mingfu; Dean, Stafford; Quan, Hude

2016-11-01

Pharmacoepidemiological research using administrative databases has become increasingly popular for chronic rhinosinusitis (CRS); however, without a validated case definition the cohort evaluated may be inaccurate resulting in biased and incorrect outcomes. The objective of this study was to develop and validate a generalizable administrative database case definition for CRS using International Classification of Diseases, 9th edition (ICD-9)-coded claims. A random sample of 100 patients with a guideline-based diagnosis of CRS and 100 control patients were selected and then linked to a Canadian physician claims database from March 31, 2010, to March 31, 2015. The proportion of CRS ICD-9-coded claims (473.x and 471.x) for each of these 200 patients were reviewed and the validity of 7 different ICD-9-based coding algorithms was evaluated. The CRS case definition of ≥2 claims with a CRS ICD-9 code (471.x or 473.x) within 2 years of the reference case provides a balanced validity with a sensitivity of 77% and specificity of 79%. Applying this CRS case definition to the claims database produced a CRS cohort of 51,000 patients with characteristics that were consistent with published demographics and rates of comorbid asthma, allergic rhinitis, and depression. This study has validated several coding algorithms; based on the results a case definition of ≥2 physician claims of CRS (ICD-9 of 471.x or 473.x) within 2 years provides an optimal level of validity. Future studies will need to validate this administrative case definition from different health system perspectives and using larger retrospective chart reviews from multiple providers. © 2016 ARS-AAOA, LLC.

Assessment of bachelor's theses in a nursing degree with a rubrics system: Development and validation study.

PubMed

González-Chordá, Víctor M; Mena-Tudela, Desirée; Salas-Medina, Pablo; Cervera-Gasch, Agueda; Orts-Cortés, Isabel; Maciá-Soler, Loreto

2016-02-01

Writing a bachelor thesis (BT) is the last step to obtain a nursing degree. In order to perform an effective assessment of a nursing BT, certain reliable and valid tools are required. To develop and validate a 3-rubric system (drafting process, dissertation, and viva) to assess final year nursing students' BT. A multi-disciplinary study of content validity and psychometric properties. The study was carried out between December 2014 and July 2015. Nursing Degree at Universitat Jaume I. Spain. Eleven experts (9 nursing professors and 2 education professors from 6 different universities) took part in the development and content validity stages. Fifty-two theses presented during the 2014-2015 academic year were included by consecutive sampling of cases in order to study the psychometric properties. First, a group of experts was created to validate the content of the assessment system based on three rubrics (drafting process, dissertation, and viva). Subsequently, a reliability and validity study of the rubrics was carried out on the 52 theses presented during the 2014-2015 academic year. The BT drafting process rubric has 8 criteria (S-CVI=0.93; α=0.837; ICC=0.614), the dissertation rubric has 7 criteria (S-CVI=0.9; α=0.893; ICC=0.74), and the viva rubric has 4 criteria (S-CVI=0.86; α=8.16; ICC=0.895). A nursing BT assessment system based on three rubrics (drafting process, dissertation, and viva) has been validated. This system may be transferred to other nursing degrees or degrees from other academic areas. It is necessary to continue with the validation process taking into account factors that may affect the results obtained. Copyright © 2015 Elsevier Ltd. All rights reserved.

Measurement properties of depression questionnaires in patients with diabetes: a systematic review.

PubMed

van Dijk, Susan E M; Adriaanse, Marcel C; van der Zwaan, Lennart; Bosmans, Judith E; van Marwijk, Harm W J; van Tulder, Maurits W; Terwee, Caroline B

2018-06-01

To conduct a systematic review on measurement properties of questionnaires measuring depressive symptoms in adult patients with type 1 or type 2 diabetes. A systematic review of the literature in MEDLINE, EMbase and PsycINFO was performed. Full text, original articles, published in any language up to October 2016 were included. Eligibility for inclusion was independently assessed by three reviewers who worked in pairs. Methodological quality of the studies was evaluated by two independent reviewers using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. Quality of the questionnaires was rated per measurement property, based on the number and quality of the included studies and the reported results. Of 6286 unique hits, 21 studies met our criteria evaluating nine different questionnaires in multiple settings and languages. The methodological quality of the included studies was variable for the different measurement properties: 9/15 studies scored 'good' or 'excellent' on internal consistency, 2/5 on reliability, 0/1 on content validity, 10/10 on structural validity, 8/11 on hypothesis testing, 1/5 on cross-cultural validity, and 4/9 on criterion validity. For the CES-D, there was strong evidence for good internal consistency, structural validity, and construct validity; moderate evidence for good criterion validity; and limited evidence for good cross-cultural validity. The PHQ-9 and WHO-5 also performed well on several measurement properties. However, the evidence for structural validity of the PHQ-9 was inconclusive. The WHO-5 was less extensively researched and originally not developed to measure depression. Currently, the CES-D is best supported for measuring depressive symptoms in diabetes patients.

Development of contextual teaching and learning based science module for junior high school for increasing creativity of students

NASA Astrophysics Data System (ADS)

Kurniasari, H.; Sukarmin; Sarwanto

2018-03-01

The purpose of this research are to analyze the the properness of contextual teaching and learning (CTL)-based science module for Junior High School for increasing students’ creativity and using CTL-based science module to increase students’ learning creativity. Development of CTL-based science module for Junior High School is Research and Development (R&D) using 4D Model consist of 4 steps: define, design, develop, and disseminate. Module is validated by 3 expert validators (Material, media, and language experts), 2 reviewer and 1 peer reviewer. . Based on the results of data analysis, it can be concluded that: the results of the validation, the average score of CTL-based science module is 88.28%, the value exceeded the value of the cut off score of 87.5%, so the media declared eligible for the study. Research shows that the gain creativity class that uses CTL-based science module has a gain of 0.72. Based on the results of the study showed that CTL-based science module effectively promotes creativity of students

Design and validation of the INICIARE instrument, for the assessment of dependency level in acutely ill hospitalised patients.

PubMed

Morales-Asencio, José Miguel; Porcel-Gálvez, Ana María; Oliveros-Valenzuela, Rosa; Rodríguez-Gómez, Susana; Sánchez-Extremera, Lucrecia; Serrano-López, Francisco Andrés; Aranda-Gallardo, Marta; Canca-Sánchez, José Carlos; Barrientos-Trigo, Sergio

2015-03-01

The aim of this study was to establish the validity and reliability of an instrument (Inventario del NIvel de Cuidados mediante IndicAdores de clasificación de Resultados de Enfermería) used to assess the dependency level in acutely hospitalised patients. This instrument is novel, and it is based on the Nursing Outcomes Classification. Multiple existing instruments for needs assessment have been poorly validated and based predominately on interventions. Standardised Nursing Languages offer an ideal framework to develop nursing sensitive instruments. A cross-sectional validation study in two acute care hospitals in Spain. This study was implemented in two phases. First, the research team developed the instrument to be validated. In the second phase, the validation process was performed by experts, and the data analysis was conducted to establish the psychometric properties of the instrument. Seven hundred and sixty-one patient ratings performed by nurses were collected during the course of the research study. Data analysis yielded a Cronbach's alpha of 0·91. An exploratory factorial analysis identified three factors (Physiological, Instrumental and Cognitive-behavioural), which explained 74% of the variance. Inventario del NIvel de Cuidados mediante IndicAdores de clasificación de Resultados de Enfermería was demonstrated to be a valid and reliable instrument based on its use in acutely hospitalised patients to assess the level of dependency. Inventario del NIvel de Cuidados mediante IndicAdores de clasificación de Resultados de Enfermería can be used as an assessment tool in hospitalised patients during the nursing process throughout the entire hospitalisation period. It contributes information to support decisions on nursing diagnoses, interventions and outcomes. It also enables data codification in large databases. © 2014 John Wiley & Sons Ltd.

Detecting symptom exaggeration in combat veterans using the MMPI-2 symptom validity scales: a mixed group validation.

PubMed

Tolin, David F; Steenkamp, Maria M; Marx, Brian P; Litz, Brett T

2010-12-01

Although validity scales of the Minnesota Multiphasic Personality Inventory-2 (MMPI-2; J. N. Butcher, W. G. Dahlstrom, J. R. Graham, A. Tellegen, & B. Kaemmer, 1989) have proven useful in the detection of symptom exaggeration in criterion-group validation (CGV) studies, usually comparing instructed feigners with known patient groups, the application of these scales has been problematic when assessing combat veterans undergoing posttraumatic stress disorder (PTSD) examinations. Mixed group validation (MGV) was employed to determine the efficacy of MMPI-2 exaggeration scales in compensation-seeking (CS) and noncompensation-seeking (NCS) veterans. Unlike CGV, MGV allows for a mix of exaggerating and nonexaggerating individuals in each group, does not require that the exaggeration versus nonexaggerating status of any individual be known, and can be adjusted for different base-rate estimates. MMPI-2 responses of 377 male veterans were examined according to CS versus NCS status. MGV was calculated using 4 sets of base-rate estimates drawn from the literature. The validity scales generally performed well (adequate sensitivity, specificity, and efficiency) under most base-rate estimations, and most produced cutoff scores that showed adequate detection of symptom exaggeration, regardless of base-rate assumptions. These results support the use of MMPI-2 validity scales for PTSD evaluations in veteran populations, even under varying base rates of symptom exaggeration.

The MicroArray Quality Control (MAQC)-II study of common practices for the development and validation of microarray-based predictive models

EPA Science Inventory

The second phase of the MicroArray Quality Control (MAQC-II) project evaluated common practices for developing and validating microarray-based models aimed at predicting toxicological and clinical endpoints. Thirty-six teams developed classifiers for 13 endpoints - some easy, som...

Development and Validation of a Christian-Based Grief Recovery Scale

ERIC Educational Resources Information Center

Jen Der Pan, Peter; Deng, Liang-Yu F.; Tsai, S. L.; Chen, Ho-Yuan J.; Yuan, Sheng-Shiou Jenny

2014-01-01

The purpose of this study was to develop and validate a Christian-based Grief Recovery Scale (CGRS) which was used to measure Christians recovering from grief after a significant loss. Taiwanese Christian participants were recruited from churches and a comprehensive university in northern Taiwan. They were affected by both the Christian faith and…

Concurrent Validity and Classification Accuracy of Curriculum-Based Measurement for Written Expression

ERIC Educational Resources Information Center

Furey, William M.; Marcotte, Amanda M.; Hintze, John M.; Shackett, Caroline M.

2016-01-01

The study presents a critical analysis of written expression curriculum-based measurement (WE-CBM) metrics derived from 3- and 10-min test lengths. Criterion validity and classification accuracy were examined for Total Words Written (TWW), Correct Writing Sequences (CWS), Percent Correct Writing Sequences (%CWS), and Correct Minus Incorrect…

Meta-Analysis of Criterion Validity for Curriculum-Based Measurement in Written Language

ERIC Educational Resources Information Center

Romig, John Elwood; Therrien, William J.; Lloyd, John W.

2017-01-01

We used meta-analysis to examine the criterion validity of four scoring procedures used in curriculum-based measurement of written language. A total of 22 articles representing 21 studies (N = 21) met the inclusion criteria. Results indicated that two scoring procedures, correct word sequences and correct minus incorrect sequences, have acceptable…

Methodological review of the quality of reach out and read: does it "work"?

PubMed

Yeager Pelatti, Christina; Pentimonti, Jill M; Justice, Laura M

2014-04-01

A considerable percentage of American children and adults fail to learn adequate literacy skills and read below a third grade level. Shared book reading is perhaps the single most important activity to prepare young children for success in reading. The primary objective of this manuscript was to critically review the methodological quality of Read Out and Read (ROR), a clinically based literacy program/intervention that teaches parents strategies to incorporate while sharing books with children as a method of preventing reading difficulties and academic struggles. A PubMed search was conducted. Articles that met three criteria were considered. First, the study must be clinically based and include parent contact with a pediatrician. Second, parental counseling ("anticipatory guidance") about the importance of parent-child book reading must be included. Third, only experimental or quasi-experimental studies were included; no additional criteria were used. Published articles from any year and peer-reviewed journal were considered. Study quality was determined using a modified version of the Downs and Black (1998) checklist assessing four categories: (1) Reporting, (2) External Validity, (3) Internal Validity-Bias, and (4) Internal Validity-Confounding. We were also interested in whether quality differed based on study design, children's age, sample size, and study outcome. Eleven studies met the inclusion criteria. The overall quality of evidence was variable across all studies; Reporting and External Validity categories were relatively strong while methodological concerns were found in the area of internal validity. Quality scores differed on the four study characteristics. Implications related to clinical practice and future studies are discussed.

Development and initial validation of a cognitive-based work-nonwork conflict scale.

PubMed

Ezzedeen, Souha R; Swiercz, Paul M

2007-06-01

Current research related to work and life outside work specifies three types of work-nonwork conflict: time, strain, and behavior-based. Overlooked in these models is a cognitive-based type of conflict whereby individuals experience work-nonwork conflict from cognitive preoccupation with work. Four studies on six different groups (N=549) were undertaken to develop and validate an initial measure of this construct. Structural equation modeling confirmed a two-factor, nine-item scale. Hypotheses regarding cognitive-based conflict's relationship with life satisfaction, work involvement, work-nonwork conflict, and work hours were supported. The relationship with knowledge work was partially supported in that only the cognitive dimension of cognitive-based conflict was related to extent of knowledge work. Hypotheses regarding cognitive-based conflict's relationship with family demands were rejected in that the cognitive dimension correlated positively rather than negatively with number of dependent children and perceived family demands. The study provides encouraging preliminary evidence of scale validity.

The reliability and validity of the Complex Task Performance Assessment: A performance-based assessment of executive function.

PubMed

Wolf, Timothy J; Dahl, Abigail; Auen, Colleen; Doherty, Meghan

2017-07-01

The objective of this study was to evaluate the inter-rater reliability, test-retest reliability, concurrent validity, and discriminant validity of the Complex Task Performance Assessment (CTPA): an ecologically valid performance-based assessment of executive function. Community control participants (n = 20) and individuals with mild stroke (n = 14) participated in this study. All participants completed the CTPA and a battery of cognitive assessments at initial testing. The control participants completed the CTPA at two different times one week apart. The intra-class correlation coefficient (ICC) for inter-rater reliability for the total score on the CTPA was .991. The ICCs for all of the sub-scores of the CTPA were also high (.889-.977). The CTPA total score was significantly correlated to Condition 4 of the DKEFS Color-Word Interference Test (p = -.425), and the Wechsler Test of Adult Reading (p  = -.493). Finally, there were significant differences between control subjects and individuals with mild stroke on the total score of the CTPA (p = .007) and all sub-scores except interpretation failures and total items incorrect. These results are also consistent with other current executive function performance-based assessments and indicate that the CTPA is a reliable and valid performance-based measure of executive function.

Use of Bayesian Networks to Probabilistically Model and Improve the Likelihood of Validation of Microarray Findings by RT-PCR

PubMed Central

English, Sangeeta B.; Shih, Shou-Ching; Ramoni, Marco F.; Smith, Lois E.; Butte, Atul J.

2014-01-01

Though genome-wide technologies, such as microarrays, are widely used, data from these methods are considered noisy; there is still varied success in downstream biological validation. We report a method that increases the likelihood of successfully validating microarray findings using real time RT-PCR, including genes at low expression levels and with small differences. We use a Bayesian network to identify the most relevant sources of noise based on the successes and failures in validation for an initial set of selected genes, and then improve our subsequent selection of genes for validation based on eliminating these sources of noise. The network displays the significant sources of noise in an experiment, and scores the likelihood of validation for every gene. We show how the method can significantly increase validation success rates. In conclusion, in this study, we have successfully added a new automated step to determine the contributory sources of noise that determine successful or unsuccessful downstream biological validation. PMID:18790084

Measuring limitations in activities of daily living: a population-based validation of a short questionnaire.

PubMed

Elfering, Achim; Cronenberg, Sonja; Grebner, Simone; Tamcan, Oezguer; Müller, Urs

2017-12-01

A newly developed questionnaire assessing limitations in activity of daily living (LADL-Q) that should improve assessment of LADL is tested in a large population-based validation study. This survey was paper-based. Overall, 16,634 individuals who were representative of the working population in the German-speaking part of Switzerland participated in the study. Item analysis was used the final version of the LADL-Q to four items per subscale that correspond to potential problems in three body regions (back and neck, upper extremities, lower extremities). Analysis included tests for reliability, internal consistency, dimensionality and convergent validity. Test-retest reliability coefficients after 2 weeks ranged from 0.82 to 0.99 (Mdn = 0.87), with no item having a coefficient below 0.60. The median item-total coefficients ranged between moderate and good. Correlation coefficients between LADL-Q subscales and three validated clinical instruments (Western Ontario and McMaster Universities osteoarthritis index, shoulder pain disability index, Oswestry) ranged from 0.63 to 0.81. In structural equation modeling the three subscales were significantly related with two important outcomes in occupational rehabilitation: self-reported general health and daily task performance. The new LADL-Q is a brief, reliable and valid tool for assessment of LADL in studies on musculoskeletal health.

Impact of the International Continence Society (ICS) report on the standardisation of terminology in nocturia on the quality of reports on nocturia and nocturnal polyuria: a systematic review.

PubMed

Hofmeester, Ilse; Kollen, Boudewijn J; Steffens, Martijn G; Bosch, J L H Ruud; Drake, Marcus J; Weiss, Jeffrey P; Blanker, Marco H

2015-04-01

To systematically review and evaluate the impact of the International Continence Society (ICS)-2002 report on standardisation of terminology in nocturia, on publications reporting on nocturia and nocturnal polyuria (NP). In 2002, the ICS defined NP as a Nocturnal Polyuria Index (nocturnal urine volume/total 24-h urine volume) of >0.2-0.33, depending on age. In April 2013 the PubMed and Embase databases were searched for studies (in English, German, French or Dutch) based on original data and adult participants, investigating the relationship between nocturia and NP. A methodological quality assessment was performed, including scores on external validity, internal validity and informativeness. Quality scores of items were compared between studies published before and after the ICS-2002 report. The search yielded 78 publications based on 66 studies. Quality scores of studies were generally high for internal validity (median 5, interquartile range [IQR] 4-6) but low for external validity. After publication of the ICS-2002 report, external validity showed a significant change from 1 (IQR 1-2) to 2 (IQR 1-2.5; P = 0.019). Nocturia remained undefined in 12 studies. In all, 19 different definitions were used for NP, most often being the ICS (or similar) definition: this covered 52% (n = 11) of studies before and 66% (n = 27) after the ICS-2002 report. Clear definitions of both nocturia and NP were identified in 67% and 76% before, and in 88% and 88% of the studies after the ICS-2002 report, respectively. The ICS-2002 report on standardisation of terminology in nocturia appears to have had a beneficial impact on reporting definitions of nocturia and NP, enabling better interpretation of results and comparisons between research projects. Because the external validity of most of the 66 studies is considered a problem, the results of these studies may not be validly extrapolated to other populations. The ICS definition of NP is used most often. However, its discriminative value seems limited due to the estimated difference of 0.6 nocturnal voids between individuals with and without NP. Refinement of current definitions based on robust research is required. Based on pathophysiological reasoning, we argue that it may be more appropriate to define NP based on nocturnal urine production or nocturnal voided volumes, rather than on a diurnal urine production pattern. © 2014 The Authors. BJU International © 2014 BJU International.

Evaluating the accuracy of the Wechsler Memory Scale-Fourth Edition (WMS-IV) logical memory embedded validity index for detecting invalid test performance.

PubMed

Soble, Jason R; Bain, Kathleen M; Bailey, K Chase; Kirton, Joshua W; Marceaux, Janice C; Critchfield, Edan A; McCoy, Karin J M; O'Rourke, Justin J F

2018-01-08

Embedded performance validity tests (PVTs) allow for continuous assessment of invalid performance throughout neuropsychological test batteries. This study evaluated the utility of the Wechsler Memory Scale-Fourth Edition (WMS-IV) Logical Memory (LM) Recognition score as an embedded PVT using the Advanced Clinical Solutions (ACS) for WAIS-IV/WMS-IV Effort System. This mixed clinical sample was comprised of 97 total participants, 71 of whom were classified as valid and 26 as invalid based on three well-validated, freestanding criterion PVTs. Overall, the LM embedded PVT demonstrated poor concordance with the criterion PVTs and unacceptable psychometric properties using ACS validity base rates (42% sensitivity/79% specificity). Moreover, 15-39% of participants obtained an invalid ACS base rate despite having a normatively-intact age-corrected LM Recognition total score. Receiving operating characteristic curve analysis revealed a Recognition total score cutoff of < 61% correct improved specificity (92%) while sensitivity remained weak (31%). Thus, results indicated the LM Recognition embedded PVT is not appropriate for use from an evidence-based perspective, and that clinicians may be faced with reconciling how a normatively intact cognitive performance on the Recognition subtest could simultaneously reflect invalid performance validity.

Measurement of COPD Severity Using a Survey-Based Score

PubMed Central

Omachi, Theodore A.; Katz, Patricia P.; Yelin, Edward H.; Iribarren, Carlos; Blanc, Paul D.

2010-01-01

Background: A comprehensive survey-based COPD severity score has usefulness for epidemiologic and health outcomes research. We previously developed and validated the survey-based COPD Severity Score without using lung function or other physiologic measurements. In this study, we aimed to further validate the severity score in a different COPD cohort and using a combination of patient-reported and objective physiologic measurements. Methods: Using data from the Function, Living, Outcomes, and Work cohort study of COPD, we evaluated the concurrent and predictive validity of the COPD Severity Score among 1,202 subjects. The survey instrument is a 35-point score based on symptoms, medication and oxygen use, and prior hospitalization or intubation for COPD. Subjects were systemically assessed using structured telephone survey, spirometry, and 6-min walk testing. Results: We found evidence to support concurrent validity of the score. Higher COPD Severity Score values were associated with poorer FEV1 (r = −0.38), FEV1% predicted (r = −0.40), Body mass, Obstruction, Dyspnea, Exercise Index (r = 0.57), and distance walked in 6 min (r = −0.43) (P < .0001 in all cases). Greater COPD severity was also related to poorer generic physical health status (r = −0.49) and disease-specific health-related quality of life (r = 0.57) (P < .0001). The score also demonstrated predictive validity. It was also associated with a greater prospective risk of acute exacerbation of COPD defined as ED visits (hazard ratio [HR], 1.31; 95% CI, 1.24-1.39), hospitalizations (HR, 1.59; 95% CI, 1.44-1.75), and either measure of hospital-based care for COPD (HR, 1.34; 95% CI, 1.26-1.41) (P < .0001 in all cases). Conclusion: The COPD Severity Score is a valid survey-based measure of disease-specific severity, both in terms of concurrent and predictive validity. The score is a psychometrically sound instrument for use in epidemiologic and outcomes research in COPD. PMID:20040611

Exploring the validity and reliability of a questionnaire for evaluating veterinary clinical teachers' supervisory skills during clinical rotations.

PubMed

Boerboom, T B B; Dolmans, D H J M; Jaarsma, A D C; Muijtjens, A M M; Van Beukelen, P; Scherpbier, A J J A

2011-01-01

Feedback to aid teachers in improving their teaching requires validated evaluation instruments. When implementing an evaluation instrument in a different context, it is important to collect validity evidence from multiple sources. We examined the validity and reliability of the Maastricht Clinical Teaching Questionnaire (MCTQ) as an instrument to evaluate individual clinical teachers during short clinical rotations in veterinary education. We examined four sources of validity evidence: (1) Content was examined based on theory of effective learning. (2) Response process was explored in a pilot study. (3) Internal structure was assessed by confirmatory factor analysis using 1086 student evaluations and reliability was examined utilizing generalizability analysis. (4) Relations with other relevant variables were examined by comparing factor scores with other outcomes. Content validity was supported by theory underlying the cognitive apprenticeship model on which the instrument is based. The pilot study resulted in an additional question about supervision time. A five-factor model showed a good fit with the data. Acceptable reliability was achievable with 10-12 questionnaires per teacher. Correlations between the factors and overall teacher judgement were strong. The MCTQ appears to be a valid and reliable instrument to evaluate clinical teachers' performance during short rotations.

Adaptation of the ESPA29 Parental Socialization Styles Scale to the Basque language: evidence of validity.

PubMed

López-Jáuregui, Alicia; Oliden, Paula Elosua

2009-11-01

The aim of this study is to adapt the ESPA29 scale of parental socialization styles in adolescence to the Basque language. The study of its psychometric properties is based on the search for evidence of internal and external validity. The first focuses on the assessment of the dimensionality of the scale by means of exploratory factor analysis. The relationship between the dimensions of parental socialization styles and gender and age guarantee the external validity of the scale. The study of the equivalence of the adapted and original versions is based on the comparisons of the reliability coefficients and on factor congruence. The results allow us to conclude the equivalence of the two scales.

Exploring a Framework for Consequential Validity for Performance-Based Assessments

ERIC Educational Resources Information Center

Kim, Su Jung

2017-01-01

This study explores a new comprehensive framework for understanding elements of validity, specifically for performance assessments that are administered within specific and dynamic contexts. The adoption of edTPA is a good empirical case for examining the concept of consequential validity because this assessment has been implemented at the state…

VALIDATION OF A METHOD FOR ESTIMATING LONG-TERM EXPOSURES BASED ON SHORT-TERM MEASUREMENTS

EPA Science Inventory

A method for estimating long-term exposures from short-term measurements is validated using data from a recent EPA study of exposure to fine particles. The method was developed a decade ago but data to validate it did not exist until recently. In this paper, data from repeated ...

Validating Accelerometry and Skinfold Measures in Youth with Down Syndrome

ERIC Educational Resources Information Center

Esposito, Phil Michael

2012-01-01

Current methods for measuring quantity and intensity of physical activity based on accelerometer output have been studied and validated in youth. These methods have been applied to youth with Down syndrome (DS) with no empirical research done to validate these measures. Similarly, individuals with DS have unique body proportions not represented by…

The Michigan Alcoholism Screening Test (MAST): A Statistical Validation Analysis

ERIC Educational Resources Information Center

Laux, John M.; Newman, Isadore; Brown, Russ

2004-01-01

This study extends the Michigan Alcoholism Screening Test (MAST; M. L. Selzer, 1971) literature base by examining 4 issues related to the validity of the MAST scores. Specifically, the authors examine the validity of the MAST scores in light of the presence of impression management, participant demographic variables, and item endorsement…

The Social Validity Assessment of Social Competence Intervention Behavior Goals

ERIC Educational Resources Information Center

Hurley, Jennifer J.; Wehby, Joseph H.; Feurer, Irene D.

2010-01-01

Social validation is the value judgment from society on the importance of a study. The social validity of behavior goals used in the social competence intervention literature was assessed using the Q-sort technique. The stimulus items were 80 different social competence behavior goals taken from 78 classroom-based social competence intervention…

Friendship Quality Scale: Conceptualization, Development and Validation

ERIC Educational Resources Information Center

Thien, Lei Mee; Razak, Nordin Abd; Jamil, Hazri

2012-01-01

The purpose of this study is twofold: (1) to initialize a new conceptualization of positive feature based Friendship Quality (FQUA) scale on the basis of four dimensions: Closeness, Help, Acceptance, and Safety; and (2) to develop and validate FQUA scale in the form of reflective measurement model. The scale development and validation procedures…

An Investigation of the Validity of Best Grading Practices

ERIC Educational Resources Information Center

McDaniel, Fred, II

2010-01-01

The validity of assessments, including grading, is paramount to education considering the high stakes decisions and inferences that are made based on grades. This investigation examined the validity of grading practices for middle school teachers in a large, suburban school district in the southeast. In this study, 118 reading and mathematics…

VALIDATION OF A METHOD FOR ESTIMATING LONG-TERM EXPOSURES BASED ON SHORT-TERM MEASUREMENTS

EPA Science Inventory

A method for estimating long-term exposures from short-term measurements is validated using data from a recent EPA study of exposure to fine particles. The method was developed a decade ago but long-term exposure data to validate it did not exist until recently. In this paper, ...

Evaluating the Content Validity of Multistage-Adaptive Tests

ERIC Educational Resources Information Center

Crotts, Katrina; Sireci, Stephen G.; Zenisky, April

2012-01-01

Validity evidence based on test content is important for educational tests to demonstrate the degree to which they fulfill their purposes. Most content validity studies involve subject matter experts (SMEs) who rate items that comprise a test form. In computerized-adaptive testing, examinees take different sets of items and test "forms"…

Exploring the Benefits of Respite Services to Family Caregivers: Methodological Issues and Current Findings

PubMed Central

Zarit, Steven H.; Liu, Yin; Bangerter, Lauren R.; Rovine, Michael J.

2017-01-01

Objectives There is growing emphasis on empirical validation of the efficacy of community-based services for older people and their families, but research on services such as respite care faces methodological challenges that have limited the growth of outcome studies. We identify problems associated with the usual research approaches for studying respite care, with the goal of stimulating use of novel and more appropriate research designs that can lead to improved studies of community-based services. Method Using the concept of research validity, we evaluate the methodological approaches in the current literature on respite services, including adult day services, in-home respite and overnight respite. Results Although randomized control trials (RCTs) are possible in community settings, validity is compromised by practical limitations of randomization and other problems. Quasi-experimental and interrupted time series designs offer comparable validity to RCTs and can be implemented effectively in community settings. Conclusion An emphasis on RCTs by funders and researchers is not supported by scientific evidence. Alternative designs can lead to development of a valid body of research on community services such as respite. PMID:26729467

Exploring the benefits of respite services to family caregivers: methodological issues and current findings.

PubMed

Zarit, Steven H; Bangerter, Lauren R; Liu, Yin; Rovine, Michael J

2017-03-01

There is growing emphasis on empirical validation of the efficacy of community-based services for older people and their families, but research on services such as respite care faces methodological challenges that have limited the growth of outcome studies. We identify problems associated with the usual research approaches for studying respite care, with the goal of stimulating use of novel and more appropriate research designs that can lead to improved studies of community-based services. Using the concept of research validity, we evaluate the methodological approaches in the current literature on respite services, including adult day services, in-home respite and overnight respite. Although randomized control trials (RCTs) are possible in community settings, validity is compromised by practical limitations of randomization and other problems. Quasi-experimental and interrupted time series designs offer comparable validity to RCTs and can be implemented effectively in community settings. An emphasis on RCTs by funders and researchers is not supported by scientific evidence. Alternative designs can lead to development of a valid body of research on community services such as respite.

Indirect Measurement of Sexual Orientation: Comparison of the Implicit Relational Assessment Procedure, Viewing Time, and Choice Reaction Time Tasks.

PubMed

Rönspies, Jelena; Schmidt, Alexander F; Melnikova, Anna; Krumova, Rosina; Zolfagari, Asadeh; Banse, Rainer

2015-07-01

The present study was conducted to validate an adaptation of the Implicit Relational Assessment Procedure (IRAP) as an indirect latency-based measure of sexual orientation. Furthermore, reliability and criterion validity of the IRAP were compared to two established indirect measures of sexual orientation: a Choice Reaction Time task (CRT) and a Viewing Time (VT) task. A sample of 87 heterosexual and 35 gay men completed all three indirect measures in an online study. The IRAP and the VT predicted sexual orientation nearly perfectly. Both measures also showed a considerable amount of convergent validity. Reliabilities (internal consistencies) reached satisfactory levels. In contrast, the CRT did not tap into sexual orientation in the present study. In sum, the VT measure performed best, with the IRAP showing only slightly lower reliability and criterion validity, whereas the CRT did not yield any evidence of reliability or criterion validity in the present research. The results were discussed in the light of specific task properties of the indirect latency-based measures (task-relevance vs. task-irrelevance).

How honest are the signals? A protocol for validating wearable sensors.

PubMed

Kayhan, Varol Onur; Chen, Zheng Chris; French, Kimberly A; Allen, Tammy D; Salomon, Kristen; Watkins, Alison

2018-02-01

There is growing interest among organizational researchers in tapping into alternative sources of data beyond self-reports to provide a new avenue for measuring behavioral constructs. Use of alternative data sources such as wearable sensors is necessary for developing theory and enhancing organizational practice. Although wearable sensors are now commercially available, the veracity of the data they capture is largely unknown and mostly based on manufacturers' claims. The goal of this research is to test the validity and reliability of data captured by one such wearable badge (by Humanyze) in the context of structured meetings where all individuals wear a badge for the duration of the encounter. We developed a series of studies, each targeting a specific sensor of this badge that is relevant for structured meetings, and we make specific recommendations for badge data usage based on our validation results. We have incorporated the insights from our studies on a website that researchers can use to conduct validation tests for their badges, upload their data, and assess the validity of the data. We discuss this website in the corresponding studies.

Reliability and validity of a tool to measure the severity of tongue thrust in children: the Tongue Thrust Rating Scale.

PubMed

Serel Arslan, S; Demir, N; Karaduman, A A

2017-02-01

This study aimed to develop a scale called Tongue Thrust Rating Scale (TTRS), which categorised tongue thrust in children in terms of its severity during swallowing, and to investigate its validity and reliability. The study describes the developmental phase of the TTRS and presented its content and criterion-based validity and interobserver and intra-observer reliability. For content validation, seven experts assessed the steps in the scale over two Delphi rounds. Two physical therapists evaluated videos of 50 children with cerebral palsy (mean age, 57·9 ± 16·8 months), using the TTRS to test criterion-based validity, interobserver and intra-observer reliability. The Karaduman Chewing Performance Scale (KCPS) and Drooling Severity and Frequency Scale (DSFS) were used for criterion-based validity. All the TTRS steps were deemed necessary. The content validity index was 0·857. A very strong positive correlation was found between two examinations by one physical therapist, which indicated intra-observer reliability (r = 0·938, P < 0·001). A very strong positive correlation was also found between the TTRS scores of two physical therapists, indicating interobserver reliability (r = 0·892, P < 0·001). There was also a strong positive correlation between the TTRS and KCPS (r = 0·724, P < 0·001) and a very strong positive correlation between the TTRS scores and DSFS (r = 0·822 and r = 0·755; P < 0·001). These results demonstrated the criterion-based validity of the TTRS. The TTRS is a valid, reliable and clinically easy-to-use functional instrument to document the severity of tongue thrust in children. © 2016 John Wiley & Sons Ltd.

Testing the psychometric properties of the Environmental Attitudes Inventory on undergraduate students in the Arab context: A test-retest approach.

PubMed

AlMenhali, Entesar Ali; Khalid, Khalizani; Iyanna, Shilpa

2018-01-01

The Environmental Attitudes Inventory (EAI) was developed to evaluate the multidimensional nature of environmental attitudes; however, it is based on a dataset from outside the Arab context. This study reinvestigated the construct validity of the EAI with a new dataset and confirmed the feasibility of applying it in the Arab context. One hundred and forty-eight subjects in Study 1 and 130 in Study 2 provided valid responses. An exploratory factor analysis (EFA) was used to extract a new factor structure in Study 1, and confirmatory factor analysis (CFA) was performed in Study 2. Both studies generated a seven-factor model, and the model fit was discussed for both the studies. Study 2 exhibited satisfactory model fit indices compared to Study 1. Factor loading values of a few items in Study 1 affected the reliability values and average variance extracted values, which demonstrated low discriminant validity. Based on the results of the EFA and CFA, this study showed sufficient model fit and suggested the feasibility of applying the EAI in the Arab context with a good construct validity and internal consistency.

Testing the psychometric properties of the Environmental Attitudes Inventory on undergraduate students in the Arab context: A test-retest approach

PubMed Central

2018-01-01

The Environmental Attitudes Inventory (EAI) was developed to evaluate the multidimensional nature of environmental attitudes; however, it is based on a dataset from outside the Arab context. This study reinvestigated the construct validity of the EAI with a new dataset and confirmed the feasibility of applying it in the Arab context. One hundred and forty-eight subjects in Study 1 and 130 in Study 2 provided valid responses. An exploratory factor analysis (EFA) was used to extract a new factor structure in Study 1, and confirmatory factor analysis (CFA) was performed in Study 2. Both studies generated a seven-factor model, and the model fit was discussed for both the studies. Study 2 exhibited satisfactory model fit indices compared to Study 1. Factor loading values of a few items in Study 1 affected the reliability values and average variance extracted values, which demonstrated low discriminant validity. Based on the results of the EFA and CFA, this study showed sufficient model fit and suggested the feasibility of applying the EAI in the Arab context with a good construct validity and internal consistency. PMID:29758021

Teachers' Cloud-Based Learning Designs: The Development of a Guiding Rubric Using the TPACK Framework

ERIC Educational Resources Information Center

Al-Harthi, Aisha Salim Ali; Campbell, Chris; Karimi, Arafeh

2018-01-01

This study aimed to develop, validate, and trial a rubric for evaluating the cloud-based learning designs (CBLD) that were developed by teachers using virtual learning environments. The rubric was developed using the technological pedagogical content knowledge (TPACK) framework, with rubric development including content and expert validation of…

Validating College Course Placement Decisions Based on CLEP Exam Scores: CLEP Placement Validity Study Results. Statistical Report

ERIC Educational Resources Information Center

Godfrey, Kelly E.; Jagesic, Sanja

2016-01-01

The College-Level Examination Program® (CLEP®) is a computer-based prior-learning assessment that allows examinees the opportunity to demonstrate mastery of knowledge and skills necessary to earn postsecondary course credit in higher education. Currently, there are 33 exams in five subject areas: composition and literature, world languages,…

A Cartoon-Based Measure of PTSD Symptomatology in Children Exposed to a Disaster

ERIC Educational Resources Information Center

Elklit, Ask; Nielsen, Louise Hjort; Lasgaard, Mathias; Duch, Christina

2013-01-01

Research on childhood posttraumatic stress disorder (PTSD) is sparse. This is partly due to the limited availability of empirically validated measures for children who are insecure readers. The present study examined the reliability and validity of a cartoon-based measure of PTSD symptoms in children exposed to a disaster. Cartoons were generated…

An Evaluation of Test Speededness in an Assessment for Third-Grade Gifted Students

ERIC Educational Resources Information Center

Hailey, Emily; Callahan, Carolyn M.; Azano, Amy; Moon, Tonya R.

2012-01-01

Reliability and validity are integral concepts in assessment design. Test speededness, the influence of time constraints on test taker performance, is often an overlooked threat to reliability and validity, especially in classroom-based testing. The purpose of this study is to evaluate the degree of test speededness of classroom-based assessments…

Examining Evidence for the Validity of PISA Learning Strategy Scales Based on Student Response Processes

ERIC Educational Resources Information Center

Hopfenbeck, Therese N.; Maul, Andrew

2011-01-01

The aim of this study was to investigate response-process based evidence for the validity of the Programme for International Student Assessment's (PISA) self-report questionnaire scales as measures of specific psychological constructs, with a focus on scales meant to measure inclination toward specific learning strategies. Cognitive interviews (N…

Understanding Outcome-Based Education Changes in Teacher Education: Evaluation of a New Instrument with Preliminary Findings

ERIC Educational Resources Information Center

Deneen, Christopher; Brown, Gavin T. L.; Bond, Trevor G.; Shroff, Ronnie

2013-01-01

Outcome-based education (OBE) is a current initiative in Hong Kong universities, with widespread backing by governments and standards bodies. However, study of students' perceptions of OBE and validation of understanding these perceptions are lacking. This paper reports on the validation of an OBE-specific instrument and resulting preliminary…

Development and initial validation of primary care provider mental illness management and team-based care self-efficacy scales.

PubMed

Loeb, Danielle F; Crane, Lori A; Leister, Erin; Bayliss, Elizabeth A; Ludman, Evette; Binswanger, Ingrid A; Kline, Danielle M; Smith, Meredith; deGruy, Frank V; Nease, Donald E; Dickinson, L Miriam

Develop and validate self-efficacy scales for primary care provider (PCP) mental illness management and team-based care participation. We developed three self-efficacy scales: team-based care (TBC), mental illness management (MIM), and chronic medical illness (CMI). We developed the scales using Bandura's Social Cognitive Theory as a guide. The survey instrument included items from previously validated scales on team-based care and mental illness management. We administered a mail survey to 900 randomly selected Colorado physicians. We conducted exploratory principal factor analysis with oblique rotation. We constructed self-efficacy scales and calculated standardized Cronbach's alpha coefficients to test internal consistency. We calculated correlation coefficients between the MIM and TBC scales and previously validated measures related to each scale to evaluate convergent validity. We tested correlations between the TBC and the measures expected to correlate with the MIM scale and vice versa to evaluate discriminant validity. PCPs (n=402, response rate=49%) from diverse practice settings completed surveys. Items grouped into factors as expected. Cronbach's alphas were 0.94, 0.88, and 0.83 for TBC, MIM, and CMI scales respectively. In convergent validity testing, the TBC scale was correlated as predicted with scales assessing communications strategies, attitudes toward teams, and other teamwork indicators (r=0.25 to 0.40, all statistically significant). Likewise, the MIM scale was significantly correlated with several items about knowledge and experience managing mental illness (r=0.24 to 41, all statistically significant). As expected in discriminant validity testing, the TBC scale had only very weak correlations with the mental illness knowledge and experience managing mental illness items (r=0.03 to 0.12). Likewise, the MIM scale was only weakly correlated with measures of team-based care (r=0.09 to.17). This validation study of MIM and TBC self-efficacy scales showed high internal validity and good construct validity. Copyright © 2016 Elsevier Inc. All rights reserved.

Validating Performance Level Descriptors (PLDs) for the AP® Environmental Science Exam

ERIC Educational Resources Information Center

Reshetar, Rosemary; Kaliski, Pamela; Chajewski, Michael; Lionberger, Karen

2012-01-01

This presentation summarizes a pilot study conducted after the May 2011 administration of the AP Environmental Science Exam. The study used analytical methods based on scaled anchoring as input to a Performance Level Descriptor validation process that solicited systematic input from subject matter experts.

Is questionnaire-based sitting time inaccurate and can it be improved? A cross-sectional investigation using accelerometer-based sitting time

PubMed Central

Gupta, Nidhi; Christiansen, Caroline Stordal; Hanisch, Christiana; Bay, Hans; Burr, Hermann; Holtermann, Andreas

2017-01-01

Objectives To investigate the differences between a questionnaire-based and accelerometer-based sitting time, and develop a model for improving the accuracy of questionnaire-based sitting time for predicting accelerometer-based sitting time. Methods 183 workers in a cross-sectional study reported sitting time per day using a single question during the measurement period, and wore 2 Actigraph GT3X+ accelerometers on the thigh and trunk for 1–4 working days to determine their actual sitting time per day using the validated Acti4 software. Least squares regression models were fitted with questionnaire-based siting time and other self-reported predictors to predict accelerometer-based sitting time. Results Questionnaire-based and accelerometer-based average sitting times were ≈272 and ≈476 min/day, respectively. A low Pearson correlation (r=0.32), high mean bias (204.1 min) and wide limits of agreement (549.8 to −139.7 min) between questionnaire-based and accelerometer-based sitting time were found. The prediction model based on questionnaire-based sitting explained 10% of the variance in accelerometer-based sitting time. Inclusion of 9 self-reported predictors in the model increased the explained variance to 41%, with 10% optimism using a resampling bootstrap validation. Based on a split validation analysis, the developed prediction model on ≈75% of the workers (n=132) reduced the mean and the SD of the difference between questionnaire-based and accelerometer-based sitting time by 64% and 42%, respectively, in the remaining 25% of the workers. Conclusions This study indicates that questionnaire-based sitting time has low validity and that a prediction model can be one solution to materially improve the precision of questionnaire-based sitting time. PMID:28093433

Validity test and its consistency in the construction of patient loyalty model

NASA Astrophysics Data System (ADS)

Yanuar, Ferra

2016-04-01

The main objective of this present study is to demonstrate the estimation of validity values and its consistency based on structural equation model. The method of estimation was then implemented to an empirical data in case of the construction the patient loyalty model. In the hypothesis model, service quality, patient satisfaction and patient loyalty were determined simultaneously, each factor were measured by any indicator variables. The respondents involved in this study were the patients who ever got healthcare at Puskesmas in Padang, West Sumatera. All 394 respondents who had complete information were included in the analysis. This study found that each construct; service quality, patient satisfaction and patient loyalty were valid. It means that all hypothesized indicator variables were significant to measure their corresponding latent variable. Service quality is the most measured by tangible, patient satisfaction is the most mesured by satisfied on service and patient loyalty is the most measured by good service quality. Meanwhile in structural equation, this study found that patient loyalty was affected by patient satisfaction positively and directly. Service quality affected patient loyalty indirectly with patient satisfaction as mediator variable between both latent variables. Both structural equations were also valid. This study also proved that validity values which obtained here were also consistence based on simulation study using bootstrap approach.

Assessing the environmental characteristics of cycling routes to school: a study on the reliability and validity of a Google Street View-based audit.

PubMed

Vanwolleghem, Griet; Van Dyck, Delfien; Ducheyne, Fabian; De Bourdeaudhuij, Ilse; Cardon, Greet

2014-06-10

Google Street View provides a valuable and efficient alternative to observe the physical environment compared to on-site fieldwork. However, studies on the use, reliability and validity of Google Street View in a cycling-to-school context are lacking. We aimed to study the intra-, inter-rater reliability and criterion validity of EGA-Cycling (Environmental Google Street View Based Audit - Cycling to school), a newly developed audit using Google Street View to assess the physical environment along cycling routes to school. Parents (n = 52) of 11-to-12-year old Flemish children, who mostly cycled to school, completed a questionnaire and identified their child's cycling route to school on a street map. Fifty cycling routes of 11-to-12-year olds were identified and physical environmental characteristics along the identified routes were rated with EGA-Cycling (5 subscales; 37 items), based on Google Street View. To assess reliability, two researchers performed the audit. Criterion validity of the audit was examined by comparing the ratings based on Google Street View with ratings through on-site assessments. Intra-rater reliability was high (kappa range 0.47-1.00). Large variations in the inter-rater reliability (kappa range -0.03-1.00) and criterion validity scores (kappa range -0.06-1.00) were reported, with acceptable inter-rater reliability values for 43% of all items and acceptable criterion validity for 54% of all items. EGA-Cycling can be used to assess physical environmental characteristics along cycling routes to school. However, to assess the micro-environment specifically related to cycling, on-site assessments have to be added.

Development and Validation of the Masculine Attributes Questionnaire

PubMed Central

Cho, Junhan; Kogan, Steven M.

2017-01-01

The present study describes the development and validation of the Masculine Attributes Questionnaire (MAQ). The purpose of this study was to develop a theoretically and empirically grounded measure of masculine attributes for sexual health research with African American young men. Consistent with Whitehead’s theory, the MAQ items were hypothesized to comprise two components representing reputation-based and respect-based attributes. The sample included 505 African American men aged 19 to 22 years (M = 20.29, SD = 1.10) living in resource-poor communities in the rural South. Convergent and discriminant validity of the MAQ were assessed by examining the associations of masculinity attributes with psychosocial factors. Criterion validity was assessed by examining the extent to which the MAQ subscales predicted sexual risk behavior outcomes. Consistent with study hypotheses, the MAQ was composed of (a) reputation-based attributes oriented toward sexual prowess, toughness, and authority-defying behavior and (b) respect-based attributes oriented toward economic independence, socially approved levels of hard work and education, and committed romantic relationships. Reputation-based attributes were associated positively with street code and negatively related to academic orientation, vocational engagement, and self-regulation, whereas respect-based attributes were associated positively with academic and vocational orientations and self-regulation. Finally, reputation-based attributes predicted sexual risk behaviors including concurrent sexual partnerships, multiple sexual partners, marijuana use, and incarceration, net of the influence of respect-based attributes. The development of the MAQ provides a new measure that permits systematic quantitative investigation of the associations between African American men’s masculinity ideology and sexual risk behavior. PMID:28413906

A validation of well-being and happiness surveys for administration via the Internet.

PubMed

Howell, Ryan T; Rodzon, Katrina S; Kurai, Mark; Sanchez, Amy H

2010-08-01

Internet research is appealing because it is a cost- and time-efficient way to access a large number of participants; however, the validity of Internet research for important subjective well-being (SWB) surveys has not been adequately assessed. The goal of the present study was to validate the Satisfaction With Life Scale (SWLS; Diener, Emmons, Larsen, & Griffin, 1985), the Positive and Negative Affect Schedule (PANAS-X; Watson & Clark, 1994), and the Subjective Happiness Scale (SHS; Lyubomirsky & Lepper, 1999) for use on the Internet. This study compared the quality of data collected using paper-based (paper-and-pencil version in a lab setting), computer-based (Web-based version in a lab setting), and Internet (Web-based version on a computer of the participant's choosing) surveys for these three measures of SWB. The paper-based and computer-based experiment recruited two college student samples; the Internet experiments recruited a college student sample and an adult sample responding to ads on different social-networking Web sites. This study provides support for the reliability, validity, and generalizability of the Internet format of the SWLS, PANAS-X, and SHS. Across the three experiments, the results indicate that the computer-based and Internet surveys had means, standard deviations, reliabilities, and factor structures that were similar to those of the paper-based versions. The discussion examines the difficulty of higher attrition for the Internet version, the need to examine reverse-coded items in the future, and the possibility that unhappy individuals are more likely to participate in Internet surveys of SWB.

Development and Validation of the Masculine Attributes Questionnaire.

PubMed

Cho, Junhan; Kogan, Steven M

2017-07-01

The present study describes the development and validation of the Masculine Attributes Questionnaire (MAQ). The purpose of this study was to develop a theoretically and empirically grounded measure of masculine attributes for sexual health research with African American young men. Consistent with Whitehead's theory, the MAQ items were hypothesized to comprise two components representing reputation-based and respect-based attributes. The sample included 505 African American men aged 19 to 22 years ( M = 20.29, SD = 1.10) living in resource-poor communities in the rural South. Convergent and discriminant validity of the MAQ were assessed by examining the associations of masculinity attributes with psychosocial factors. Criterion validity was assessed by examining the extent to which the MAQ subscales predicted sexual risk behavior outcomes. Consistent with study hypotheses, the MAQ was composed of (a) reputation-based attributes oriented toward sexual prowess, toughness, and authority-defying behavior and (b) respect-based attributes oriented toward economic independence, socially approved levels of hard work and education, and committed romantic relationships. Reputation-based attributes were associated positively with street code and negatively related to academic orientation, vocational engagement, and self-regulation, whereas respect-based attributes were associated positively with academic and vocational orientations and self-regulation. Finally, reputation-based attributes predicted sexual risk behaviors including concurrent sexual partnerships, multiple sexual partners, marijuana use, and incarceration, net of the influence of respect-based attributes. The development of the MAQ provides a new measure that permits systematic quantitative investigation of the associations between African American men's masculinity ideology and sexual risk behavior.

Validation of standard operating procedures in a multicenter retrospective study to identify -omics biomarkers for chronic low back pain.

PubMed

Dagostino, Concetta; De Gregori, Manuela; Gieger, Christian; Manz, Judith; Gudelj, Ivan; Lauc, Gordan; Divizia, Laura; Wang, Wei; Sim, Moira; Pemberton, Iain K; MacDougall, Jane; Williams, Frances; Van Zundert, Jan; Primorac, Dragan; Aulchenko, Yurii; Kapural, Leonardo; Allegri, Massimo

2017-01-01

Chronic low back pain (CLBP) is one of the most common medical conditions, ranking as the greatest contributor to global disability and accounting for huge societal costs based on the Global Burden of Disease 2010 study. Large genetic and -omics studies provide a promising avenue for the screening, development and validation of biomarkers useful for personalized diagnosis and treatment (precision medicine). Multicentre studies are needed for such an effort, and a standardized and homogeneous approach is vital for recruitment of large numbers of participants among different centres (clinical and laboratories) to obtain robust and reproducible results. To date, no validated standard operating procedures (SOPs) for genetic/-omics studies in chronic pain have been developed. In this study, we validated an SOP model that will be used in the multicentre (5 centres) retrospective "PainOmics" study, funded by the European Community in the 7th Framework Programme, which aims to develop new biomarkers for CLBP through three different -omics approaches: genomics, glycomics and activomics. The SOPs describe the specific procedures for (1) blood collection, (2) sample processing and storage, (3) shipping details and (4) cross-check testing and validation before assays that all the centres involved in the study have to follow. Multivariate analysis revealed the absolute specificity and homogeneity of the samples collected by the five centres for all genetics, glycomics and activomics analyses. The SOPs used in our multicenter study have been validated. Hence, they could represent an innovative tool for the correct management and collection of reliable samples in other large-omics-based multicenter studies.

Clinical Validity of the ADI-R in a US-Based Latino Population.

PubMed

Vanegas, Sandra B; Magaña, Sandra; Morales, Miguel; McNamara, Ellyn

2016-05-01

The Autism Diagnostic Interview-Revised (ADI-R) has been validated as a tool to aid in the diagnosis of Autism; however, given the growing diversity in the United States, the ADI-R must be validated for different languages and cultures. This study evaluates the validity of the ADI-R in a US-based Latino, Spanish-speaking population of 50 children and adolescents with ASD and developmental disability. Sensitivity and specificity of the ADI-R as a diagnostic tool were moderate, but lower than previously reported values. Validity of the social reciprocity and restrictive and repetitive behaviors domains was high, but low in the communication domain. Findings suggest that language discordance between caregiver and child may influence reporting of communication symptoms and contribute to lower sensitivity and specificity.

Factor structure, validity and reliability of the Cambridge Worry Scale in a pregnant population.

PubMed

Green, Josephine M; Kafetsios, Konstantinos; Statham, Helen E; Snowdon, Claire M

2003-11-01

This article presents the Cambridge Worry Scale (CWS), a content-based measure for assessing worries, and discusses its psychometric properties based on a longitudinal study of 1,207 pregnant women. Principal components analysis revealed a four-factor structure of women's concerns during pregnancy: socio-medical, own health, socio-economic and relational. The measure demonstrated good reliability and validity. Total CWS scores were strongly associated with state and trait anxiety (convergent validity) but also had significant and unique predictive value for mood outcomes (discriminant validity). The CWS discriminated better between women with different reproductive histories than measures of state and trait anxiety. We conclude that the CWS is a reliable and valid tool for assessing the extent and content of worries in specific situations.

MotiveValidator: interactive web-based validation of ligand and residue structure in biomolecular complexes.

PubMed

Vařeková, Radka Svobodová; Jaiswal, Deepti; Sehnal, David; Ionescu, Crina-Maria; Geidl, Stanislav; Pravda, Lukáš; Horský, Vladimír; Wimmerová, Michaela; Koča, Jaroslav

2014-07-01

Structure validation has become a major issue in the structural biology community, and an essential step is checking the ligand structure. This paper introduces MotiveValidator, a web-based application for the validation of ligands and residues in PDB or PDBx/mmCIF format files provided by the user. Specifically, MotiveValidator is able to evaluate in a straightforward manner whether the ligand or residue being studied has a correct annotation (3-letter code), i.e. if it has the same topology and stereochemistry as the model ligand or residue with this annotation. If not, MotiveValidator explicitly describes the differences. MotiveValidator offers a user-friendly, interactive and platform-independent environment for validating structures obtained by any type of experiment. The results of the validation are presented in both tabular and graphical form, facilitating their interpretation. MotiveValidator can process thousands of ligands or residues in a single validation run that takes no more than a few minutes. MotiveValidator can be used for testing single structures, or the analysis of large sets of ligands or fragments prepared for binding site analysis, docking or virtual screening. MotiveValidator is freely available via the Internet at http://ncbr.muni.cz/MotiveValidator. © The Author(s) 2014. Published by Oxford University Press on behalf of Nucleic Acids Research.

The Validity of the Comparative Interrupted Time Series Design for Evaluating the Effect of School-Level Interventions.

PubMed

Jacob, Robin; Somers, Marie-Andree; Zhu, Pei; Bloom, Howard

2016-06-01

In this article, we examine whether a well-executed comparative interrupted time series (CITS) design can produce valid inferences about the effectiveness of a school-level intervention. This article also explores the trade-off between bias reduction and precision loss across different methods of selecting comparison groups for the CITS design and assesses whether choosing matched comparison schools based only on preintervention test scores is sufficient to produce internally valid impact estimates. We conduct a validation study of the CITS design based on the federal Reading First program as implemented in one state using results from a regression discontinuity design as a causal benchmark. Our results contribute to the growing base of evidence regarding the validity of nonexperimental designs. We demonstrate that the CITS design can, in our example, produce internally valid estimates of program impacts when multiple years of preintervention outcome data (test scores in the present case) are available and when a set of reasonable criteria are used to select comparison organizations (schools in the present case). © The Author(s) 2016.

Systematic review of measurement properties of questionnaires measuring somatization in primary care patients.

PubMed

Sitnikova, Kate; Dijkstra-Kersten, Sandra M A; Mokkink, Lidwine B; Terluin, Berend; van Marwijk, Harm W J; Leone, Stephanie S; van der Horst, Henriëtte E; van der Wouden, Johannes C

2017-12-01

The aim of this review is to critically appraise the evidence on measurement properties of self-report questionnaires measuring somatization in adult primary care patients and to provide recommendations about which questionnaires are most useful for this purpose. We assessed the methodological quality of included studies using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. To draw overall conclusions about the quality of the questionnaires, we conducted an evidence synthesis using predefined criteria for judging the measurement properties. We found 24 articles on 9 questionnaires. Studies on the Patient Health Questionnaire-15 (PHQ-15) and the Four-Dimensional Symptom Questionnaire (4DSQ) somatization subscale prevailed and covered the broadest range of measurement properties. These questionnaires had the best internal consistency, test-retest reliability, structural validity, and construct validity. The PHQ-15 also had good criterion validity, whereas the 4DSQ somatization subscale was validated in several languages. The Bodily Distress Syndrome (BDS) checklist had good internal consistency and structural validity. Some evidence was found for good construct validity and criterion validity of the Physical Symptom Checklist (PSC-51) and good construct validity of the Symptom Check-List (SCL-90-R) somatization subscale. However, these three questionnaires were only studied in a small number of primary care studies. Based on our findings, we recommend the use of either the PHQ-15 or 4DSQ somatization subscale for somatization in primary care. Other questionnaires, such as the BDS checklist, PSC-51 and the SCL-90-R somatization subscale show promising results but have not been studied extensively in primary care. Copyright © 2017 Elsevier Inc. All rights reserved.

Methodological convergence of program evaluation designs.

PubMed

Chacón-Moscoso, Salvador; Anguera, M Teresa; Sanduvete-Chaves, Susana; Sánchez-Martín, Milagrosa

2014-01-01

Nowadays, the confronting dichotomous view between experimental/quasi-experimental and non-experimental/ethnographic studies still exists but, despite the extensive use of non-experimental/ethnographic studies, the most systematic work on methodological quality has been developed based on experimental and quasi-experimental studies. This hinders evaluators and planners' practice of empirical program evaluation, a sphere in which the distinction between types of study is changing continually and is less clear. Based on the classical validity framework of experimental/quasi-experimental studies, we carry out a review of the literature in order to analyze the convergence of design elements in methodological quality in primary studies in systematic reviews and ethnographic research. We specify the relevant design elements that should be taken into account in order to improve validity and generalization in program evaluation practice in different methodologies from a practical methodological and complementary view. We recommend ways to improve design elements so as to enhance validity and generalization in program evaluation practice.

A verification library for multibody simulation software

NASA Technical Reports Server (NTRS)

Kim, Sung-Soo; Haug, Edward J.; Frisch, Harold P.

1989-01-01

A multibody dynamics verification library, that maintains and manages test and validation data is proposed, based on RRC Robot arm and CASE backhoe validation and a comparitive study of DADS, DISCOS, and CONTOPS that are existing public domain and commercial multibody dynamic simulation programs. Using simple representative problems, simulation results from each program are cross checked, and the validation results are presented. Functionalities of the verification library are defined, in order to automate validation procedure.

Developing a Psychometric Instrument to Measure Physical Education Teachers' Job Demands and Resources

ERIC Educational Resources Information Center

Zhang, Tan; Chen, Ang

2017-01-01

Based on the job demands-resources model, the study developed and validated an instrument that measures physical education teachers' job demands-resources perception. Expert review established content validity with the average item rating of 3.6/5.0. Construct validity and reliability were determined with a teacher sample (n = 397). Exploratory…

Predictive Validity of a Student Self-Report Screener of Behavioral and Emotional Risk in an Urban High School

ERIC Educational Resources Information Center

Dowdy, Erin; Harrell-Williams, Leigh; Dever, Bridget V.; Furlong, Michael J.; Moore, Stephanie; Raines, Tara; Kamphaus, Randy W.

2016-01-01

Increasingly, schools are implementing school-based screening for risk of behavioral and emotional problems; hence, foundational evidence supporting the predictive validity of screening instruments is important to assess. This study examined the predictive validity of the Behavior Assessment System for Children-2 Behavioral and Emotional Screening…

Development and Validation of a New Measure of Faculty Assessments of Reformed Teaching and Learning Practices

ERIC Educational Resources Information Center

Ellett, Chad D.; Monsaas, Judy; Martin-Hansen, Lisa; Demir, Abdulkadir

2012-01-01

This study reports on the continued large-sample validation of the Inventory for Teaching and Learning (ITAL), a new teacher perception measure of "reformed (inquiry- and standards-based) and traditional teaching and learning" developed for use in science and mathematics classrooms. The continued validation of the ITAL used large samples…

Reliability and Validity of the Devereux Early Childhood Assessment (DECA) as a Function of Parent and Teacher Ratings

ERIC Educational Resources Information Center

Barbu, Otilia C.; Levine-Donnerstein, Deborah; Marx, Ronald W.; Yaden, David B., Jr.

2013-01-01

This study examined reliability and validity of the Devereux Early Childhood Assessment (DECA), based on samples of parents and teachers' ratings of 1,145 entering kindergartners in the Southwest. Confirmatory factor analysis showed that DECA presented good reliability and validity for manifest variables, corroborating previous findings. Three…

Tracing Professional Proficiencies Back to Curricula in Professional Education: A Content Validation Based on the Convergent Validity of Multiple Job Analysis Measures.

ERIC Educational Resources Information Center

Burgar, Paul S.

A study was commissioned by a large petrochemical concern in order to validate professional degrees as a job entry requirement. The investigations considered two issues: (1) "Are activities performed by professionals (chemists and engineers) measurably different from the activities of subordinate technical personnel?" and "What…

Development and Validation of a Measure of Elementary Teachers' Science Content Knowledge in Two Multiyear Teacher Professional Development Intervention Projects

ERIC Educational Resources Information Center

Maerten-Rivera, Jaime Lynn; Huggins-Manley, Anne Corinne; Adamson, Karen; Lee, Okhee; Llosa, Lorena

2015-01-01

Using data collected from two multiyear teacher professional development projects employing randomized control trials, this study describes the development and validation of a paper-based test of elementary teachers' science content knowledge (SCK). Evidence of construct validity is presented, including evidence on internal structural…

The Perceived Leadership Communication Questionnaire (PLCQ): Development and Validation.

PubMed

Schneider, Frank M; Maier, Michaela; Lovrekovic, Sara; Retzbach, Andrea

2015-01-01

The Perceived Leadership Communication Questionnaire (PLCQ) is a short, reliable, and valid instrument for measuring leadership communication from both perspectives of the leader and the follower. Drawing on a communication-based approach to leadership and following a theoretical framework of interpersonal communication processes in organizations, this article describes the development and validation of a one-dimensional 6-item scale in four studies (total N = 604). Results from Study 1 and 2 provide evidence for the internal consistency and factorial validity of the PLCQ's self-rating version (PLCQ-SR)-a version for measuring how leaders perceive their own communication with their followers. Results from Study 3 and 4 show internal consistency, construct validity, and criterion validity of the PLCQ's other-rating version (PLCQ-OR)-a version for measuring how followers perceive the communication of their leaders. Cronbach's α had an average of.80 over the four studies. All confirmatory factor analyses yielded good to excellent model fit indices. Convergent validity was established by average positive correlations of.69 with subdimensions of transformational leadership and leader-member exchange scales. Furthermore, nonsignificant correlations with socially desirable responding indicated discriminant validity. Last, criterion validity was supported by a moderately positive correlation with job satisfaction (r =.31).

Alternative methods to evaluate trial level surrogacy.

PubMed

Abrahantes, Josè Cortiñas; Shkedy, Ziv; Molenberghs, Geert

2008-01-01

The evaluation and validation of surrogate endpoints have been extensively studied in the last decade. Prentice [1] and Freedman, Graubard and Schatzkin [2] laid the foundations for the evaluation of surrogate endpoints in randomized clinical trials. Later, Buyse et al. [5] proposed a meta-analytic methodology, producing different methods for different settings, which was further studied by Alonso and Molenberghs [9], in their unifying approach based on information theory. In this article, we focus our attention on the trial-level surrogacy and propose alternative procedures to evaluate such surrogacy measure, which do not pre-specify the type of association. A promising correction based on cross-validation is investigated. As well as the construction of confidence intervals for this measure. In order to avoid making assumption about the type of relationship between the treatment effects and its distribution, a collection of alternative methods, based on regression trees, bagging, random forests, and support vector machines, combined with bootstrap-based confidence interval and, should one wish, in conjunction with a cross-validation based correction, will be proposed and applied. We apply the various strategies to data from three clinical studies: in opthalmology, in advanced colorectal cancer, and in schizophrenia. The results obtained for the three case studies are compared; they indicate that using random forest or bagging models produces larger estimated values for the surrogacy measure, which are in general stabler and the confidence interval narrower than linear regression and support vector regression. For the advanced colorectal cancer studies, we even found the trial-level surrogacy is considerably different from what has been reported. In general the alternative methods are more computationally demanding, and specially the calculation of the confidence intervals, require more computational time that the delta-method counterpart. First, more flexible modeling techniques can be used, allowing for other type of association. Second, when no cross-validation-based correction is applied, overly optimistic trial-level surrogacy estimates will be found, thus cross-validation is highly recommendable. Third, the use of the delta method to calculate confidence intervals is not recommendable since it makes assumptions valid only in very large samples. It may also produce range-violating limits. We therefore recommend alternatives: bootstrap methods in general. Also, the information-theoretic approach produces comparable results with the bagging and random forest approaches, when cross-validation correction is applied. It is also important to observe that, even for the case in which the linear model might be a good option too, bagging methods perform well too, and their confidence intervals were more narrow.

Development and Validation of Triarchic Construct Scales from the Psychopathic Personality Inventory

PubMed Central

Hall, Jason R.; Drislane, Laura E.; Patrick, Christopher J.; Morano, Mario; Lilienfeld, Scott O.; Poythress, Norman G.

2014-01-01

The Triarchic model of psychopathy describes this complex condition in terms of distinct phenotypic components of boldness, meanness, and disinhibition. Brief self-report scales designed specifically to index these psychopathy facets have thus far demonstrated promising construct validity. The present study sought to develop and validate scales for assessing facets of the Triarchic model using items from a well-validated existing measure of psychopathy—the Psychopathic Personality Inventory (PPI). A consensus rating approach was used to identify PPI items relevant to each Triarchic facet, and the convergent and discriminant validity of the resulting PPI-based Triarchic scales were evaluated in relation to multiple criterion variables (i.e., other psychopathy inventories, antisocial personality disorder features, personality traits, psychosocial functioning) in offender and non-offender samples. The PPI-based Triarchic scales showed good internal consistency and related to criterion variables in ways consistent with predictions based on the Triarchic model. Findings are discussed in terms of implications for conceptualization and assessment of psychopathy. PMID:24447280

Development and validation of Triarchic construct scales from the psychopathic personality inventory.

PubMed

Hall, Jason R; Drislane, Laura E; Patrick, Christopher J; Morano, Mario; Lilienfeld, Scott O; Poythress, Norman G

2014-06-01

The Triarchic model of psychopathy describes this complex condition in terms of distinct phenotypic components of boldness, meanness, and disinhibition. Brief self-report scales designed specifically to index these psychopathy facets have thus far demonstrated promising construct validity. The present study sought to develop and validate scales for assessing facets of the Triarchic model using items from a well-validated existing measure of psychopathy-the Psychopathic Personality Inventory (PPI). A consensus-rating approach was used to identify PPI items relevant to each Triarchic facet, and the convergent and discriminant validity of the resulting PPI-based Triarchic scales were evaluated in relation to multiple criterion variables (i.e., other psychopathy inventories, antisocial personality disorder features, personality traits, psychosocial functioning) in offender and nonoffender samples. The PPI-based Triarchic scales showed good internal consistency and related to criterion variables in ways consistent with predictions based on the Triarchic model. Findings are discussed in terms of implications for conceptualization and assessment of psychopathy.

Dietary Assessment in the MetaCardis Study: Development and Relative Validity of an Online Food Frequency Questionnaire.

PubMed

Verger, Eric O; Armstrong, Patrice; Nielsen, Trine; Chakaroun, Rima; Aron-Wisnewsky, Judith; Gøbel, Rikke Juul; Schütz, Tatjana; Delaere, Fabien; Gausseres, Nicolas; Clément, Karine; Holmes, Bridget A

2017-06-01

The European study MetaCardis aims to investigate the role of the gut microbiota in health and cardiometabolic diseases in France, Germany, and Denmark. To evaluate long-term diet-disease relationships, a food frequency questionnaire (FFQ) was found to be the most relevant dietary assessment method for the MetaCardis study. The objectives of this study were to describe the development of three semiquantitative online FFQs used in the MetaCardis study-one FFQ per country-and to assess the relative validity of the French MetaCardis FFQ. The layout and format of the MetaCardis FFQ was based on the European Prospective Investigation of Cancer (EPIC)-Norfolk FFQ and the content was based on relevant European FFQs. Portion size and nutrient composition were derived from national food consumption surveys and food composition databases. To assess the validity of the French MetaCardis FFQ, a cross-sectional study design was utilized. The validation study included 324 adults recruited between September 2013 and June 2015 from different hospitals in Paris, France. Food intakes were measured with both the French MetaCardis FFQ and 3 consecutive self-administered web-based 24-hour dietary recalls (DRs). Several measures of validity of the French MetaCardis FFQ were evaluated: estimations of food groups, energy, and nutrient intakes from the DRs and the FFQ, Spearman and Pearson correlations, cross-classification, and Bland-Altman analyses. The French MetaCardis FFQ tended to report higher food, energy, and nutrient intakes compared with the DRs. Mean correlation coefficient was 0.429 for food, 0.460 for energy, 0.544 for macronutrients, 0.640 for alcohol, and 0.503 for micronutrient intakes. Almost half of participants (44.4%) were correctly classified within tertiles of consumption, whereas 12.9% were misclassified in the opposite tertile. Performance of the FFQ was relatively similar after stratification by sex. The French MetaCardis FFQ was found to have an acceptable level of validity and may be a useful instrument to rank individuals based on their food and nutrient intakes. Copyright © 2017 Academy of Nutrition and Dietetics. Published by Elsevier Inc. All rights reserved.

Validation of a Formula for Assigning Continuing Education Credit to Printed Home Study Courses

PubMed Central

Hanson, Alan L.

2007-01-01

Objectives To reevaluate and validate the use of a formula for calculating the amount of continuing education credit to be awarded for printed home study courses. Methods Ten home study courses were selected for inclusion in a study to validate the formula, which is based on the number of words, number of final examination questions, and estimated difficulty level of the course. The amount of estimated credit calculated using the a priori formula was compared to the average amount of time required to complete each article based on pharmacists' self-reporting. Results A strong positive relationship between the amount of time required to complete the home study courses based on the a priori calculation and the times reported by pharmacists completing the 10 courses was found (p < 0.001). The correlation accounted for 86.2% of the total variability in the average pharmacist reported completion times (p < 0.001). Conclusions The formula offers an efficient and accurate means of determining the amount of continuing education credit that should be assigned to printed home study courses. PMID:19503705

PHEV Energy Use Estimation: Validating the Gamma Distribution for Representing the Random Daily Driving Distance

DOE Office of Scientific and Technical Information (OSTI.GOV)

Lin, Zhenhong; Dong, Jing; Liu, Changzheng

2012-01-01

The petroleum and electricity consumptions of plug-in hybrid electric vehicles (PHEVs) are sensitive to the variation of daily vehicle miles traveled (DVMT). Some studies assume DVMT to follow a Gamma distribution, but such a Gamma assumption is yet to be validated. This study finds the Gamma assumption valid in the context of PHEV energy analysis, based on continuous GPS travel data of 382 vehicles, each tracked for at least 183 days. The validity conclusion is based on the found small prediction errors, resulting from the Gamma assumption, in PHEV petroleum use, electricity use, and energy cost. The finding that themore » Gamma distribution is valid and reliable is important. It paves the way for the Gamma distribution to be assumed for analyzing energy uses of PHEVs in the real world. The Gamma distribution can be easily specified with very few pieces of driver information and is relatively easy for mathematical manipulation. Given the validation in this study, the Gamma distribution can now be used with better confidence in a variety of applications, such as improving vehicle consumer choice models, quantifying range anxiety for battery electric vehicles, investigating roles of charging infrastructure, and constructing online calculators that provide personal estimates of PHEV energy use.« less

A Model-Based Method for Content Validation of Automatically Generated Test Items

ERIC Educational Resources Information Center

Zhang, Xinxin; Gierl, Mark

2016-01-01

The purpose of this study is to describe a methodology to recover the item model used to generate multiple-choice test items with a novel graph theory approach. Beginning with the generated test items and working backward to recover the original item model provides a model-based method for validating the content used to automatically generate test…

Validation of a New Conceptual Model of School Connectedness and Its Assessment Measure

ERIC Educational Resources Information Center

Hirao, Katsura

2011-01-01

A self-report assessment scale of school connectedness was validated in this study based on the data from middle-school children in a northeastern state of the United States (n = 145). The scale was based on the School Bonding Model (Morita, 1991), which was derived reductively from the social control (bond) theory (Hirschi, 1969). This validation…

Drive: Theory and Construct Validation

PubMed Central

Petrides, K. V.

2016-01-01

This article explicates the theory of drive and describes the development and validation of two measures. A representative set of drive facets was derived from an extensive corpus of human attributes (Study 1). Operationalised using an International Personality Item Pool version (the Drive:IPIP), a three-factor model was extracted from the facets in two samples and confirmed on a third sample (Study 2). The multi-item IPIP measure showed congruence with a short form, based on single-item ratings of the facets, and both demonstrated cross-informant reliability. Evidence also supported the measures’ convergent, discriminant, concurrent, and incremental validity (Study 3). Based on very promising findings, the authors hope to initiate a stream of research in what is argued to be a rather neglected niche of individual differences and non-cognitive assessment. PMID:27409773

Developing evaluation instrument based on CIPP models on the implementation of portfolio assessment

NASA Astrophysics Data System (ADS)

Kurnia, Feni; Rosana, Dadan; Supahar

2017-08-01

This study aimed to develop an evaluation instrument constructed by CIPP model on the implementation of portfolio assessment in science learning. This study used research and development (R & D) method; adapting 4-D by the development of non-test instrument, and the evaluation instrument constructed by CIPP model. CIPP is the abbreviation of Context, Input, Process, and Product. The techniques of data collection were interviews, questionnaires, and observations. Data collection instruments were: 1) the interview guidelines for the analysis of the problems and the needs, 2) questionnaire to see level of accomplishment of portfolio assessment instrument, and 3) observation sheets for teacher and student to dig up responses to the portfolio assessment instrument. The data obtained was quantitative data obtained from several validators. The validators consist of two lecturers as the evaluation experts, two practitioners (science teachers), and three colleagues. This paper shows the results of content validity obtained from the validators and the analysis result of the data obtained by using Aikens' V formula. The results of this study shows that the evaluation instrument based on CIPP models is proper to evaluate the implementation of portfolio assessment instruments. Based on the experts' judgments, practitioners, and colleagues, the Aikens' V coefficient was between 0.86-1,00 which means that it is valid and can be used in the limited trial and operational field trial.

Validation of a DNA IQ-based extraction method for TECAN robotic liquid handling workstations for processing casework.

PubMed

Frégeau, Chantal J; Lett, C Marc; Fourney, Ron M

2010-10-01

A semi-automated DNA extraction process for casework samples based on the Promega DNA IQ™ system was optimized and validated on TECAN Genesis 150/8 and Freedom EVO robotic liquid handling stations configured with fixed tips and a TECAN TE-Shake™ unit. The use of an orbital shaker during the extraction process promoted efficiency with respect to DNA capture, magnetic bead/DNA complex washes and DNA elution. Validation studies determined the reliability and limitations of this shaker-based process. Reproducibility with regards to DNA yields for the tested robotic workstations proved to be excellent and not significantly different than that offered by the manual phenol/chloroform extraction. DNA extraction of animal:human blood mixtures contaminated with soil demonstrated that a human profile was detectable even in the presence of abundant animal blood. For exhibits containing small amounts of biological material, concordance studies confirmed that DNA yields for this shaker-based extraction process are equivalent or greater to those observed with phenol/chloroform extraction as well as our original validated automated magnetic bead percolation-based extraction process. Our data further supports the increasing use of robotics for the processing of casework samples. Crown Copyright © 2009. Published by Elsevier Ireland Ltd. All rights reserved.

Social identity and worldview validation: the effects of ingroup identity primes and mortality salience on value endorsement.

PubMed

Halloran, Michael J; Kashima, Emiko S

2004-07-01

In this article, the authors report an investigation of the relationship between terror management and social identity processes by testing for the effects of social identity salience on worldview validation. Two studies, with distinct populations, were conducted to test the hypothesis that mortality salience would lead to worldview validation of values related to a salient social identity. In Study 1, reasonable support for this hypothesis was found with bicultural Aboriginal Australian participants (N = 97). It was found that thoughts of death led participants to validate ingroup and reject outgroup values depending on the social identity that had been made salient. In Study 2, when their student and Australian identities were primed, respectively, Anglo-Australian students (N = 119) validated values related to those identities, exclusively. The implications of the findings for identity-based worldview validation are discussed.

Validation techniques for fault emulation of SRAM-based FPGAs

DOE PAGES

Quinn, Heather; Wirthlin, Michael

2015-08-07

A variety of fault emulation systems have been created to study the effect of single-event effects (SEEs) in static random access memory (SRAM) based field-programmable gate arrays (FPGAs). These systems are useful for augmenting radiation-hardness assurance (RHA) methodologies for verifying the effectiveness for mitigation techniques; understanding error signatures and failure modes in FPGAs; and failure rate estimation. For radiation effects researchers, it is important that these systems properly emulate how SEEs manifest in FPGAs. If the fault emulation systems does not mimic the radiation environment, the system will generate erroneous data and incorrect predictions of behavior of the FPGA inmore » a radiation environment. Validation determines whether the emulated faults are reasonable analogs to the radiation-induced faults. In this study we present methods for validating fault emulation systems and provide several examples of validated FPGA fault emulation systems.« less

Design of capability measurement instruments pedagogic content knowledge (PCK) for prospective mathematics teachers

NASA Astrophysics Data System (ADS)

Aminah, N.; Wahyuni, I.

2018-05-01

The purpose of this study is to find out how the process of designing a tool of measurement Pedagogical Content Knowledge (PCK) capabilities, especially for prospective mathematics teachers are valid and practical. The design study of this measurement appliance uses modified Plomp development step, which consists of (1) initial assessment stage, (2) design stage at this stage, the researcher designs the measuring grille of PCK capability, (3) realization stage that is making measurement tool ability of PCK, (4) test phase, evaluation, and revision that is testing validation of measurement tools conducted by experts. Based on the results obtained that the design of PCK capability measurement tool is valid as indicated by the assessment of expert validator, and the design of PCK capability measurement tool, shown based on the assessment of teachers and lecturers as users of states strongly agree the design of PCK measurement tools can be used.

Smartphone-Based Self-Assessment of Stress in Healthy Adult Individuals: A Systematic Review

PubMed Central

Þórarinsdóttir, Helga; Kessing, Lars Vedel

2017-01-01

Background Stress is a common experience in today’s society. Smartphone ownership is widespread, and smartphones can be used to monitor health and well-being. Smartphone-based self-assessment of stress can be done in naturalistic settings and may potentially reflect real-time stress level. Objective The objectives of this systematic review were to evaluate (1) the use of smartphones to measure self-assessed stress in healthy adult individuals, (2) the validity of smartphone-based self-assessed stress compared with validated stress scales, and (3) the association between smartphone-based self-assessed stress and smartphone generated objective data. Methods A systematic review of the scientific literature was reported and conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement. The scientific databases PubMed, PsycINFO, Embase, IEEE, and ACM were searched and supplemented by a hand search of reference lists. The databases were searched for original studies involving healthy individuals older than 18 years, measuring self-assessed stress using smartphones. Results A total of 35 published articles comprising 1464 individuals were included for review. According to the objectives, (1) study designs were heterogeneous, and smartphone-based self-assessed stress was measured using various methods (e.g., dichotomized questions on stress, yes or no; Likert scales on stress; and questionnaires); (2) the validity of smartphone-based self-assessed stress compared with validated stress scales was investigated in 3 studies, and of these, only 1 study found a moderate statistically significant positive correlation (r=.4; P<.05); and (3) in exploratory analyses, smartphone-based self-assessed stress was found to correlate with some of the reported smartphone generated objective data, including voice features and data on activity and phone usage. Conclusions Smartphones are being used to measure self-assessed stress in different contexts. The evidence of the validity of smartphone-based self-assessed stress is limited and should be investigated further. Smartphone generated objective data can potentially be used to monitor, predict, and reduce stress levels. PMID:28193600

Smartphone-Based Self-Assessment of Stress in Healthy Adult Individuals: A Systematic Review.

PubMed

Þórarinsdóttir, Helga; Kessing, Lars Vedel; Faurholt-Jepsen, Maria

2017-02-13

Stress is a common experience in today's society. Smartphone ownership is widespread, and smartphones can be used to monitor health and well-being. Smartphone-based self-assessment of stress can be done in naturalistic settings and may potentially reflect real-time stress level. The objectives of this systematic review were to evaluate (1) the use of smartphones to measure self-assessed stress in healthy adult individuals, (2) the validity of smartphone-based self-assessed stress compared with validated stress scales, and (3) the association between smartphone-based self-assessed stress and smartphone generated objective data. A systematic review of the scientific literature was reported and conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement. The scientific databases PubMed, PsycINFO, Embase, IEEE, and ACM were searched and supplemented by a hand search of reference lists. The databases were searched for original studies involving healthy individuals older than 18 years, measuring self-assessed stress using smartphones. A total of 35 published articles comprising 1464 individuals were included for review. According to the objectives, (1) study designs were heterogeneous, and smartphone-based self-assessed stress was measured using various methods (e.g., dichotomized questions on stress, yes or no; Likert scales on stress; and questionnaires); (2) the validity of smartphone-based self-assessed stress compared with validated stress scales was investigated in 3 studies, and of these, only 1 study found a moderate statistically significant positive correlation (r=.4; P<.05); and (3) in exploratory analyses, smartphone-based self-assessed stress was found to correlate with some of the reported smartphone generated objective data, including voice features and data on activity and phone usage. Smartphones are being used to measure self-assessed stress in different contexts. The evidence of the validity of smartphone-based self-assessed stress is limited and should be investigated further. Smartphone generated objective data can potentially be used to monitor, predict, and reduce stress levels. ©Helga Þórarinsdóttir, Lars Vedel Kessing, Maria Faurholt-Jepsen. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 13.02.2017.

Reliability and Validity of Bedside Version of Persian WAB (P-WAB-1).

PubMed

Nilipour, Reza; Pourshahbaz, Abbas; Ghoreyshi, Zahra Sadat

2014-10-01

In this study, we reported the reliability and validity of Bedside version of Persian WAB (P-WAB-1) adapted from Western Aphasia Battery (WAB-R) (1,2). P-WAB-1 is a clinical linguistic measuring tool to determine severity and type of aphasia in brain damaged patients based on Aphasia Quotient (AQ) as a functional measure. For the purposes of a quick clinical screening of aphasia in Persian, we adapted the bedside version of WAB-R to assess the performance of Persian aphasic patients. The data we reported on adaptation, validity and reliability of P-WAB-1 are based on faithful translation and criterion validity ratio (CVR) taken from the expert panel and the performance of 60 consecutive brain damaged patients referred to different university clinics for rehabilitation and 30 healthy subjects as norms and 40 age-matched epileptic patients as the control group. Based on the results of this study, P-WAB-1 has internal consistency (a=0.71) and test-retest reliability (r=.65 P<0.001) and the subtests are sensitive enough to contribute to Aphasia Quotient (AQ) as a functional measure of severity of aphasia in Iranian brain damaged patients. Based on AQ results, our aphasic patients were classified into four distinct groups of severity. P-WAB-1 is the first clinical linguistic test to determine severity of aphasia based on an operational index and can be considered as a valid baseline for screening and diagnosis of aphasia among Persian speaking brain damaged patients. This study is the initial step on adaptation of different versions of WAB-R to measure the severity of aphasia using AQ, LQ and CQ as operational measures and to classify Persian speaking aphasic patients into different types.

Validation approach for a fast and simple targeted screening method for 75 antibiotics in meat and aquaculture products using LC-MS/MS.

PubMed

Dubreil, Estelle; Gautier, Sophie; Fourmond, Marie-Pierre; Bessiral, Mélaine; Gaugain, Murielle; Verdon, Eric; Pessel, Dominique

2017-04-01

An approach is described to validate a fast and simple targeted screening method for antibiotic analysis in meat and aquaculture products by LC-MS/MS. The strategy of validation was applied for a panel of 75 antibiotics belonging to different families, i.e., penicillins, cephalosporins, sulfonamides, macrolides, quinolones and phenicols. The samples were extracted once with acetonitrile, concentrated by evaporation and injected into the LC-MS/MS system. The approach chosen for the validation was based on the Community Reference Laboratory (CRL) guidelines for the validation of screening qualitative methods. The aim of the validation was to prove sufficient sensitivity of the method to detect all the targeted antibiotics at the level of interest, generally the maximum residue limit (MRL). A robustness study was also performed to test the influence of different factors. The validation showed that the method is valid to detect and identify 73 antibiotics of the 75 antibiotics studied in meat and aquaculture products at the validation levels.

Improving the governance of patient safety in emergency care: a systematic review of interventions

PubMed Central

Hesselink, Gijs; Berben, Sivera; Beune, Thimpe

2016-01-01

Objectives To systematically review interventions that aim to improve the governance of patient safety within emergency care on effectiveness, reliability, validity and feasibility. Design A systematic review of the literature. Methods PubMed, EMBASE, Cumulative Index to Nursing and Allied Health Literature, the Cochrane Database of Systematic Reviews and PsychInfo were searched for studies published between January 1990 and July 2014. We included studies evaluating interventions relevant for higher management to oversee and manage patient safety, in prehospital emergency medical service (EMS) organisations and hospital-based emergency departments (EDs). Two reviewers independently selected candidate studies, extracted data and assessed study quality. Studies were categorised according to study quality, setting, sample, intervention characteristics and findings. Results Of the 18 included studies, 13 (72%) were non-experimental. Nine studies (50%) reported data on the reliability and/or validity of the intervention. Eight studies (44%) reported on the feasibility of the intervention. Only 4 studies (22%) reported statistically significant effects. The use of a simulation-based training programme and well-designed incident reporting systems led to a statistically significant improvement of safety knowledge and attitudes by ED staff and an increase of incident reports within EDs, respectively. Conclusions Characteristics of the interventions included in this review (eg, anonymous incident reporting and validation of incident reports by an independent party) could provide useful input for the design of an effective tool to govern patient safety in EMS organisations and EDs. However, executives cannot rely on a robust set of evidence-based and feasible tools to govern patient safety within their emergency care organisation and in the chain of emergency care. Established strategies from other high-risk sectors need to be evaluated in emergency care settings, using an experimental design with valid outcome measures to strengthen the evidence base. PMID:26826151

The Safety Culture Enactment Questionnaire (SCEQ): Theoretical model and empirical validation.

PubMed

de Castro, Borja López; Gracia, Francisco J; Tomás, Inés; Peiró, José M

2017-06-01

This paper presents the Safety Culture Enactment Questionnaire (SCEQ), designed to assess the degree to which safety is an enacted value in the day-to-day running of nuclear power plants (NPPs). The SCEQ is based on a theoretical safety culture model that is manifested in three fundamental components of the functioning and operation of any organization: strategic decisions, human resources practices, and daily activities and behaviors. The extent to which the importance of safety is enacted in each of these three components provides information about the pervasiveness of the safety culture in the NPP. To validate the SCEQ and the model on which it is based, two separate studies were carried out with data collection in 2008 and 2014, respectively. In Study 1, the SCEQ was administered to the employees of two Spanish NPPs (N=533) belonging to the same company. Participants in Study 2 included 598 employees from the same NPPs, who completed the SCEQ and other questionnaires measuring different safety outcomes (safety climate, safety satisfaction, job satisfaction and risky behaviors). Study 1 comprised item formulation and examination of the factorial structure and reliability of the SCEQ. Study 2 tested internal consistency and provided evidence of factorial validity, validity based on relationships with other variables, and discriminant validity between the SCEQ and safety climate. Exploratory Factor Analysis (EFA) carried out in Study 1 revealed a three-factor solution corresponding to the three components of the theoretical model. Reliability analyses showed strong internal consistency for the three scales of the SCEQ, and each of the 21 items on the questionnaire contributed to the homogeneity of its theoretically developed scale. Confirmatory Factor Analysis (CFA) carried out in Study 2 supported the internal structure of the SCEQ; internal consistency of the scales was also supported. Furthermore, the three scales of the SCEQ showed the expected correlation patterns with the measured safety outcomes. Finally, results provided evidence of discriminant validity between the SCEQ and safety climate. We conclude that the SCEQ is a valid, reliable instrument supported by a theoretical framework, and it is useful to measure the enactment of safety culture in NPPs. Copyright © 2017 Elsevier Ltd. All rights reserved.

Reliability and validity of the Pragmatics Observational Measure (POM): a new observational measure of pragmatic language for children.

PubMed

Cordier, Reinie; Munro, Natalie; Wilkes-Gillan, Sarah; Speyer, Renée; Pearce, Wendy M

2014-07-01

There is a need for a reliable and valid assessment of childhood pragmatic language skills during peer-peer interactions. This study aimed to evaluate the psychometric properties of a newly developed pragmatic assessment, the Pragmatic Observational Measure (POM). The psychometric properties of the POM were investigated from observational data of two studies - study 1 involved 342 children aged 5-11 years (108 children with ADHD; 108 typically developing playmates; 126 children in the control group), and study 2 involved 9 children with ADHD who attended a 7-week play-based intervention. The psychometric properties of the POM were determined based on the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) taxonomy of psychometric properties and definitions for health-related outcomes; the Pragmatic Protocol was used as the reference tool against which the POM was evaluated. The POM demonstrated sound psychometric properties in all the reliability, validity and interpretability criteria against which it was assessed. The findings showed that the POM is a reliable and valid measure of pragmatic language skills of children with ADHD between the age of 5 and 11 years and has clinical utility in identifying children with pragmatic language difficulty. Copyright © 2014 The Authors. Published by Elsevier Ltd.. All rights reserved.

Reliability and validity of advanced theory-of-mind measures in middle childhood and adolescence.

PubMed

Hayward, Elizabeth O; Homer, Bruce D

2017-09-01

Although theory-of-mind (ToM) development is well documented for early childhood, there is increasing research investigating changes in ToM reasoning in middle childhood and adolescence. However, the psychometric properties of most advanced ToM measures for use with older children and adolescents have not been firmly established. We report on the reliability and validity of widely used, conventional measures of advanced ToM with this age group. Notable issues with both reliability and validity of several of the measures were evident in the findings. With regard to construct validity, results do not reveal a clear empirical commonality between tasks, and, after accounting for comprehension, developmental trends were evident in only one of the tasks investigated. Statement of contribution What is already known on this subject? Second-order false belief tasks have acceptable internal consistency. The Eyes Test has poor internal consistency. Validity of advanced theory-of-mind tasks is often based on the ability to distinguish clinical from typical groups. What does this study add? This study examines internal consistency across six widely used advanced theory-of-mind tasks. It investigates validity of tasks based on comprehension of items by typically developing individuals. It further assesses construct validity, or commonality between tasks. © 2017 The British Psychological Society.

Validation of persuasive messages for the promotion of physical activity among people with coronary heart disease.

PubMed

Mendez, Roberto Della Rosa; Rodrigues, Roberta Cunha Matheus; Spana, Thaís Moreira; Cornélio, Marília Estevam; Gallani, Maria Cecília Bueno Jayme; Pérez-Nebra, Amalia Raquel

2012-01-01

to validate the content of persuasive messages for promoting walking among patients with coronary heart disease (CHD). The messages were constructed to strengthen or change patients' attitudes to walking. the selection of persuasive arguments was based on behavioral beliefs (determinants of attitude) related to walking. The messages were constructed based in the Elaboration Likelihood Model and were submitted to content validation. the data was analyzed with the content validity index and by the importance which the patients attributed to the messages' persuasive arguments. Positive behavioral beliefs (i.e. positive and negative reinforcement) and self-efficacy were the appeals which the patients considered important. The messages with validation evidence will be tested in an intervention study for the promotion of the practice of physical activity among patients with CHD.

Developing evaluation scales for horticultural therapy.

PubMed

Im, Eun-Ae; Park, Sin-Ae; Son, Ki-Cheol

2018-04-01

This study developed evaluation scales for measuring the effects of horticultural therapy in practical settings. Qualitative and quantitative research, including three preliminary studies and a main study, were conducted. In the first study, a total of 779 horticultural therapists answered an open-end questionnaire based on 58 items about elements of occupational therapy and seven factors about singularity of horticultural therapy. In the second study, 20 horticultural therapists participated in in-depth interviews. In the third study, a Delphi method was conducted with 24 horticultural therapists to build a model of assessment indexes and ensure the validity. In the final study, the reserve scales were tested by 121 horticultural therapists in their practical settings for 1045 clients, to verify their reliability and validity. Preliminary questions in the effects area of horticultural therapy were developed in the first study, and validity for the components in the second study. In the third study, an expert Delphi survey was conducted as part of content validity verification of the preliminary tool of horticultural therapy for physical, cognitive, psychological-emotional, and social areas. In the final study, the evaluation tool, which verified the construct, convergence, discriminant, and predictive validity and reliability test, was used to finalise the evaluation tool. The effects of horticultural therapy were classified as four different aspects, namely, physical, cognitive, psycho-emotional, and social, based on previous studies on the effects of horticultural therapy. 98 questions in the four aspects were selected as reserve scales. The reliability of each scale was calculated as 0.982 in physical, 0.980 in cognitive, 0.965 in psycho-emotional, and 0.972 in social aspects based on the Cronbach's test of intra-item internal consistency and half reliability of Spearman-Brown. This study was the first to demonstrate validity and reliability by simultaneously developing four measures of horticultural therapy effectiveness, namely, physical, cognitive, psychological-emotional, and social, both locally and externally. It is especially worthwhile in that it can be applied in common to people. Copyright © 2018 Elsevier Ltd. All rights reserved.

Design and Development Computer-Based E-Learning Teaching Material for Improving Mathematical Understanding Ability and Spatial Sense of Junior High School Students

NASA Astrophysics Data System (ADS)

Nurjanah; Dahlan, J. A.; Wibisono, Y.

2017-02-01

This paper aims to make a design and development computer-based e-learning teaching material for improving mathematical understanding ability and spatial sense of junior high school students. Furthermore, the particular aims are (1) getting teaching material design, evaluation model, and intrument to measure mathematical understanding ability and spatial sense of junior high school students; (2) conducting trials computer-based e-learning teaching material model, asessment, and instrument to develop mathematical understanding ability and spatial sense of junior high school students; (3) completing teaching material models of computer-based e-learning, assessment, and develop mathematical understanding ability and spatial sense of junior high school students; (4) resulting research product is teaching materials of computer-based e-learning. Furthermore, the product is an interactive learning disc. The research method is used of this study is developmental research which is conducted by thought experiment and instruction experiment. The result showed that teaching materials could be used very well. This is based on the validation of computer-based e-learning teaching materials, which is validated by 5 multimedia experts. The judgement result of face and content validity of 5 validator shows that the same judgement result to the face and content validity of each item test of mathematical understanding ability and spatial sense. The reliability test of mathematical understanding ability and spatial sense are 0,929 and 0,939. This reliability test is very high. While the validity of both tests have a high and very high criteria.

An Argument-Based Approach to the Validation of UHTRUST: Can We Measure How Recent Graduates Can Be Trusted with Unfamiliar Tasks?

ERIC Educational Resources Information Center

Wijnen-Meijer, M.; Van der Schaaf, M.; Booij, E.; Harendza, S.; Boscardin, C.; Wijngaarden, J. Van; Ten Cate, Th. J.

2013-01-01

There is a need for valid methods to assess the readiness for clinical practice of medical graduates. This study evaluates the validity of Utrecht Hamburg Trainee Responsibility for Unfamiliar Situations Test (UHTRUST), an authentic simulation procedure to assess whether medical trainees are ready to be entrusted with unfamiliar clinical tasks…

The Use of Authentic Assessment to Report Accountability Data on Young Children's Language, Literacy and Pre-Math Competency

ERIC Educational Resources Information Center

Gao, Xin; Grisham-Brown, Jennifer

2011-01-01

This validity study examined the validity of Assessment, Evaluation, and Programming System, 2nd Edition (AEPS®), a curriculum-based, authentic assessment for infants and young children. The primary purposes were to: a) examine whether the AEPS® is a concurrently valid tool for measuring young children's language, literacy and pre-math skills for…

Development and Validation of the Meaning of Work Inventory among French Workers

ERIC Educational Resources Information Center

Arnoux-Nicolas, Caroline; Sovet, Laurent; Lhotellier, Lin; Bernaud, Jean-Luc

2017-01-01

The purpose of this study was to validate a psychometric instrument among French workers for assessing the meaning of work. Following an empirical framework, a two-step procedure consisted of exploring and then validating the scale among distinctive samples. The consequent Meaning of Work Inventory is a 15-item scale based on a four-factor model,…

Grid workflow validation using ontology-based tacit knowledge: A case study for quantitative remote sensing applications

NASA Astrophysics Data System (ADS)

Liu, Jia; Liu, Longli; Xue, Yong; Dong, Jing; Hu, Yingcui; Hill, Richard; Guang, Jie; Li, Chi

2017-01-01

Workflow for remote sensing quantitative retrieval is the ;bridge; between Grid services and Grid-enabled application of remote sensing quantitative retrieval. Workflow averts low-level implementation details of the Grid and hence enables users to focus on higher levels of application. The workflow for remote sensing quantitative retrieval plays an important role in remote sensing Grid and Cloud computing services, which can support the modelling, construction and implementation of large-scale complicated applications of remote sensing science. The validation of workflow is important in order to support the large-scale sophisticated scientific computation processes with enhanced performance and to minimize potential waste of time and resources. To research the semantic correctness of user-defined workflows, in this paper, we propose a workflow validation method based on tacit knowledge research in the remote sensing domain. We first discuss the remote sensing model and metadata. Through detailed analysis, we then discuss the method of extracting the domain tacit knowledge and expressing the knowledge with ontology. Additionally, we construct the domain ontology with Protégé. Through our experimental study, we verify the validity of this method in two ways, namely data source consistency error validation and parameters matching error validation.

The Measurement and Dimensionality of Mobile Learning Systems Success: Two-Stage Development and Validation

ERIC Educational Resources Information Center

Lin, Hsin-Hui; Wang, Yi-Shun; Li, Ci-Rong; Shih, Ying-Wei; Lin, Shin-Jeng

2017-01-01

The main purpose of this study is to develop and validate a multidimensional instrument for measuring mobile learning systems success (MLSS) based on the previous research. This study defines the construct of MLSS, develops a generic MLSS instrument with desirable psychometric properties, and explores the instrument's theoretical and practical…

Validity of Adult Retrospective Reports of Adverse Childhood Experiences: Review of the Evidence

ERIC Educational Resources Information Center

Hardt, Jochen; Rutter, Michael

2004-01-01

Background: Influential studies have cast doubt on the validity of retrospective reports by adults of their own adverse experiences in childhood. Accordingly, many researchers view retrospective reports with scepticism. Method: A computer-based search, supplemented by hand searches, was used to identify studies reported between 1980 and 2001 in…

Incremental Validity of Thinking Styles in Predicting Academic Achievements: An Experimental Study in Hypermedia Learning Environments

ERIC Educational Resources Information Center

Fan, Weiqiao; Zhang, Li-Fang; Watkins, David

2010-01-01

The study examined the incremental validity of thinking styles in predicting academic achievement after controlling for personality and achievement motivation in the hypermedia-based learning environment. Seventy-two Chinese college students from Shanghai, the People's Republic of China, took part in this instructional experiment. The…

Examining construct and predictive validity of the Health-IT Usability Evaluation Scale: confirmatory factor analysis and structural equation modeling results

PubMed Central

Yen, Po-Yin; Sousa, Karen H; Bakken, Suzanne

2014-01-01

Background In a previous study, we developed the Health Information Technology Usability Evaluation Scale (Health-ITUES), which is designed to support customization at the item level. Such customization matches the specific tasks/expectations of a health IT system while retaining comparability at the construct level, and provides evidence of its factorial validity and internal consistency reliability through exploratory factor analysis. Objective In this study, we advanced the development of Health-ITUES to examine its construct validity and predictive validity. Methods The health IT system studied was a web-based communication system that supported nurse staffing and scheduling. Using Health-ITUES, we conducted a cross-sectional study to evaluate users’ perception toward the web-based communication system after system implementation. We examined Health-ITUES's construct validity through first and second order confirmatory factor analysis (CFA), and its predictive validity via structural equation modeling (SEM). Results The sample comprised 541 staff nurses in two healthcare organizations. The CFA (n=165) showed that a general usability factor accounted for 78.1%, 93.4%, 51.0%, and 39.9% of the explained variance in ‘Quality of Work Life’, ‘Perceived Usefulness’, ‘Perceived Ease of Use’, and ‘User Control’, respectively. The SEM (n=541) supported the predictive validity of Health-ITUES, explaining 64% of the variance in intention for system use. Conclusions The results of CFA and SEM provide additional evidence for the construct and predictive validity of Health-ITUES. The customizability of Health-ITUES has the potential to support comparisons at the construct level, while allowing variation at the item level. We also illustrate application of Health-ITUES across stages of system development. PMID:24567081

Validating Remotely Sensed Land Surface Evapotranspiration Based on Multi-scale Field Measurements

NASA Astrophysics Data System (ADS)

Jia, Z.; Liu, S.; Ziwei, X.; Liang, S.

2012-12-01

The land surface evapotranspiration plays an important role in the surface energy balance and the water cycle. There have been significant technical and theoretical advances in our knowledge of evapotranspiration over the past two decades. Acquisition of the temporally and spatially continuous distribution of evapotranspiration using remote sensing technology has attracted the widespread attention of researchers and managers. However, remote sensing technology still has many uncertainties coming from model mechanism, model inputs, parameterization schemes, and scaling issue in the regional estimation. Achieving remotely sensed evapotranspiration (RS_ET) with confident certainty is required but difficult. As a result, it is indispensable to develop the validation methods to quantitatively assess the accuracy and error sources of the regional RS_ET estimations. This study proposes an innovative validation method based on multi-scale evapotranspiration acquired from field measurements, with the validation results including the accuracy assessment, error source analysis, and uncertainty analysis of the validation process. It is a potentially useful approach to evaluate the accuracy and analyze the spatio-temporal properties of RS_ET at both the basin and local scales, and is appropriate to validate RS_ET in diverse resolutions at different time-scales. An independent RS_ET validation using this method was presented over the Hai River Basin, China in 2002-2009 as a case study. Validation at the basin scale showed good agreements between the 1 km annual RS_ET and the validation data such as the water balanced evapotranspiration, MODIS evapotranspiration products, precipitation, and landuse types. Validation at the local scale also had good results for monthly, daily RS_ET at 30 m and 1 km resolutions, comparing to the multi-scale evapotranspiration measurements from the EC and LAS, respectively, with the footprint model over three typical landscapes. Although some validation experiments demonstrated that the models yield accurate estimates at flux measurement sites, the question remains whether they are performing well over the broader landscape. Moreover, a large number of RS_ET products have been released in recent years. Thus, we also pay attention to the cross-validation method of RS_ET derived from multi-source models. "The Multi-scale Observation Experiment on Evapotranspiration over Heterogeneous Land Surfaces: Flux Observation Matrix" campaign is carried out at the middle reaches of the Heihe River Basin, China in 2012. Flux measurements from an observation matrix composed of 22 EC and 4 LAS are acquired to investigate the cross-validation of multi-source models over different landscapes. In this case, six remote sensing models, including the empirical statistical model, the one-source and two-source models, the Penman-Monteith equation based model, the Priestley-Taylor equation based model, and the complementary relationship based model, are used to perform an intercomparison. All the results from the two cases of RS_ET validation showed that the proposed validation methods are reasonable and feasible.

Test-retest reliability and validity of a web-based food-frequency questionnaire for adolescents aged 13-14 to be used in the Norwegian Mother and Child Cohort Study (MoBa).

PubMed

Overby, Nina Cecilie; Johannesen, Elisabeth; Jensen, Grete; Skjaevesland, Anne-Kirsti; Haugen, Margaretha

2014-01-01

The assessment of food intake is challenging and prone to errors; it is therefore important to consider the reliability and validity of the assessment methods. The aim of this study was to analyze the reproducibility and validity of a developed food-frequency questionnaire (FFQ) for use among adolescents. In total, 58 students (aged 13-14) from four different schools in the southern part of Norway participated in the reproducibility study of filling out the FFQ 4 weeks apart. In addition, 93 students participated in the relative validity study where the FFQ was compared to 2×24-hour dietary recalls, while 92 students participated in the absolute validity study where the intakes of fatty acids and vitamin D from the FFQ were compared to fatty acids and 25-hydroxy-vitamin D3 in whole blood. The median Spearman correlation coefficient for all nutrients in the test-retest reliability study was 0.57. The median Spearman correlation for all nutrients in the relative validity study was 0.26, while the correlations coefficients were low in the absolute validity study with n-3 fatty acid coefficients ranging from 0.05 to 0.25, and absent for vitamin D (r=0.000). The test-retest reproducibility was considered good, the relative validity was considered poor to good, and the absolute validity was considered poor. However, the results are comparable to other studies among adolescents.

Family-Based Benchmarking of Copy Number Variation Detection Software.

PubMed

Nutsua, Marcel Elie; Fischer, Annegret; Nebel, Almut; Hofmann, Sylvia; Schreiber, Stefan; Krawczak, Michael; Nothnagel, Michael

2015-01-01

The analysis of structural variants, in particular of copy-number variations (CNVs), has proven valuable in unraveling the genetic basis of human diseases. Hence, a large number of algorithms have been developed for the detection of CNVs in SNP array signal intensity data. Using the European and African HapMap trio data, we undertook a comparative evaluation of six commonly used CNV detection software tools, namely Affymetrix Power Tools (APT), QuantiSNP, PennCNV, GLAD, R-gada and VEGA, and assessed their level of pair-wise prediction concordance. The tool-specific CNV prediction accuracy was assessed in silico by way of intra-familial validation. Software tools differed greatly in terms of the number and length of the CNVs predicted as well as the number of markers included in a CNV. All software tools predicted substantially more deletions than duplications. Intra-familial validation revealed consistently low levels of prediction accuracy as measured by the proportion of validated CNVs (34-60%). Moreover, up to 20% of apparent family-based validations were found to be due to chance alone. Software using Hidden Markov models (HMM) showed a trend to predict fewer CNVs than segmentation-based algorithms albeit with greater validity. PennCNV yielded the highest prediction accuracy (60.9%). Finally, the pairwise concordance of CNV prediction was found to vary widely with the software tools involved. We recommend HMM-based software, in particular PennCNV, rather than segmentation-based algorithms when validity is the primary concern of CNV detection. QuantiSNP may be used as an additional tool to detect sets of CNVs not detectable by the other tools. Our study also reemphasizes the need for laboratory-based validation, such as qPCR, of CNVs predicted in silico.

A simplified approach to the pooled analysis of calibration of clinical prediction rules for systematic reviews of validation studies

PubMed Central

Dimitrov, Borislav D; Motterlini, Nicola; Fahey, Tom

2015-01-01

Objective Estimating calibration performance of clinical prediction rules (CPRs) in systematic reviews of validation studies is not possible when predicted values are neither published nor accessible or sufficient or no individual participant or patient data are available. Our aims were to describe a simplified approach for outcomes prediction and calibration assessment and evaluate its functionality and validity. Study design and methods: Methodological study of systematic reviews of validation studies of CPRs: a) ABCD2 rule for prediction of 7 day stroke; and b) CRB-65 rule for prediction of 30 day mortality. Predicted outcomes in a sample validation study were computed by CPR distribution patterns (“derivation model”). As confirmation, a logistic regression model (with derivation study coefficients) was applied to CPR-based dummy variables in the validation study. Meta-analysis of validation studies provided pooled estimates of “predicted:observed” risk ratios (RRs), 95% confidence intervals (CIs), and indexes of heterogeneity (I2) on forest plots (fixed and random effects models), with and without adjustment of intercepts. The above approach was also applied to the CRB-65 rule. Results Our simplified method, applied to ABCD2 rule in three risk strata (low, 0–3; intermediate, 4–5; high, 6–7 points), indicated that predictions are identical to those computed by univariate, CPR-based logistic regression model. Discrimination was good (c-statistics =0.61–0.82), however, calibration in some studies was low. In such cases with miscalibration, the under-prediction (RRs =0.73–0.91, 95% CIs 0.41–1.48) could be further corrected by intercept adjustment to account for incidence differences. An improvement of both heterogeneities and P-values (Hosmer-Lemeshow goodness-of-fit test) was observed. Better calibration and improved pooled RRs (0.90–1.06), with narrower 95% CIs (0.57–1.41) were achieved. Conclusion Our results have an immediate clinical implication in situations when predicted outcomes in CPR validation studies are lacking or deficient by describing how such predictions can be obtained by everyone using the derivation study alone, without any need for highly specialized knowledge or sophisticated statistics. PMID:25931829

Psychometric properties of the Bulgarian translation of noise sensitivity scale short form (NSS-SF): implementation in the field of noise control.

PubMed

Dzhambov, Angel M; Dimitrova, Donka D

2014-01-01

The Noise Sensitivity Scale Short Form (NSS-SF), developed in English as a more practical form of the classical Weinstein NSS, has not to date been validated in other cultures, and its validity and reliability have not yet been confirmed. This study aimed to validate NSS-SF in Bulgarian and to demonstrate its applicability. The study comprised test-retest (n = 115) and a field-testing (n = 71) of the newly validated scale. Its construct validity was examined with confirmatory factor analysis, and very good model-fit was observed. Temporal stability was assessed in a test-retest (r = 0.990), convergent validity was examined with single-item susceptibility to the noise scale (r = 0.906) and discriminant validity was confirmed with single-item noise annoyance scale (r = 0.718). The lowest observed McDonald's omega across the studies was 0.923. The cross-cultural validation of NSS-SF was successful but it proved to be somewhat problematic with respect to its annoyance-based items.

Cognitive Decline in Down Syndrome: A Validity/Reliability Study of the Test for Severe Impairment.

ERIC Educational Resources Information Center

Cosgrave, Mary P.; McCarron, Mary; Anderson, Mary; Tyrrell, Janette; Gill, Michael; Lawlor, Brian A.

1998-01-01

The utility of the Test for Severe Impairment was studied with 60 older persons who had Down Syndrome. Construct validity, test-retest reliability, and interrater reliability were established for the full study group and for subgroups based on degree of mental retardation and dementia status. Some possible applications and limitations of the test…

Test Takers' Beliefs and Experiences of a High-Stakes Computer-Based English Listening and Speaking Test

ERIC Educational Resources Information Center

Zhan, Ying; Wan, Zhi Hong

2016-01-01

Test takers' beliefs or experiences have been overlooked in most validation studies in language education. Meanwhile, a mutual exclusion has been observed in the literature, with little or no dialogue between validation studies and studies concerning the uses and consequences of testing. To help fill these research gaps, a group of Senior III…

Measuring metacognitive ability based on science literacy in dynamic electricity topic

NASA Astrophysics Data System (ADS)

Warni; Sunyono; Rosidin

2018-01-01

This study aims to produce an instrument of metacognition ability assessment based on science literacy on theoretically and empirically feasible dynamic electrical material. The feasibility of the assessment instrument includes theoretical validity on material, construction, and language aspects, as well as empirical validity, reliability, difficulty, distinguishing, and distractor indices. The development of assessment instruments refers to the Dick and Carey development model which includes the preliminary study stage, initial product development, validation and revision, and piloting. The instrument was tested to 32 students of class IX in SMP Negeri 20 Bandar Lampung, using the design of One Group Pretest-Postest Design. The result shows that the metacognition ability assessment instrument based on science literacy is feasible theoretically with theoretical validity percentage of 95.44% and empirical validity of 43.75% for the high category, 43.75% for the medium category, and 12.50 % for low category questions; Reliability of assessment instruments of 0.83 high categories; Difficulty level of difficult item is about 31.25% and medium category is equal to 68.75%. Item that has very good distinguishing power is 12.50%, 62.50% for good stage, and medium category is 25.00%; As well as the duplexing function on a matter of multiple choice is 80.00% including good category and 20.00% for medium category.

Critical-Inquiry-Based-Learning: Model of Learning to Promote Critical Thinking Ability of Pre-service Teachers

NASA Astrophysics Data System (ADS)

Prayogi, S.; Yuanita, L.; Wasis

2018-01-01

This study aimed to develop Critical-Inquiry-Based-Learning (CIBL) learning model to promote critical thinking (CT) ability of preservice teachers. The CIBL learning model was developed by meeting the criteria of validity, practicality, and effectiveness. Validation of the model involves 4 expert validators through the mechanism of the focus group discussion (FGD). CIBL learning model declared valid to promote CT ability, with the validity level (Va) of 4.20 and reliability (r) of 90,1% (very reliable). The practicality of the model was evaluated when it was implemented that involving 17 of preservice teachers. The CIBL learning model had been declared practice, its measuring from learning feasibility (LF) with very good criteria (LF-score = 4.75). The effectiveness of the model was evaluated from the improvement CT ability after the implementation of the model. CT ability were evaluated using the scoring technique adapted from Ennis-Weir Critical Thinking Essay Test. The average score of CT ability on pretest is - 1.53 (uncritical criteria), whereas on posttest is 8.76 (critical criteria), with N-gain score of 0.76 (high criteria). Based on the results of this study, it can be concluded that developed CIBL learning model is feasible to promote CT ability of preservice teachers.

A Study on the Data Compression Technology-Based Intelligent Data Acquisition (IDAQ) System for Structural Health Monitoring of Civil Structures

PubMed Central

Jeon, Joonryong

2017-01-01

In this paper, a data compression technology-based intelligent data acquisition (IDAQ) system was developed for structural health monitoring of civil structures, and its validity was tested using random signals (El-Centro seismic waveform). The IDAQ system was structured to include a high-performance CPU with large dynamic memory for multi-input and output in a radio frequency (RF) manner. In addition, the embedded software technology (EST) has been applied to it to implement diverse logics needed in the process of acquiring, processing and transmitting data. In order to utilize IDAQ system for the structural health monitoring of civil structures, this study developed an artificial filter bank by which structural dynamic responses (acceleration) were efficiently acquired, and also optimized it on the random El-Centro seismic waveform. All techniques developed in this study have been embedded to our system. The data compression technology-based IDAQ system was proven valid in acquiring valid signals in a compressed size. PMID:28704945

A Study on the Data Compression Technology-Based Intelligent Data Acquisition (IDAQ) System for Structural Health Monitoring of Civil Structures.

PubMed

Heo, Gwanghee; Jeon, Joonryong

2017-07-12

In this paper, a data compression technology-based intelligent data acquisition (IDAQ) system was developed for structural health monitoring of civil structures, and its validity was tested using random signals (El-Centro seismic waveform). The IDAQ system was structured to include a high-performance CPU with large dynamic memory for multi-input and output in a radio frequency (RF) manner. In addition, the embedded software technology (EST) has been applied to it to implement diverse logics needed in the process of acquiring, processing and transmitting data. In order to utilize IDAQ system for the structural health monitoring of civil structures, this study developed an artificial filter bank by which structural dynamic responses (acceleration) were efficiently acquired, and also optimized it on the random El-Centro seismic waveform. All techniques developed in this study have been embedded to our system. The data compression technology-based IDAQ system was proven valid in acquiring valid signals in a compressed size.

Characteristics of Residual Symptoms in Korean Patients with Major Depressive Disorder: A Validation Study for the Korean Version of Depression Residual Symptom Scale.

PubMed

Park, Sol A; Jeon, Sang Won; Yoon, Ho-Kyoung; Yoon, Seo Young; Shin, Cheolmin; Ko, Young-Hoon

2018-02-01

Residual symptoms of depression are related to more severe and chronic course of functional impairment with higher risk of relapse. The objective of this study was to validate, and determine psychometric properties of the Korean version of Depression Residual Symptom Scale (KDRSS). A total of 203 outpatients with recent episode of major depression based on DSM-IV criteria were enrolled in this study. They had been treated with antidepressants and assessed by KDRSS, Hamilton Depression Rating Scale-24 (HDRS-24), and Montgomery-Åsberg Depression Rating Scale (MARDS). The validity and reliability of KDRSS were assessed, including internal consistency reliability, concurrent validity, temporal stability, factorial validity, and discriminative validity. Internal consistency (Cronbach's alpha=0.961), concurrent validity (MADRS: r=0.731, p<0.01, HDRS-24: r=0.663, p<0.01), and temporal stability (r=0.726, p<0.01) of KDRSS were all excellent. KDRSS showed good discriminative validity based on MARDS. KDRSS consisted of one-factor structure accounting for 63.8% of total variance. All subjects except two in full remission group had one or more residual symptoms. In 7 subscales of KDRSS consisting of similar items respectively, 'lack of energy' was the most commonly reported, followed by 'increased emotionalism' in this group. KDRSS is a useful and sensitive instrument for measuring residual depressive symptoms. Since some depressive symptoms including 'lack of energy' and 'increased emotionalism' in patients with full remission might be persistent during psychiatric intervention, these symptoms need to be focused on in clinical practice.

Development and community-based validation of the IDEA study Instrumental Activities of Daily Living (IDEA-IADL) questionnaire.

PubMed

Collingwood, Cecilia; Paddick, Stella-Maria; Kisoli, Aloyce; Dotchin, Catherine L; Gray, William K; Mbowe, Godfrey; Mkenda, Sarah; Urasa, Sarah; Mushi, Declare; Chaote, Paul; Walker, Richard W

2014-01-01

The dementia diagnosis gap in sub-Saharan Africa (SSA) is large, partly due to difficulties in assessing function, an essential step in diagnosis. As part of the Identification and Intervention for Dementia in Elderly Africans (IDEA) study, to develop, pilot, and validate an Instrumental Activities of Daily Living (IADL) questionnaire for use in a rural Tanzanian population to assist in the identification of people with dementia alongside cognitive screening. The questionnaire was developed at a workshop for rural primary healthcare workers, based on culturally appropriate roles and usual activities of elderly people in this community. It was piloted in 52 individuals under follow-up from a dementia prevalence study. Validation subsequently took place during a community dementia-screening programme. Construct validation against gold standard clinical dementia diagnosis using DSM-IV criteria was carried out on a stratified sample of the cohort and validity assessed using area under the receiver operating characteristic (AUROC) curve analysis. An 11-item questionnaire (IDEA-IADL) was developed after pilot testing. During formal validation on 130 community-dwelling elderly people who presented for screening, the AUROC curve was 0.896 for DSM-IV dementia when used in isolation and 0.937 when used in conjunction with the IDEA cognitive screen, previously validated in Tanzania. The internal consistency was 0.959. Performance on the IDEA-IADL was not biased with regard to age, gender or education level. The IDEA-IADL questionnaire appears to be a useful aid to dementia screening in this setting. Further validation in other healthcare settings in SSA is required.

Predictive Validity of Curriculum-Based Measures in the Reading Assessment of Students Who Are English Language Learners

ERIC Educational Resources Information Center

Muyskens, Paul; Betts, Joseph; Lau, Matthew Y.; Marston, Doug

2009-01-01

The inclusion of English Language Learners as a subgroup in the No Child Left Behind legislation has leant additional importance to the need for valid and efficient measures of reading for students whose first language is not English. This study examines the use of Curriculum-Based Measurement (CBM) reading fluency as a predictor of later reading…

Toward a New Process-Based Indicator for Measuring Writing Fluency: Evidence from L2 Writers' Think-Aloud Protocols

ERIC Educational Resources Information Center

Abdel Latif, Muhammad M.

2009-01-01

This article reports on a study aimed at testing the hypothesis that, because of strategic and temporal variables, composing rate and text quantity may not be valid measures of writing fluency. A second objective was to validate the mean length of writers' translating episodes as a process-based indicator that mirrors their fluent written…

Computer Literacy and the Construct Validity of a High-Stakes Computer-Based Writing Assessment

ERIC Educational Resources Information Center

Jin, Yan; Yan, Ming

2017-01-01

One major threat to validity in high-stakes testing is construct-irrelevant variance. In this study we explored whether the transition from a paper-and-pencil to a computer-based test mode in a high-stakes test in China, the College English Test, has brought about variance irrelevant to the construct being assessed in this test. Analyses of the…

Validity and Reliability of the Turkish Version of Needs Based Biopsychosocial Distress Instrument for Cancer Patients (CANDI)

PubMed Central

Beyhun, Nazim Ercument; Can, Gamze; Tiryaki, Ahmet; Karakullukcu, Serdar; Bulut, Bekir; Yesilbas, Sehbal; Kavgaci, Halil; Topbas, Murat

2016-01-01

Background Needs based biopsychosocial distress instrument for cancer patients (CANDI) is a scale based on needs arising due to the effects of cancer. Objectives The aim of this research was to determine the reliability and validity of the CANDI scale in the Turkish language. Patients and Methods The study was performed with the participation of 172 cancer patients aged 18 and over. Factor analysis (principal components analysis) was used to assess construct validity. Criterion validities were tested by computing Spearman correlation between CANDI and hospital anxiety depression scale (HADS), and brief symptom inventory (BSI) (convergent validity) and quality of life scales (FACT-G) (divergent validity). Test-retest reliabilities and internal consistencies were measured with intraclass correlation (ICC) and Cronbach-α. Results A three-factor solution (emotional, physical and social) was found with factor analysis. Internal reliability (α = 0.94) and test-retest reliability (ICC = 0.87) were significantly high. Correlations between CANDI and HADS (rs = 0.67), and BSI (rs = 0.69) and FACT-G (rs = -0.76) were moderate and significant in the expected direction. Conclusions CANDI is a valid and reliable scale in cancer patients with a three-factor structure (emotional, physical and social) in the Turkish language. PMID:27621931

Using entropy measures to characterize human locomotion.

PubMed

Leverick, Graham; Szturm, Tony; Wu, Christine Q

2014-12-01

Entropy measures have been widely used to quantify the complexity of theoretical and experimental dynamical systems. In this paper, the value of using entropy measures to characterize human locomotion is demonstrated based on their construct validity, predictive validity in a simple model of human walking and convergent validity in an experimental study. Results show that four of the five considered entropy measures increase meaningfully with the increased probability of falling in a simple passive bipedal walker model. The same four entropy measures also experienced statistically significant increases in response to increasing age and gait impairment caused by cognitive interference in an experimental study. Of the considered entropy measures, the proposed quantized dynamical entropy (QDE) and quantization-based approximation of sample entropy (QASE) offered the best combination of sensitivity to changes in gait dynamics and computational efficiency. Based on these results, entropy appears to be a viable candidate for assessing the stability of human locomotion.

Circulating tumor cells: clinical validity and utility.

PubMed

Cabel, Luc; Proudhon, Charlotte; Gortais, Hugo; Loirat, Delphine; Coussy, Florence; Pierga, Jean-Yves; Bidard, François-Clément

2017-06-01

Circulating tumor cells (CTCs) are rare tumor cells and have been investigated as diagnostic, prognostic and predictive biomarkers in many types of cancer. Although CTCs are not currently used in clinical practice, CTC studies have accumulated a high level of clinical validity, especially in breast, lung, prostate and colorectal cancers. In this review, we present an overview of the current clinical validity of CTCs in metastatic and non-metastatic disease, and the main concepts and studies investigating the clinical utility of CTCs. In particular, this review will focus on breast, lung, colorectal and prostate cancer. Three major topics concerning the clinical utility of CTC are discussed-(1) treatment based on CTCs used as liquid biopsy, (2) treatment based on CTC count or CTC variations, and (3) treatment based on CTC biomarker expression. A summary of published or ongoing phase II and III trials is also presented.

The development of performance-based practical assessment model at civil engineering workshop in state polytechnic

NASA Astrophysics Data System (ADS)

Kristinayanti, W. S.; Mas Pertiwi, I. G. A. I.; Evin Yudhi, S.; Lokantara, W. D.

2018-01-01

Assessment is an important element in education that shall oversees students’ competence not only in terms of cognitive aspect, but alsothe students’ psychomotorin a comprehensive way. Civil Engineering Department at Bali State Polytechnic,as a vocational education institution, emphasizes on not only the theoretical foundation of the study, but also the application throughpracticum in workshop-based learning. We are aware of a need for performance-based assessment for these students, which would be essential for the student’s all-round performance in their studies.We try to develop a performance-based practicum assessment model that is needed to assess student’s ability in workshop-based learning. This research was conducted in three stages, 1) learning needs analysis, 2) instruments development, and 3) testing of instruments. The study uses rubrics set-up to test students’ competence in the workshop and test the validity. We obtained 34-point valid statement out of 35, and resulted in value of Cronbach’s alpha equal to 0.977. In expert test we obtained a value of CVI = 0.75 which means that the drafted assessment is empirically valid within thetrial group.

How has the economic downturn affected communities and implementation of science-based prevention in the randomized trial of communities that care?

PubMed

Kuklinski, Margaret R; Hawkins, J David; Plotnick, Robert D; Abbott, Robert D; Reid, Carolina K

2013-06-01

This study examined implications of the economic downturn that began in December 2007 for the Community Youth Development Study (CYDS), a longitudinal randomized controlled trial of the Communities That Care (CTC) prevention system. The downturn had the potential to affect the internal validity of the CYDS research design and implementation of science-based prevention in study communities. We used archival economic indicators and community key leader reports of economic conditions to assess the extent of the economic downturn in CYDS communities and potential internal validity threats. We also examined whether stronger economic downturn effects were associated with a decline in science-based prevention implementation. Economic indicators suggested the downturn affected CYDS communities to different degrees. We found no evidence of systematic differences in downturn effects in CTC compared to control communities that would threaten internal validity of the randomized trial. The Community Economic Problems scale was a reliable measure of community economic conditions, and it showed criterion validity in relation to several objective economic indicators. CTC coalitions continued to implement science-based prevention to a significantly greater degree than control coalitions 2 years after the downturn began. However, CTC implementation levels declined to some extent as unemployment, the percentage of students qualifying for free lunch, and community economic problems worsened. Control coalition implementation levels were not related to economic conditions before or after the downturn, but mean implementation levels of science-based prevention were also relatively low in both periods.

How Has the Economic Downturn Affected Communities and Implementation of Science-Based Prevention in the Randomized Trial of Communities That Care?

PubMed Central

Kuklinski, Margaret R.; Hawkins, J. David; Plotnick, Robert D.; Abbott, Robert D.; Reid, Carolina K.

2013-01-01

This study examined implications of the economic downturn that began in December 2007 for the Community Youth Development Study (CYDS), a longitudinal randomized controlled trial of the Communities That Care (CTC) prevention system. The downturn had the potential to affect the internal validity of the CYDS research design and implementation of science-based prevention in study communities. We used archival economic indicators and community key leader reports of economic conditions to assess the extent of the economic downturn in CYDS communities and potential internal validity threats. We also examined whether stronger economic downturn effects were associated with a decline in science-based prevention implementation. Economic indicators suggested the downturn affected CYDS communities to different degrees. We found no evidence of systematic differences in downturn effects in CTC compared to control communities that would threaten internal validity of the randomized trial. The Community Economic Problems scale was a reliable measure of community economic conditions, and it showed criterion validity in relation to several objective economic indicators. CTC coalitions continued to implement science-based prevention to a significantly greater degree than control coalitions 2 years after the downturn began. However, CTC implementation levels declined to some extent as unemployment, the percentage of students qualifying for free lunch, and community economic problems worsened. Control coalition implementation levels were not related to economic conditions before or after the downturn, but mean implementation levels of science-based prevention were also relatively low in both periods. PMID:23054169

NASA GPM GV Science Implementation

NASA Technical Reports Server (NTRS)

Petersen, W. A.

2009-01-01

Pre-launch algorithm development & post-launch product evaluation: The GPM GV paradigm moves beyond traditional direct validation/comparison activities by incorporating improved algorithm physics & model applications (end-to-end validation) in the validation process. Three approaches: 1) National Network (surface): Operational networks to identify and resolve first order discrepancies (e.g., bias) between satellite and ground-based precipitation estimates. 2) Physical Process (vertical column): Cloud system and microphysical studies geared toward testing and refinement of physically-based retrieval algorithms. 3) Integrated (4-dimensional): Integration of satellite precipitation products into coupled prediction models to evaluate strengths/limitations of satellite precipitation producers.

NCI-FDA Interagency Oncology Task Force Workshop Provides Guidance for Analytical Validation of Protein-based Multiplex Assays | Office of Cancer Clinical Proteomics Research

Cancer.gov

An NCI-FDA Interagency Oncology Task Force (IOTF) Molecular Diagnostics Workshop was held on October 30, 2008 in Cambridge, MA, to discuss requirements for analytical validation of protein-based multiplex technologies in the context of its intended use. This workshop developed through NCI's Clinical Proteomic Technologies for Cancer initiative and the FDA focused on technology-specific analytical validation processes to be addressed prior to use in clinical settings. In making this workshop unique, a case study approach was used to discuss issues related to

Content validation of the nursing diagnosis acute pain in the Czech Republic and Slovakia.

PubMed

Zeleníková, Renáta; Žiaková, Katarína; Čáp, Juraj; Jarošová, Darja

2014-10-01

The main purpose of the study was to validate the defining characteristics of the nursing diagnosis acute pain in the Czech Republic and Slovakia. This is a descriptive study. The validation process involved was based on Fehring's diagnostic content validity model. Four defining characteristics were classified as major by Slovak nurses and eight defining characteristics were classified as major by Czech nurses. Validation of the nursing diagnosis acute pain in the Czech and Slovak sociocultural context has shown that nurses prioritize characteristics that are behavioral in nature as well as patients' verbal reports of pain. Verbal reports of pain and behavioral indicators are important for arriving at the nursing diagnosis acute pain. © 2014 NANDA International, Inc.

Validation of an administrative claims-based diagnostic code for pneumonia in a US-based commercially insured COPD population

PubMed Central

Kern, David M; Davis, Jill; Williams, Setareh A; Tunceli, Ozgur; Wu, Bingcao; Hollis, Sally; Strange, Charlie; Trudo, Frank

2015-01-01

Objective To estimate the accuracy of claims-based pneumonia diagnoses in COPD patients using clinical information in medical records as the reference standard. Methods Selecting from a repository containing members’ data from 14 regional United States health plans, this validation study identified pneumonia diagnoses within a group of patients initiating treatment for COPD between March 1, 2009 and March 31, 2012. Patients with ≥1 claim for pneumonia (International Classification of Diseases Version 9-CM code 480.xx–486.xx) were identified during the 12 months following treatment initiation. A subset of 800 patients was randomly selected to abstract medical record data (paper based and electronic) for a target sample of 400 patients, to estimate validity within 5% margin of error. Positive predictive value (PPV) was calculated for the claims diagnosis of pneumonia relative to the reference standard, defined as a documented diagnosis in the medical record. Results A total of 388 records were reviewed; 311 included a documented pneumonia diagnosis, indicating 80.2% (95% confidence interval [CI]: 75.8% to 84.0%) of claims-identified pneumonia diagnoses were validated by the medical charts. Claims-based diagnoses in inpatient or emergency departments (n=185) had greater PPV versus outpatient settings (n=203), 87.6% (95% CI: 81.9%–92.0%) versus 73.4% (95% CI: 66.8%–79.3%), respectively. Claims-diagnoses verified with paper-based charts had similar PPV as the overall study sample, 80.2% (95% CI: 71.1%–87.5%), and higher PPV than those linked to electronic medical records, 73.3% (95% CI: 65.5%–80.2%). Combined paper-based and electronic records had a higher PPV, 87.6% (95% CI: 80.9%–92.6%). Conclusion Administrative claims data indicating a diagnosis of pneumonia in COPD patients are supported by medical records. The accuracy of a medical record diagnosis of pneumonia remains unknown. With increased use of claims data in medical research, COPD researchers can study pneumonia with confidence that claims data are a valid tool when studying the safety of COPD therapies that could potentially lead to increased pneumonia susceptibility or severity. PMID:26229461

Validation of an administrative claims-based diagnostic code for pneumonia in a US-based commercially insured COPD population.

PubMed

Kern, David M; Davis, Jill; Williams, Setareh A; Tunceli, Ozgur; Wu, Bingcao; Hollis, Sally; Strange, Charlie; Trudo, Frank

2015-01-01

To estimate the accuracy of claims-based pneumonia diagnoses in COPD patients using clinical information in medical records as the reference standard. Selecting from a repository containing members' data from 14 regional United States health plans, this validation study identified pneumonia diagnoses within a group of patients initiating treatment for COPD between March 1, 2009 and March 31, 2012. Patients with ≥1 claim for pneumonia (International Classification of Diseases Version 9-CM code 480.xx-486.xx) were identified during the 12 months following treatment initiation. A subset of 800 patients was randomly selected to abstract medical record data (paper based and electronic) for a target sample of 400 patients, to estimate validity within 5% margin of error. Positive predictive value (PPV) was calculated for the claims diagnosis of pneumonia relative to the reference standard, defined as a documented diagnosis in the medical record. A total of 388 records were reviewed; 311 included a documented pneumonia diagnosis, indicating 80.2% (95% confidence interval [CI]: 75.8% to 84.0%) of claims-identified pneumonia diagnoses were validated by the medical charts. Claims-based diagnoses in inpatient or emergency departments (n=185) had greater PPV versus outpatient settings (n=203), 87.6% (95% CI: 81.9%-92.0%) versus 73.4% (95% CI: 66.8%-79.3%), respectively. Claims-diagnoses verified with paper-based charts had similar PPV as the overall study sample, 80.2% (95% CI: 71.1%-87.5%), and higher PPV than those linked to electronic medical records, 73.3% (95% CI: 65.5%-80.2%). Combined paper-based and electronic records had a higher PPV, 87.6% (95% CI: 80.9%-92.6%). Administrative claims data indicating a diagnosis of pneumonia in COPD patients are supported by medical records. The accuracy of a medical record diagnosis of pneumonia remains unknown. With increased use of claims data in medical research, COPD researchers can study pneumonia with confidence that claims data are a valid tool when studying the safety of COPD therapies that could potentially lead to increased pneumonia susceptibility or severity.

Relative validity of micronutrient and fiber intake assessed with two new interactive meal- and Web-based food frequency questionnaires.

PubMed

Christensen, Sara E; Möller, Elisabeth; Bonn, Stephanie E; Ploner, Alexander; Bälter, Olle; Lissner, Lauren; Bälter, Katarina

2014-02-21

The meal- and Web-based food frequency questionnaires, Meal-Q and MiniMeal-Q, were developed for cost-efficient assessment of dietary intake in epidemiological studies. The objective of this study was to evaluate the relative validity of micronutrient and fiber intake assessed with Meal-Q and MiniMeal-Q. The reproducibility of Meal-Q was also evaluated. A total of 163 volunteer men and women aged between 20 and 63 years were recruited from Stockholm County, Sweden. Assessment of micronutrient and fiber intake with the 174-item Meal-Q was compared to a Web-based 7-day weighed food record (WFR). Two administered Meal-Q questionnaires were compared for reproducibility. The 126-item MiniMeal-Q, developed after the validation study, was evaluated in a simulated validation by using truncated Meal-Q data. The study population consisted of approximately 80% women (129/163) with a mean age of 33 years (SD 12) who were highly educated (130/163, 80% with >12 years of education) on average. Cross-classification of quartiles with the WFR placed 69% to 90% in the same/adjacent quartile for Meal-Q and 67% to 89% for MiniMeal-Q. Bland-Altman plots with the WFR and the questionnaires showed large variances and a trend of increasing underestimation with increasing intakes. Deattenuated and energy-adjusted Spearman rank correlations between the questionnaires and the WFR were in the range ρ=.25-.69, excluding sodium that was not statistically significant. Cross-classifications of quartiles of the 2 Meal-Q administrations placed 86% to 97% in the same/adjacent quartile. Intraclass correlation coefficients for energy-adjusted intakes were in the range of .50-.76. With the exception of sodium, this validation study demonstrates Meal-Q and MiniMeal-Q to be useful methods for ranking micronutrient and fiber intake in epidemiological studies with Web-based data collection.

Design and validation of general biology learning program based on scientific inquiry skills

NASA Astrophysics Data System (ADS)

Cahyani, R.; Mardiana, D.; Noviantoro, N.

2018-03-01

Scientific inquiry is highly recommended to teach science. The reality in the schools and colleges is that many educators still have not implemented inquiry learning because of their lack of understanding. The study aims to1) analyze students’ difficulties in learning General Biology, 2) design General Biology learning program based on multimedia-assisted scientific inquiry learning, and 3) validate the proposed design. The method used was Research and Development. The subjects of the study were 27 pre-service students of general elementary school/Islamic elementary schools. The workflow of program design includes identifying learning difficulties of General Biology, designing course programs, and designing instruments and assessment rubrics. The program design is made for four lecture sessions. Validation of all learning tools were performed by expert judge. The results showed that: 1) there are some problems identified in General Biology lectures; 2) the designed products include learning programs, multimedia characteristics, worksheet characteristics, and, scientific attitudes; and 3) expert validation shows that all program designs are valid and can be used with minor revisions. The first section in your paper.

Patterns of Cognitive Strengths and Weaknesses: Identification Rates, Agreement, and Validity for Learning Disabilities Identification

PubMed Central

Miciak, Jeremy; Fletcher, Jack M.; Stuebing, Karla; Vaughn, Sharon; Tolar, Tammy D.

2014-01-01

Purpose Few empirical investigations have evaluated LD identification methods based on a pattern of cognitive strengths and weaknesses (PSW). This study investigated the reliability and validity of two proposed PSW methods: the concordance/discordance method (C/DM) and cross battery assessment (XBA) method. Methods Cognitive assessment data for 139 adolescents demonstrating inadequate response to intervention was utilized to empirically classify participants as meeting or not meeting PSW LD identification criteria using the two approaches, permitting an analysis of: (1) LD identification rates; (2) agreement between methods; and (3) external validity. Results LD identification rates varied between the two methods depending upon the cut point for low achievement, with low agreement for LD identification decisions. Comparisons of groups that met and did not meet LD identification criteria on external academic variables were largely null, raising questions of external validity. Conclusions This study found low agreement and little evidence of validity for LD identification decisions based on PSW methods. An alternative may be to use multiple measures of academic achievement to guide intervention. PMID:24274155

Validation study of an electronic method of condensed outcomes tools reporting in orthopaedics.

PubMed

Farr, Jack; Verma, Nikhil; Cole, Brian J

2013-12-01

Patient-reported outcomes (PRO) instruments are a vital source of data for evaluating the efficacy of medical treatments. Historically, outcomes instruments have been designed, validated, and implemented as paper-based questionnaires. The collection of paper-based outcomes information may result in patients becoming fatigued as they respond to redundant questions. This problem is exacerbated when multiple PRO measures are provided to a single patient. In addition, the management and analysis of data collected in paper format involves labor-intensive processes to score and render the data analyzable. Computer-based outcomes systems have the potential to mitigate these problems by reformatting multiple outcomes tools into a single, user-friendly tool.The study aimed to determine whether the electronic outcomes system presented produces results comparable with the test-retest correlations reported for the corresponding orthopedic paper-based outcomes instruments.The study is designed as a crossover study based on consecutive orthopaedic patients arriving at one of two designated orthopedic knee clinics.Patients were assigned to complete either a paper or a computer-administered questionnaire based on a similar set of questions (Knee injury and Osteoarthritis Outcome Score, International Knee Documentation Committee form, 36-Item Short Form survey, version 1, Lysholm Knee Scoring Scale). Each patient completed the same surveys using the other instrument, so that all patients had completed both paper and electronic versions. Correlations between the results from the two modes were studied and compared with test-retest data from the original validation studies.The original validation studies established test-retest reliability by computing correlation coefficients for two administrations of the paper instrument. Those correlation coefficients were all in the range of 0.7 to 0.9, which was deemed satisfactory. The present study computed correlation coefficients between the paper and electronic modes of administration. These correlation coefficients demonstrated similar results with an overall value of 0.86.On the basis of the correlation coefficients, the electronic application of commonly used knee outcome scores compare variably to the traditional paper variants with a high rate of test-retest correlation. This equivalence supports the use of the condensed electronic outcomes system and validates comparison of scores between electronic and paper modes. Thieme Medical Publishers 333 Seventh Avenue, New York, NY 10001, USA.

Reliability and validity of the Outcome Expectations for Exercise Scale-2.

PubMed

Resnick, Barbara

2005-10-01

Development of a reliable and valid measure of outcome expectations for exercise for older adults will help establish the relationship between outcome expectations and exercise and facilitate the development of interventions to increase physical activity in older adults. The purpose of this study was to test the reliability and validity of the Outcome Expectations for Exercise-2 Scale (OEE-2), a 13-item measure with two subscales: positive OEE (POEE) and negative OEE (NOEE). The OEE-2 scale was given to 161 residents in a continuing-care retirement community. There was some evidence of validity based on confirmatory factor analysis, Rasch-analysis INFIT and OUTFIT statistics, and convergent validity and test criterion relationships. There was some evidence for reliability of the OEE-2 based on alpha coefficients, person- and item-separation reliability indexes, and R(2)values. Based on analyses, suggested revisions are provided for future use of the OEE-2. Although ongoing reliability and validity testing are needed, the OEE-2 scale can be used to identify older adults with low outcome expectations for exercise, and interventions can then be implemented to strengthen these expectations and improve exercise behavior.

Measuring activities and participation in persons with haemophilia: A systematic review of commonly used instruments.

PubMed

Timmer, M A; Gouw, S C; Feldman, B M; Zwagemaker, A; de Kleijn, P; Pisters, M F; Schutgens, R E G; Blanchette, V; Srivastava, A; David, J A; Fischer, K; van der Net, J

2018-03-01

Monitoring clinical outcome in persons with haemophilia (PWH) is essential in order to provide optimal treatment for individual patients and compare effectiveness of treatment strategies. Experience with measurement of activities and participation in haemophilia is limited and consensus on preferred tools is lacking. The aim of this study was to give a comprehensive overview of the measurement properties of a selection of commonly used tools developed to assess activities and participation in PWH. Electronic databases were searched for articles that reported on reliability, validity or responsiveness of predetermined measurement tools (5 self-reported and 4 performance based measurement tools). Methodological quality of the studies was assessed according to the COSMIN checklist. Best evidence synthesis was used to summarize evidence on the measurement properties. The search resulted in 3453 unique hits. Forty-two articles were included. The self-reported Haemophilia Acitivity List (HAL), Pediatric HAL (PedHAL) and the performance based Functional Independence Score in Haemophilia (FISH) were studied most extensively. Methodological quality of the studies was limited. Measurement error, cross-cultural validity and responsiveness have been insufficiently evaluated. Albeit based on limited evidence, the measurement properties of the PedHAL, HAL and FISH are currently considered most satisfactory. Further research needs to focus on measurement error, responsiveness, interpretability and cross-cultural validity of the self-reported tools and validity of performance based tools which are able to assess limitations in sports and leisure activities. © 2018 The Authors. Haemophilia Published by John Wiley & Sons Ltd.

Validity evidence for an OSCE to assess competency in systems-based practice and practice-based learning and improvement: a preliminary investigation.

PubMed

Varkey, Prathibha; Natt, Neena; Lesnick, Timothy; Downing, Steven; Yudkowsky, Rachel

2008-08-01

To determine the psychometric properties and validity of an OSCE to assess the competencies of Practice-Based Learning and Improvement (PBLI) and Systems-Based Practice (SBP) in graduate medical education. An eight-station OSCE was piloted at the end of a three-week Quality Improvement elective for nine preventive medicine and endocrinology fellows at Mayo Clinic. The stations assessed performance in quality measurement, root cause analysis, evidence-based medicine, insurance systems, team collaboration, prescription errors, Nolan's model, and negotiation. Fellows' performance in each of the stations was assessed by three faculty experts using checklists and a five-point global competency scale. A modified Angoff procedure was used to set standards. Evidence for the OSCE's validity, feasibility, and acceptability was gathered. Evidence for content and response process validity was judged as excellent by institutional content experts. Interrater reliability of scores ranged from 0.85 to 1 for most stations. Interstation correlation coefficients ranged from -0.62 to 0.99, reflecting case specificity. Implementation cost was approximately $255 per fellow. All faculty members agreed that the OSCE was realistic and capable of providing accurate assessments. The OSCE provides an opportunity to systematically sample the different subdomains of Quality Improvement. Furthermore, the OSCE provides an opportunity for the demonstration of skills rather than the testing of knowledge alone, thus making it a potentially powerful assessment tool for SBP and PBLI. The study OSCE was well suited to assess SBP and PBLI. The evidence gathered through this study lays the foundation for future validation work.

Investigating the validity and usability of an interactive computer programme for assessing competence in telephone-based mental health triage.

PubMed

Sands, Natisha; Elsom, Stephen; Keppich-Arnold, Sandra; Henderson, Kathryn; King, Peter; Bourke-Finn, Karen; Brunning, Debra

2016-02-01

Telephone-based mental health triage services are frontline health-care providers that operate 24/7 to facilitate access to psychiatric assessment and intervention for people requiring assistance with a mental health problem. The mental health triage clinical role is complex, and the populations triage serves are typically high risk; yet to date, no evidence-based methods have been available to assess clinician competence to practice telephone-based mental health triage. The present study reports the findings of a study that investigated the validity and usability of the Mental Health Triage Competency Assessment Tool, an evidence-based, interactive computer programme designed to assist clinicians in developing and assessing competence to practice telephone-based mental health triage. © 2015 Australian College of Mental Health Nurses Inc.

The Deaf Acculturation Scale (DAS): Development and Validation of a 58-Item Measure

PubMed Central

Maxwell-McCaw, Deborah; Zea, Maria Cecilia

2011-01-01

This study involved the development and validation of the Deaf Acculturation Scale (DAS), a new measure of cultural identity for Deaf and hard-of-hearing (hh) populations. Data for this study were collected online and involved a nation-wide sample of 3,070 deaf/hh individuals. Results indicated strong internal reliabilities for all the subscales, and construct validity was established by demonstrating that the DAS could discriminate groups based on parental hearing status, school background, and use of self-labels. Construct validity was further demonstrated through factorial analyses, and findings resulted in a final 58-item measure. Directions for future research are discussed. PMID:21263041

Translation into Brazilian Portuguese and validation of the "Quantitative Global Scarring Grading System for Post-acne Scarring" *

PubMed Central

Cachafeiro, Thais Hofmann; Escobar, Gabriela Fortes; Maldonado, Gabriela; Cestari, Tania Ferreira

2014-01-01

The "Quantitative Global Scarring Grading System for Postacne Scarring" was developed in English for acne scar grading, based on the number and severity of each type of scar. The aims of this study were to translate this scale into Brazilian Portuguese and verify its reliability and validity. The study followed five steps: Translation, Expert Panel, Back Translation, Approval of authors and Validation. The translated scale showed high internal consistency and high test-retest reliability, confirming its reproducibility. Therefore, it has been validated for our population and can be recommended as a reliable instrument to assess acne scarring. PMID:25184939

Patient-completed or symptom-based screening tools for endometriosis: a scoping review.

PubMed

Surrey, Eric; Carter, Cathryn M; Soliman, Ahmed M; Khan, Shahnaz; DiBenedetti, Dana B; Snabes, Michael C

2017-08-01

The objective of this review was to evaluate existing patient-completed screening questionnaires and/or symptom-based predictive models with respect to their potential for use as screening tools for endometriosis in adult women. Validated instruments were of particular interest. We conducted structured searches of PubMed and targeted searches of the gray literature to identify studies reporting on screening instruments used in endometriosis. Studies were screened according to inclusion and exclusion criteria that followed the PICOS (population, intervention, comparison, outcomes, study design) framework. A total of 16 studies were identified, of which 10 described measures for endometriosis in general, 2 described measures for endometriosis at specific sites, and 4 described measures for deep-infiltrating endometriosis. Only 1 study evaluated a questionnaire that was solely patient-completed. Most measures required physician, imaging, or laboratory assessments in addition to patient-completed questionnaires, and several measures relied on complex scoring. Validation for use as a screening tool in adult women with potential endometriosis was lacking in all studies, as most studies focused on diagnosis versus screening. This literature review did not identify any fully validated, symptom-based, patient-reported questionnaires for endometriosis screening in adult women.

Code-based Diagnostic Algorithms for Idiopathic Pulmonary Fibrosis. Case Validation and Improvement.

PubMed

Ley, Brett; Urbania, Thomas; Husson, Gail; Vittinghoff, Eric; Brush, David R; Eisner, Mark D; Iribarren, Carlos; Collard, Harold R

2017-06-01

Population-based studies of idiopathic pulmonary fibrosis (IPF) in the United States have been limited by reliance on diagnostic code-based algorithms that lack clinical validation. To validate a well-accepted International Classification of Diseases, Ninth Revision, code-based algorithm for IPF using patient-level information and to develop a modified algorithm for IPF with enhanced predictive value. The traditional IPF algorithm was used to identify potential cases of IPF in the Kaiser Permanente Northern California adult population from 2000 to 2014. Incidence and prevalence were determined overall and by age, sex, and race/ethnicity. A validation subset of cases (n = 150) underwent expert medical record and chest computed tomography review. A modified IPF algorithm was then derived and validated to optimize positive predictive value. From 2000 to 2014, the traditional IPF algorithm identified 2,608 cases among 5,389,627 at-risk adults in the Kaiser Permanente Northern California population. Annual incidence was 6.8/100,000 person-years (95% confidence interval [CI], 6.1-7.7) and was higher in patients with older age, male sex, and white race. The positive predictive value of the IPF algorithm was only 42.2% (95% CI, 30.6 to 54.6%); sensitivity was 55.6% (95% CI, 21.2 to 86.3%). The corrected incidence was estimated at 5.6/100,000 person-years (95% CI, 2.6-10.3). A modified IPF algorithm had improved positive predictive value but reduced sensitivity compared with the traditional algorithm. A well-accepted International Classification of Diseases, Ninth Revision, code-based IPF algorithm performs poorly, falsely classifying many non-IPF cases as IPF and missing a substantial proportion of IPF cases. A modification of the IPF algorithm may be useful for future population-based studies of IPF.

Design, testing and validation of an innovative web-based instrument to evaluate school meal quality.

PubMed

Patterson, Emma; Quetel, Anna-Karin; Lilja, Karin; Simma, Marit; Olsson, Linnea; Elinder, Liselotte Schäfer

2013-06-01

To develop a feasible, valid, reliable web-based instrument to objectively evaluate school meal quality in Swedish primary schools. The construct 'school meal quality' was operationalized by an expert panel into six domains, one of which was nutritional quality. An instrument was drafted and pilot-tested. Face validity was evaluated by the panel. Feasibility was established via a large national study. Food-based criteria to predict the nutritional adequacy of school meals in terms of fat quality, iron, vitamin D and fibre content were developed. Predictive validity was evaluated by comparing the nutritional adequacy of school menus based on these criteria with the results from a nutritional analysis. Inter-rater reliability was also assessed. The instrument was developed between 2010 and 2012. It is designed for use in all primary schools by school catering and/or management representatives. A pilot-test of eighty schools in Stockholm (autumn 2010) and a further test of feasibility in 191 schools nationally (spring 2011). The four nutrient-specific food-based criteria predicted nutritional adequacy with sensitivity ranging from 0.85 to 1.0, specificity from 0.45 to 1.0 and accuracy from 0.67 to 1.0. The sample in the national study was statistically representative and the majority of users rated the questionnaire positively, suggesting the instrument is feasible. The inter-rater reliability was fair to almost perfect for continuous variables and agreement was ≥ 67 % for categorical variables. An innovative web-based system to comprehensively monitor school meal quality across several domains, with validated questions in the nutritional domain, is available in Sweden for the first time.

A design of experiments approach to validation sampling for logistic regression modeling with error-prone medical records.

PubMed

Ouyang, Liwen; Apley, Daniel W; Mehrotra, Sanjay

2016-04-01

Electronic medical record (EMR) databases offer significant potential for developing clinical hypotheses and identifying disease risk associations by fitting statistical models that capture the relationship between a binary response variable and a set of predictor variables that represent clinical, phenotypical, and demographic data for the patient. However, EMR response data may be error prone for a variety of reasons. Performing a manual chart review to validate data accuracy is time consuming, which limits the number of chart reviews in a large database. The authors' objective is to develop a new design-of-experiments-based systematic chart validation and review (DSCVR) approach that is more powerful than the random validation sampling used in existing approaches. The DSCVR approach judiciously and efficiently selects the cases to validate (i.e., validate whether the response values are correct for those cases) for maximum information content, based only on their predictor variable values. The final predictive model will be fit using only the validation sample, ignoring the remainder of the unvalidated and unreliable error-prone data. A Fisher information based D-optimality criterion is used, and an algorithm for optimizing it is developed. The authors' method is tested in a simulation comparison that is based on a sudden cardiac arrest case study with 23 041 patients' records. This DSCVR approach, using the Fisher information based D-optimality criterion, results in a fitted model with much better predictive performance, as measured by the receiver operating characteristic curve and the accuracy in predicting whether a patient will experience the event, than a model fitted using a random validation sample. The simulation comparisons demonstrate that this DSCVR approach can produce predictive models that are significantly better than those produced from random validation sampling, especially when the event rate is low. © The Author 2015. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For Permissions, please email: journals.permissions@oup.com.

Use of multiple cluster analysis methods to explore the validity of a community outcomes concept map.

PubMed

Orsi, Rebecca

2017-02-01

Concept mapping is now a commonly-used technique for articulating and evaluating programmatic outcomes. However, research regarding validity of knowledge and outcomes produced with concept mapping is sparse. The current study describes quantitative validity analyses using a concept mapping dataset. We sought to increase the validity of concept mapping evaluation results by running multiple cluster analysis methods and then using several metrics to choose from among solutions. We present four different clustering methods based on analyses using the R statistical software package: partitioning around medoids (PAM), fuzzy analysis (FANNY), agglomerative nesting (AGNES) and divisive analysis (DIANA). We then used the Dunn and Davies-Bouldin indices to assist in choosing a valid cluster solution for a concept mapping outcomes evaluation. We conclude that the validity of the outcomes map is high, based on the analyses described. Finally, we discuss areas for further concept mapping methods research. Copyright © 2016 Elsevier Ltd. All rights reserved.

Prevalence of Invalid Performance on Baseline Testing for Sport-Related Concussion by Age and Validity Indicator.

PubMed

Abeare, Christopher A; Messa, Isabelle; Zuccato, Brandon G; Merker, Bradley; Erdodi, Laszlo

2018-03-12

Estimated base rates of invalid performance on baseline testing (base rates of failure) for the management of sport-related concussion range from 6.1% to 40.0%, depending on the validity indicator used. The instability of this key measure represents a challenge in the clinical interpretation of test results that could undermine the utility of baseline testing. To determine the prevalence of invalid performance on baseline testing and to assess whether the prevalence varies as a function of age and validity indicator. This retrospective, cross-sectional study included data collected between January 1, 2012, and December 31, 2016, from a clinical referral center in the Midwestern United States. Participants included 7897 consecutively tested, equivalently proportioned male and female athletes aged 10 to 21 years, who completed baseline neurocognitive testing for the purpose of concussion management. Baseline assessment was conducted with the Immediate Postconcussion Assessment and Cognitive Testing (ImPACT), a computerized neurocognitive test designed for assessment of concussion. Base rates of failure on published ImPACT validity indicators were compared within and across age groups. Hypotheses were developed after data collection but prior to analyses. Of the 7897 study participants, 4086 (51.7%) were male, mean (SD) age was 14.71 (1.78) years, 7820 (99.0%) were primarily English speaking, and the mean (SD) educational level was 8.79 (1.68) years. The base rate of failure ranged from 6.4% to 47.6% across individual indicators. Most of the sample (55.7%) failed at least 1 of 4 validity indicators. The base rate of failure varied considerably across age groups (117 of 140 [83.6%] for those aged 10 years to 14 of 48 [29.2%] for those aged 21 years), representing a risk ratio of 2.86 (95% CI, 2.60-3.16; P < .001). The results for base rate of failure were surprisingly high overall and varied widely depending on the specific validity indicator and the age of the examinee. The strong age association, with 3 of 4 participants aged 10 to 12 years failing validity indicators, suggests that the clinical interpretation and utility of baseline testing in this age group is questionable. These findings underscore the need for close scrutiny of performance validity indicators on baseline testing across age groups.

Individual Differences in Base Rate Neglect: A Fuzzy Processing Preference Index

PubMed Central

Wolfe, Christopher R.; Fisher, Christopher R.

2013-01-01

Little is known about individual differences in integrating numeric base-rates and qualitative text in making probability judgments. Fuzzy-Trace Theory predicts a preference for fuzzy processing. We conducted six studies to develop the FPPI, a reliable and valid instrument assessing individual differences in this fuzzy processing preference. It consists of 19 probability estimation items plus 4 "M-Scale" items that distinguish simple pattern matching from “base rate respect.” Cronbach's Alpha was consistently above 0.90. Validity is suggested by significant correlations between FPPI scores and three other measurers: "Rule Based" Process Dissociation Procedure scores; the number of conjunction fallacies in joint probability estimation; and logic index scores on syllogistic reasoning. Replicating norms collected in a university study with a web-based study produced negligible differences in FPPI scores, indicating robustness. The predicted relationships between individual differences in base rate respect and both conjunction fallacies and syllogistic reasoning were partially replicated in two web-based studies. PMID:23935255

Development and validation of the Evidence Based Medicine Questionnaire (EBMQ) to assess doctors' knowledge, practice and barriers regarding the implementation of evidence-based medicine in primary care.

PubMed

Hisham, Ranita; Ng, Chirk Jenn; Liew, Su May; Lai, Pauline Siew Mei; Chia, Yook Chin; Khoo, Ee Ming; Hanafi, Nik Sherina; Othman, Sajaratulnisah; Lee, Ping Yein; Abdullah, Khatijah Lim; Chinna, Karuthan

2018-06-23

Evidence-Based Medicine (EBM) integrates best available evidence from literature and patients' values, which then informs clinical decision making. However, there is a lack of validated instruments to assess the knowledge, practice and barriers of primary care physicians in the implementation of EBM. This study aimed to develop and validate an Evidence-Based Medicine Questionnaire (EBMQ) in Malaysia. The EBMQ was developed based on a qualitative study, literature review and an expert panel. Face and content validity was verified by the expert panel and piloted among 10 participants. Primary care physicians with or without EBM training who could understand English were recruited from December 2015 to January 2016. The EBMQ was administered at baseline and two weeks later. A higher score indicates better knowledge, better practice of EBM and less barriers towards the implementation of EBM. We hypothesized that the EBMQ would have three domains: knowledge, practice and barriers. The final version of the EBMQ consists of 80 items: 62 items were measured on a nominal scale, 22 items were measured on a 5 point Likert-scale. Flesch reading ease was 61.2. A total of 343 participants were approached; of whom 320 agreed to participate (response rate = 93.2%). Factor analysis revealed that the EBMQ had eight domains after 13 items were removed: "EBM websites", "evidence-based journals", "types of studies", "terms related to EBM", "practice", "access", "patient preferences" and "support". Cronbach alpha for the overall EBMQ was 0.909, whilst the Cronbach alpha for the individual domain ranged from 0.657-0.940. The EBMQ was able to discriminate between doctors with and without EBM training for 24 out of 42 items. At test-retest, kappa values ranged from 0.155 to 0.620. The EBMQ was found to be a valid and reliable instrument to assess the knowledge, practice and barriers towards the implementation of EBM among primary care physicians in Malaysia.

Applicability of Monte Carlo cross validation technique for model development and validation using generalised least squares regression

NASA Astrophysics Data System (ADS)

Haddad, Khaled; Rahman, Ataur; A Zaman, Mohammad; Shrestha, Surendra

2013-03-01

SummaryIn regional hydrologic regression analysis, model selection and validation are regarded as important steps. Here, the model selection is usually based on some measurements of goodness-of-fit between the model prediction and observed data. In Regional Flood Frequency Analysis (RFFA), leave-one-out (LOO) validation or a fixed percentage leave out validation (e.g., 10%) is commonly adopted to assess the predictive ability of regression-based prediction equations. This paper develops a Monte Carlo Cross Validation (MCCV) technique (which has widely been adopted in Chemometrics and Econometrics) in RFFA using Generalised Least Squares Regression (GLSR) and compares it with the most commonly adopted LOO validation approach. The study uses simulated and regional flood data from the state of New South Wales in Australia. It is found that when developing hydrologic regression models, application of the MCCV is likely to result in a more parsimonious model than the LOO. It has also been found that the MCCV can provide a more realistic estimate of a model's predictive ability when compared with the LOO.

Development and validation of the Spanish-English Language Proficiency Scale (SELPS).

PubMed

Smyk, Ekaterina; Restrepo, M Adelaida; Gorin, Joanna S; Gray, Shelley

2013-07-01

This study examined the development and validation of a criterion-referenced Spanish-English Language Proficiency Scale (SELPS) that was designed to assess the oral language skills of sequential bilingual children ages 4-8. This article reports results for the English proficiency portion of the scale. The SELPS assesses syntactic complexity, grammatical accuracy, verbal fluency, and lexical diversity based on 2 story retell tasks. In Study 1, 40 children were given 2 story retell tasks to evaluate the reliability of parallel forms. In Study 2, 76 children participated in the validation of the scale against language sample measures and teacher ratings of language proficiency. Study 1 indicated no significant differences between the SELPS scores on the 2 stories. Study 2 indicated that the SELPS scores correlated significantly with their counterpart language sample measures. Correlations between the SELPS and teacher ratings were moderate. The 2 story retells elicited comparable SELPS scores, providing a valuable tool for test-retest conditions in the assessment of language proficiency. Correlations between the SELPS scores and external variables indicated that these measures assessed the same language skills. Results provided empirical evidence regarding the validity of inferences about language proficiency based on the SELPS score.

Validity of a computerized population registry of dementia based on clinical databases.

PubMed

Mar, J; Arrospide, A; Soto-Gordoa, M; Machón, M; Iruin, Á; Martinez-Lage, P; Gabilondo, A; Moreno-Izco, F; Gabilondo, A; Arriola, L

2018-05-08

The handling of information through digital media allows innovative approaches for identifying cases of dementia through computerized searches within the clinical databases that include systems for coding diagnoses. The aim of this study was to analyze the validity of a dementia registry in Gipuzkoa based on the administrative and clinical databases existing in the Basque Health Service. This is a descriptive study based on the evaluation of available data sources. First, through review of medical records, the diagnostic validity was evaluated in 2 samples of cases identified and not identified as dementia. The sensitivity, specificity and positive and negative predictive value of the diagnosis of dementia were measured. Subsequently, the cases of living dementia in December 31, 2016 were searched in the entire Gipuzkoa population to collect sociodemographic and clinical variables. The validation samples included 986 cases and 327 no cases. The calculated sensitivity was 80.2% and the specificity was 99.9%. The negative predictive value was 99.4% and positive value was 95.1%. The cases in Gipuzkoa were 10,551, representing 65% of the cases predicted according to the literature. Antipsychotic medication were taken by a 40% and a 25% of the cases were institutionalized. A registry of dementias based on clinical and administrative databases is valid and feasible. Its main contribution is to show the dimension of dementia in the health system. Copyright © 2018 Sociedad Española de Neurología. Publicado por Elsevier España, S.L.U. All rights reserved.

Development and Validation of the Personal Strengths Inventory Using Exploratory and Confirmatory Factor Analyses

ERIC Educational Resources Information Center

Liau, Albert Kienfie; Chow, Daryl; Tan, Teck Kiang; Senf, Konrad

2011-01-01

The purpose of this study was to establish the reliability and validity of the scores on a brief strengths-based assessment, the 22-item Personal Strengths Inventory (PSI). In Study 1, findings from exploratory factor analysis of 410 adolescents provided evidence for a five-factor solution--social competence (four items), emotional awareness (five…

Cultural Validity of the Minnesota Multiphasic Personality Inventory-2 Empirical Correlates: Is This the Best We Can Do?

ERIC Educational Resources Information Center

Hill, Jill S.; Robbins, Rockey R.; Pace, Terry M.

2012-01-01

This article critically reviews empirical correlates of the Minnesota Multiphasic Personality Inventory-2 (MMPI-2; Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989), based on several validation studies conducted with different racial, ethnic, and cultural groups. A major critique of the reviewed MMPI-2 studies was focused on the use of…

On Using Simulations to Inform Decision Making during Instrument Development

ERIC Educational Resources Information Center

Morgan, Grant B.; Moore, Courtney A.; Floyd, Harlee S.

2018-01-01

Although content validity--how well each item of an instrument represents the construct being measured--is foundational in the development of an instrument, statistical validity is also important to the decisions that are made based on the instrument. The primary purpose of this study is to demonstrate how simulation studies can be used to assist…

Open and Distance Education Accreditation Standards Scale: Validity and Reliability Studies

ERIC Educational Resources Information Center

Can, Ertug

2016-01-01

The purpose of this study is to develop, and test the validity and reliability of a scale for the use of researchers to determine the accreditation standards of open and distance education based on the views of administrators, teachers, staff and students. This research was designed according to the general descriptive survey model since it aims…

An Application-Based Discussion of Construct Validity and Internal Consistency Reliability.

ERIC Educational Resources Information Center

Taylor, Dianne L.; Campbell, Kathleen T.

Several techniques for conducting studies of measurement integrity are explained and illustrated using a heuristic data set from a study of teachers' participation in decision making (D. L. Taylor, 1991). The sample consisted of 637 teachers. It is emphasized that validity and reliability are characteristics of data, and do not inure to tests as…

Evidences of Validity of the Brazilian Scale of Learner's Attitude towards Distance Education Programs

ERIC Educational Resources Information Center

Coelho, Francisco Antonio, Jr.; Cortat, Mariane; Flores, Clarissa Leite; Santos, Flávio Augusto Mendes; Alves, Gleidilson Costa; Faiad, Cristiane; Ramos, Wilsa Maria; Rodrigues da Silva, Alan

2018-01-01

Online learning is one of the fastest growing trends in educational uses of technology. In this study, an instrument to measure the social attitudes of the Brazilian students based on distance education was developed and validated. The study population consisted of public administration undergraduate students that has been providing by distance…

A Study of Developing an Attitude Scale towards Authentic Learning Environments and Evaluation

ERIC Educational Resources Information Center

Çetinkaya, Murat

2018-01-01

The aim of the research is to improve a valid and reliable attributing scale which identifies authentic learning environments and evaluation attributes of the science teacher candidates. The study has been designed on the base of validity and reliability of the scale developed to evaluate the authentic learning environments. The research group is…

MLS Multipath Studies. Phase 3. Volume I. Overview and Propagation Model Validation/Refinement Studies.

DTIC Science & Technology

1979-04-25

Airport (Bedford, MA ) and Ft. Devens, MA. (2) validation of the models for building reflections based on elevation field measurements at JFK airport and...angles. 2-60 III. BUILDING REFLECTIONS A. Van Measurements at John F. Kennedy (JFK) International Airport, New York Figure 3-1 shows a map of JFK airport with

An Interactive Whiteboard Student Survey: Development, Validity and Reliability

ERIC Educational Resources Information Center

Turel, Yalin Kilic

2011-01-01

The interactive whiteboard (IWB) has become a popular technology for instructors over the last decade. Though research asserts that the IWBs facilitate learning in different ways, there is a lack of studies examining actual IWB use in classroom settings based on learners' perspectives by means of valid instruments. The purpose of this study is to…

The Physician Values in Practice Scale: Construction and Initial Validation

ERIC Educational Resources Information Center

Hartung, Paul J.; Taber, Brian J.; Richard, George V.

2005-01-01

Measures of values typically appraise the construct globally, across life domains or relative to a broad life domain such as work. We conducted two studies to construct and initially validate an occupation- and context-specific values measure. Study 1, based on a sample of 192 medical students, describes the initial construction and item analysis…

A New Method for Analyzing Content Validity Data Using Multidimensional Scaling

ERIC Educational Resources Information Center

Li, Xueming; Sireci, Stephen G.

2013-01-01

Validity evidence based on test content is of essential importance in educational testing. One source for such evidence is an alignment study, which helps evaluate the congruence between tested objectives and those specified in the curriculum. However, the results of an alignment study do not always sufficiently capture the degree to which a test…

Improving the Residency Admissions Process by Integrating a Professionalism Assessment: A Validity and Feasibility Study

ERIC Educational Resources Information Center

Bajwa, Nadia M.; Yudkowsky, Rachel; Belli, Dominique; Vu, Nu Viet; Park, Yoon Soo

2017-01-01

The purpose of this study was to provide validity and feasibility evidence in measuring professionalism using the Professionalism Mini-Evaluation Exercise (P-MEX) scores as part of a residency admissions process. In 2012 and 2013, three standardized-patient-based P-MEX encounters were administered to applicants invited for an interview at the…

Is questionnaire-based sitting time inaccurate and can it be improved? A cross-sectional investigation using accelerometer-based sitting time.

PubMed

Gupta, Nidhi; Christiansen, Caroline Stordal; Hanisch, Christiana; Bay, Hans; Burr, Hermann; Holtermann, Andreas

2017-01-16

To investigate the differences between a questionnaire-based and accelerometer-based sitting time, and develop a model for improving the accuracy of questionnaire-based sitting time for predicting accelerometer-based sitting time. 183 workers in a cross-sectional study reported sitting time per day using a single question during the measurement period, and wore 2 Actigraph GT3X+ accelerometers on the thigh and trunk for 1-4 working days to determine their actual sitting time per day using the validated Acti4 software. Least squares regression models were fitted with questionnaire-based siting time and other self-reported predictors to predict accelerometer-based sitting time. Questionnaire-based and accelerometer-based average sitting times were ≈272 and ≈476 min/day, respectively. A low Pearson correlation (r=0.32), high mean bias (204.1 min) and wide limits of agreement (549.8 to -139.7 min) between questionnaire-based and accelerometer-based sitting time were found. The prediction model based on questionnaire-based sitting explained 10% of the variance in accelerometer-based sitting time. Inclusion of 9 self-reported predictors in the model increased the explained variance to 41%, with 10% optimism using a resampling bootstrap validation. Based on a split validation analysis, the developed prediction model on ≈75% of the workers (n=132) reduced the mean and the SD of the difference between questionnaire-based and accelerometer-based sitting time by 64% and 42%, respectively, in the remaining 25% of the workers. This study indicates that questionnaire-based sitting time has low validity and that a prediction model can be one solution to materially improve the precision of questionnaire-based sitting time. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.

Early Prediction of Intensive Care Unit-Acquired Weakness: A Multicenter External Validation Study.

PubMed

Witteveen, Esther; Wieske, Luuk; Sommers, Juultje; Spijkstra, Jan-Jaap; de Waard, Monique C; Endeman, Henrik; Rijkenberg, Saskia; de Ruijter, Wouter; Sleeswijk, Mengalvio; Verhamme, Camiel; Schultz, Marcus J; van Schaik, Ivo N; Horn, Janneke

2018-01-01

An early diagnosis of intensive care unit-acquired weakness (ICU-AW) is often not possible due to impaired consciousness. To avoid a diagnostic delay, we previously developed a prediction model, based on single-center data from 212 patients (development cohort), to predict ICU-AW at 2 days after ICU admission. The objective of this study was to investigate the external validity of the original prediction model in a new, multicenter cohort and, if necessary, to update the model. Newly admitted ICU patients who were mechanically ventilated at 48 hours after ICU admission were included. Predictors were prospectively recorded, and the outcome ICU-AW was defined by an average Medical Research Council score <4. In the validation cohort, consisting of 349 patients, we analyzed performance of the original prediction model by assessment of calibration and discrimination. Additionally, we updated the model in this validation cohort. Finally, we evaluated a new prediction model based on all patients of the development and validation cohort. Of 349 analyzed patients in the validation cohort, 190 (54%) developed ICU-AW. Both model calibration and discrimination of the original model were poor in the validation cohort. The area under the receiver operating characteristics curve (AUC-ROC) was 0.60 (95% confidence interval [CI]: 0.54-0.66). Model updating methods improved calibration but not discrimination. The new prediction model, based on all patients of the development and validation cohort (total of 536 patients) had a fair discrimination, AUC-ROC: 0.70 (95% CI: 0.66-0.75). The previously developed prediction model for ICU-AW showed poor performance in a new independent multicenter validation cohort. Model updating methods improved calibration but not discrimination. The newly derived prediction model showed fair discrimination. This indicates that early prediction of ICU-AW is still challenging and needs further attention.

Chronic Pain: Content Validation of Nursing Diagnosis in Slovakia and the Czech Republic.

PubMed

Zeleníková, Renáta; Maniaková, Lenka

2015-10-01

The main purpose of the study was to validate the defining characteristics and related factors of the nursing diagnosis "chronic pain" in Slovakia and the Czech Republic. This is a descriptive study. The validation process involved was based on Fehring's Diagnostic Content Validity Model. Three defining characteristics (reports pain, altered ability to continue previous activities, and depression) were classified as major by Slovak nurses, and one defining characteristic (reports pain) was classified as major by Czech nurses. The results of the study provide guidance in devising strategies of pain assessment and can aid in the formulation of accurate nursing diagnoses. The defining characteristic "reports pain" is important for arriving at the nursing diagnosis "chronic pain." © 2014 NANDA International, Inc.

Validation testing of a soil macronutrient sensing system

USDA-ARS?s Scientific Manuscript database

Rapid on-site measurements of soil macronutrients (i.e., nitrogen, phosphorus, and potassium) are needed for site-specific crop management, where fertilizer nutrient application rates are adjusted spatially based on local requirements. This study reports on validation testing of a previously develop...

Assessing the Validity of a Stage Measure on Physical Activity in a Population-Based Sample of Individuals with Type 1 or Type 2 Diabetes

ERIC Educational Resources Information Center

Plotnikoff, Ronald C.; Lippke, Sonia; Reinbold-Matthews, Melissa; Courneya, Kerry S.; Karunamuni, Nandini; Sigal, Ronald J.; Birkett, Nicholas

2007-01-01

This study was designed to test the validity of a transtheoretical model's physical activity (PA) stage measure with intention and different intensities of behavior in a large population-based sample of adults living with diabetes (Type 1 diabetes, n = 697; Type 2 diabetes, n = 1,614) and examine different age groups. The overall…

Nutrition screening tools: an analysis of the evidence.

PubMed

Skipper, Annalynn; Ferguson, Maree; Thompson, Kyle; Castellanos, Victoria H; Porcari, Judy

2012-05-01

In response to questions about tools for nutrition screening, an evidence analysis project was developed to identify the most valid and reliable nutrition screening tools for use in acute care and hospital-based ambulatory care settings. An oversight group defined nutrition screening and literature search criteria. A trained analyst conducted structured searches of the literature for studies of nutrition screening tools according to predetermined criteria. Eleven nutrition screening tools designed to detect undernutrition in patients in acute care and hospital-based ambulatory care were identified. Trained analysts evaluated articles for quality using criteria specified by the American Dietetic Association's Evidence Analysis Library. Members of the oversight group assigned quality grades to the tools based on the quality of the supporting evidence, including reliability and validity data. One tool, the NRS-2002, received a grade I, and 4 tools-the Simple Two-Part Tool, the Mini-Nutritional Assessment-Short Form (MNA-SF), the Malnutrition Screening Tool (MST), and Malnutrition Universal Screening Tool (MUST)-received a grade II. The MST was the only tool shown to be both valid and reliable for identifying undernutrition in the settings studied. Thus, validated nutrition screening tools that are simple and easy to use are available for application in acute care and hospital-based ambulatory care settings.

A pilot study to validate measures of the theory of reasoned action for organ donation behavior.

PubMed

Wong, Shui Hung; Chow, Amy Yin Man

2018-04-01

The present study aimed at taking the first attempt in validating the measures generated based on the theory of reasoned action (TRA). A total of 211 university students participated in the study, 95 were included in the exploratory factor analysis and 116 were included in the confirmatory factor analysis. The TRA measurements were established with adequate psychometric properties, internal consistency, and construct validity. Findings also suggested that attitude toward organ donation has both a cognitive and affective nature, while the subjective norm of the family seems to be important to students' views on organ donation.

Development and Validation of Targeted Next-Generation Sequencing Panels for Detection of Germline Variants in Inherited Diseases.

PubMed

Santani, Avni; Murrell, Jill; Funke, Birgit; Yu, Zhenming; Hegde, Madhuri; Mao, Rong; Ferreira-Gonzalez, Andrea; Voelkerding, Karl V; Weck, Karen E

2017-06-01

- The number of targeted next-generation sequencing (NGS) panels for genetic diseases offered by clinical laboratories is rapidly increasing. Before an NGS-based test is implemented in a clinical laboratory, appropriate validation studies are needed to determine the performance characteristics of the test. - To provide examples of assay design and validation of targeted NGS gene panels for the detection of germline variants associated with inherited disorders. - The approaches used by 2 clinical laboratories for the development and validation of targeted NGS gene panels are described. Important design and validation considerations are examined. - Clinical laboratories must validate performance specifications of each test prior to implementation. Test design specifications and validation data are provided, outlining important steps in validation of targeted NGS panels by clinical diagnostic laboratories.

Validation plays the role of a "bridge" in connecting remote sensing research and applications

NASA Astrophysics Data System (ADS)

Wang, Zhiqiang; Deng, Ying; Fan, Yida

2018-07-01

Remote sensing products contribute to improving earth observations over space and time. Uncertainties exist in products of different levels; thus, validation of these products before and during their applications is critical. This study discusses the meaning of validation in depth and proposes a new definition of reliability for use with such products. In this context, validation should include three aspects: a description of the relevant uncertainties, quantitative measurement results and a qualitative judgment that considers the needs of users. A literature overview is then presented evidencing improvements in the concepts associated with validation. It shows that the root mean squared error (RMSE) is widely used to express accuracy; increasing numbers of remote sensing products have been validated; research institutes contribute most validation efforts; and sufficient validation studies encourage the application of remote sensing products. Validation plays a connecting role in the distribution and application of remote sensing products. Validation connects simple remote sensing subjects with other disciplines, and it connects primary research with practical applications. Based on the above findings, it is suggested that validation efforts that include wider cooperation among research institutes and full consideration of the needs of users should be promoted.

Validation of a track repeating algorithm for intensity modulated proton therapy: clinical cases study

NASA Astrophysics Data System (ADS)

Yepes, Pablo P.; Eley, John G.; Liu, Amy; Mirkovic, Dragan; Randeniya, Sharmalee; Titt, Uwe; Mohan, Radhe

2016-04-01

Monte Carlo (MC) methods are acknowledged as the most accurate technique to calculate dose distributions. However, due its lengthy calculation times, they are difficult to utilize in the clinic or for large retrospective studies. Track-repeating algorithms, based on MC-generated particle track data in water, accelerate dose calculations substantially, while essentially preserving the accuracy of MC. In this study, we present the validation of an efficient dose calculation algorithm for intensity modulated proton therapy, the fast dose calculator (FDC), based on a track-repeating technique. We validated the FDC algorithm for 23 patients, which included 7 brain, 6 head-and-neck, 5 lung, 1 spine, 1 pelvis and 3 prostate cases. For validation, we compared FDC-generated dose distributions with those from a full-fledged Monte Carlo based on GEANT4 (G4). We compared dose-volume-histograms, 3D-gamma-indices and analyzed a series of dosimetric indices. More than 99% of the voxels in the voxelized phantoms describing the patients have a gamma-index smaller than unity for the 2%/2 mm criteria. In addition the difference relative to the prescribed dose between the dosimetric indices calculated with FDC and G4 is less than 1%. FDC reduces the calculation times from 5 ms per proton to around 5 μs.

Elbow-specific clinical rating systems: extent of established validity, reliability, and responsiveness.

PubMed

The, Bertram; Reininga, Inge H F; El Moumni, Mostafa; Eygendaal, Denise

2013-10-01

The modern standard of evaluating treatment results includes the use of rating systems. Elbow-specific rating systems are frequently used in studies aiming at elbow-specific pathology. However, proper validation studies seem to be relatively sparse. In addition, these scoring systems might not always be used for appropriate populations of interest. Both of these issues might give rise to invalid conclusions being reported in the literature. Our aim was to investigate the extent to which the available elbow-specific outcome measurement tools have been validated and the quality of the validation itself. We also aimed to provide characteristics of the populations used for validation of these scales to enable clinicians to use them appropriately. A literature search identified 17 studies of 12 different elbow-specific scoring systems. These were assessed for validity, reliability, and responsiveness characteristics. The quality of these assessments was rated according to the Consensus Based Standards for the Selection of Health Measurement Instruments (COSMIN) checklist criteria, a standardized and validated tool developed specifically for this purpose. Currently, the only elbow-specific rating system that is validated using high-quality methodology is the Oxford Elbow Score, a patient-administered outcome measure tool that has been validated on heterogeneous study populations. Other rating systems still have to be proven in the future to be as good as the Oxford Elbow Score for clinical or research purposes. Additional validation studies are needed. Copyright © 2013 Journal of Shoulder and Elbow Surgery Board of Trustees. Published by Mosby, Inc. All rights reserved.

Validation of a new mortality risk prediction model for people 65 years and older in northwest Russia: The Crystal risk score.

PubMed

Turusheva, Anna; Frolova, Elena; Bert, Vaes; Hegendoerfer, Eralda; Degryse, Jean-Marie

2017-07-01

Prediction models help to make decisions about further management in clinical practice. This study aims to develop a mortality risk score based on previously identified risk predictors and to perform internal and external validations. In a population-based prospective cohort study of 611 community-dwelling individuals aged 65+ in St. Petersburg (Russia), all-cause mortality risks over 2.5 years follow-up were determined based on the results obtained from anthropometry, medical history, physical performance tests, spirometry and laboratory tests. C-statistic, risk reclassification analysis, integrated discrimination improvement analysis, decision curves analysis, internal validation and external validation were performed. Older adults were at higher risk for mortality [HR (95%CI)=4.54 (3.73-5.52)] when two or more of the following components were present: poor physical performance, low muscle mass, poor lung function, and anemia. If anemia was combined with high C-reactive protein (CRP) and high B-type natriuretic peptide (BNP) was added the HR (95%CI) was slightly higher (5.81 (4.73-7.14)) even after adjusting for age, sex and comorbidities. Our models were validated in an external population of adults 80+. The extended model had a better predictive capacity for cardiovascular mortality [HR (95%CI)=5.05 (2.23-11.44)] compared to the baseline model [HR (95%CI)=2.17 (1.18-4.00)] in the external population. We developed and validated a new risk prediction score that may be used to identify older adults at higher risk for mortality in Russia. Additional studies need to determine which targeted interventions improve the outcomes of these at-risk individuals. Copyright © 2017 Elsevier B.V. All rights reserved.

Test–retest reliability and validity of a web-based food-frequency questionnaire for adolescents aged 13–14 to be used in the Norwegian Mother and Child Cohort Study (MoBa)

PubMed Central

Øverby, Nina Cecilie; Johannesen, Elisabeth; Jensen, Grete; Skjaevesland, Anne-Kirsti; Haugen, Margaretha

2014-01-01

Background The assessment of food intake is challenging and prone to errors; it is therefore important to consider the reliability and validity of the assessment methods. Objective The aim of this study was to analyze the reproducibility and validity of a developed food-frequency questionnaire (FFQ) for use among adolescents. Design In total, 58 students (aged 13–14) from four different schools in the southern part of Norway participated in the reproducibility study of filling out the FFQ 4 weeks apart. In addition, 93 students participated in the relative validity study where the FFQ was compared to 2×24-hour dietary recalls, while 92 students participated in the absolute validity study where the intakes of fatty acids and vitamin D from the FFQ were compared to fatty acids and 25-hydroxy-vitamin D3 in whole blood. Results The median Spearman correlation coefficient for all nutrients in the test–retest reliability study was 0.57. The median Spearman correlation for all nutrients in the relative validity study was 0.26, while the correlations coefficients were low in the absolute validity study with n-3 fatty acid coefficients ranging from 0.05 to 0.25, and absent for vitamin D (r=0.000). Conclusion The test–retest reproducibility was considered good, the relative validity was considered poor to good, and the absolute validity was considered poor. However, the results are comparable to other studies among adolescents. PMID:25371661

Reliability and Validity of the Alberta Context Tool (ACT) with Professional Nurses: Findings from a Multi-Study Analysis

PubMed Central

Squires, Janet E.; Hayduk, Leslie; Hutchinson, Alison M.; Mallick, Ranjeeta; Norton, Peter G.; Cummings, Greta G.; Estabrooks, Carole A.

2015-01-01

Although organizational context is central to evidence-based practice, underdeveloped measurement hindersitsassessment. The Alberta Context Tool, comprised of 59 items that tap10 modifiable contextual concepts, was developed to address this gap. The purpose of this study to examine the reliability and validity of scores obtained when the Alberta Context Tool is completed by professional nurses across different healthcare settings. Five separate studies (N = 2361 nurses across different care settings) comprised the study sample. Reliability and validity were assessed. Cronbach’s alpha exceeded 0.70 for9/10 Alberta Context Tool concepts. Item-total correlations exceeded acceptable standards for 56/59items. Confirmatory Factor Analysescoordinated acceptably with the Alberta Context Tool’s proposed latent structure. The mean values for each Alberta Context Tool concept increased from low to high levels of research utilization(as hypothesized) further supporting its validity. This study provides robust evidence forreliability and validity of scores obtained with the Alberta Context Tool when administered to professional nurses. PMID:26098857

Issues in developing valid assessments of speech pathology students' performance in the workplace.

PubMed

McAllister, Sue; Lincoln, Michelle; Ferguson, Alison; McAllister, Lindy

2010-01-01

Workplace-based learning is a critical component of professional preparation in speech pathology. A validated assessment of this learning is seen to be 'the gold standard', but it is difficult to develop because of design and validation issues. These issues include the role and nature of judgement in assessment, challenges in measuring quality, and the relationship between assessment and learning. Valid assessment of workplace-based performance needs to capture the development of competence over time and account for both occupation specific and generic competencies. This paper reviews important conceptual issues in the design of valid and reliable workplace-based assessments of competence including assessment content, process, impact on learning, measurement issues, and validation strategies. It then goes on to share what has been learned about quality assessment and validation of a workplace-based performance assessment using competency-based ratings. The outcomes of a four-year national development and validation of an assessment tool are described. A literature review of issues in conceptualizing, designing, and validating workplace-based assessments was conducted. Key factors to consider in the design of a new tool were identified and built into the cycle of design, trialling, and data analysis in the validation stages of the development process. This paper provides an accessible overview of factors to consider in the design and validation of workplace-based assessment tools. It presents strategies used in the development and national validation of a tool COMPASS, used in an every speech pathology programme in Australia, New Zealand, and Singapore. The paper also describes Rasch analysis, a model-based statistical approach which is useful for establishing validity and reliability of assessment tools. Through careful attention to conceptual and design issues in the development and trialling of workplace-based assessments, it has been possible to develop the world's first valid and reliable national assessment tool for the assessment of performance in speech pathology.

Reliability and validity of a Tutorial Group Effectiveness Instrument.

PubMed

Singaram, Veena S; Van Der Vleuten, Cees P M; Van Berkel, Henk; Dolmans, Diana H J M

2010-01-01

Tutorial group effectiveness is essential for the success of learning in problem-based learning (PBL). Less effective and dysfunctional groups compromise the quality of students learning in PBL. This article aims to report on the reliability and validity of an instrument aimed at measuring tutorial group effectiveness in PBL. The items within the instrument are clustered around motivational and cognitive factors based on Slavin's theoretical framework. A confirmatory factor analysis (CFA) was carried out to estimate the validity of the instrument. Furthermore, generalizability studies were conducted and alpha coefficients were computed to determine the reliability and homogeneity of each factor. The CFA indicated that a three-factor model comprising 19 items showed a good fit with the data. Alpha coefficients per factor were high. The findings of the generalizability studies indicated that at least 9-10 student responses are needed in order to obtain reliable data at the tutorial group level. The instrument validated in this study has the potential to provide faculty and students with diagnostic information and feedback about student behaviors that enhance and hinder tutorial group effectiveness.

Impact of Learning Model Based on Cognitive Conflict toward Student’s Conceptual Understanding

NASA Astrophysics Data System (ADS)

Mufit, F.; Festiyed, F.; Fauzan, A.; Lufri, L.

2018-04-01

The problems that often occur in the learning of physics is a matter of misconception and low understanding of the concept. Misconceptions do not only happen to students, but also happen to college students and teachers. The existing learning model has not had much impact on improving conceptual understanding and remedial efforts of student misconception. This study aims to see the impact of cognitive-based learning model in improving conceptual understanding and remediating student misconceptions. The research method used is Design / Develop Research. The product developed is a cognitive conflict-based learning model along with its components. This article reports on product design results, validity tests, and practicality test. The study resulted in the design of cognitive conflict-based learning model with 4 learning syntaxes, namely (1) preconception activation, (2) presentation of cognitive conflict, (3) discovery of concepts & equations, (4) Reflection. The results of validity tests by some experts on aspects of content, didactic, appearance or language, indicate very valid criteria. Product trial results also show a very practical product to use. Based on pretest and posttest results, cognitive conflict-based learning models have a good impact on improving conceptual understanding and remediating misconceptions, especially in high-ability students.

Validity of "Hi_Science" as instructional media based-android refer to experiential learning model

NASA Astrophysics Data System (ADS)

Qamariah, Jumadi, Senam, Wilujeng, Insih

2017-08-01

Hi_Science is instructional media based-android in learning science on material environmental pollution and global warming. This study is aimed: (a) to show the display of Hi_Science that will be applied in Junior High School, and (b) to describe the validity of Hi_Science. Hi_Science as instructional media created with colaboration of innovative learning model and development of technology at the current time. Learning media selected is based-android and collaborated with experiential learning model as an innovative learning model. Hi_Science had adapted student worksheet by Taufiq (2015). Student worksheet had very good category by two expert lecturers and two science teachers (Taufik, 2015). This student worksheet is refined and redeveloped in android as an instructional media which can be used by students for learning science not only in the classroom, but also at home. Therefore, student worksheet which has become instructional media based-android must be validated again. Hi_Science has been validated by two experts. The validation is based on assessment of meterials aspects and media aspects. The data collection was done by media assessment instrument. The result showed the assessment of material aspects has obtained the average value 4,72 with percentage of agreement 96,47%, that means Hi_Science on the material aspects is in excellent category or very valid category. The assessment of media aspects has obtained the average value 4,53 with percentage of agreement 98,70%, that means Hi_Science on the media aspects is in excellent category or very valid category. It was concluded that Hi_Science as instructional media can be applied in the junior high school.

Development and validation of a registry-based definition of eosinophilic esophagitis in Denmark

PubMed Central

Dellon, Evan S; Erichsen, Rune; Pedersen, Lars; Shaheen, Nicholas J; Baron, John A; Sørensen, Henrik T; Vyberg, Mogens

2013-01-01

AIM: To develop and validate a case definition of eosinophilic esophagitis (EoE) in the linked Danish health registries. METHODS: For case definition development, we queried the Danish medical registries from 2006-2007 to identify candidate cases of EoE in Northern Denmark. All International Classification of Diseases-10 (ICD-10) and prescription codes were obtained, and archived pathology slides were obtained and re-reviewed to determine case status. We used an iterative process to select inclusion/exclusion codes, refine the case definition, and optimize sensitivity and specificity. We then re-queried the registries from 2008-2009 to yield a validation set. The case definition algorithm was applied, and sensitivity and specificity were calculated. RESULTS: Of the 51 and 49 candidate cases identified in both the development and validation sets, 21 and 24 had EoE, respectively. Characteristics of EoE cases in the development set [mean age 35 years; 76% male; 86% dysphagia; 103 eosinophils per high-power field (eos/hpf)] were similar to those in the validation set (mean age 42 years; 83% male; 67% dysphagia; 77 eos/hpf). Re-review of archived slides confirmed that the pathology coding for esophageal eosinophilia was correct in greater than 90% of cases. Two registry-based case algorithms based on pathology, ICD-10, and pharmacy codes were successfully generated in the development set, one that was sensitive (90%) and one that was specific (97%). When these algorithms were applied to the validation set, they remained sensitive (88%) and specific (96%). CONCLUSION: Two registry-based definitions, one highly sensitive and one highly specific, were developed and validated for the linked Danish national health databases, making future population-based studies feasible. PMID:23382628

Validated Questionnaire of Maternal Attitude and Knowledge for Predicting Caries Risk in Children: Epidemiological Study in North Jakarta, Indonesia.

PubMed

Laksmiastuti, Sri Ratna; Budiardjo, Sarworini Bagio; Sutadi, Heriandi

2017-06-01

Predicting caries risk in children can be done by identifying caries risk factors. It is an important measure which contributes to best understanding of the cariogenic profile of the patient. Identification could be done by clinical examination and answering the questionnaire. We arrange the study to verify the questionnaire validation for predicting caries risk in children. The study was conducted on 62 pairs of mothers and their children, aged between 3 and 5 years. The questionnaire consists of 10 questions concerning mothers' attitude and knowledge about oral health. The reliability and validity test is based on Cronbach's alpha and correlation coefficient value. All question are reliable (Cronbach's alpha = 0.873) and valid (Corrected item-total item correlation >0.4). Five questionnaires of mother's attitude about oral health and five questionnaires of mother's knowledge about oral health are reliable and valid for predicting caries risk in children.

The use of video clips in teleconsultation for preschool children with movement disorders.

PubMed

Gorter, Hetty; Lucas, Cees; Groothuis-Oudshoorn, Karin; Maathuis, Carel; van Wijlen-Hempel, Rietje; Elvers, Hans

2013-01-01

To investigate the reliability and validity of video clips in assessing movement disorders in preschool children. The study group included 27 children with neuromotor concerns. The explorative validity group included children with motor problems (n = 21) or with typical development (n = 9). Hempel screening was used for live observation of the child, full recording, and short video clips. The explorative study tested the validity of the clinical classifications "typical" or "suspect." Agreement between live observation and the full recording was almost perfect; Agreement for the clinical classification "typical" or "suspect" was substantial. Agreement between the full recording and short video clips was substantial to moderate. The explorative validity study, based on short video clips and the presence of a neuromotor developmental disorder, showed substantial agreement. Hempel screening enables reliable and valid observation of video clips, but further research is necessary to demonstrate the predictive value.

A validation method for near-infrared spectroscopy based tissue oximeters for cerebral and somatic tissue oxygen saturation measurements.

PubMed

Benni, Paul B; MacLeod, David; Ikeda, Keita; Lin, Hung-Mo

2018-04-01

We describe the validation methodology for the NIRS based FORE-SIGHT ELITE ® (CAS Medical Systems, Inc., Branford, CT, USA) tissue oximeter for cerebral and somatic tissue oxygen saturation (StO 2 ) measurements for adult subjects submitted to the United States Food and Drug Administration (FDA) to obtain clearance for clinical use. This validation methodology evolved from a history of NIRS validations in the literature and FDA recommended use of Deming regression and bootstrapping statistical validation methods. For cerebral validation, forehead cerebral StO 2 measurements were compared to a weighted 70:30 reference (REF CX B ) of co-oximeter internal jugular venous and arterial blood saturation of healthy adult subjects during a controlled hypoxia sequence, with a sensor placed on the forehead. For somatic validation, somatic StO 2 measurements were compared to a weighted 70:30 reference (REF CX S ) of co-oximetry central venous and arterial saturation values following a similar protocol, with sensors place on the flank, quadriceps muscle, and calf muscle. With informed consent, 25 subjects successfully completed the cerebral validation study. The bias and precision (1 SD) of cerebral StO 2 compared to REF CX B was -0.14 ± 3.07%. With informed consent, 24 subjects successfully completed the somatic validation study. The bias and precision of somatic StO 2 compared to REF CX S was 0.04 ± 4.22% from the average of flank, quadriceps, and calf StO 2 measurements to best represent the global whole body REF CX S . The NIRS validation methods presented potentially provide a reliable means to test NIRS monitors and qualify them for clinical use.

Development and community-based validation of the IDEA study Instrumental Activities of Daily Living (IDEA-IADL) questionnaire

PubMed Central

Collingwood, Cecilia; Paddick, Stella-Maria; Kisoli, Aloyce; Dotchin, Catherine L.; Gray, William K.; Mbowe, Godfrey; Mkenda, Sarah; Urasa, Sarah; Mushi, Declare; Chaote, Paul; Walker, Richard W.

2014-01-01

Background The dementia diagnosis gap in sub-Saharan Africa (SSA) is large, partly due to difficulties in assessing function, an essential step in diagnosis. Objectives As part of the Identification and Intervention for Dementia in Elderly Africans (IDEA) study, to develop, pilot, and validate an Instrumental Activities of Daily Living (IADL) questionnaire for use in a rural Tanzanian population to assist in the identification of people with dementia alongside cognitive screening. Design The questionnaire was developed at a workshop for rural primary healthcare workers, based on culturally appropriate roles and usual activities of elderly people in this community. It was piloted in 52 individuals under follow-up from a dementia prevalence study. Validation subsequently took place during a community dementia-screening programme. Construct validation against gold standard clinical dementia diagnosis using DSM-IV criteria was carried out on a stratified sample of the cohort and validity assessed using area under the receiver operating characteristic (AUROC) curve analysis. Results An 11-item questionnaire (IDEA-IADL) was developed after pilot testing. During formal validation on 130 community-dwelling elderly people who presented for screening, the AUROC curve was 0.896 for DSM-IV dementia when used in isolation and 0.937 when used in conjunction with the IDEA cognitive screen, previously validated in Tanzania. The internal consistency was 0.959. Performance on the IDEA-IADL was not biased with regard to age, gender or education level. Conclusions The IDEA-IADL questionnaire appears to be a useful aid to dementia screening in this setting. Further validation in other healthcare settings in SSA is required. PMID:25537940

Continual Response Measurement: Design and Validation.

ERIC Educational Resources Information Center

Baggaley, Jon

1987-01-01

Discusses reliability and validity of continual response measurement (CRM), a computer-based measurement technique, and its use in social science research. Highlights include the importance of criterion-referencing the data, guidelines for designing studies using CRM, examples typifying their deductive and inductive functions, and a discussion of…

Validation of a short measure of effort-reward imbalance in the workplace: evidence from China.

PubMed

Li, Jian; Loerbroks, Adrian; Shang, Li; Wege, Natalia; Wahrendorf, Morten; Siegrist, Johannes

2012-01-01

Work stress is an emergent risk in occupational health in China, and its measurement is still a critical issue. The aim of this study was to examine the reliability and validity of a short version of the effort-reward imbalance (ERI) questionnaire in a sample of Chinese workers. A community-based survey was conducted in 1,916 subjects aged 30-65 years with paid employment (971 men and 945 women). Acceptable internal consistencies of the three scales, effort, reward and overcommitment, were obtained. Confirmatory factor analysis showed a good model fit of the data with the theoretical structure (goodness-of-fit index = 0.95). Evidence of criterion validity was demonstrated, as all three scales were independently associated with elevated odds ratios of both poor physical and mental health. Based on the findings of our study, this short version of the ERI questionnaire is considered to be a reliable and valid tool for measuring psychosocial work environment in Chinese working populations.

A multiple-group measurement scale for interprofessional collaboration: Adaptation and validation into Italian and German languages.

PubMed

Vittadello, Fabio; Mischo-Kelling, Maria; Wieser, Heike; Cavada, Luisa; Lochner, Lukas; Naletto, Carla; Fink, Verena; Reeves, Scott

2018-05-01

This article presents a study that aimed to validate a translation of a multiple-group measurement scale for interprofessional collaboration (IPC). We used survey data gathered over a three month period as part of a mixed methods study that explored the nature of IPC in Northern Italy. Following a translation from English into Italian and German the survey was distributed online to over 5,000 health professionals (dieticians, nurses, occupational therapists, physicians, physiotherapists, speech therapists and psychologists) based in one regional health trust. In total, 2,238 different health professions completed the survey. Based on the original scale, three principal components were extracted and confirmed as relevant factors for IPC (communication, accommodation and isolation). A confirmatory analysis (3-factor model) was applied to the data of physicians and nurses by language group. In conclusion, the validation of the German and Italian IPC scale has provided an instrument of acceptable reliability and validity for the assessment of IPC involving physicians and nurses.

Health risk behaviors in adolescents and emerging adults with congenital heart disease: psychometric properties of the Health Behavior Scale-Congenital Heart Disease.

PubMed

Goossens, Eva; Luyckx, Koen; Mommen, Nele; Gewillig, Marc; Budts, Werner; Zupancic, Nele; Moons, Philip

2013-12-01

To optimize long-term outcomes, patients with congenital heart disease (CHD) should adopt health-promoting behaviors. Studies on health behavior in afflicted patients are scarce and comparability of study results is limited. To enlarge the body of evidence, we have developed the Health Behavior Scale-Congenital Heart Disease (HBS-CHD). We examined the psychometric properties of the HBS-CHD by providing evidence for (a) the content validity; (b) validity based on the relationships with other variables; (c) reliability in terms of stability; and (d) responsiveness. Ten experts rated the relevance of the HBS-CHD items. The item content validity index (I-CVI) and the averaged scale content validity index (S-CVI/Ave); the modified multi-rater Kappa and proportion of missing values for each question were calculated. Relationships with other variables were evaluated using six hypotheses that were tested in 429 adolescents with CHD. Stability of the instrument was assessed using Heise's method; and responsiveness was tested by calculating the Guyatt's Responsiveness Index (GRI). Overall, 86.3% of the items had a good to excellent content validity; the S-CVI/Ave (0.81) and multi-rater Kappa (0.78) were adequate. The average proportion of missing values was low (1.2%). Because five out of six hypotheses were confirmed, evidence for the validity of the HBS-CHD based on relationships with other variables was provided. The stability of the instrument could not be confirmed based on our data. The GRI showed good to excellent capacity of the HBS-CHD to detect clinical changes in the health behavior over time. We found that the HBS-CHD is a valid and responsive questionnaire to assess health behaviors in patients with CHD.

Validating Pseudo-dynamic Source Models against Observed Ground Motion Data at the SCEC Broadband Platform, Ver 16.5

NASA Astrophysics Data System (ADS)

Song, S. G.

2016-12-01

Simulation-based ground motion prediction approaches have several benefits over empirical ground motion prediction equations (GMPEs). For instance, full 3-component waveforms can be produced and site-specific hazard analysis is also possible. However, it is important to validate them against observed ground motion data to confirm their efficiency and validity before practical uses. There have been community efforts for these purposes, which are supported by the Broadband Platform (BBP) project at the Southern California Earthquake Center (SCEC). In the simulation-based ground motion prediction approaches, it is a critical element to prepare a possible range of scenario rupture models. I developed a pseudo-dynamic source model for Mw 6.5-7.0 by analyzing a number of dynamic rupture models, based on 1-point and 2-point statistics of earthquake source parameters (Song et al. 2014; Song 2016). In this study, the developed pseudo-dynamic source models were tested against observed ground motion data at the SCEC BBP, Ver 16.5. The validation was performed at two stages. At the first stage, simulated ground motions were validated against observed ground motion data for past events such as the 1992 Landers and 1994 Northridge, California, earthquakes. At the second stage, they were validated against the latest version of empirical GMPEs, i.e., NGA-West2. The validation results show that the simulated ground motions produce ground motion intensities compatible with observed ground motion data at both stages. The compatibility of the pseudo-dynamic source models with the omega-square spectral decay and the standard deviation of the simulated ground motion intensities are also discussed in the study

Sensor data validation and reconstruction. Phase 1: System architecture study

NASA Technical Reports Server (NTRS)

1991-01-01

The sensor validation and data reconstruction task reviewed relevant literature and selected applicable validation and reconstruction techniques for further study; analyzed the selected techniques and emphasized those which could be used for both validation and reconstruction; analyzed Space Shuttle Main Engine (SSME) hot fire test data to determine statistical and physical relationships between various parameters; developed statistical and empirical correlations between parameters to perform validation and reconstruction tasks, using a computer aided engineering (CAE) package; and conceptually designed an expert system based knowledge fusion tool, which allows the user to relate diverse types of information when validating sensor data. The host hardware for the system is intended to be a Sun SPARCstation, but could be any RISC workstation with a UNIX operating system and a windowing/graphics system such as Motif or Dataviews. The information fusion tool is intended to be developed using the NEXPERT Object expert system shell, and the C programming language.

Development and validation of the Alcohol Myopia Scale.

PubMed

Lac, Andrew; Berger, Dale E

2013-09-01

Alcohol myopia theory conceptualizes the ability of alcohol to narrow attention and how this demand on mental resources produces the impairments of self-inflation, relief, and excess. The current research was designed to develop and validate a scale based on this framework. People who were alcohol users rated items representing myopic experiences arising from drinking episodes in the past month. In Study 1 (N = 260), the preliminary 3-factor structure was supported by exploratory factor analysis. In Study 2 (N = 289), the 3-factor structure was substantiated with confirmatory factor analysis, and it was superior in fit to an empirically indefensible 1-factor structure. The final 14-item scale was evaluated with internal consistency reliability, discriminant validity, convergent validity, criterion validity, and incremental validity. The alcohol myopia scale (AMS) illuminates conceptual underpinnings of this theory and yields insights for understanding the tunnel vision that arises from intoxication.

Earlinet validation of CATS L2 product

NASA Astrophysics Data System (ADS)

Proestakis, Emmanouil; Amiridis, Vassilis; Kottas, Michael; Marinou, Eleni; Binietoglou, Ioannis; Ansmann, Albert; Wandinger, Ulla; Yorks, John; Nowottnick, Edward; Makhmudov, Abduvosit; Papayannis, Alexandros; Pietruczuk, Aleksander; Gialitaki, Anna; Apituley, Arnoud; Muñoz-Porcar, Constantino; Bortoli, Daniele; Dionisi, Davide; Althausen, Dietrich; Mamali, Dimitra; Balis, Dimitris; Nicolae, Doina; Tetoni, Eleni; Luigi Liberti, Gian; Baars, Holger; Stachlewska, Iwona S.; Voudouri, Kalliopi-Artemis; Mona, Lucia; Mylonaki, Maria; Rita Perrone, Maria; João Costa, Maria; Sicard, Michael; Papagiannopoulos, Nikolaos; Siomos, Nikolaos; Burlizzi, Pasquale; Engelmann, Ronny; Abdullaev, Sabur F.; Hofer, Julian; Pappalardo, Gelsomina

2018-04-01

The Cloud-Aerosol Transport System (CATS) onboard the International Space Station (ISS), is a lidar system providing vertically resolved aerosol and cloud profiles since February 2015. In this study, the CATS aerosol product is validated against the aerosol profiles provided by the European Aerosol Research Lidar Network (EARLINET). This validation activity is based on collocated CATS-EARLINET measurements and the comparison of the particle backscatter coefficient at 1064nm.

Using Office Discipline Referral Data for Decision Making about Student Behavior in Elementary and Middle Schools: An Empirical Evaluation of Validity

ERIC Educational Resources Information Center

Irvin, Larry K.; Horner, Robert H.; Ingram, Kimberly; Todd, Anne W.; Sugai, George; Sampson, Nadia Katul; Boland, Joseph B.

2006-01-01

In this evaluation we used Messick's construct validity as a conceptual framework for an empirical study assessing the validity of use, utility, and impact of office discipline referral (ODR) measures for data-based decision making about student behavior in schools. The Messick approach provided a rubric for testing the fit of our theory of use of…

Development of a proficiency-based virtual reality simulation training curriculum for laparoscopic appendicectomy.

PubMed

Sirimanna, Pramudith; Gladman, Marc A

2017-10-01

Proficiency-based virtual reality (VR) training curricula improve intraoperative performance, but have not been developed for laparoscopic appendicectomy (LA). This study aimed to develop an evidence-based training curriculum for LA. A total of 10 experienced (>50 LAs), eight intermediate (10-30 LAs) and 20 inexperienced (<10 LAs) operators performed guided and unguided LA tasks on a high-fidelity VR simulator using internationally relevant techniques. The ability to differentiate levels of experience (construct validity) was measured using simulator-derived metrics. Learning curves were analysed. Proficiency benchmarks were defined by the performance of the experienced group. Intermediate and experienced participants completed a questionnaire to evaluate the realism (face validity) and relevance (content validity). Of 18 surgeons, 16 (89%) considered the VR model to be visually realistic and 17 (95%) believed that it was representative of actual practice. All 'guided' modules demonstrated construct validity (P < 0.05), with learning curves that plateaued between sessions 6 and 9 (P < 0.01). When comparing inexperienced to intermediates to experienced, the 'unguided' LA module demonstrated construct validity for economy of motion (5.00 versus 7.17 versus 7.84, respectively; P < 0.01) and task time (864.5 s versus 477.2 s versus 352.1 s, respectively, P < 0.01). Construct validity was also confirmed for number of movements, path length and idle time. Validated modules were used for curriculum construction, with proficiency benchmarks used as performance goals. A VR LA model was realistic and representative of actual practice and was validated as a training and assessment tool. Consequently, the first evidence-based internationally applicable training curriculum for LA was constructed, which facilitates skill acquisition to proficiency. © 2017 Royal Australasian College of Surgeons.

Reliability and validity of a novel Kinect-based software program for measuring posture, balance and side-bending.

PubMed

Grooten, Wilhelmus Johannes Andreas; Sandberg, Lisa; Ressman, John; Diamantoglou, Nicolas; Johansson, Elin; Rasmussen-Barr, Eva

2018-01-08

Clinical examinations are subjective and often show a low validity and reliability. Objective and highly reliable quantitative assessments are available in laboratory settings using 3D motion analysis, but these systems are too expensive to use for simple clinical examinations. Qinematic™ is an interactive movement analyses system based on the Kinect camera and is an easy-to-use clinical measurement system for assessing posture, balance and side-bending. The aim of the study was to test the test-retest the reliability and construct validity of Qinematic™ in a healthy population, and to calculate the minimal clinical differences for the variables of interest. A further aim was to identify the discriminative validity of Qinematic™ in people with low-back pain (LBP). We performed a test-retest reliability study (n = 37) with around 1 week between the occasions, a construct validity study (n = 30) in which Qinematic™ was tested against a 3D motion capture system, and a discriminative validity study, in which a group of people with LBP (n = 20) was compared to healthy controls (n = 17). We tested a large range of psychometric properties of 18 variables in three sections: posture (head and pelvic position, weight distribution), balance (sway area and velocity in single- and double-leg stance), and side-bending. The majority of the variables in the posture and balance sections, showed poor/fair reliability (ICC < 0.4) and poor/fair validity (Spearman <0.4), with significant differences between occasions, between Qinematic™ and the 3D-motion capture system. In the clinical study, Qinematic™ did not differ between people with LPB and healthy for these variables. For one variable, side-bending to the left, there was excellent reliability (ICC =0.898), excellent validity (r = 0.943), and Qinematic™ could differentiate between LPB and healthy individuals (p = 0.012). This paper shows that a novel software program (Qinematic™) based on the Kinect camera for measuring balance, posture and side-bending has poor psychometric properties, indicating that the variables on balance and posture should not be used for monitoring individual changes over time or in research. Future research on the dynamic tasks of Qinematic™ is warranted.

The development of learning materials based on core model to improve students’ learning outcomes in topic of Chemical Bonding

NASA Astrophysics Data System (ADS)

Avianti, R.; Suyatno; Sugiarto, B.

2018-04-01

This study aims to create an appropriate learning material based on CORE (Connecting, Organizing, Reflecting, Extending) model to improve students’ learning achievement in Chemical Bonding Topic. This study used 4-D models as research design and one group pretest-posttest as design of the material treatment. The subject of the study was teaching materials based on CORE model, conducted on 30 students of Science class grade 10. The collecting data process involved some techniques such as validation, observation, test, and questionnaire. The findings were that: (1) all the contents were valid, (2) the practicality and the effectiveness of all the contents were good. The conclusion of this research was that the CORE model is appropriate to improve students’ learning outcomes for studying Chemical Bonding.

[Measurement properties of self-report questionnaires published in Korean nursing journals].

PubMed

Lee, Eun-Hyun; Kim, Chun-Ja; Kim, Eun Jung; Chae, Hyun-Ju; Cho, Soo-Yeon

2013-02-01

The purpose of this study was to evaluate measurement properties of self-report questionnaires for studies published in Korean nursing journals. Of 424 Korean nursing articles initially identified, 168 articles met the inclusion criteria. The methodological quality of the measurements used in the studies and interpretability were assessed using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. It consists of items on internal consistency, reliability, measurement error, content validity, construct validity including structural validity, hypothesis testing, cross-cultural validity, and criterion validity, and responsiveness. For each item of the COSMIN checklist, measurement properties are rated on a four-point scale: excellent, good, fair, and poor. Each measurement property is scored with worst score counts. All articles used the classical test theory for measurement properties. Internal consistency (72.6%), construct validity (56.5%), and content validity (38.2%) were most frequently reported properties being rated as 'excellent' by COSMIN checklist, whereas other measurement properties were rarely reported. A systematic review of measurement properties including interpretability of most instruments warrants further research and nursing-focused checklists assessing measurement properties should be developed to facilitate intervention outcomes across Korean studies.

The development of learning material using learning cycle 5E model based stem to improve students’ learning outcomes in Thermochemistry

NASA Astrophysics Data System (ADS)

sugiarti, A. C.; suyatno, S.; Sanjaya, I. G. M.

2018-04-01

The objective of this study is describing the feasibility of Learning Cycle 5E STEM (Science, Technology, Engineering, and Mathematics) based learning material which is appropriate to improve students’ learning achievement in Thermochemistry. The study design used 4-D models and one group pretest-posttest design to obtain the information about the improvement of sudents’ learning outcomes. The subject was learning cycle 5E based STEM learning materials which the data were collected from 30 students of Science class at 11th Grade. The techniques used in this study were validation, observation, test, and questionnaire. Some result attain: (1) all the learning materials contents were valid, (2) the practicality and the effectiveness of all the learning materials contents were classified as good. The conclution of this study based on those three condition, the Learnig Cycle 5E based STEM learning materials is appropriate to improve students’ learning outcomes in studying Thermochemistry.

Objective assessment based on motion-related metrics and technical performance in laparoscopic suturing.

PubMed

Sánchez-Margallo, Juan A; Sánchez-Margallo, Francisco M; Oropesa, Ignacio; Enciso, Silvia; Gómez, Enrique J

2017-02-01

The aim of this study is to present the construct and concurrent validity of a motion-tracking method of laparoscopic instruments based on an optical pose tracker and determine its feasibility as an objective assessment tool of psychomotor skills during laparoscopic suturing. A group of novice ([Formula: see text] laparoscopic procedures), intermediate (11-100 laparoscopic procedures) and experienced ([Formula: see text] laparoscopic procedures) surgeons performed three intracorporeal sutures on an ex vivo porcine stomach. Motion analysis metrics were recorded using the proposed tracking method, which employs an optical pose tracker to determine the laparoscopic instruments' position. Construct validation was measured for all 10 metrics across the three groups and between pairs of groups. Concurrent validation was measured against a previously validated suturing checklist. Checklists were completed by two independent surgeons over blinded video recordings of the task. Eighteen novices, 15 intermediates and 11 experienced surgeons took part in this study. Execution time and path length travelled by the laparoscopic dissector presented construct validity. Experienced surgeons required significantly less time ([Formula: see text]), travelled less distance using both laparoscopic instruments ([Formula: see text]) and made more efficient use of the work space ([Formula: see text]) compared with novice and intermediate surgeons. Concurrent validation showed strong correlation between both the execution time and path length and the checklist score ([Formula: see text] and [Formula: see text], [Formula: see text]). The suturing performance was successfully assessed by the motion analysis method. Construct and concurrent validity of the motion-based assessment method has been demonstrated for the execution time and path length metrics. This study demonstrates the efficacy of the presented method for objective evaluation of psychomotor skills in laparoscopic suturing. However, this method does not take into account the quality of the suture. Thus, future works will focus on developing new methods combining motion analysis and qualitative outcome evaluation to provide a complete performance assessment to trainees.

Validation of the Vanderbilt Holistic Face Processing Test.

PubMed

Wang, Chao-Chih; Ross, David A; Gauthier, Isabel; Richler, Jennifer J

2016-01-01

The Vanderbilt Holistic Face Processing Test (VHPT-F) is a new measure of holistic face processing with better psychometric properties relative to prior measures developed for group studies (Richler et al., 2014). In fields where psychologists study individual differences, validation studies are commonplace and the concurrent validity of a new measure is established by comparing it to an older measure with established validity. We follow this approach and test whether the VHPT-F measures the same construct as the composite task, which is group-based measure at the center of the large literature on holistic face processing. In Experiment 1, we found a significant correlation between holistic processing measured in the VHPT-F and the composite task. Although this correlation was small, it was comparable to the correlation between holistic processing measured in the composite task with the same faces, but different target parts (top or bottom), which represents a reasonable upper limit for correlations between the composite task and another measure of holistic processing. These results confirm the validity of the VHPT-F by demonstrating shared variance with another measure of holistic processing based on the same operational definition. These results were replicated in Experiment 2, but only when the demographic profile of our sample matched that of Experiment 1.

Validation of the Vanderbilt Holistic Face Processing Test

PubMed Central

Wang, Chao-Chih; Ross, David A.; Gauthier, Isabel; Richler, Jennifer J.

2016-01-01

The Vanderbilt Holistic Face Processing Test (VHPT-F) is a new measure of holistic face processing with better psychometric properties relative to prior measures developed for group studies (Richler et al., 2014). In fields where psychologists study individual differences, validation studies are commonplace and the concurrent validity of a new measure is established by comparing it to an older measure with established validity. We follow this approach and test whether the VHPT-F measures the same construct as the composite task, which is group-based measure at the center of the large literature on holistic face processing. In Experiment 1, we found a significant correlation between holistic processing measured in the VHPT-F and the composite task. Although this correlation was small, it was comparable to the correlation between holistic processing measured in the composite task with the same faces, but different target parts (top or bottom), which represents a reasonable upper limit for correlations between the composite task and another measure of holistic processing. These results confirm the validity of the VHPT-F by demonstrating shared variance with another measure of holistic processing based on the same operational definition. These results were replicated in Experiment 2, but only when the demographic profile of our sample matched that of Experiment 1. PMID:27933014

School-based behavioral assessment tools are reliable and valid for measurement of fruit and vegetable intake, physical activity, and television viewing in young children.

PubMed

Economos, Christina D; Sacheck, Jennifer M; Kwan Ho Chui, Kenneth; Irizarry, Laura; Irizzary, Laura; Guillemont, Juliette; Collins, Jessica J; Hyatt, Raymond R

2008-04-01

Interventions aiming to modify the dietary and physical activity behaviors of young children require precise and accurate measurement tools. As part of a larger community-based project, three school-based questionnaires were developed to assess (a) fruit and vegetable intake, (b) physical activity and television (TV) viewing, and (c) perceived parental support for diet and physical activity. Test-retest reliability was performed on all questionnaires and validity was measured for fruit and vegetable intake, physical activity, and TV viewing. Eighty-four school children (8.3+/-1.1 years) were studied. Test-retest reliability was performed by administering questionnaires twice, 1 to 2 hours apart. Validity of the fruit and vegetable questionnaire was measured by direct observation, while the physical activity and TV questionnaire was validated by a parent phone interview. All three questionnaires yielded excellent test-retest reliability (P<0.001). The majority of fruit and vegetable questions and the questions regarding specific physical activities and TV viewing were valid. Low validity scores were found for questions on watching TV during breakfast or dinner. These questionnaires are reliable and valid tools to assess fruit and vegetable intake, physical activity, and TV viewing behaviors in early elementary school-aged children. Methods for assessment of children's TV viewing during meals should be further investigated because of parent-child discrepancies.

Exploring geo-tagged photos for land cover validation with deep learning

NASA Astrophysics Data System (ADS)

Xing, Hanfa; Meng, Yuan; Wang, Zixuan; Fan, Kaixuan; Hou, Dongyang

2018-07-01

Land cover validation plays an important role in the process of generating and distributing land cover thematic maps, which is usually implemented by high cost of sample interpretation with remotely sensed images or field survey. With an increasing availability of geo-tagged landscape photos, the automatic photo recognition methodologies, e.g., deep learning, can be effectively utilised for land cover applications. However, they have hardly been utilised in validation processes, as challenges remain in sample selection and classification for highly heterogeneous photos. This study proposed an approach to employ geo-tagged photos for land cover validation by using the deep learning technology. The approach first identified photos automatically based on the VGG-16 network. Then, samples for validation were selected and further classified by considering photos distribution and classification probabilities. The implementations were conducted for the validation of the GlobeLand30 land cover product in a heterogeneous area, western California. Experimental results represented promises in land cover validation, given that GlobeLand30 showed an overall accuracy of 83.80% with classified samples, which was close to the validation result of 80.45% based on visual interpretation. Additionally, the performances of deep learning based on ResNet-50 and AlexNet were also quantified, revealing no substantial differences in final validation results. The proposed approach ensures geo-tagged photo quality, and supports the sample classification strategy by considering photo distribution, with accuracy improvement from 72.07% to 79.33% compared with solely considering the single nearest photo. Consequently, the presented approach proves the feasibility of deep learning technology on land cover information identification of geo-tagged photos, and has a great potential to support and improve the efficiency of land cover validation.

Assessing South China (Guangzhou) High School Students' Views on Nature of Science: A Validation Study

ERIC Educational Resources Information Center

Deng, Feng; Chai, Ching Sing; Tsai, Chin-Chung; Lin, Tzung-Jin

2014-01-01

Research on students' views on nature of science (VNOS) in Asian countries such as China is notably lacking. This study aimed to develop and validate an instrument to measure South China high school students' VNOS. Based on the previously acquired qualitative data, the instrument included seven VNOS dimensions which reflect the crucial aspects of…

The Development, Validity and Reliability of TPACK-Deep: A Technological Pedagogical Content Knowledge Scale

ERIC Educational Resources Information Center

Yurdakul, Isil Kabakci; Odabasi, Hatice Ferhan; Kilicer, Kerem; Coklar, Ahmet Naci; Birinci, Gurkay; Kurt, Adile Askim

2012-01-01

The purpose of this study is to develop a TPACK (technological pedagogical content knowledge) scale based on the centered component of TPACK framework in order to measure preservice teachers' TPACK. A systematic and step-by-step approach was followed for the development of the scale. The validity and reliability studies of the scale were carried…

Testing the Validity of the Uses and Gratifications Measures: Do They Work?

ERIC Educational Resources Information Center

Sherer, Michael D.; Ervin, R. Ferrell

Based on data gathered in a larger study of audience response to the debates held during the 1976 campaign for the presidency of the United States, a study examined the validity of uses and gratifications measures and explored how these measures were affected by differing levels of audience media activity. The data was drawn from telephone…

Assessing motivation orientations in schizophrenia: Scale development and validation

PubMed Central

Cooper, Shanna; Lavaysse, Lindsey M.; Gard, David E.

2014-01-01

Motivation deficits are common in several disorders including schizophrenia, and are an important factor in both functioning and treatment adherence. Self-Determination Theory (SDT), a leading macro-theory of motivation, has contributed a number of insights into how motivation is impaired in schizophrenia. Nonetheless, self-report measures of motivation appropriate for people with severe mental illness (including those that emphasize SDT) are generally lacking in literature. To fill this gap, we adapted and abbreviated the well-validated General Causality Orientation Scale for use with people with schizophrenia and with other severe mental disorders (GCOS-clinical populations; GCOS-CP). In Study 1, we tested the similarity of our measure to the existing GCOS (using a college sample) and then validated this new measure in a schizophrenia and healthy control sample (Study 2). Results from Study 1 (N=360) indicated that the GCOS-CP was psychometrically similar to the original GCOS and provided good convergent and discriminant validity. In Study 2, the GCOS-CP was given to individuals with (N=44) and without schizophrenia (N=42). In line with both laboratory-based and observer-based research, people with schizophrenia showed lower motivational autonomy and higher impersonal/amotivated orientations. Additional applications of the GCOS-CP are discussed. PMID:25454115

Analysis of Flowfields over Four-Engine DC-X Rockets

NASA Technical Reports Server (NTRS)

Wang, Ten-See; Cornelison, Joni

1996-01-01

The objective of this study is to validate a computational methodology for the aerodynamic performance of an advanced conical launch vehicle configuration. The computational methodology is based on a three-dimensional, viscous flow, pressure-based computational fluid dynamics formulation. Both wind-tunnel and ascent flight-test data are used for validation. Emphasis is placed on multiple-engine power-on effects. Computational characterization of the base drag in the critical subsonic regime is the focus of the validation effort; until recently, almost no multiple-engine data existed for a conical launch vehicle configuration. Parametric studies using high-order difference schemes are performed for the cold-flow tests, whereas grid studies are conducted for the flight tests. The computed vehicle axial force coefficients, forebody, aftbody, and base surface pressures compare favorably with those of tests. The results demonstrate that with adequate grid density and proper distribution, a high-order difference scheme, finite rate afterburning kinetics to model the plume chemistry, and a suitable turbulence model to describe separated flows, plume/air mixing, and boundary layers, computational fluid dynamics is a tool that can be used to predict the low-speed aerodynamic performance for rocket design and operations.

Evaluation of a physically based quasi-linear and a conceptually based nonlinear Muskingum methods

NASA Astrophysics Data System (ADS)

Perumal, Muthiah; Tayfur, Gokmen; Rao, C. Madhusudana; Gurarslan, Gurhan

2017-03-01

Two variants of the Muskingum flood routing method formulated for accounting nonlinearity of the channel routing process are investigated in this study. These variant methods are: (1) The three-parameter conceptual Nonlinear Muskingum (NLM) method advocated by Gillin 1978, and (2) The Variable Parameter McCarthy-Muskingum (VPMM) method recently proposed by Perumal and Price in 2013. The VPMM method does not require rigorous calibration and validation procedures as required in the case of NLM method due to established relationships of its parameters with flow and channel characteristics based on hydrodynamic principles. The parameters of the conceptual nonlinear storage equation used in the NLM method were calibrated using the Artificial Intelligence Application (AIA) techniques, such as the Genetic Algorithm (GA), the Differential Evolution (DE), the Particle Swarm Optimization (PSO) and the Harmony Search (HS). The calibration was carried out on a given set of hypothetical flood events obtained by routing a given inflow hydrograph in a set of 40 km length prismatic channel reaches using the Saint-Venant (SV) equations. The validation of the calibrated NLM method was investigated using a different set of hypothetical flood hydrographs obtained in the same set of channel reaches used for calibration studies. Both the sets of solutions obtained in the calibration and validation cases using the NLM method were compared with the corresponding solutions of the VPMM method based on some pertinent evaluation measures. The results of the study reveal that the physically based VPMM method is capable of accounting for nonlinear characteristics of flood wave movement better than the conceptually based NLM method which requires the use of tedious calibration and validation procedures.

77 FR 31027 - Submission for OMB Review; Comment Request; Web-Based Assessment of the Clinical Studies Support...

Federal Register 2010, 2011, 2012, 2013, 2014

2012-05-24

...; Comment Request; Web-Based Assessment of the Clinical Studies Support Center (CSSC) Summary: Under the... current valid OMB control number. Proposed Collection: Title: Web-Based Assessment of the Clinical Studies... Operations and Procedures (MOP); coordinating meeting space and logistics for in-person meetings, Web...

Validity and Reliability Study of the Scale for Determining the Civic-Mindedness Levels of Teaching Staff

ERIC Educational Resources Information Center

Yesil, Rüstü

2017-01-01

The purpose of this study was to develop a valid and reliable scale that can be used in determining the civic-mindedness levels of teaching staff working at universities. The study group of the research consisted of 758 students, 256 of whom were male and 524 were female. The item list, which was based on the literature and expert opinions, was…

Opportunity integrated assessment facilitating critical thinking and science process skills measurement on acid base matter

NASA Astrophysics Data System (ADS)

Sari, Anggi Ristiyana Puspita; Suyanta, LFX, Endang Widjajanti; Rohaeti, Eli

2017-05-01

Recognizing the importance of the development of critical thinking and science process skills, the instrument should give attention to the characteristics of chemistry. Therefore, constructing an accurate instrument for measuring those skills is important. However, the integrated instrument assessment is limited in number. The purpose of this study is to validate an integrated assessment instrument for measuring students' critical thinking and science process skills on acid base matter. The development model of the test instrument adapted McIntire model. The sample consisted of 392 second grade high school students in the academic year of 2015/2016 in Yogyakarta. Exploratory Factor Analysis (EFA) was conducted to explore construct validity, whereas content validity was substantiated by Aiken's formula. The result shows that the KMO test is 0.714 which indicates sufficient items for each factor and the Bartlett test is significant (a significance value of less than 0.05). Furthermore, content validity coefficient which is based on 8 experts is obtained at 0.85. The findings support the integrated assessment instrument to measure critical thinking and science process skills on acid base matter.

Validation of Ocean Color Remote Sensing Reflectance Using Autonomous Floats

NASA Technical Reports Server (NTRS)

Gerbi, Gregory P.; Boss, Emanuel; Werdell, P. Jeremy; Proctor, Christopher W.; Haentjens, Nils; Lewis, Marlon R.; Brown, Keith; Sorrentino, Diego; Zaneveld, J. Ronald V.; Barnard, Andrew H.;

External Validation Study of First Trimester Obstetric Prediction Models (Expect Study I): Research Protocol and Population Characteristics.

PubMed

Meertens, Linda Jacqueline Elisabeth; Scheepers, Hubertina Cj; De Vries, Raymond G; Dirksen, Carmen D; Korstjens, Irene; Mulder, Antonius Lm; Nieuwenhuijze, Marianne J; Nijhuis, Jan G; Spaanderman, Marc Ea; Smits, Luc Jm

2017-10-26

A number of first-trimester prediction models addressing important obstetric outcomes have been published. However, most models have not been externally validated. External validation is essential before implementing a prediction model in clinical practice. The objective of this paper is to describe the design of a study to externally validate existing first trimester obstetric prediction models, based upon maternal characteristics and standard measurements (eg, blood pressure), for the risk of pre-eclampsia (PE), gestational diabetes mellitus (GDM), spontaneous preterm birth (PTB), small-for-gestational-age (SGA) infants, and large-for-gestational-age (LGA) infants among Dutch pregnant women (Expect Study I). The results of a pilot study on the feasibility and acceptability of the recruitment process and the comprehensibility of the Pregnancy Questionnaire 1 are also reported. A multicenter prospective cohort study was performed in The Netherlands between July 1, 2013 and December 31, 2015. First trimester obstetric prediction models were systematically selected from the literature. Predictor variables were measured by the Web-based Pregnancy Questionnaire 1 and pregnancy outcomes were established using the Postpartum Questionnaire 1 and medical records. Information about maternal health-related quality of life, costs, and satisfaction with Dutch obstetric care was collected from a subsample of women. A pilot study was carried out before the official start of inclusion. External validity of the models will be evaluated by assessing discrimination and calibration. Based on the pilot study, minor improvements were made to the recruitment process and online Pregnancy Questionnaire 1. The validation cohort consists of 2614 women. Data analysis of the external validation study is in progress. This study will offer insight into the generalizability of existing, non-invasive first trimester prediction models for various obstetric outcomes in a Dutch obstetric population. An impact study for the evaluation of the best obstetric prediction models in the Dutch setting with respect to their effect on clinical outcomes, costs, and quality of life-Expect Study II-is being planned. Netherlands Trial Registry (NTR): NTR4143; http://www.trialregister.nl/trialreg/admin/rctview.asp?TC=4143 (Archived by WebCite at http://www.webcitation.org/6t8ijtpd9). ©Linda Jacqueline Elisabeth Meertens, Hubertina CJ Scheepers, Raymond G De Vries, Carmen D Dirksen, Irene Korstjens, Antonius LM Mulder, Marianne J Nieuwenhuijze, Jan G Nijhuis, Marc EA Spaanderman, Luc JM Smits. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 26.10.2017.

20 CFR 404.346 - Your relationship as wife, husband, widow, or widower based upon a deemed valid marriage.

Code of Federal Regulations, 2010 CFR

2010-04-01

... widower based upon a deemed valid marriage. 404.346 Section 404.346 Employees' Benefits SOCIAL SECURITY... relationship as wife, husband, widow, or widower based upon a deemed valid marriage. (a) General. If your... explained in § 404.345, you may be eligible for benefits based upon a deemed valid marriage. You will be...

Reliability and validity of expert assessment based on airborne and urinary measures of nickel and chromium exposure in the electroplating industry.

PubMed

Chen, Yu-Cheng; Coble, Joseph B; Deziel, Nicole C; Ji, Bu-Tian; Xue, Shouzheng; Lu, Wei; Stewart, Patricia A; Friesen, Melissa C

2014-11-01

The reliability and validity of six experts' exposure ratings were evaluated for 64 nickel-exposed and 72 chromium-exposed workers from six Shanghai electroplating plants based on airborne and urinary nickel and chromium measurements. Three industrial hygienists and three occupational physicians independently ranked the exposure intensity of each metal on an ordinal scale (1-4) for each worker's job in two rounds: the first round was based on responses to an occupational history questionnaire and the second round also included responses to an electroplating industry-specific questionnaire. The Spearman correlation (r(s)) was used to compare each rating's validity to its corresponding subject-specific arithmetic mean of four airborne or four urinary measurements. Reliability was moderately high (weighted kappa range=0.60-0.64). Validity was poor to moderate (r(s)=-0.37-0.46) for both airborne and urinary concentrations of both metals. For airborne nickel concentrations, validity differed by plant. For dichotomized metrics, sensitivity and specificity were higher based on urinary measurements (47-78%) than airborne measurements (16-50%). Few patterns were observed by metal, assessment round, or expert type. These results suggest that, for electroplating exposures, experts can achieve moderately high agreement and (reasonably) distinguish between low and high exposures when reviewing responses to in-depth questionnaires used in population-based case-control studies.

Reliability and validity of expert assessment based on airborne and urinary measures of nickel and chromium exposure in the electroplating industry

PubMed Central

Chen, Yu-Cheng; Coble, Joseph B; Deziel, Nicole C.; Ji, Bu-Tian; Xue, Shouzheng; Lu, Wei; Stewart, Patricia A; Friesen, Melissa C

2014-01-01

The reliability and validity of six experts’ exposure ratings were evaluated for 64 nickel-exposed and 72 chromium-exposed workers from six Shanghai electroplating plants based on airborne and urinary nickel and chromium measurements. Three industrial hygienists and three occupational physicians independently ranked the exposure intensity of each metal on an ordinal scale (1–4) for each worker's job in two rounds: the first round was based on responses to an occupational history questionnaire and the second round also included responses to an electroplating industry-specific questionnaire. Spearman correlation (rs) was used to compare each rating's validity to its corresponding subject-specific arithmetic mean of four airborne or four urinary measurements. Reliability was moderately-high (weighted kappa range=0.60–0.64). Validity was poor to moderate (rs= -0.37–0.46) for both airborne and urinary concentrations of both metals. For airborne nickel concentrations, validity differed by plant. For dichotomized metrics, sensitivity and specificity were higher based on urinary measurements (47–78%) than airborne measurements (16–50%). Few patterns were observed by metal, assessment round, or expert type. These results suggest that, for electroplating exposures, experts can achieve moderately-high agreement and (reasonably) distinguish between low and high exposures when reviewing responses to in-depth questionnaires used in population-based case-control studies. PMID:24736099

Validating a biometric authentication system: sample size requirements.

PubMed

Dass, Sarat C; Zhu, Yongfang; Jain, Anil K

2006-12-01

Authentication systems based on biometric features (e.g., fingerprint impressions, iris scans, human face images, etc.) are increasingly gaining widespread use and popularity. Often, vendors and owners of these commercial biometric systems claim impressive performance that is estimated based on some proprietary data. In such situations, there is a need to independently validate the claimed performance levels. System performance is typically evaluated by collecting biometric templates from n different subjects, and for convenience, acquiring multiple instances of the biometric for each of the n subjects. Very little work has been done in 1) constructing confidence regions based on the ROC curve for validating the claimed performance levels and 2) determining the required number of biometric samples needed to establish confidence regions of prespecified width for the ROC curve. To simplify the analysis that address these two problems, several previous studies have assumed that multiple acquisitions of the biometric entity are statistically independent. This assumption is too restrictive and is generally not valid. We have developed a validation technique based on multivariate copula models for correlated biometric acquisitions. Based on the same model, we also determine the minimum number of samples required to achieve confidence bands of desired width for the ROC curve. We illustrate the estimation of the confidence bands as well as the required number of biometric samples using a fingerprint matching system that is applied on samples collected from a small population.

Validation of learning style measures: implications for medical education practice.

PubMed

Chapman, Dane M; Calhoun, Judith G

2006-06-01

It is unclear which learners would most benefit from the more individualised, student-structured, interactive approaches characteristic of problem-based and computer-assisted learning. The validity of learning style measures is uncertain, and there is no unifying learning style construct identified to predict such learners. This study was conducted to validate learning style constructs and to identify the learners most likely to benefit from problem-based and computer-assisted curricula. Using a cross-sectional design, 3 established learning style inventories were administered to 97 post-Year 2 medical students. Cognitive personality was measured by the Group Embedded Figures Test, information processing by the Learning Styles Inventory, and instructional preference by the Learning Preference Inventory. The 11 subscales from the 3 inventories were factor-analysed to identify common learning constructs and to verify construct validity. Concurrent validity was determined by intercorrelations of the 11 subscales. A total of 94 pre-clinical medical students completed all 3 inventories. Five meaningful learning style constructs were derived from the 11 subscales: student- versus teacher-structured learning; concrete versus abstract learning; passive versus active learning; individual versus group learning, and field-dependence versus field-independence. The concurrent validity of 10 of 11 subscales was supported by correlation analysis. Medical students most likely to thrive in a problem-based or computer-assisted learning environment would be expected to score highly on abstract, active and individual learning constructs and would be more field-independent. Learning style measures were validated in a medical student population and learning constructs were established for identifying learners who would most likely benefit from a problem-based or computer-assisted curriculum.

Validation of Screening Assays for Developmental Toxicity: An Exposure-Based Approach

EPA Science Inventory

There continue to be widespread efforts to develop assay methods for developmental toxicity that are shorter than the traditional Segment 2 study and use fewer or no animals. As with any alternative test method, novel developmental toxicity assays must be validated by evaluating ...

A Validation Study of the "School Leader Dispositions Inventory"[C

ERIC Educational Resources Information Center

Melton, Teri Denlea; Tysinger, Dawn; Mallory, Barbara; Green, James

2011-01-01

Although university-based school administrator preparation programs are required by accreditation agencies to assess the dispositions of candidates, valid and reliable methods for doing so remain scarce. "The School Leaders Disposition Inventory"[C] (SDLI) is proposed as an instrument that has promise for identifying leadership…

AIRS Retrieval Validation During the EAQUATE

NASA Technical Reports Server (NTRS)

Zhou, Daniel K.; Smith, William L.; Cuomo, Vincenzo; Taylor, Jonathan P.; Barnet, Christopher D.; DiGirolamo, Paolo; Pappalardo, Gelsomina; Larar, Allen M.; Liu, Xu; Newman, Stuart M.

2006-01-01

Atmospheric and surface thermodynamic parameters retrieved with advanced hyperspectral remote sensors of Earth observing satellites are critical for weather prediction and scientific research. The retrieval algorithms and retrieved parameters from satellite sounders must be validated to demonstrate the capability and accuracy of both observation and data processing systems. The European AQUA Thermodynamic Experiment (EAQUATE) was conducted mainly for validation of the Atmospheric InfraRed Sounder (AIRS) on the AQUA satellite, but also for assessment of validation systems of both ground-based and aircraft-based instruments which will be used for other satellite systems such as the Infrared Atmospheric Sounding Interferometer (IASI) on the European MetOp satellite, the Cross-track Infrared Sounder (CrIS) from the NPOESS Preparatory Project and the following NPOESS series of satellites. Detailed inter-comparisons were conducted and presented using different retrieval methodologies: measurements from airborne ultraspectral Fourier transform spectrometers, aircraft in-situ instruments, dedicated dropsondes and radiosondes, and ground based Raman Lidar, as well as from the European Center for Medium range Weather Forecasting (ECMWF) modeled thermal structures. The results of this study not only illustrate the quality of the measurements and retrieval products but also demonstrate the capability of these validation systems which are put in place to validate current and future hyperspectral sounding instruments and their scientific products.

Designing and validation of a yoga-based intervention for obsessive compulsive disorder.

PubMed

Bhat, Shubha; Varambally, Shivarama; Karmani, Sneha; Govindaraj, Ramajayam; Gangadhar, B N

2016-06-01

Some yoga-based practices have been found to be useful for patients with obsessive compulsive disorder (OCD). The authors could not find a validated yoga therapy module available for OCD. This study attempted to formulate a generic yoga-based intervention module for OCD. A yoga module was designed based on traditional and contemporary yoga literature. The module was sent to 10 yoga experts for content validation. The experts rated the usefulness of the practices on a scale of 1-5 (5 = extremely useful). The final version of the module was pilot-tested on patients with OCD (n = 17) for both feasibility and effect on symptoms. Eighty-eight per cent (22 out of 25) of the items in the initial module were retained, with modifications in the module as suggested by the experts along with patients' inputs and authors' experience. The module was found to be feasible and showed an improvement in symptoms of OCD on total Yale-Brown Obsessive-Compulsive Scale (YBOCS) score (p = 0.001). A generic yoga therapy module for OCD was validated by experts in the field and found feasible to practice in patients. A decrease in the symptom scores was also found following yoga practice of 2 weeks. Further clinical validation is warranted to confirm efficacy.

Development and Validation of an Instrument Measuring Theory-Based Determinants of Monitoring Obesogenic Behaviors of Pre-Schoolers among Hispanic Mothers.

PubMed

Branscum, Paul; Lora, Karina R

2016-06-02

Public health interventions are greatly needed for obesity prevention, and planning for such strategies should include community participation. The study's purpose was to develop and validate a theory-based instrument with low-income, Hispanic mothers of preschoolers, to assess theory-based determinants of maternal monitoring of child's consumption of fruits and vegetables and sugar-sweetened beverages (SSB). Nine focus groups with mothers were conducted to determine nutrition-related behaviors that mothers found as most obesogenic for their children. Next, behaviors were operationally defined and rated for importance and changeability. Two behaviors were selected for investigation (fruits and vegetable and SSB). Twenty semi-structured interviews with mothers were conducted next to develop culturally appropriate items for the instrument. Afterwards, face and content validity were established using a panel of six experts. Finally, the instrument was tested with a sample of 238 mothers. Psychometric properties evaluated included construct validity (using the maximum likelihood extraction method of factor analysis), and internal consistency reliability (Cronbach's alpha). Results suggested that all scales on the instrument were valid and reliable, except for the autonomy scales. Researchers and community planners working with Hispanic families can use this instrument to measure theory-based determinants of parenting behaviors related to preschoolers' consumption of fruits and vegetables, and SSB.

Model-Based Verification and Validation of the SMAP Uplink Processes

NASA Technical Reports Server (NTRS)

Khan, M. Omair; Dubos, Gregory F.; Tirona, Joseph; Standley, Shaun

2013-01-01

This case study stands as an example of how a project can validate a system-level design earlier in the project life cycle than traditional V&V processes by using simulation on a system model. Specifically, this paper describes how simulation was added to a system model of the Soil Moisture Active-Passive (SMAP) mission's uplink process.Also discussed are the advantages and disadvantages of the methods employed and the lessons learned; which are intended to benefit future model-based and simulation-based V&V development efforts.

External validation of preexisting first trimester preeclampsia prediction models.

PubMed

Allen, Rebecca E; Zamora, Javier; Arroyo-Manzano, David; Velauthar, Luxmilar; Allotey, John; Thangaratinam, Shakila; Aquilina, Joseph

2017-10-01

To validate the increasing number of prognostic models being developed for preeclampsia using our own prospective study. A systematic review of literature that assessed biomarkers, uterine artery Doppler and maternal characteristics in the first trimester for the prediction of preeclampsia was performed and models selected based on predefined criteria. Validation was performed by applying the regression coefficients that were published in the different derivation studies to our cohort. We assessed the models discrimination ability and calibration. Twenty models were identified for validation. The discrimination ability observed in derivation studies (Area Under the Curves) ranged from 0.70 to 0.96 when these models were validated against the validation cohort, these AUC varied importantly, ranging from 0.504 to 0.833. Comparing Area Under the Curves obtained in the derivation study to those in the validation cohort we found statistically significant differences in several studies. There currently isn't a definitive prediction model with adequate ability to discriminate for preeclampsia, which performs as well when applied to a different population and can differentiate well between the highest and lowest risk groups within the tested population. The pre-existing large number of models limits the value of further model development and future research should be focussed on further attempts to validate existing models and assessing whether implementation of these improves patient care. Crown Copyright © 2017. Published by Elsevier B.V. All rights reserved.

Landscape unit based digital elevation model development for the freshwater wetlands within the Arthur C. Marshall Loxahatchee National Wildlife Refuge, Southeastern Florida

USGS Publications Warehouse

Xie, Zhixiao; Liu, Zhongwei; Jones, John W.; Higer, Aaron L.; Telis, Pamela A.

2011-01-01

The hydrologic regime is a critical limiting factor in the delicate ecosystem of the greater Everglades freshwater wetlands in south Florida that has been severely altered by management activities in the past several decades. "Getting the water right" is regarded as the key to successful restoration of this unique wetland ecosystem. An essential component to represent and model its hydrologic regime, specifically water depth, is an accurate ground Digital Elevation Model (DEM). The Everglades Depth Estimation Network (EDEN) supplies important hydrologic data, and its products (including a ground DEM) have been well received by scientists and resource managers involved in Everglades restoration. This study improves the EDEN DEMs of the Loxahatchee National Wildlife Refuge, also known as Water Conservation Area 1 (WCA1), by adopting a landscape unit (LU) based interpolation approach. The study first filtered the input elevation data based on newly available vegetation data, and then created a separate geostatistical model (universal kriging) for each LU. The resultant DEMs have encouraging cross-validation and validation results, especially since the validation is based on an independent elevation dataset (derived by subtracting water depth measurements from EDEN water surface elevations). The DEM product of this study will directly benefit hydrologic and ecological studies as well as restoration efforts. The study will also be valuable for a broad range of wetland studies.

Development and validation of a preference based measure derived from the Cambridge Pulmonary Hypertension Outcome Review (CAMPHOR) for use in cost utility analyses.

PubMed

McKenna, Stephen P; Ratcliffe, Julie; Meads, David M; Brazier, John E

2008-08-21

Pulmonary Hypertension is a severe and incurable disease with poor prognosis. A suite of new disease-specific measures--the Cambridge Pulmonary Hypertension Outcome Review (CAMPHOR) - was recently developed for use in this condition. The purpose of this study was to develop and validate a preference based measure from the CAMPHOR that could be used in cost-utility analyses. Items were selected that covered major issues covered by the CAMPHOR QoL scale (activities, travelling, dependence and communication). These were used to create 36 health states that were valued by 249 people representative of the UK adult population, using the time trade-off (TTO) technique. Data from the TTO interviews were analysed using both aggregate and individual level modelling. Finally, the original CAMPHOR validation data were used to validate the new preference based model. The predicted health state values ranged from 0.962 to 0.136. The mean level model selected for analyzing the data had good explanatory power (0.936), did not systematically over- or underestimate the observed mean health state values and showed no evidence of auto correlation in the prediction errors. The value of less than 1 reflects a background level of ill health in state 1111, as judged by the respondents. Scores derived from the new measure had excellent test-retest reliability (0.85) and construct validity. The CAMPHOR utility score appears better able to distinguish between WHO functional classes (II and III) than the EQ-5D and SF-6D. The tariff derived in this study can be used to classify an individual into a health state based on their responses to the CAMPHOR. The results of this study widen the evidence base for conducting economic evaluations of interventions designed to improve QoL for patients with PH.

Development and Psychometric Properties of the Decision Making on the Type of Delivery Questionnaire in Iranian Women.

PubMed

Latifnejad Roudsari, Robab; Zakerihamidi, Maryam; Merghati Khoei, Effat; Kazemnejad, Anoshirvan

2017-07-01

A tool which can help to decide on the determinants in selecting the delivery type is an effective step towards the goals of the World Health. This study aimed to develop and evaluate the psychometric properties of a scale based on Iranian culture to make decision on the type of delivery. This is a methodological study using a questionnaire proposed by Schneider. The following steps were used to design the project. In the first step, perceptions and experiences of 45 pregnant women, postpartum women, midwives, gynecologists and non-pregnant women were determined based on interviews and observations using focused ethnography. In the second stage, the terms in the questionnaire based on qualitative study was assessed. Then, in the third stage, psychometric testing of the decision making on the type of delivery scale (DMTDS) based on the cultural concepts of decision making towards the type of delivery and its influencing factors based on focused ethnography using face validity, content validity, construct validity, internal consistency and reliability was done on400 pregnant and postpartum women. The initially developed scale consisted of 60 items on a 5-point Likert scale, which reduced to 43 items following measurement of the face and content validity. The results of the exploratory factor analysis elicited 36 items and a seven-factor structure including motivational beliefs on vaginal delivery, social beliefs towards childbirth, motivational beliefs on cesarean delivery, personal beliefs, sources of information, catastrophic thinking and child birth experiences. Cronbach's alpha coefficient (0.80) confirmed the high internal consistency of the scale. The developed questionnaire appears to be a valid and reliable tool for health care providers to measure the women's decision making towards type of delivery. Therefore, this tool can be used in the Iranian community. The scale may help the midwives and obstetricians to be aware of the women's decision regarding their choice of delivery and as a result to plan appropriately in order to reduce unnecessary cesarean sections.

Attitudes Toward Transgender Men and Women: Development and Validation of a New Measure

PubMed Central

Billard, Thomas J

2018-01-01

A series of three studies were conducted to generate, develop, and validate the Attitudes toward Transgender Men and Women (ATTMW) scale. In Study 1, 120 American adults responded to an open-ended questionnaire probing various dimensions of their perceptions of transgender individuals and identity. Qualitative thematic analysis generated 200 items based on their responses. In Study 2, 238 American adults completed a questionnaire consisting of the generated items. Exploratory factor analysis (EFA) revealed two non-identical 12-item subscales (ATTM and ATTW) of the full 24-item scale. In Study 3, 150 undergraduate students completed a survey containing the ATTMW and a number of validity-testing variables. Confirmatory factor analysis (CFA) verified the single-factor structures of the ATTM and ATTW subscales, and the convergent, discriminant, predictive, and concurrent validities of the ATTMW were also established. Together, our results demonstrate that the ATTMW is a reliable and valid measure of attitudes toward transgender individuals. PMID:29666595

TA [B] Predicting Microstructure-Creep Resistance Correlation in High Temperature Alloys over Multiple Time Scales

DOE Office of Scientific and Technical Information (OSTI.GOV)

Tomar, Vikas

2017-03-06

DoE-NETL partnered with Purdue University to predict the creep and associated microstructure evolution of tungsten-based refractory alloys. Researchers use grain boundary (GB) diagrams, a new concept, to establish time-dependent creep resistance and associated microstructure evolution of grain boundaries/intergranular films GB/IGF controlled creep as a function of load, environment, and temperature. The goal was to conduct a systematic study that includes the development of a theoretical framework, multiscale modeling, and experimental validation using W-based body-centered-cubic alloys, doped/alloyed with one or two of the following elements: nickel, palladium, cobalt, iron, and copper—typical refractory alloys. Prior work has already established and validated amore » basic theory for W-based binary and ternary alloys; the study conducted under this project extended this proven work. Based on interface diagrams phase field models were developed to predict long term microstructural evolution. In order to validate the models nanoindentation creep data was used to elucidate the role played by the interface properties in predicting long term creep strength and microstructure evolution.« less

Development of an Instrument for Measuring Clinicians’ Power Perceptions in the Workplace

PubMed Central

Bartos, Christa E.; Fridsma, Douglas B.; Butler, Brian S.; Penrod, Louis E.; Becich, Michael J.; Crowley, Rebecca S.

2008-01-01

We report on the development of an instrument to measure clinicians’ perceptions of their personal power in the workplace in relation to resistance to computerized physician order entry (CPOE). The instrument is based on French and Raven’s six bases of social power and uses a semantic differential methodology. A measurement study was conducted to determine the reliability and validity of the survey. The survey was administered online and distributed via a URL by email to 19 physicians, nurses, and health unit coordinators from a university hospital. Acceptable reliability was achieved by removing or moving some semantic differential word pairs used to represent the six power bases (alpha range from 0.76–0.89). The Semantic Differential Power Perception (SDPP) survey validity was tested against an already validated instrument and found to be acceptable (correlation range from 0.51–0.81). The SDPP survey instrument was determined to be both reliable and valid. PMID:18375189

Validity and reliability of the Turkish version of the Optimality Index-US (OI-US) to assess maternity care outcomes.

PubMed

Yucel, Cigdem; Taskin, Lale; Low, Lisa Kane

2015-12-01

Although obstetrical interventions are used commonly in Turkey, there is no standardized evidence-based assessment tool to evaluate maternity care outcomes. The Optimality Index-US (OI-US) is an evidence-based tool that was developed for the purpose of measuring aggregate perinatal care processes and outcomes against an optimal or best possible standard. This index has been validated and used in Netherlands, USA and UK until now. The objective of this study was to adapt the OI-US to assess maternity care outcomes in Turkey. Translation and back translation were used to develop the Optimality Index-Turkey (OI-TR) version. To evaluate the content validity of the OI-TR, an expert panel group (n=10) reviewed the items and evidence-based quality of the OI-TR for application in Turkey. Following the content validity process, the OI-TR was used to assess 150 healthy and 150 high-risk pregnant women who gave birth at a high volume, urban maternity hospital in Turkey. The scores between the two groups were compared to assess the discriminant validity of the OI-TR. The percentage of agreement between two raters and the Kappa statistic were calculated to evaluate the reliability. Content validity was established for the OI-TR by an expert group. Discriminant validity was confirmed by comparing the OI scores of healthy pregnant women (mean OI score=77.65%) and those of high-risk pregnant women (mean OI score=78.60%). The percentage of agreement between the two raters was 96.19, and inter-rater agreement was provided for each item in the OI-TR. OI-TR is a valid and reliable tool that can be used to assess maternity care outcomes in Turkey. The results of this study indicate that although the risk statuses of the women differed, the type of care they received was essentially the same, as measured by the OI-TR. Care was not individualised based on risk and for a majority of items was inconsistent with evidence based practice, which is not optimal. Use of the OI-TR will help to provide a standardized way to assess maternity care process and outcomes of maternity care in Turkey which can inform future research aimed at improving maternity care outcomes. Copyright © 2015 Elsevier Ltd. All rights reserved.

Development and External Validation of a Melanoma Risk Prediction Model Based on Self-assessed Risk Factors.

PubMed

Vuong, Kylie; Armstrong, Bruce K; Weiderpass, Elisabete; Lund, Eiliv; Adami, Hans-Olov; Veierod, Marit B; Barrett, Jennifer H; Davies, John R; Bishop, D Timothy; Whiteman, David C; Olsen, Catherine M; Hopper, John L; Mann, Graham J; Cust, Anne E; McGeechan, Kevin

2016-08-01

Identifying individuals at high risk of melanoma can optimize primary and secondary prevention strategies. To develop and externally validate a risk prediction model for incident first-primary cutaneous melanoma using self-assessed risk factors. We used unconditional logistic regression to develop a multivariable risk prediction model. Relative risk estimates from the model were combined with Australian melanoma incidence and competing mortality rates to obtain absolute risk estimates. A risk prediction model was developed using the Australian Melanoma Family Study (629 cases and 535 controls) and externally validated using 4 independent population-based studies: the Western Australia Melanoma Study (511 case-control pairs), Leeds Melanoma Case-Control Study (960 cases and 513 controls), Epigene-QSkin Study (44 544, of which 766 with melanoma), and Swedish Women's Lifestyle and Health Cohort Study (49 259 women, of which 273 had melanoma). We validated model performance internally and externally by assessing discrimination using the area under the receiver operating curve (AUC). Additionally, using the Swedish Women's Lifestyle and Health Cohort Study, we assessed model calibration and clinical usefulness. The risk prediction model included hair color, nevus density, first-degree family history of melanoma, previous nonmelanoma skin cancer, and lifetime sunbed use. On internal validation, the AUC was 0.70 (95% CI, 0.67-0.73). On external validation, the AUC was 0.66 (95% CI, 0.63-0.69) in the Western Australia Melanoma Study, 0.67 (95% CI, 0.65-0.70) in the Leeds Melanoma Case-Control Study, 0.64 (95% CI, 0.62-0.66) in the Epigene-QSkin Study, and 0.63 (95% CI, 0.60-0.67) in the Swedish Women's Lifestyle and Health Cohort Study. Model calibration showed close agreement between predicted and observed numbers of incident melanomas across all deciles of predicted risk. In the external validation setting, there was higher net benefit when using the risk prediction model to classify individuals as high risk compared with classifying all individuals as high risk. The melanoma risk prediction model performs well and may be useful in prevention interventions reliant on a risk assessment using self-assessed risk factors.

Using wound care algorithms: a content validation study.

PubMed

Beitz, J M; van Rijswijk, L

1999-09-01

Valid and reliable heuristic devices facilitating optimal wound care are lacking. The objectives of this study were to establish content validation data for a set of wound care algorithms, to identify their associated strengths and weaknesses, and to gain insight into the wound care decision-making process. Forty-four registered nurse wound care experts were surveyed and interviewed at national and regional educational meetings. Using a cross-sectional study design and an 83-item, 4-point Likert-type scale, this purposive sample was asked to quantify the degree of validity of the algorithms' decisions and components. Participants' comments were tape-recorded, transcribed, and themes were derived. On a scale of 1 to 4, the mean score of the entire instrument was 3.47 (SD +/- 0.87), the instrument's Content Validity Index was 0.86, and the individual Content Validity Index of 34 of 44 participants was > 0.8. Item scores were lower for those related to packing deep wounds (P < .001). No other significant differences were observed. Qualitative data analysis revealed themes of difficulty associated with wound assessment and care issues, that is, the absence of valid and reliable definitions. The wound care algorithms studied proved valid. However, the lack of valid and reliable wound assessment and care definitions hinders optimal use of these instruments. Further research documenting their clinical use is warranted. Research-based practice recommendations should direct the development of future valid and reliable algorithms designed to help nurses provide optimal wound care.

A Simulation Study of Threats to Validity in Quasi-Experimental Designs: Interrelationship between Design, Measurement, and Analysis.

PubMed

Holgado-Tello, Fco P; Chacón-Moscoso, Salvador; Sanduvete-Chaves, Susana; Pérez-Gil, José A

2016-01-01

The Campbellian tradition provides a conceptual framework to assess threats to validity. On the other hand, different models of causal analysis have been developed to control estimation biases in different research designs. However, the link between design features, measurement issues, and concrete impact estimation analyses is weak. In order to provide an empirical solution to this problem, we use Structural Equation Modeling (SEM) as a first approximation to operationalize the analytical implications of threats to validity in quasi-experimental designs. Based on the analogies established between the Classical Test Theory (CTT) and causal analysis, we describe an empirical study based on SEM in which range restriction and statistical power have been simulated in two different models: (1) A multistate model in the control condition (pre-test); and (2) A single-trait-multistate model in the control condition (post-test), adding a new mediator latent exogenous (independent) variable that represents a threat to validity. Results show, empirically, how the differences between both the models could be partially or totally attributed to these threats. Therefore, SEM provides a useful tool to analyze the influence of potential threats to validity.

A Simulation Study of Threats to Validity in Quasi-Experimental Designs: Interrelationship between Design, Measurement, and Analysis

PubMed Central

Holgado-Tello, Fco. P.; Chacón-Moscoso, Salvador; Sanduvete-Chaves, Susana; Pérez-Gil, José A.

2016-01-01

The Campbellian tradition provides a conceptual framework to assess threats to validity. On the other hand, different models of causal analysis have been developed to control estimation biases in different research designs. However, the link between design features, measurement issues, and concrete impact estimation analyses is weak. In order to provide an empirical solution to this problem, we use Structural Equation Modeling (SEM) as a first approximation to operationalize the analytical implications of threats to validity in quasi-experimental designs. Based on the analogies established between the Classical Test Theory (CTT) and causal analysis, we describe an empirical study based on SEM in which range restriction and statistical power have been simulated in two different models: (1) A multistate model in the control condition (pre-test); and (2) A single-trait-multistate model in the control condition (post-test), adding a new mediator latent exogenous (independent) variable that represents a threat to validity. Results show, empirically, how the differences between both the models could be partially or totally attributed to these threats. Therefore, SEM provides a useful tool to analyze the influence of potential threats to validity. PMID:27378991

Validation of the German version of the Ford Insomnia Response to Stress Test.

PubMed

Dieck, Arne; Helbig, Susanne; Drake, Christopher L; Backhaus, Jutta

2018-06-01

The purpose of this study was to assess the psychometric properties of a German version of the Ford Insomnia Response to Stress Test with groups with and without sleep problems. Three studies were analysed. Data set 1 was based on an initial screening for a sleep training program (n = 393), data set 2 was based on a study to test the test-retest reliability of the Ford Insomnia Response to Stress Test (n = 284) and data set 3 was based on a study to examine the influence of competitive sport on sleep (n = 37). Data sets 1 and 2 were used to test internal consistency, factor structure, convergent validity, discriminant validity and test-retest reliability of the Ford Insomnia Response to Stress Test. Content validity was tested using data set 3. Cronbach's alpha of the Ford Insomnia Response to Stress Test was good (α = 0.80) and test-retest reliability was satisfactory (r = 0.72). Overall, the one-factor model showed the best fit. Furthermore, significant positive correlations between the Ford Insomnia Response to Stress Test and impaired sleep quality, depression and stress reactivity were in line with the expectations regarding the convergent validity. Subjects with sleep problems had significantly higher scores in the Ford Insomnia Response to Stress Test than subjects without sleep problems (P < 0.01). Competitive athletes with higher scores in the Ford Insomnia Response to Stress Test had significantly lower sleep quality (P = 0.01), demonstrating that vulnerability for stress-induced sleep disturbances accompanies poorer sleep quality in stressful episodes. The findings show that the German version of the Ford Insomnia Response to Stress Test is a reliable and valid questionnaire to assess the vulnerability to stress-induced sleep disturbances. © 2017 European Sleep Research Society.

Basic Scale on Insomnia complaints and Quality of Sleep (BaSIQS): reliability, initial validity and normative scores in higher education students.

PubMed

Allen Gomes, Ana; Ruivo Marques, Daniel; Meia-Via, Ana Maria; Meia-Via, Mariana; Tavares, José; Fernandes da Silva, Carlos; Pinto de Azevedo, Maria Helena

2015-04-01

Based on successive samples totaling more than 5000 higher education students, we scrutinized the reliability, structure, initial validity and normative scores of a brief self-report seven-item scale to screen for the continuum of nighttime insomnia complaints/perceived sleep quality, used by our team for more than a decade, henceforth labeled the Basic Scale on Insomnia complaints and Quality of Sleep (BaSIQS). In study/sample 1 (n = 1654), the items were developed based on part of a larger survey on higher education sleep-wake patterns. The test-retest study was conducted in an independent small group (n = 33) with a 2-8 week gap. In study/sample 2 (n = 360), focused mainly on validity, the BaSIQS was completed together with the Pittsburgh Sleep Quality Index (PSQI). In study 3, a large recent sample of students from universities all over the country (n = 2995) answered the BaSIQS items, based on which normative scores were determined, and an additional question on perceived sleep problems in order to further analyze the scale's validity. Regarding reliability, Cronbach alpha coefficients were systematically higher than 0.7, and the test-retest correlation coefficient was greater than 0.8. Structure analyses revealed consistently satisfactory two-factor and single-factor solutions. Concerning validity analyses, BaSIQS scores were significantly correlated with PSQI component scores and overall score (r = 0.652 corresponding to a large association); mean scores were significantly higher in those students classifying themselves as having sleep problems (p < 0.0001, d = 0.99 corresponding to a large effect size). In conclusion, the BaSIQS is very easy to administer, and appears to be a reliable and valid scale in higher education students. It might be a convenient short tool in research and applied settings to rapidly assess sleep quality or screen for insomnia complaints, and it may be easily used in other populations with minor adaptations.

Daily home-based spirometry during withdrawal of inhaled corticosteroid in severe to very severe chronic obstructive pulmonary disease

PubMed Central

Rodriguez-Roisin, Roberto; Tetzlaff, Kay; Watz, Henrik; Wouters, Emiel FM; Disse, Bernd; Finnigan, Helen; Magnussen, Helgo; Calverley, Peter MA

2016-01-01

The WISDOM study (NCT00975195) reported a change in lung function following withdrawal of fluticasone propionate in patients with severe to very severe COPD treated with tiotropium and salmeterol. However, little is known about the validity of home-based spirometry measurements of lung function in COPD. Therefore, as part of this study, following suitable training, patients recorded daily home-based spirometry measurements in addition to undergoing periodic in-clinic spirometric testing throughout the study duration. We subsequently determined the validity of home-based spirometry for detecting changes in lung function by comparing in-clinic and home-based forced expiratory volume in 1 second in patients who underwent stepwise fluticasone propionate withdrawal over 12 weeks versus patients remaining on fluticasone propionate for 52 weeks. Bland–Altman analysis of these data confirmed good agreement between in-clinic and home-based measurements, both across all visits and at the individual visits at study weeks 6, 12, 18, and 52. There was a measurable difference between the forced expiratory volume in 1 second values recorded at home and in the clinic (mean difference of −0.05 L), which may be due to suboptimal patient effort in performing unsupervised recordings. However, this difference remained consistent over time. Overall, these data demonstrate that home-based and in-clinic spirometric measurements were equally valid and reliable for assessing lung function in patients with COPD, and suggest that home-based spirometry may be a useful tool to facilitate analysis of changes in lung function on a day-to-day basis. PMID:27578972

Workforce Perceptions of Hospital Safety Culture: Development and Validation of the Patient Safety Climate in Healthcare Organizations Survey

PubMed Central

Singer, Sara; Meterko, Mark; Baker, Laurence; Gaba, David; Falwell, Alyson; Rosen, Amy

2007-01-01

Objective To describe the development of an instrument for assessing workforce perceptions of hospital safety culture and to assess its reliability and validity. Data Sources/Study Setting Primary data collected between March 2004 and May 2005. Personnel from 105 U.S. hospitals completed a 38-item paper and pencil survey. We received 21,496 completed questionnaires, representing a 51 percent response rate. Study Design Based on review of existing safety climate surveys, we developed a list of key topics pertinent to maintaining a culture of safety in high-reliability organizations. We developed a draft questionnaire to address these topics and pilot tested it in four preliminary studies of hospital personnel. We modified the questionnaire based on experience and respondent feedback, and distributed the revised version to 42,249 hospital workers. Data Collection We randomly divided respondents into derivation and validation samples. We applied exploratory factor analysis to responses in the derivation sample. We used those results to create scales in the validation sample, which we subjected to multitrait analysis (MTA). Principal Findings We identified nine constructs, three organizational factors, two unit factors, three individual factors, and one additional factor. Constructs demonstrated substantial convergent and discriminant validity in the MTA. Cronbach's α coefficients ranged from 0.50 to 0.89. Conclusions It is possible to measure key salient features of hospital safety climate using a valid and reliable 38-item survey and appropriate hospital sample sizes. This instrument may be used in further studies to better understand the impact of safety climate on patient safety outcomes. PMID:17850530

Discriminant Validity Assessment: Use of Fornell & Larcker criterion versus HTMT Criterion

NASA Astrophysics Data System (ADS)

Hamid, M. R. Ab; Sami, W.; Mohmad Sidek, M. H.

2017-09-01

Assessment of discriminant validity is a must in any research that involves latent variables for the prevention of multicollinearity issues. Fornell and Larcker criterion is the most widely used method for this purpose. However, a new method has emerged for establishing the discriminant validity assessment through heterotrait-monotrait (HTMT) ratio of correlations method. Therefore, this article presents the results of discriminant validity assessment using these methods. Data from previous study was used that involved 429 respondents for empirical validation of value-based excellence model in higher education institutions (HEI) in Malaysia. From the analysis, the convergent, divergent and discriminant validity were established and admissible using Fornell and Larcker criterion. However, the discriminant validity is an issue when employing the HTMT criterion. This shows that the latent variables under study faced the issue of multicollinearity and should be looked into for further details. This also implied that the HTMT criterion is a stringent measure that could detect the possible indiscriminant among the latent variables. In conclusion, the instrument which consisted of six latent variables was still lacking in terms of discriminant validity and should be explored further.

Knowledge-Driven Analysis Identifies a Gene–Gene Interaction Affecting High-Density Lipoprotein Cholesterol Levels in Multi-Ethnic Populations

PubMed Central

Ma, Li; Brautbar, Ariel; Boerwinkle, Eric; Sing, Charles F.

2012-01-01

Total cholesterol, low-density lipoprotein cholesterol, triglyceride, and high-density lipoprotein cholesterol (HDL-C) levels are among the most important risk factors for coronary artery disease. We tested for gene–gene interactions affecting the level of these four lipids based on prior knowledge of established genome-wide association study (GWAS) hits, protein–protein interactions, and pathway information. Using genotype data from 9,713 European Americans from the Atherosclerosis Risk in Communities (ARIC) study, we identified an interaction between HMGCR and a locus near LIPC in their effect on HDL-C levels (Bonferroni corrected P c = 0.002). Using an adaptive locus-based validation procedure, we successfully validated this gene–gene interaction in the European American cohorts from the Framingham Heart Study (P c = 0.002) and the Multi-Ethnic Study of Atherosclerosis (MESA; P c = 0.006). The interaction between these two loci is also significant in the African American sample from ARIC (P c = 0.004) and in the Hispanic American sample from MESA (P c = 0.04). Both HMGCR and LIPC are involved in the metabolism of lipids, and genome-wide association studies have previously identified LIPC as associated with levels of HDL-C. However, the effect on HDL-C of the novel gene–gene interaction reported here is twice as pronounced as that predicted by the sum of the marginal effects of the two loci. In conclusion, based on a knowledge-driven analysis of epistasis, together with a new locus-based validation method, we successfully identified and validated an interaction affecting a complex trait in multi-ethnic populations. PMID:22654671

Automatic Detection of Whole Night Snoring Events Using Non-Contact Microphone

PubMed Central

Dafna, Eliran; Tarasiuk, Ariel; Zigel, Yaniv

2013-01-01

Objective Although awareness of sleep disorders is increasing, limited information is available on whole night detection of snoring. Our study aimed to develop and validate a robust, high performance, and sensitive whole-night snore detector based on non-contact technology. Design Sounds during polysomnography (PSG) were recorded using a directional condenser microphone placed 1 m above the bed. An AdaBoost classifier was trained and validated on manually labeled snoring and non-snoring acoustic events. Patients Sixty-seven subjects (age 52.5±13.5 years, BMI 30.8±4.7 kg/m2, m/f 40/27) referred for PSG for obstructive sleep apnea diagnoses were prospectively and consecutively recruited. Twenty-five subjects were used for the design study; the validation study was blindly performed on the remaining forty-two subjects. Measurements and Results To train the proposed sound detector, >76,600 acoustic episodes collected in the design study were manually classified by three scorers into snore and non-snore episodes (e.g., bedding noise, coughing, environmental). A feature selection process was applied to select the most discriminative features extracted from time and spectral domains. The average snore/non-snore detection rate (accuracy) for the design group was 98.4% based on a ten-fold cross-validation technique. When tested on the validation group, the average detection rate was 98.2% with sensitivity of 98.0% (snore as a snore) and specificity of 98.3% (noise as noise). Conclusions Audio-based features extracted from time and spectral domains can accurately discriminate between snore and non-snore acoustic events. This audio analysis approach enables detection and analysis of snoring sounds from a full night in order to produce quantified measures for objective follow-up of patients. PMID:24391903

The quality of instruments to assess the process of shared decision making: A systematic review.

PubMed

Gärtner, Fania R; Bomhof-Roordink, Hanna; Smith, Ian P; Scholl, Isabelle; Stiggelbout, Anne M; Pieterse, Arwen H

2018-01-01

To inventory instruments assessing the process of shared decision making and appraise their measurement quality, taking into account the methodological quality of their validation studies. In a systematic review we searched seven databases (PubMed, Embase, Emcare, Cochrane, PsycINFO, Web of Science, Academic Search Premier) for studies investigating instruments measuring the process of shared decision making. Per identified instrument, we assessed the level of evidence separately for 10 measurement properties following a three-step procedure: 1) appraisal of the methodological quality using the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist, 2) appraisal of the psychometric quality of the measurement property using three possible quality scores, 3) best-evidence synthesis based on the number of studies, their methodological and psychometrical quality, and the direction and consistency of the results. The study protocol was registered at PROSPERO: CRD42015023397. We included 51 articles describing the development and/or evaluation of 40 shared decision-making process instruments: 16 patient questionnaires, 4 provider questionnaires, 18 coding schemes and 2 instruments measuring multiple perspectives. There is an overall lack of evidence for their measurement quality, either because validation is missing or methods are poor. The best-evidence synthesis indicated positive results for a major part of instruments for content validity (50%) and structural validity (53%) if these were evaluated, but negative results for a major part of instruments when inter-rater reliability (47%) and hypotheses testing (59%) were evaluated. Due to the lack of evidence on measurement quality, the choice for the most appropriate instrument can best be based on the instrument's content and characteristics such as the perspective that they assess. We recommend refinement and validation of existing instruments, and the use of COSMIN-guidelines to help guarantee high-quality evaluations.

Automatic detection of whole night snoring events using non-contact microphone.

PubMed

Dafna, Eliran; Tarasiuk, Ariel; Zigel, Yaniv

2013-01-01

Although awareness of sleep disorders is increasing, limited information is available on whole night detection of snoring. Our study aimed to develop and validate a robust, high performance, and sensitive whole-night snore detector based on non-contact technology. Sounds during polysomnography (PSG) were recorded using a directional condenser microphone placed 1 m above the bed. An AdaBoost classifier was trained and validated on manually labeled snoring and non-snoring acoustic events. Sixty-seven subjects (age 52.5 ± 13.5 years, BMI 30.8 ± 4.7 kg/m(2), m/f 40/27) referred for PSG for obstructive sleep apnea diagnoses were prospectively and consecutively recruited. Twenty-five subjects were used for the design study; the validation study was blindly performed on the remaining forty-two subjects. To train the proposed sound detector, >76,600 acoustic episodes collected in the design study were manually classified by three scorers into snore and non-snore episodes (e.g., bedding noise, coughing, environmental). A feature selection process was applied to select the most discriminative features extracted from time and spectral domains. The average snore/non-snore detection rate (accuracy) for the design group was 98.4% based on a ten-fold cross-validation technique. When tested on the validation group, the average detection rate was 98.2% with sensitivity of 98.0% (snore as a snore) and specificity of 98.3% (noise as noise). Audio-based features extracted from time and spectral domains can accurately discriminate between snore and non-snore acoustic events. This audio analysis approach enables detection and analysis of snoring sounds from a full night in order to produce quantified measures for objective follow-up of patients.

Identification of Heterogeneous Cognitive Subgroups in Community-Dwelling Older Adults: A Latent Class Analysis of the Einstein Aging Study.

PubMed

Zammit, Andrea R; Hall, Charles B; Lipton, Richard B; Katz, Mindy J; Muniz-Terrera, Graciela

2018-05-01

The aim of this study was to identify natural subgroups of older adults based on cognitive performance, and to establish each subgroup's characteristics based on demographic factors, physical function, psychosocial well-being, and comorbidity. We applied latent class (LC) modeling to identify subgroups in baseline assessments of 1345 Einstein Aging Study (EAS) participants free of dementia. The EAS is a community-dwelling cohort study of 70+ year-old adults living in the Bronx, NY. We used 10 neurocognitive tests and 3 covariates (age, sex, education) to identify latent subgroups. We used goodness-of-fit statistics to identify the optimal class solution and assess model adequacy. We also validated our model using two-fold split-half cross-validation. The sample had a mean age of 78.0 (SD=5.4) and a mean of 13.6 years of education (SD=3.5). A 9-class solution based on cognitive performance at baseline was the best-fitting model. We characterized the 9 identified classes as (i) disadvantaged, (ii) poor language, (iii) poor episodic memory and fluency, (iv) poor processing speed and executive function, (v) low average, (vi) high average, (vii) average, (viii) poor executive and poor working memory, (ix) elite. The cross validation indicated stable class assignment with the exception of the average and high average classes. LC modeling in a community sample of older adults revealed 9 cognitive subgroups. Assignment of subgroups was reliable and associated with external validators. Future work will test the predictive validity of these groups for outcomes such as Alzheimer's disease, vascular dementia and death, as well as markers of biological pathways that contribute to cognitive decline. (JINS, 2018, 24, 511-523).

Combined 3D-QSAR, molecular docking and molecular dynamics study on thyroid hormone activity of hydroxylated polybrominated diphenyl ethers to thyroid receptors β

DOE Office of Scientific and Technical Information (OSTI.GOV)

Li, Xiaolin; Ye, Li; Wang, Xiaoxiang

2012-12-15

Several recent reports suggested that hydroxylated polybrominated diphenyl ethers (HO-PBDEs) may disturb thyroid hormone homeostasis. To illuminate the structural features for thyroid hormone activity of HO-PBDEs and the binding mode between HO-PBDEs and thyroid hormone receptor (TR), the hormone activity of a series of HO-PBDEs to thyroid receptors β was studied based on the combination of 3D-QSAR, molecular docking, and molecular dynamics (MD) methods. The ligand- and receptor-based 3D-QSAR models were obtained using Comparative Molecular Similarity Index Analysis (CoMSIA) method. The optimum CoMSIA model with region focusing yielded satisfactory statistical results: leave-one-out cross-validation correlation coefficient (q{sup 2}) was 0.571 andmore » non-cross-validation correlation coefficient (r{sup 2}) was 0.951. Furthermore, the results of internal validation such as bootstrapping, leave-many-out cross-validation, and progressive scrambling as well as external validation indicated the rationality and good predictive ability of the best model. In addition, molecular docking elucidated the conformations of compounds and key amino acid residues at the docking pocket, MD simulation further determined the binding process and validated the rationality of docking results. -- Highlights: ► The thyroid hormone activities of HO-PBDEs were studied by 3D-QSAR. ► The binding modes between HO-PBDEs and TRβ were explored. ► 3D-QSAR, molecular docking, and molecular dynamics (MD) methods were performed.« less

Validation of the translation of an instrument to measure reliability of written information on treatment choices: a study on attention deficit/hyperactivity disorder (ADHD).

PubMed

Montoya, A; Llopis, N; Gilaberte, I

2011-12-01

DISCERN is an instrument designed to help patients assess the reliability of written information on treatment choices. Originally created in English, there is no validated Spanish version of this instrument. This study seeks to validate the Spanish translation of the DISCERN instrument used as a primary measure on a multicenter study aimed to assess the reliability of web-based information on treatment choices for attention deficit/hyperactivity disorder (ADHD). We used a modified version of a method for validating translated instruments in which the original source-language version is formally compared with the back-translated source-language version. Each item was ranked in terms of comparability of language, similarity of interpretability, and degree of understandability. Responses used Likert scales ranging from 1 to 7, where 1 indicates the best interpretability, language and understandability, and 7 indicates the worst. Assessments were performed by 20 raters fluent in the source language. The Spanish translation of DISCERN, based on ratings of comparability, interpretability and degree of understandability (mean score (SD): 1.8 (1.1), 1.4 (0.9) and 1.6 (1.1), respectively), was considered extremely comparable. All items received a score of less than three, therefore no further revision of the translation was needed. The validation process showed that the quality of DISCERN translation was high, validating the comparable language of the tool translated on assessing written information on treatment choices for ADHD.

Investigating the technical adequacy of curriculum-based measurement in written expression for students who are deaf or hard of hearing.

PubMed

Cheng, Shu-Fen; Rose, Susan

2009-01-01

This study investigated the technical adequacy of curriculum-based measures of written expression (CBM-W) in terms of writing prompts and scoring methods for deaf and hard-of-hearing students. Twenty-two students at the secondary school-level completed 3-min essays within two weeks, which were scored for nine existing and alternative curriculum-based measurement (CBM) scoring methods. The technical features of the nine scoring methods were examined for interrater reliability, alternate-form reliability, and criterion-related validity. The existing CBM scoring method--number of correct minus incorrect word sequences--yielded the highest reliability and validity coefficients. The findings from this study support the use of the CBM-W as a reliable and valid tool for assessing general writing proficiency with secondary students who are deaf or hard of hearing. The CBM alternative scoring methods that may serve as additional indicators of written expression include correct subject-verb agreements, correct clauses, and correct morphemes.

How reliable is internet-based self-reported identity, socio-demographic and obesity measures in European adults?

PubMed

Celis-Morales, Carlos; Livingstone, Katherine M; Woolhead, Clara; Forster, Hannah; O'Donovan, Clare B; Macready, Anna L; Fallaize, Rosalind; Marsaux, Cyril F M; Tsirigoti, Lydia; Efstathopoulou, Eirini; Moschonis, George; Navas-Carretero, Santiago; San-Cristobal, Rodrigo; Kolossa, Silvia; Klein, Ulla L; Hallmann, Jacqueline; Godlewska, Magdalena; Surwiłło, Agnieszka; Drevon, Christian A; Bouwman, Jildau; Grimaldi, Keith; Parnell, Laurence D; Manios, Yannis; Traczyk, Iwona; Gibney, Eileen R; Brennan, Lorraine; Walsh, Marianne C; Lovegrove, Julie A; Martinez, J Alfredo; Daniel, Hannelore; Saris, Wim H M; Gibney, Mike; Mathers, John C

2015-09-01

In e-health intervention studies, there are concerns about the reliability of internet-based, self-reported (SR) data and about the potential for identity fraud. This study introduced and tested a novel procedure for assessing the validity of internet-based, SR identity and validated anthropometric and demographic data via measurements performed face-to-face in a validation study (VS). Participants (n = 140) from seven European countries, participating in the Food4Me intervention study which aimed to test the efficacy of personalised nutrition approaches delivered via the internet, were invited to take part in the VS. Participants visited a research centre in each country within 2 weeks of providing SR data via the internet. Participants received detailed instructions on how to perform each measurement. Individual's identity was checked visually and by repeated collection and analysis of buccal cell DNA for 33 genetic variants. Validation of identity using genomic information showed perfect concordance between SR and VS. Similar results were found for demographic data (age and sex verification). We observed strong intra-class correlation coefficients between SR and VS for anthropometric data (height 0.990, weight 0.994 and BMI 0.983). However, internet-based SR weight was under-reported (Δ -0.70 kg [-3.6 to 2.1], p < 0.0001) and, therefore, BMI was lower for SR data (Δ -0.29 kg m(-2) [-1.5 to 1.0], p < 0.0001). BMI classification was correct in 93 % of cases. We demonstrate the utility of genotype information for detection of possible identity fraud in e-health studies and confirm the reliability of internet-based, SR anthropometric and demographic data collected in the Food4Me study. NCT01530139 ( http://clinicaltrials.gov/show/NCT01530139 ).

Development and Validation of Methodology to Model Flow in Ventilation Systems Commonly Found in Nuclear Facilities. Phase I

DOE Office of Scientific and Technical Information (OSTI.GOV)

Strons, Philip; Bailey, James L.; Davis, John

2016-03-01

In this work, we apply the CFD in modeling airflow and particulate transport. This modeling is then compared to field validation studies to both inform and validate the modeling assumptions. Based on the results of field tests, modeling assumptions and boundary conditions are refined and the process is repeated until the results are found to be reliable with a high level of confidence.

Reliability and validity assessment of gastrointestinal dystemperaments questionnaire: a novel scale in Persian traditional medicine

PubMed Central

Hoseinzadeh, Hamidreza; Taghipour, Ali; Yousefi, Mahdi

2018-01-01

Background Development of a questionnaire based on the resources of Persian traditional medicine seems necessary. One of the problems faced by practitioners of traditional medicine is the different opinions regarding the diagnosis of general temperament or temperament of member. One of the reasons is the lack of validity tools, and it has led to difficulties in training the student of traditional medicine and the treatment of patients. The differences in the detection methods, have given rise to several treatment methods. Objective The present study aimed to develop a questionnaire and standard software for diagnosis of gastrointestinal dystemperaments. Methods The present research is a tool developing study which included 8 stages of developing the items, determining the statements based on items, assessing the face validity, assessing the content validity, assessing the reliability, rating the items, developing a software for calculation of the total score of the questionnaire named GDS v.1.1, and evaluating the concurrent validity using statistical tests including Cronbach’s alpha coefficient, Cohen’s kappa coefficient. Results Based on the results, 112 notes including 62 symptoms were extracted from resources, and 58 items were obtained from in-person interview sessions with a panel of experts. A statement was selected for each item and, after merging a number of statements, a total of 49 statements were finally obtained. By calculating the score of statement impact and determining the content validity, respectively, 6 and 10 other items were removed from the list of statements. Standardized Cronbach’s alpha for this questionnaire was obtained 0.795 and its concurrent validity was equal to 0.8. Conclusion A quantitative tool was developed for diagnosis and examination of gastrointestinal dystemperaments. The developed questionnaire is adequately reliable and valid for this purpose. In addition, the software can be used for clinical diagnosis. PMID:29629060

Examining construct and predictive validity of the Health-IT Usability Evaluation Scale: confirmatory factor analysis and structural equation modeling results.

PubMed

Yen, Po-Yin; Sousa, Karen H; Bakken, Suzanne

2014-10-01

In a previous study, we developed the Health Information Technology Usability Evaluation Scale (Health-ITUES), which is designed to support customization at the item level. Such customization matches the specific tasks/expectations of a health IT system while retaining comparability at the construct level, and provides evidence of its factorial validity and internal consistency reliability through exploratory factor analysis. In this study, we advanced the development of Health-ITUES to examine its construct validity and predictive validity. The health IT system studied was a web-based communication system that supported nurse staffing and scheduling. Using Health-ITUES, we conducted a cross-sectional study to evaluate users' perception toward the web-based communication system after system implementation. We examined Health-ITUES's construct validity through first and second order confirmatory factor analysis (CFA), and its predictive validity via structural equation modeling (SEM). The sample comprised 541 staff nurses in two healthcare organizations. The CFA (n=165) showed that a general usability factor accounted for 78.1%, 93.4%, 51.0%, and 39.9% of the explained variance in 'Quality of Work Life', 'Perceived Usefulness', 'Perceived Ease of Use', and 'User Control', respectively. The SEM (n=541) supported the predictive validity of Health-ITUES, explaining 64% of the variance in intention for system use. The results of CFA and SEM provide additional evidence for the construct and predictive validity of Health-ITUES. The customizability of Health-ITUES has the potential to support comparisons at the construct level, while allowing variation at the item level. We also illustrate application of Health-ITUES across stages of system development. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Validation of the TTM processes of change measure for physical activity in an adult French sample.

PubMed

Bernard, Paquito; Romain, Ahmed-Jérôme; Trouillet, Raphael; Gernigon, Christophe; Nigg, Claudio; Ninot, Gregory

2014-04-01

Processes of change (POC) are constructs from the transtheoretical model that propose to examine how people engage in a behavior. However, there is no consensus about a leading model explaining POC and there is no validated French POC scale in physical activity This study aimed to compare the different existing models to validate a French POC scale. Three studies, with 748 subjects included, were carried out to translate the items and evaluate their clarity (study 1, n = 77), to assess the factorial validity (n = 200) and invariance/equivalence (study 2, n = 471), and to analyze the concurrent validity by stage × process analyses (study 3, n = 671). Two models displayed adequate fit to the data; however, based on the Akaike information criterion, the fully correlated five-factor model appeared as the most appropriate to measure POC in physical activity. The invariance/equivalence was also confirmed across genders and student status. Four of the five existing factors discriminated pre-action and post-action stages. These data support the validation of the POC questionnaire in physical activity among a French sample. More research is needed to explore the longitudinal properties of this scale.

Creation and validation of web-based food allergy audiovisual educational materials for caregivers.

PubMed

Rosen, Jamie; Albin, Stephanie; Sicherer, Scott H

2014-01-01

Studies reveal deficits in caregivers' ability to prevent and treat food-allergic reactions with epinephrine and a consumer preference for validated educational materials in audiovisual formats. This study was designed to create brief, validated educational videos on food allergen avoidance and emergency management of anaphylaxis for caregivers of children with food allergy. The study used a stepwise iterative process including creation of a needs assessment survey consisting of 25 queries administered to caregivers and food allergy experts to identify curriculum content. Preliminary videos were drafted, reviewed, and revised based on knowledge and satisfaction surveys given to another cohort of caregivers and health care professionals. The final materials were tested for validation of their educational impact and user satisfaction using pre- and postknowledge tests and satisfaction surveys administered to a convenience sample of 50 caretakers who had not participated in the development stages. The needs assessment identified topics of importance including treatment of allergic reactions and food allergen avoidance. Caregivers in the final validation included mothers (76%), fathers (22%), and other caregivers (2%). Race/ethnicity were white (66%), black (12%), Asian (12%), Hispanic (8%), and other (2%). Knowledge tests (maximum score = 18) increased from a mean score of 12.4 preprogram to 16.7 postprogram (p < 0.0001). On a 7-point Likert scale, all satisfaction categories remained above a favorable mean score of 6, indicating participants were overall very satisfied, learned a lot, and found the materials to be informative, straightforward, helpful, and interesting. This web-based audiovisual curriculum on food allergy improved knowledge scores and was well received.

Design of a Competency Evaluation Model for Clinical Nursing Practicum, Based on Standardized Language Systems: Psychometric Validation Study.

PubMed

Iglesias-Parra, Maria Rosa; García-Guerrero, Alfonso; García-Mayor, Silvia; Kaknani-Uttumchandani, Shakira; León-Campos, Álvaro; Morales-Asencio, José Miguel

2015-07-01

To develop an evaluation system of clinical competencies for the practicum of nursing students based on the Nursing Interventions Classification (NIC). Psychometric validation study: the first two phases addressed definition and content validation, and the third phase consisted of a cross-sectional study for analyzing reliability. The study population was undergraduate nursing students and clinical tutors. Through the Delphi technique, 26 competencies and 91 interventions were isolated. Cronbach's α was 0.96. Factor analysis yielded 18 factors that explained 68.82% of the variance. Overall inter-item correlation was 0.26, and total-item correlation ranged between 0.66 and 0.19. A competency system for the nursing practicum, structured on the NIC, is a reliable method for assessing and evaluating clinical competencies. Further evaluations in other contexts are needed. The availability of standardized language systems in the nursing discipline supposes an ideal framework to develop the nursing curricula. © 2015 Sigma Theta Tau International.

In Infants' Hands: Identification of Preverbal Infants at Risk for Primary Language Delay

ERIC Educational Resources Information Center

Lüke, Carina; Grimminger, Angela; Rohlfing, Katharina J.; Liszkowski, Ulf; Ritterfeld, Ute

2017-01-01

Early identification of primary language delay is crucial to implement effective prevention programs. Available screening instruments are based on parents' reports and have only insufficient predictive validity. This study employed observational measures of preverbal infants' gestural communication to test its predictive validity for identifying…

Development and Validation of the Educational Technologist Multimedia Competency Survey

ERIC Educational Resources Information Center

Ritzhaupt, Albert D.; Martin, Florence

2014-01-01

The purpose of this research study was to identify the multimedia competencies of an educational technologist by creating a valid and reliable survey instrument to administer to educational technology professionals. The educational technology multimedia competency survey developed through this research is based on a conceptual framework that…

Assessing Clinical Reasoning (ASCLIRE): Instrument Development and Validation

ERIC Educational Resources Information Center

Kunina-Habenicht, Olga; Hautz, Wolf E.; Knigge, Michel; Spies, Claudia; Ahlers, Olaf

2015-01-01

Clinical reasoning is an essential competency in medical education. This study aimed at developing and validating a test to assess diagnostic accuracy, collected information, and diagnostic decision time in clinical reasoning. A norm-referenced computer-based test for the assessment of clinical reasoning (ASCLIRE) was developed, integrating the…

Developing and Validating a Metacognitive Writing Questionnaire for EFL Learners

ERIC Educational Resources Information Center

Farahian, Majid

2017-01-01

In an attempt to develop a metacognitive writing questionnaire, Farahian (2015) conducted a study which was based on the results obtained from a semi-structured interview (Maftoon, Birjandi & Farahian, 2014). After running various exploratory factor analyses (EFA) to validate the questionnaire two general scales of knowledge and regulation of…

Validating an Observation Protocol to Measure Special Education Teacher Effectiveness

ERIC Educational Resources Information Center

Johnson, Evelyn S.; Semmelroth, Carrie L.

2015-01-01

This study used Kane's (2013) Interpretation/Use Argument (IUA) to measure validity on the Recognizing Effective Special Education Teachers (RESET) observation tool. The RESET observation tool is designed to evaluate special education teacher effectiveness using evidence-based instructional practices as the basis for evaluation. In alignment with…

Reliability and Validity of Curriculum-Based Informal Reading Inventories.

ERIC Educational Resources Information Center

Fuchs, Lynn; And Others

A study was conducted to explore the reliability and validity of three prominent procedures used in informal reading inventories (IRIs): (1) choosing a 95% word recognition accuracy standard for determining student instructional level, (2) arbitrarily selecting a passage to represent the difficulty level of a basal reader, and (3) employing…

Codification and Validation of Professional Development Questionnaire of Teachers

ERIC Educational Resources Information Center

Ayyoobi, Fatemah; Pourshafei, Hadi; Asgari, Ali

2016-01-01

Teacher in the educational system and the teaching-learning process, as a main leading should need to knowledge and professional skills. Therefore, evaluation of professional development is important. This study aims to design and modify Construction and Validation of professional development questionnaire of teachers. This research based on…

Does Linguistic Analysis Confirm the Validity of Facilitated Communication?

ERIC Educational Resources Information Center

Saloviita, Timo

2018-01-01

Facilitated communication (FC) has been interpreted as an ideomotor phenomenon, in which one person physically supports another person's hand and unconsciously affects the content of the writing. Despite the strong experimental evidence against the authenticity of FC output, several studies claim to support its validity based on idiosyncrasies…

Teaching Satisfaction Scale

ERIC Educational Resources Information Center

Ho, Chung-Lim; Au, Wing-Tung

2006-01-01

The present study proposes a teaching satisfaction measure and examines the validity of its scores. The measure is based on the Life Satisfaction Scale (LSS). Scores on the five-item Teaching Satisfaction Scale (TSS) were validated on a sample of 202 primary and secondary school teachers and favorable psychometric properties were found. As…

Are implicit motives revealed in mere words? Testing the marker-word hypothesis with computer-based text analysis

PubMed Central

Schultheiss, Oliver C.

2013-01-01

Traditionally, implicit motives (i.e., non-conscious preferences for specific classes of incentives) are assessed through semantic coding of imaginative stories. The present research tested the marker-word hypothesis, which states that implicit motives are reflected in the frequencies of specific words. Using Linguistic Inquiry and Word Count (LIWC; Pennebaker et al., 2001), Study 1 identified word categories that converged with a content-coding measure of the implicit motives for power, achievement, and affiliation in picture stories collected in German and US student samples, showed discriminant validity with self-reported motives, and predicted well-validated criteria of implicit motives (gender difference for the affiliation motive; in interaction with personal-goal progress: emotional well-being). Study 2 demonstrated LIWC-based motive scores' causal validity by documenting their sensitivity to motive arousal. PMID:24137149

Development of Chemistry Triangle Oriented Module on Topic of Reaction Rate for Senior High School Level Grade XI Chemistry Learning.

NASA Astrophysics Data System (ADS)

Sari, D. R.; Hardeli; Bayharti

2018-04-01

This study aims to produce chemistry triangle oriented module on topic of reaction rate, and to reveal the validity and practicality level of the generated module. The type of research used is EducationalDesign Research (EDR) with development model is Plompmodel. This model consists of three phases, which are preliminary research, prototyping phase, and assessment phase. The instrument used in this research is questionnaire validity and practicality. The data of the research were analyzed by using Kappa Cohen formula. The chemistry triangle oriented module validation sheet was given to 5 validators consisting of 3 chemistry lecturers and 2 high school chemistry teachers, while the practicality sheet was given to 2 chemistry teachers, 6 students of SMAN 10 Padang grade XII MIA 5 on the small groupevaluation and 25 students of SMAN 10 Padang grade XII MIA 6 on the field test. Based on the questionnaire validity analysis, the validity level of the module is very high with the value of kappa moment 0.87. The level of practicality based on teacher questionnaire response is very high category with a kappa moment value 0.96. Based on the questionnaire of student responses on small group evaluation, the level of practicality is very high category with a kappa moment 0.81, and the practicality is very high category with kappa moment value 0.83 based on questionnaire of student response on field test.

GOSAT validation out standing in the field: A case study of satellite validation using the SSEC Portable Atmospheric Research Center (SPARC)

NASA Astrophysics Data System (ADS)

Wagner, T. J.; Borg, L. A.; Feltz, M.; Gero, P. J.; Knuteson, R. O.; Olson, E.

2016-12-01

The Space Science and Engineering Center (SSEC) at the University of Wisconsin-Madison has developed the SSEC Portable Atmospheric Research Center (SPARC), a mobile 11 m trailer that houses numerous in situ and ground-based remote sensing instruments. Available instrumentation includes the Atmospheric Emitted Radiance Interferometer (AERI), a hyperspectral infrared radiometer from which trace gas concentrations and profiles of temperature and water vapor can be retrieved; the High Spectral Resolution Lidar (HSRL), a multichannel lidar capable of directly retrieving profiles of optical depth and backscatter depolarization; and a Doppler lidar wind profiler. The remote instrumentation suite is complemented by surface meteorology observations and a radiosonde ground station. Collectively, these instruments enable SPARC to participate in a wide variety of field studies, including meteorological field experiments and ground-based satellite calibration and validation studies. In August 2016, SPARC traveled to the Chequamegon National Forest in northern Wisconsin for a two week long deployment alongside the WLEF-TV tower. This 447 m tower houses long-term observations of thermodynamic and atmospheric composition at multiple heights, enabling studies of phenomena like atmospheric/land surface interactions and carbon uptake. During this deployment, SPARC launched radiosondes coincident with clear-sky overpasses of the Greenhouse gases Observing SATellite (GOSAT). Thermodynamic profiles from the radiosondes and AERI combined with the trace gas observations from the tower were used to validate the GOSAT observations of carbon dioxide and methane. The on-site presence of SPARC allowed for better characterization of the environment and greater observational certainty than was possible with the tower alone. Examples from this particular validation study as well as a discussion of how SPARC can contribute to other satellite calibration and validation investigations will be presented.

The Development and Validation of the Game User Experience Satisfaction Scale (GUESS).

PubMed

Phan, Mikki H; Keebler, Joseph R; Chaparro, Barbara S

2016-12-01

The aim of this study was to develop and psychometrically validate a new instrument that comprehensively measures video game satisfaction based on key factors. Playtesting is often conducted in the video game industry to help game developers build better games by providing insight into the players' attitudes and preferences. However, quality feedback is difficult to obtain from playtesting sessions without a quality gaming assessment tool. There is a need for a psychometrically validated and comprehensive gaming scale that is appropriate for playtesting and game evaluation purposes. The process of developing and validating this new scale followed current best practices of scale development and validation. As a result, a mixed-method design that consisted of item pool generation, expert review, questionnaire pilot study, exploratory factor analysis (N = 629), and confirmatory factor analysis (N = 729) was implemented. A new instrument measuring video game satisfaction, called the Game User Experience Satisfaction Scale (GUESS), with nine subscales emerged. The GUESS was demonstrated to have content validity, internal consistency, and convergent and discriminant validity. The GUESS was developed and validated based on the assessments of over 450 unique video game titles across many popular genres. Thus, it can be applied across many types of video games in the industry both as a way to assess what aspects of a game contribute to user satisfaction and as a tool to aid in debriefing users on their gaming experience. The GUESS can be administered to evaluate user satisfaction of different types of video games by a variety of users. © 2016, Human Factors and Ergonomics Society.

Methodology Series Module 9: Designing Questionnaires and Clinical Record Forms - Part II.

PubMed

Setia, Maninder Singh

2017-01-01

This article is a continuation of the previous module on designing questionnaires and clinical record form in which we have discussed some basic points about designing the questionnaire and clinical record forms. In this section, we will discuss the reliability and validity of questionnaires. The different types of validity are face validity, content validity, criterion validity, and construct validity. The different types of reliability are test-retest reliability, inter-rater reliability, and intra-rater reliability. Some of these parameters are assessed by subject area experts. However, statistical tests should be used for evaluation of other parameters. Once the questionnaire has been designed, the researcher should pilot test the questionnaire. The items in the questionnaire should be changed based on the feedback from the pilot study participants and the researcher's experience. After the basic structure of the questionnaire has been finalized, the researcher should assess the validity and reliability of the questionnaire or the scale. If an existing standard questionnaire is translated in the local language, the researcher should assess the reliability and validity of the translated questionnaire, and these values should be presented in the manuscript. The decision to use a self- or interviewer-administered, paper- or computer-based questionnaire depends on the nature of the questions, literacy levels of the target population, and resources.

Methodology Series Module 9: Designing Questionnaires and Clinical Record Forms – Part II

PubMed Central

Setia, Maninder Singh

2017-01-01

This article is a continuation of the previous module on designing questionnaires and clinical record form in which we have discussed some basic points about designing the questionnaire and clinical record forms. In this section, we will discuss the reliability and validity of questionnaires. The different types of validity are face validity, content validity, criterion validity, and construct validity. The different types of reliability are test-retest reliability, inter-rater reliability, and intra-rater reliability. Some of these parameters are assessed by subject area experts. However, statistical tests should be used for evaluation of other parameters. Once the questionnaire has been designed, the researcher should pilot test the questionnaire. The items in the questionnaire should be changed based on the feedback from the pilot study participants and the researcher's experience. After the basic structure of the questionnaire has been finalized, the researcher should assess the validity and reliability of the questionnaire or the scale. If an existing standard questionnaire is translated in the local language, the researcher should assess the reliability and validity of the translated questionnaire, and these values should be presented in the manuscript. The decision to use a self- or interviewer-administered, paper- or computer-based questionnaire depends on the nature of the questions, literacy levels of the target population, and resources. PMID:28584367

The Predictive Validity of Using Admissions Testing and Multiple Mini-Interviews in Undergraduate University Admissions

ERIC Educational Resources Information Center

Makransky, Guido; Havmose, Philip; Vang, Maria Louison; Andersen, Tonny Elmose; Nielsen, Tine

2017-01-01

The aim of this study was to evaluate the predictive validity of a two-step admissions procedure that included a cognitive ability test followed by multiple mini-interviews (MMIs) used to assess non-cognitive skills, compared to grade-based admissions relative to subsequent drop-out rates and academic achievement after one and two years of study.…

Construct Validity of "e-rater"® in Scoring TOEFL® Essays. Research Report. ETS RR-07-21

ERIC Educational Resources Information Center

Attali, Yigal

2007-01-01

This study examined the construct validity of the "e-rater"® automated essay scoring engine as an alternative to human scoring in the context of TOEFL® essay writing. Analyses were based on a sample of students who repeated the TOEFL within a short time period. Two "e-rater" scores were investigated in this study, the first…

Adaptation of the Boundary Violations Scale Developed Based on Structural Family Therapy to the Turkish Context: A Study of Validity and Reliability

ERIC Educational Resources Information Center

Avci, Rasit; Çolakkadioglu, Oguzhan; Öz, Aysegül Sükran; Akbas, Turan

2015-01-01

The purpose of this study was to adapt "The Boundary Violations Scale" (Madden et al., 2002), which was created to measure the intergenerational boundary violations in families from the perspective of children, to Turkish and to test the validity and reliability of the Turkish version of this instrument. This instrument was developed…

The Development of a Secondary-Level Solo Wind Instrument Performance Rubric Using the Multifaceted Rasch Partial Credit Measurement Model

ERIC Educational Resources Information Center

Wesolowski, Brian C.; Amend, Ross M.; Barnstead, Thomas S.; Edwards, Andrew S.; Everhart, Matthew; Goins, Quentin R.; Grogan, Robert J., III; Herceg, Amanda M.; Jenkins, S. Ira; Johns, Paul M.; McCarver, Christopher J.; Schaps, Robin E.; Sorrell, Gary W.; Williams, Jonathan D.

2017-01-01

The purpose of this study was to describe the development of a valid and reliable rubric to assess secondary-level solo instrumental music performance based on principles of invariant measurement. The research questions that guided this study included (1) What is the psychometric quality (i.e., validity, reliability, and precision) of a scale…

Validation of the openEHR archetype library by using OWL reasoning.

PubMed

Menárguez-Tortosa, Marcos; Fernández-Breis, Jesualdo Tomás

2011-01-01

Electronic Health Record architectures based on the dual model architecture use archetypes for representing clinical knowledge. Therefore, ensuring their correctness and consistency is a fundamental research goal. In this work, we explore how an approach based on OWL technologies can be used for such purpose. This method has been applied to the openEHR archetype repository, which is the largest available one nowadays. The results of this validation are also reported in this study.

Development and validation of the ASPIRE-VA coaching fidelity checklist (ACFC): a tool to help ensure delivery of high-quality weight management interventions.

PubMed

Damschroder, Laura J; Goodrich, David E; Kim, Hyungjin Myra; Holleman, Robert; Gillon, Leah; Kirsh, Susan; Richardson, Caroline R; Lutes, Lesley D

2016-09-01

Practical and valid instruments are needed to assess fidelity of coaching for weight loss. The purpose of this study was to develop and validate the ASPIRE Coaching Fidelity Checklist (ACFC). Classical test theory guided ACFC development. Principal component analyses were used to determine item groupings. Psychometric properties, internal consistency, and inter-rater reliability were evaluated for each subscale. Criterion validity was tested by predicting weight loss as a function of coaching fidelity. The final 19-item ACFC consists of two domains (session process and session structure) and five subscales (sets goals and monitor progress, assess and personalize self-regulatory content, manages the session, creates a supportive and empathetic climate, and stays on track). Four of five subscales showed high internal consistency (Cronbach alphas > 0.70) for group-based coaching; only two of five subscales had high internal reliability for phone-based coaching. All five sub-scales were positively and significantly associated with weight loss for group- but not for phone-based coaching. The ACFC is a reliable and valid instrument that can be used to assess fidelity and guide skill-building for weight management interventionists.

Modeling river total bed material load discharge using artificial intelligence approaches (based on conceptual inputs)

NASA Astrophysics Data System (ADS)

Roushangar, Kiyoumars; Mehrabani, Fatemeh Vojoudi; Shiri, Jalal

2014-06-01

This study presents Artificial Intelligence (AI)-based modeling of total bed material load through developing the accuracy level of the predictions of traditional models. Gene expression programming (GEP) and adaptive neuro-fuzzy inference system (ANFIS)-based models were developed and validated for estimations. Sediment data from Qotur River (Northwestern Iran) were used for developing and validation of the applied techniques. In order to assess the applied techniques in relation to traditional models, stream power-based and shear stress-based physical models were also applied in the studied case. The obtained results reveal that developed AI-based models using minimum number of dominant factors, give more accurate results than the other applied models. Nonetheless, it was revealed that k-fold test is a practical but high-cost technique for complete scanning of applied data and avoiding the over-fitting.

Developing Guided Inquiry-Based Student Lab Worksheet for Laboratory Knowledge Course

NASA Astrophysics Data System (ADS)

Rahmi, Y. L.; Novriyanti, E.; Ardi, A.; Rifandi, R.

2018-04-01

The course of laboratory knowledge is an introductory course for biology students to follow various lectures practicing in the biology laboratory. Learning activities of laboratory knowledge course at this time in the Biology Department, Universitas Negeri Padang has not been completed by supporting learning media such as student lab worksheet. Guided inquiry learning model is one of the learning models that can be integrated into laboratory activity. The study aimed to produce student lab worksheet based on guided inquiry for laboratory knowledge course and to determine the validity of lab worksheet. The research was conducted using research and developmet (R&D) model. The instruments used in data collection in this research were questionnaire for student needed analysis and questionnaire to measure the student lab worksheet validity. The data obtained was quantitative from several validators. The validators consist of three lecturers. The percentage of a student lab worksheet validity was 94.18 which can be categorized was very good.

Development of self and peer performance assessment on iodometric titration experiment

NASA Astrophysics Data System (ADS)

Nahadi; Siswaningsih, W.; Kusumaningtyas, H.

2018-05-01

This study aims to describe the process in developing of reliable and valid assessment to measure students’ performance on iodometric titration and the effect of the self and peer assessment on students’ performance. The self and peer-instrument provides valuable feedback for the student performance improvement. The developed assessment contains rubric and task for facilitating self and peer assessment. The participants are 24 students at the second-grade student in certain vocational high school in Bandung. The participants divided into two groups. The first 12 students involved in the validity test of the developed assessment, while the remain 12 students participated for the reliability test. The content validity was evaluated based on the judgment experts. Test result of content validity based on judgment expert show that the developed performance assessment instrument categorized as valid on each task with the realibity classified as very good. Analysis of the impact of the self and peer assessment implementation showed that the peer instrument supported the self assessment.

Validity evidence as a key marker of quality of technical skill assessment in OTL-HNS.

PubMed

Labbé, Mathilde; Young, Meredith; Nguyen, Lily H P

2018-01-13

Quality monitoring of assessment practices should be a priority in all residency programs. Validity evidence is one of the main hallmarks of assessment quality and should be collected to support the interpretation and use of assessment data. Our objective was to identify, synthesize, and present the validity evidence reported supporting different technical skill assessment tools in otolaryngology-head and neck surgery (OTL-HNS). We performed a secondary analysis of data generated through a systematic review of all published tools for assessing technical skills in OTL-HNS (n = 16). For each tool, we coded validity evidence according to the five types of evidence described by the American Educational Research Association's interpretation of Messick's validity framework. Descriptive statistical analyses were conducted. All 16 tools included in our analysis were supported by internal structure and relationship to variables validity evidence. Eleven articles presented evidence supporting content. Response process was discussed only in one article, and no study reported on evidence exploring consequences. We present the validity evidence reported for 16 rater-based tools that could be used for work-based assessment of OTL-HNS residents in the operating room. The articles included in our review were consistently deficient in evidence for response process and consequences. Rater-based assessment tools that support high-stakes decisions that impact the learner and programs should include several sources of validity evidence. Thus, use of any assessment should be done with careful consideration of the context-specific validity evidence supporting score interpretation, and we encourage deliberate continual assessment quality-monitoring. NA. Laryngoscope, 2018. © 2018 The American Laryngological, Rhinological and Otological Society, Inc.

A systematic review of a functional assessment Tool: UCSD Performance-based skill assessment (UPSA).

PubMed

Becattini-Oliveira, Ana Claudia; Dutra, Douglas de Farias; Spenciere de Oliveira Campos, Bárbara; de Araujo, Verônica Carvalho; Charchat-Fichman, Helenice

2018-05-18

Performance based assessment instruments have been employed in functional capacity measurement of mental disorders. The aim of this systematic review was to identify the psychometric properties of the UCSD Performance-based Skill Assessment (UPSA). A search was conducted using the PRISMA protocol and 'UPSA' as key word term on electronic databases, with a date range for articles published from 2001-2017. Published studies involving community-dwelling adults were included. Pharmacological and/or clinical interventions involving clinical outcomes and/or institutionalized samples were excluded. Data related to construct validity, test-retest reliability and sensitivity/specificity were extracted, summarized and analyzed according to UPSA versions and psychiatric disorders. Fifty-eight studies including 8782 Community-dwelling adults met selection criteria. Data supporting the construct and known-groups validity were extracted from 41 studies involving Schizophrenia and schizoaffective disorders and 17 studies involving other metal illness. The UPSA was culturally adapted to 8 different languages and employed in 17 countries. Few studies reported sensitivity and specificity and the cut-off points could not be generalized. Moderate to strong evidence of construct validity and test-retest reliability was found. Few studies proposed cut-off points. The UPSA showed good psychometric properties in different versions including those culturally adapted. Copyright © 2018 Elsevier B.V. All rights reserved.

Measurement properties of tools measuring mental health knowledge: a systematic review.

PubMed

Wei, Yifeng; McGrath, Patrick J; Hayden, Jill; Kutcher, Stan

2016-08-23

Mental health literacy has received great attention recently to improve mental health knowledge, decrease stigma and enhance help-seeking behaviors. We conducted a systematic review to critically appraise the qualities of studies evaluating the measurement properties of mental health knowledge tools and the quality of included measurement properties. We searched PubMed, PsycINFO, EMBASE, CINAHL, the Cochrane Library, and ERIC for studies addressing psychometrics of mental health knowledge tools and published in English. We applied the COSMIN checklist to assess the methodological quality of each study as "excellent", "good", "fair", or "indeterminate". We ranked the level of evidence of the overall quality of each measurement property across studies as "strong", "moderate", "limited", "conflicting", or "unknown". We identified 16 mental health knowledge tools in 17 studies, addressing reliability, validity, responsiveness or measurement errors. The methodological quality of included studies ranged from "poor" to "excellent" including 6 studies addressing the content validity, internal consistency or structural validity demonstrating "excellent" quality. We found strong evidence of the content validity or internal consistency of 6 tools; moderate evidence of the internal consistency, the content validity or the reliability of 8 tools; and limited evidence of the reliability, the structural validity, the criterion validity, or the construct validity of 12 tools. Both the methodological qualities of included studies and the overall evidence of measurement properties are mixed. Based on the current evidence, we recommend that researchers consider using tools with measurement properties of strong or moderate evidence that also reached the threshold for positive ratings according to COSMIN checklist.

Geriatric Patient Safety Indicators Based on Linked Administrative Health Data to Assess Anticoagulant-Related Thromboembolic and Hemorrhagic Adverse Events in Older Inpatients: A Study Proposal

PubMed Central

Quantin, Catherine; Reich, Oliver; Tuppin, Philippe; Fagot-Campagna, Anne; Paccaud, Fred; Peytremann-Bridevaux, Isabelle; Burnand, Bernard

2017-01-01

Background Frail older people with multiple interacting conditions, polypharmacy, and complex care needs are particularly exposed to health care-related adverse events. Among these, anticoagulant-related thromboembolic and hemorrhagic events are particularly frequent and serious in older inpatients. The growing use of anticoagulants in this population and their substantial risk of toxicity and inefficacy have therefore become an important patient safety and public health concern worldwide. Anticoagulant-related adverse events and the quality of anticoagulation management should thus be routinely assessed to improve patient safety in vulnerable older inpatients. Objective This project aims to develop and validate a set of outcome and process indicators based on linked administrative health data (ie, insurance claims data linked to hospital discharge data) assessing older inpatient safety related to anticoagulation in both Switzerland and France, and enabling comparisons across time and among hospitals, health territories, and countries. Geriatric patient safety indicators (GPSIs) will assess anticoagulant-related adverse events. Geriatric quality indicators (GQIs) will evaluate the management of anticoagulants for the prevention and treatment of arterial or venous thromboembolism in older inpatients. Methods GPSIs will measure cumulative incidences of thromboembolic and bleeding adverse events based on hospital discharge data linked to insurance claims data. Using linked administrative health data will improve GPSI risk adjustment on patients’ conditions that are present at admission and will capture in-hospital and postdischarge adverse events. GQIs will estimate the proportion of index hospital stays resulting in recommended anticoagulation at discharge and up to various time frames based on the same electronic health data. The GPSI and GQI development and validation process will comprise 6 stages: (1) selection and specification of candidate indicators, (2) definition of administrative data-based algorithms, (3) empirical measurement of indicators using linked administrative health data, (4) validation of indicators, (5) analyses of geographic and temporal variations for reliable and valid indicators, and (6) data visualization. Results Study populations will consist of 166,670 Swiss and 5,902,037 French residents aged 65 years and older admitted to an acute care hospital at least once during the 2012-2014 period and insured for at least 1 year before admission and 1 year after discharge. We will extract Swiss data from the Helsana Group data warehouse and French data from the national health insurance information system (SNIIR-AM). The study has been approved by Swiss and French ethics committees and regulatory organizations for data protection. Conclusions Validated GPSIs and GQIs should help support and drive quality and safety improvement in older inpatients, inform health care stakeholders, and enable international comparisons. We discuss several limitations relating to the representativeness of study populations, accuracy of administrative health data, methods used for GPSI criterion validity assessment, and potential confounding bias in comparisons based on GQIs, and we address these limitations to strengthen study feasibility and validity. PMID:28495660

Infant polysomnography: reliability and validity of infant arousal assessment.

PubMed

Crowell, David H; Kulp, Thomas D; Kapuniai, Linda E; Hunt, Carl E; Brooks, Lee J; Weese-Mayer, Debra E; Silvestri, Jean; Ward, Sally Davidson; Corwin, Michael; Tinsley, Larry; Peucker, Mark

2002-10-01

Infant arousal scoring based on the Atlas Task Force definition of transient EEG arousal was evaluated to determine (1). whether transient arousals can be identified and assessed reliably in infants and (2). whether arousal and no-arousal epochs scored previously by trained raters can be validated reliably by independent sleep experts. Phase I for inter- and intrarater reliability scoring was based on two datasets of sleep epochs selected randomly from nocturnal polysomnograms of healthy full-term, preterm, idiopathic apparent life-threatening event cases, and siblings of Sudden Infant Death Syndrome infants of 35 to 64 weeks postconceptional age. After training, test set 1 reliability was assessed and discrepancies identified. After retraining, test set 2 was scored by the same raters to determine interrater reliability. Later, three raters from the trained group rescored test set 2 to assess inter- and intrarater reliabilities. Interrater and intrarater reliability kappa's, with 95% confidence intervals, ranged from substantial to almost perfect levels of agreement. Interrater reliabilities for spontaneous arousals were initially moderate and then substantial. During the validation phase, 315 previously scored epochs were presented to four sleep experts to rate as containing arousal or no-arousal events. Interrater expert agreements were diverse and considered as noninterpretable. Concordance in sleep experts' agreements, based on identification of the previously sampled arousal and no-arousal epochs, was used as a secondary evaluative technique. Results showed agreement by two or more experts on 86% of the Collaborative Home Infant Monitoring Evaluation Study arousal scored events. Conversely, only 1% of the Collaborative Home Infant Monitoring Evaluation Study-scored no-arousal epochs were rated as an arousal. In summary, this study presents an empirically tested model with procedures and criteria for attaining improved reliability in transient EEG arousal assessments in infants using the modified Atlas Task Force standards. With training based on specific criteria, substantial inter- and intrarater agreement in identifying infant arousals was demonstrated. Corroborative validation results were too disparate for meaningful interpretation. Alternate evaluation based on concordance agreements supports reliance on infant EEG criteria for assessment. Results mandate additional confirmatory validation studies with specific training on infant EEG arousal assessment criteria.

Developing and validating an instrument for measuring mobile computing self-efficacy.

PubMed

Wang, Yi-Shun; Wang, Hsiu-Yuan

2008-08-01

IT-related self-efficacy has been found to have a critical influence on system use. However, traditional measures of computer self-efficacy and Internet-related self-efficacy are perceived to be inapplicable in the context of mobile computing and commerce because they are targeted primarily at either desktop computer or wire-based technology contexts. Based on previous research, this study develops and validates a multidimensional instrument for measuring mobile computing self-efficacy (MCSE). This empirically validated instrument will be useful to researchers in developing and testing the theories of mobile user behavior, and to practitioners in assessing the mobile computing self-efficacy of users and promoting the use of mobile commerce systems.

External validation of type 2 diabetes computer simulation models: definitions, approaches, implications and room for improvement-a protocol for a systematic review.

PubMed

Ogurtsova, Katherine; Heise, Thomas L; Linnenkamp, Ute; Dintsios, Charalabos-Markos; Lhachimi, Stefan K; Icks, Andrea

2017-12-29

Type 2 diabetes mellitus (T2DM), a highly prevalent chronic disease, puts a large burden on individual health and health care systems. Computer simulation models, used to evaluate the clinical and economic effectiveness of various interventions to handle T2DM, have become a well-established tool in diabetes research. Despite the broad consensus about the general importance of validation, especially external validation, as a crucial instrument of assessing and controlling for the quality of these models, there are no systematic reviews comparing such validation of diabetes models. As a result, the main objectives of this systematic review are to identify and appraise the different approaches used for the external validation of existing models covering the development and progression of T2DM. We will perform adapted searches by applying respective search strategies to identify suitable studies from 14 electronic databases. Retrieved study records will be included or excluded based on predefined eligibility criteria as defined in this protocol. Among others, a publication filter will exclude studies published before 1995. We will run abstract and full text screenings and then extract data from all selected studies by filling in a predefined data extraction spreadsheet. We will undertake a descriptive, narrative synthesis of findings to address the study objectives. We will pay special attention to aspects of quality of these models in regard to the external validation based upon ISPOR and ADA recommendations as well as Mount Hood Challenge reports. All critical stages within the screening, data extraction and synthesis processes will be conducted by at least two authors. This protocol adheres to PRISMA and PRISMA-P standards. The proposed systematic review will provide a broad overview of the current practice in the external validation of models with respect to T2DM incidence and progression in humans built on simulation techniques. PROSPERO CRD42017069983 .

Validity of Basic Electronic 1 Module Integrated Character Value Based on Conceptual Change Teaching Model to Increase Students Physics Competency in STKIP PGRI West Sumatera

NASA Astrophysics Data System (ADS)

Hidayati, A.; Rahmi, A.; Yohandri; Ratnawulan

2018-04-01

The importance of teaching materials in accordance with the characteristics of students became the main reason for the development of basic electronics I module integrated character values based on conceptual change teaching model. The module development in this research follows the development procedure of Plomp which includes preliminary research, prototyping phase and assessment phase. In the first year of this research, the module is validated. Content validity is seen from the conformity of the module with the development theory in accordance with the demands of learning model characteristics. The validity of the construct is seen from the linkage and consistency of each module component developed with the characteristic of the integrated learning model of character values obtained through validator assessment. The average validation value assessed by the validator belongs to a very valid category. Based on the validator assessment then revised the basic electronics I module integrated character values based on conceptual change teaching model.

Development and initial validation of an instrument to assess stressors among South African sports coaches.

PubMed

Kubayi, Alliance; Toriola, Abel; Didymus, Faye

2018-06-01

The aim of this series of studies was to develop and initially validate an instrument to assess stressors among South African sports coaches. In study one, a preliminary pool of 45 items was developed based on existing literature and an expert panel was employed to assess the content validity and applicability of these items. In study two, the 32 items that were retained after study one were analysed using principal component analysis (PCA). The resultant factorial structure comprised four components: environmental stressors, performance stressors, task-related stressors, and athlete stressors. These four components were made up of 26 items and, together, the components and items comprised the provisional Stressors in Sports Coaching Questionnaire (SSCQ). The results show that the SSCQ demonstrates acceptable internal consistency (.73-.89). The findings provide preliminary evidence that SSCQ is a valid tool to assess stressors among South African sports coaches.

All framing effects are not created equal: Low convergent validity between two classic measurements of framing

PubMed Central

Zhen, Shanshan; Yu, Rongjun

2016-01-01

Human risk-taking attitudes can be influenced by two logically equivalent but descriptively different frames, termed the framing effect. The classic hypothetical vignette-based task (Asian disease problem) and a recently developed reward-based gambling task have been widely used to assess individual differences in the framing effect. Previous studies treat framing bias as a stable trait that has genetic basis. However, these two paradigms differ in terms of task domain (loss vs. gain) and task context (vignette-based vs. reward-based) and the convergent validity of these measurements remains unknown. Here, we developed a vignette-based task and a gambling task in both gain and loss domains and tested correlations of the framing effect among these tasks in 159 young adults. Our results revealed no significant correlation between the vignette-based task in the loss domain and the gambling task in the gain domain, indicating low convergent validity. The current findings raise the question of how to measure the framing effect precisely, especially in individual difference studies using large samples and expensive neuroscience methods. Our results suggest that the framing effect is influenced by both task domain and task context and future research should be cautious about the operationalization of the framing effect. PMID:27436680

All framing effects are not created equal: Low convergent validity between two classic measurements of framing.

PubMed

Zhen, Shanshan; Yu, Rongjun

2016-07-20

Human risk-taking attitudes can be influenced by two logically equivalent but descriptively different frames, termed the framing effect. The classic hypothetical vignette-based task (Asian disease problem) and a recently developed reward-based gambling task have been widely used to assess individual differences in the framing effect. Previous studies treat framing bias as a stable trait that has genetic basis. However, these two paradigms differ in terms of task domain (loss vs. gain) and task context (vignette-based vs. reward-based) and the convergent validity of these measurements remains unknown. Here, we developed a vignette-based task and a gambling task in both gain and loss domains and tested correlations of the framing effect among these tasks in 159 young adults. Our results revealed no significant correlation between the vignette-based task in the loss domain and the gambling task in the gain domain, indicating low convergent validity. The current findings raise the question of how to measure the framing effect precisely, especially in individual difference studies using large samples and expensive neuroscience methods. Our results suggest that the framing effect is influenced by both task domain and task context and future research should be cautious about the operationalization of the framing effect.

Validation of a personalized dosimetric evaluation tool (Oedipe) for targeted radiotherapy based on the Monte Carlo MCNPX code

NASA Astrophysics Data System (ADS)

Chiavassa, S.; Aubineau-Lanièce, I.; Bitar, A.; Lisbona, A.; Barbet, J.; Franck, D.; Jourdain, J. R.; Bardiès, M.

2006-02-01

Dosimetric studies are necessary for all patients treated with targeted radiotherapy. In order to attain the precision required, we have developed Oedipe, a dosimetric tool based on the MCNPX Monte Carlo code. The anatomy of each patient is considered in the form of a voxel-based geometry created using computed tomography (CT) images or magnetic resonance imaging (MRI). Oedipe enables dosimetry studies to be carried out at the voxel scale. Validation of the results obtained by comparison with existing methods is complex because there are multiple sources of variation: calculation methods (different Monte Carlo codes, point kernel), patient representations (model or specific) and geometry definitions (mathematical or voxel-based). In this paper, we validate Oedipe by taking each of these parameters into account independently. Monte Carlo methodology requires long calculation times, particularly in the case of voxel-based geometries, and this is one of the limits of personalized dosimetric methods. However, our results show that the use of voxel-based geometry as opposed to a mathematically defined geometry decreases the calculation time two-fold, due to an optimization of the MCNPX2.5e code. It is therefore possible to envisage the use of Oedipe for personalized dosimetry in the clinical context of targeted radiotherapy.

Validity and reliability of portfolio assessment of student competence in two dental school populations: a four-year study.

PubMed

Gadbury-Amyot, Cynthia C; McCracken, Michael S; Woldt, Janet L; Brennan, Robert L

2014-05-01

The purpose of this study was to empirically investigate the validity and reliability of portfolio assessment in two U.S. dental schools using a unified framework for validity. In the process of validation, it is not the test that is validated but rather the claims (interpretations and uses) about test scores that are validated. Kane's argument-based validation framework provided the structure for reporting results where validity claims are followed by evidence to support the argument. This multivariate generalizability theory study found that the greatest source of variance was attributable to faculty raters, suggesting that portfolio assessment would benefit from two raters' evaluating each portfolio independently. The results are generally supportive of holistic scoring, but analytical scoring deserves further research. Correlational analyses between student portfolios and traditional measures of student competence and readiness for licensure resulted in significant correlations between portfolios and National Board Dental Examination Part I (r=0.323, p<0.01) and Part II scores (r=0.268, p<0.05) and small and non-significant correlations with grade point average and scores on the Western Regional Examining Board (WREB) exam. It is incumbent upon the users of portfolio assessment to determine if the claims and evidence arguments set forth in this study support the proposed claims for and decisions about portfolio assessment in their respective institutions.

Systematic literature review and assessment of patient-reported outcome instruments in sickle cell disease.

PubMed

Sarri, Grammati; Bhor, Menaka; Abogunrin, Seye; Farmer, Caroline; Nandal, Savita; Halloway, Rashid; Revicki, Dennis A

2018-05-21

Sickle cell disease (SCD) is a chronic condition associated with high mortality and morbidity. It is characterized by acute clinical symptoms such as painful vaso-occlusive crises, which can impair health-related quality of life (HRQL). This study was conducted to identify validated patient-reported outcome (PRO) instruments for use in future trials of potential treatments for SCD. A systematic literature review (SLR) was performed using MEDLINE and EMBASE to identify United States (US)-based studies published in English between 1997 and 2017 that reported on validated PRO instruments used in randomized controlled trials and real-world settings. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist was used to assess the quality of PRO instruments. The SLR included 21 studies assessing the psychometric properties of 24 PRO instruments. Fifteen of those instruments were developed and validated for adults and 10 for children (one instrument was used in both children and young adults aged up to 21 years). Only five of the 15 adult instruments and three of the 10 pediatric instruments were developed specifically for SCD. For most instruments, there were few or no data on validation conducted in SCD development cohorts. Of the 24 PRO instruments identified, 16 had strong internal reliability (Cronbach's α ≥0.80). There was often insufficient information to assess the content validity, construct validity, responsiveness, or test-retest reliability of the instruments identified for both child and adult populations. No validated PRO instruments measuring caregiver burden in SCD were identified. The evidence on the psychometric properties of PRO instruments was limited. However, the results of this SLR provide key information on such tools to help inform the design of future clinical trials for patients with SCD in the US.

Improving the quality of discrete-choice experiments in health: how can we assess validity and reliability?

PubMed

Janssen, Ellen M; Marshall, Deborah A; Hauber, A Brett; Bridges, John F P

2017-12-01

The recent endorsement of discrete-choice experiments (DCEs) and other stated-preference methods by regulatory and health technology assessment (HTA) agencies has placed a greater focus on demonstrating the validity and reliability of preference results. Areas covered: We present a practical overview of tests of validity and reliability that have been applied in the health DCE literature and explore other study qualities of DCEs. From the published literature, we identify a variety of methods to assess the validity and reliability of DCEs. We conceptualize these methods to create a conceptual model with four domains: measurement validity, measurement reliability, choice validity, and choice reliability. Each domain consists of three categories that can be assessed using one to four procedures (for a total of 24 tests). We present how these tests have been applied in the literature and direct readers to applications of these tests in the health DCE literature. Based on a stakeholder engagement exercise, we consider the importance of study characteristics beyond traditional concepts of validity and reliability. Expert commentary: We discuss study design considerations to assess the validity and reliability of a DCE, consider limitations to the current application of tests, and discuss future work to consider the quality of DCEs in healthcare.

Development and validation of an online interactive, multimedia wound care algorithms program.

PubMed

Beitz, Janice M; van Rijswijk, Lia

2012-01-01

To provide education based on evidence-based and validated wound care algorithms we designed and implemented an interactive, Web-based learning program for teaching wound care. A mixed methods quantitative pilot study design with qualitative components was used to test and ascertain the ease of use, validity, and reliability of the online program. A convenience sample of 56 RN wound experts (formally educated, certified in wound care, or both) participated. The interactive, online program consists of a user introduction, interactive assessment of 15 acute and chronic wound photos, user feedback about the percentage correct, partially correct, or incorrect algorithm and dressing choices and a user survey. After giving consent, participants accessed the online program, provided answers to the demographic survey, and completed the assessment module and photographic test, along with a posttest survey. The construct validity of the online interactive program was strong. Eighty-five percent (85%) of algorithm and 87% of dressing choices were fully correct even though some programming design issues were identified. Online study results were consistently better than previously conducted comparable paper-pencil study results. Using a 5-point Likert-type scale, participants rated the program's value and ease of use as 3.88 (valuable to very valuable) and 3.97 (easy to very easy), respectively. Similarly the research process was described qualitatively as "enjoyable" and "exciting." This digital program was well received indicating its "perceived benefits" for nonexpert users, which may help reduce barriers to implementing safe, evidence-based care. Ongoing research using larger sample sizes may help refine the program or algorithms while identifying clinician educational needs. Initial design imperfections and programming problems identified also underscored the importance of testing all paper and Web-based programs designed to educate health care professionals or guide patient care.

Full immersion simulation: validation of a distributed simulation environment for technical and non-technical skills training in Urology.

PubMed

Brewin, James; Tang, Jessica; Dasgupta, Prokar; Khan, Muhammad S; Ahmed, Kamran; Bello, Fernando; Kneebone, Roger; Jaye, Peter

2015-07-01

To evaluate the face, content and construct validity of the distributed simulation (DS) environment for technical and non-technical skills training in endourology. To evaluate the educational impact of DS for urology training. DS offers a portable, low-cost simulated operating room environment that can be set up in any open space. A prospective mixed methods design using established validation methodology was conducted in this simulated environment with 10 experienced and 10 trainee urologists. All participants performed a simulated prostate resection in the DS environment. Outcome measures included surveys to evaluate the DS, as well as comparative analyses of experienced and trainee urologist's performance using real-time and 'blinded' video analysis and validated performance metrics. Non-parametric statistical methods were used to compare differences between groups. The DS environment demonstrated face, content and construct validity for both non-technical and technical skills. Kirkpatrick level 1 evidence for the educational impact of the DS environment was shown. Further studies are needed to evaluate the effect of simulated operating room training on real operating room performance. This study has shown the validity of the DS environment for non-technical, as well as technical skills training. DS-based simulation appears to be a valuable addition to traditional classroom-based simulation training. © 2014 The Authors BJU International © 2014 BJU International Published by John Wiley & Sons Ltd.

Feasibility, reliability, and validity of a smartphone based application for the assessment of cognitive function in the elderly.

PubMed

Brouillette, Robert M; Foil, Heather; Fontenot, Stephanie; Correro, Anthony; Allen, Ray; Martin, Corby K; Bruce-Keller, Annadora J; Keller, Jeffrey N

2013-01-01

While considerable knowledge has been gained through the use of established cognitive and motor assessment tools, there is a considerable interest and need for the development of a battery of reliable and validated assessment tools that provide real-time and remote analysis of cognitive and motor function in the elderly. Smartphones appear to be an obvious choice for the development of these "next-generation" assessment tools for geriatric research, although to date no studies have reported on the use of smartphone-based applications for the study of cognition in the elderly. The primary focus of the current study was to assess the feasibility, reliability, and validity of a smartphone-based application for the assessment of cognitive function in the elderly. A total of 57 non-demented elderly individuals were administered a newly developed smartphone application-based Color-Shape Test (CST) in order to determine its utility in measuring cognitive processing speed in the elderly. Validity of this novel cognitive task was assessed by correlating performance on the CST with scores on widely accepted assessments of cognitive function. Scores on the CST were significantly correlated with global cognition (Mini-Mental State Exam: r = 0.515, p<0.0001) and multiple measures of processing speed and attention (Digit Span: r = 0.427, p<0.0001; Trail Making Test: r = -0.651, p<0.00001; Digit Symbol Test: r = 0.508, p<0.0001). The CST was not correlated with naming and verbal fluency tasks (Boston Naming Test, Vegetable/Animal Naming) or memory tasks (Logical Memory Test). Test re-test reliability was observed to be significant (r = 0.726; p = 0.02). Together, these data are the first to demonstrate the feasibility, reliability, and validity of using a smartphone-based application for the purpose of assessing cognitive function in the elderly. The importance of these findings for the establishment of smartphone-based assessment batteries of cognitive and motor function in the elderly is discussed.

Development and Validation of a Symptom-Based Activity Index for Adults with Eosinophilic Esophagitis

PubMed Central

Schoepfer, Alain M.; Straumann, Alex; Panczak, Radoslaw; Coslovsky, Michael; Kuehni, Claudia E.; Maurer, Elisabeth; Haas, Nadine A.; Romero, Yvonne; Hirano, Ikuo; Alexander, Jeffrey A.; Gonsalves, Nirmala; Furuta, Glenn T.; Dellon, Evan S.; Leung, John; Collins, Margaret H.; Bussmann, Christian; Netzer, Peter; Gupta, Sandeep K.; Aceves, Seema S.; Chehade, Mirna; Moawad, Fouad J.; Enders, Felicity T.; Yost, Kathleen J.; Taft, Tiffany H.; Kern, Emily; Zwahlen, Marcel; Safroneeva, Ekaterina

2015-01-01

BACKGROUND & AIMS Standardized instruments are needed to assess the activity of eosinophilic esophagitis (EoE), to provide endpoints for clinical trials and observational studies. We aimed to develop and validate a patient-reported outcome (PRO) instrument and score, based on items that could account for variations in patients’ assessments of disease severity. We also evaluated relationships between patients’ assessment of disease severity and EoE-associated endoscopic, histologic, and laboratory findings. METHODS We collected information from 186 patients with EoE in Switzerland and the US (69.4% male; median age, 43 years) via surveys (n = 135), focus groups (n = 27), and semi-structured interviews (n = 24). Items were generated for the instruments to assess biologic activity based on physician input. Linear regression was used to quantify the extent to which variations in patient-reported disease characteristics could account for variations in patients’ assessment of EoE severity. The PRO instrument was prospectively used in 153 adult patients with EoE (72.5% male; median age, 38 years), and validated in an independent group of 120 patients with EoE (60.8% male; median age, 40.5 years). RESULTS Seven PRO factors that are used to assess characteristics of dysphagia, behavioral adaptations to living with dysphagia, and pain while swallowing accounted for 67% of the variation in patients’ assessment of disease severity. Based on statistical consideration and patient input, a 7-day recall period was selected. Highly active EoE, based on endoscopic and histologic findings, was associated with an increase in patient-assessed disease severity. In the validation study, the mean difference between patient assessment of EoE severity and PRO score was 0.13 (on a scale from 0 to 10). CONCLUSIONS We developed and validated an EoE scoring system based on 7 PRO items that assesses symptoms over a 7-day recall period. Clinicaltrials.gov number: NCT00939263. PMID:25160980

A review of preference-based measures for the assessment of quality of life in children and adolescents with cerebral palsy.

PubMed

Mpundu-Kaambwa, Christine; Chen, Gang; Huynh, Elisabeth; Russo, Remo; Ratcliffe, Julie

2018-07-01

To examine the psychometric properties and suitability for use within the context of cerebral palsy research in children and adolescents of generic preference-based outcome measures (PROMs). Nine electronic databases were searched in this systematic review. The consensus-based standards for the selection of health measurement instruments (COSMIN) checklist were used to measure the psychometric properties of the PROMs. A meta-analysis was used to pool correlation coefficients for convergent validity using the Schmidt-Hunter method. Heterogeneity was assessed using the I-squared statistic (I 2 ). Four preference-based PROMs were identified from eight studies: Health Utilities Index-Mark 2 and 3 (HUI-2 and HUI-3, respectively), the Assessment Quality of Life-4 dimension (AQoL-4D) and the EuroQol-5 dimension 3 level (EQ-5D-3L). Only the HUI system was primarily developed for application with children/adolescents though health-state values for scoring the PROM were elicited from adults. The HUI-3 covered the most relevant constructs though it excludes important modules of health-related quality of life (HRQOL) such as activity limitations and participation restrictions. In terms of psychometric properties, evidence was presented for only five of COSMIN measurement properties: reliability (HUI3), measurement error (HUI-3), content validity (HUI-2 and HUI-3), Hypotheses testing (HUI-3 and AQoL-4D) and criterion validity (HUI-3). No papers reported on internal consistency, structural validity, cross-cultural validity or responsiveness of the preference-based measures in children and adolescents with cerebral palsy. This review highlights the dearth in studies using preference-based PROMs to measure HRQOL associated with cerebral palsy in children and adolescents. The HUI-3 demonstrated the strongest psychometric properties, though it does not cover all dimensions relevant to this population.

Instruments to assess self-care among healthy children: A systematic review of measurement properties.

PubMed

Urpí-Fernández, Ana-María; Zabaleta-Del-Olmo, Edurne; Montes-Hidalgo, Javier; Tomás-Sábado, Joaquín; Roldán-Merino, Juan-Francisco; Lluch-Canut, María-Teresa

2017-12-01

To identify, critically appraise and summarize the measurement properties of instruments to assess self-care in healthy children. Assessing self-care is a proper consideration for nursing practice and nursing research. No systematic review summarizes instruments of measurement validated in healthy children. Psychometric review in accordance with the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) panel. MEDLINE, CINAHL, PsycINFO, Web of Science and Open Grey were searched from their inception to December 2016. Validation studies with a healthy child population were included. Search was not restricted by language. Two reviewers independently assessed the methodological quality of included studies using the COSMIN checklist. Eleven studies were included in the review assessing the measurement properties of ten instruments. There was a maximum of two studies per instrument. None of the studies evaluated the properties of test-retest reliability, measurement error, criterion validity and responsiveness. Internal consistency and structural validity were rated as "excellent" or "good" in four studies. Four studies were rated as "excellent" in content validity. Cross-cultural validity was rated as "poor" in the two studies (three instruments) which cultural adaptation was carried out. The evidence available does not allow firm conclusions about the instruments identified in terms of reliability and validity. Future research should focus on generate evidence about a wider range of measurement properties of these instruments using a rigorous methodology, as well as instrument testing on different countries and child population. © 2017 John Wiley & Sons Ltd.

Brazilian Center for the Validation of Alternative Methods (BraCVAM) and the process of validation in Brazil.

PubMed

Presgrave, Octavio; Moura, Wlamir; Caldeira, Cristiane; Pereira, Elisabete; Bôas, Maria H Villas; Eskes, Chantra

2016-03-01

The need for the creation of a Brazilian centre for the validation of alternative methods was recognised in 2008, and members of academia, industry and existing international validation centres immediately engaged with the idea. In 2012, co-operation between the Oswaldo Cruz Foundation (FIOCRUZ) and the Brazilian Health Surveillance Agency (ANVISA) instigated the establishment of the Brazilian Center for the Validation of Alternative Methods (BraCVAM), which was officially launched in 2013. The Brazilian validation process follows OECD Guidance Document No. 34, where BraCVAM functions as the focal point to identify and/or receive requests from parties interested in submitting tests for validation. BraCVAM then informs the Brazilian National Network on Alternative Methods (RENaMA) of promising assays, which helps with prioritisation and contributes to the validation studies of selected assays. A Validation Management Group supervises the validation study, and the results obtained are peer-reviewed by an ad hoc Scientific Review Committee, organised under the auspices of BraCVAM. Based on the peer-review outcome, BraCVAM will prepare recommendations on the validated test method, which will be sent to the National Council for the Control of Animal Experimentation (CONCEA). CONCEA is in charge of the regulatory adoption of all validated test methods in Brazil, following an open public consultation. 2016 FRAME.

JaCVAM-organized international validation study of the in vivo rodent alkaline comet assay for detection of genotoxic carcinogens: II. Summary of definitive validation study results.

PubMed

Uno, Yoshifumi; Kojima, Hajime; Omori, Takashi; Corvi, Raffaella; Honma, Masamistu; Schechtman, Leonard M; Tice, Raymond R; Beevers, Carol; De Boeck, Marlies; Burlinson, Brian; Hobbs, Cheryl A; Kitamoto, Sachiko; Kraynak, Andrew R; McNamee, James; Nakagawa, Yuzuki; Pant, Kamala; Plappert-Helbig, Ulla; Priestley, Catherine; Takasawa, Hironao; Wada, Kunio; Wirnitzer, Uta; Asano, Norihide; Escobar, Patricia A; Lovell, David; Morita, Takeshi; Nakajima, Madoka; Ohno, Yasuo; Hayashi, Makoto

2015-07-01

The in vivo rodent alkaline comet assay (comet assay) is used internationally to investigate the in vivo genotoxic potential of test chemicals. This assay, however, has not previously been formally validated. The Japanese Center for the Validation of Alternative Methods (JaCVAM), with the cooperation of the U.S. NTP Interagency Center for the Evaluation of Alternative Toxicological Methods (NICEATM)/the Interagency Coordinating Committee on the Validation of Alternative Methods (ICCVAM), the European Centre for the Validation of Alternative Methods (ECVAM), and the Japanese Environmental Mutagen Society/Mammalian Mutagenesis Study Group (JEMS/MMS), organized an international validation study to evaluate the reliability and relevance of the assay for identifying genotoxic carcinogens, using liver and stomach as target organs. The ultimate goal of this exercise was to establish an Organisation for Economic Co-operation and Development (OECD) test guideline. The study protocol was optimized in the pre-validation studies, and then the definitive (4th phase) validation study was conducted in two steps. In the 1st step, assay reproducibility was confirmed among laboratories using four coded reference chemicals and the positive control ethyl methanesulfonate. In the 2nd step, the predictive capability was investigated using 40 coded chemicals with known genotoxic and carcinogenic activity (i.e., genotoxic carcinogens, genotoxic non-carcinogens, non-genotoxic carcinogens, and non-genotoxic non-carcinogens). Based on the results obtained, the in vivo comet assay is concluded to be highly capable of identifying genotoxic chemicals and therefore can serve as a reliable predictor of rodent carcinogenicity. Copyright © 2015 Elsevier B.V. All rights reserved.

Driving simulator validation of driver behavior with limited safe vantage points for data collection in work zones.

PubMed

Bham, Ghulam H; Leu, Ming C; Vallati, Manoj; Mathur, Durga R

2014-06-01

This study is aimed at validating a driving simulator (DS) for the study of driver behavior in work zones. A validation study requires field data collection. For studies conducted in highway work zones, the availability of safe vantage points for data collection at critical locations can be a significant challenge. A validation framework is therefore proposed in this paper, demonstrated using a fixed-based DS that addresses the issue by using a global positioning system (GPS). The validation of the DS was conducted using objective and subjective evaluations. The objective validation was divided into qualitative and quantitative evaluations. The DS was validated by comparing the results of simulation with the field data, which were collected using a GPS along the highway and video recordings at specific locations in a work zone. The constructed work zone scenario in the DS was subjectively evaluated with 46 participants. The objective evaluation established the absolute and relative validity of the DS. The mean speeds from the DS data showed excellent agreement with the field data. The subjective evaluation indicated realistic driving experience by the participants. The use of GPS showed that continuous data collected along the highway can overcome the challenges of unavailability of safe vantage points especially at critical locations. Further, a validated DS can be used for examining driver behavior in complex situations by replicating realistic scenarios. Copyright © 2014 Elsevier Ltd. All rights reserved.

Assessing the validity of discourse analysis: transdisciplinary convergence

NASA Astrophysics Data System (ADS)

Jaipal-Jamani, Kamini

2014-12-01

Research studies using discourse analysis approaches make claims about phenomena or issues based on interpretation of written or spoken text, which includes images and gestures. How are findings/interpretations from discourse analysis validated? This paper proposes transdisciplinary convergence as a way to validate discourse analysis approaches to research. The argument is made that discourse analysis explicitly grounded in semiotics, systemic functional linguistics, and critical theory, offers a credible research methodology. The underlying assumptions, constructs, and techniques of analysis of these three theoretical disciplines can be drawn on to show convergence of data at multiple levels, validating interpretations from text analysis.

The psychometric properties of the WHOQOL-BREF in Japanese couples

PubMed Central

Sun, Yi; Sugawara, Masumi; Matsumoto, Satoko; Sakai, Atsushi; Takaoka, Junko; Goto, Noriko

2015-01-01

This study investigated the psychometric properties of the Japanese version of the WHOQOL-BREF among 10,693 community-based married Japanese men and women (4376 couples) who were either expecting or raising a child. Analyses of item-response distributions, internal consistency, criterion validity, and discriminant validity indicated that the scale had acceptable reliability and performed well in preliminary tests of validity. Furthermore, dyadic confirmatory factor analysis revealed that the theoretical factor structure was valid and similar across partners, suggesting that men and women define and value quality of life in a similar way. PMID:28070365

Developing group investigation-based book on numerical analysis to increase critical thinking student’s ability

NASA Astrophysics Data System (ADS)

Maharani, S.; Suprapto, E.

2018-03-01

Critical thinking is very important in Mathematics; it can make student more understanding mathematics concept. Critical thinking is also needed in numerical analysis. The Numerical analysis's book is not yet including critical thinking in them. This research aims to develop group investigation-based book on numerical analysis to increase critical thinking student’s ability, to know the quality of the group investigation-based book on numerical analysis is valid, practical, and effective. The research method is Research and Development (R&D) with the subject are 30 student college department of Mathematics education at Universitas PGRI Madiun. The development model used is 4-D modified to 3-D until the stage development. The type of data used is descriptive qualitative data. Instruments used are sheets of validation, test, and questionnaire. Development results indicate that group investigation-based book on numerical analysis in the category of valid a value 84.25%. Students response to the books very positive, so group investigation-based book on numerical analysis category practical, i.e., 86.00%. The use of group investigation-based book on numerical analysis has been meeting the completeness criteria classical learning that is 84.32 %. Based on research result of this study concluded that group investigation-based book on numerical analysis is feasible because it meets the criteria valid, practical, and effective. So, the book can be used by every mathematics academician. The next research can be observed that book based group investigation in other subjects.

A long-term validation of the modernised DC-ARC-OES solid-sample method.

PubMed

Flórián, K; Hassler, J; Förster, O

2001-12-01

The validation procedure based on ISO 17025 standard has been used to study and illustrate both the longterm stability of the calibration process of the DC-ARC solid sample spectrometric method and the main validation criteria of the method. In the calculation of the validation characteristics depending on the linearity(calibration), also the fulfilment of predetermining criteria such as normality and homoscedasticity was checked. In order to decide whether there are any trends in the time-variation of the analytical signal or not, also the Neumann test of trend was applied and evaluated. Finally, a comparison with similar validation data of the ETV-ICP-OES method was carried out.

The risk of bias in systematic reviews tool showed fair reliability and good construct validity.

PubMed

Bühn, Stefanie; Mathes, Tim; Prengel, Peggy; Wegewitz, Uta; Ostermann, Thomas; Robens, Sibylle; Pieper, Dawid

2017-11-01

There is a movement from generic quality checklists toward a more domain-based approach in critical appraisal tools. This study aimed to report on a first experience with the newly developed risk of bias in systematic reviews (ROBIS) tool and compare it with A Measurement Tool to Assess Systematic Reviews (AMSTAR), that is, the most common used tool to assess methodological quality of systematic reviews while assessing validity, reliability, and applicability. Validation study with four reviewers based on 16 systematic reviews in the field of occupational health. Interrater reliability (IRR) of all four raters was highest for domain 2 (Fleiss' kappa κ = 0.56) and lowest for domain 4 (κ = 0.04). For ROBIS, median IRR was κ = 0.52 (range 0.13-0.88) for the experienced pair of raters compared to κ = 0.32 (range 0.12-0.76) for the less experienced pair of raters. The percentage of "yes" scores of each review of ROBIS ratings was strongly correlated with the AMSTAR ratings (r s  = 0.76; P = 0.01). ROBIS has fair reliability and good construct validity to assess the risk of bias in systematic reviews. More validation studies are needed to investigate reliability and applicability, in particular. Copyright © 2017 Elsevier Inc. All rights reserved.

Validity and reliability of Persian version of Listening Styles Profile-Revised (LSP- R) in Iranian students.

PubMed

Fatehi, Zahra; Baradaran, Hamid Reza; Asadpour, Mohamad; Rezaeian, Mohsen

2017-01-01

Background: Individuals' listening styles differs based on their characters, professions and situations. This study aimed to assess the validity and reliability of Listening Styles Profile- Revised (LSP- R) in Iranian students. Methods: After translating into Persian, LSP-R was employed in a sample of 240 medical and nursing Persian speaking students in Iran. Statistical analysis was performed to test the reliability and validity of the LSP-R. Results: The study revealed high internal consistency and good test-retest reliability for the Persian version of the questionnaire. The Cronbach's alpha coefficient was 0.72 and intra-class correlation coefficient 0.87. The means for the content validity index and the content validity ratio (CVR) were 0.90 and 0.83, respectively. Exploratory factor analysis (EFA) yielded a four-factor solution accounted for 60.8% of the observed variance. Majority of medical students (73%) as well as majority of nursing students (70%) stated that their listening styles were task-oriented. Conclusion: In general, the study finding suggests that the Persian version of LSP-R is a valid and reliable instrument for assessing listening styles profile in the studied sample.

Students' Perceptions of Their ICT-Based College English Course in China: A Case Study

ERIC Educational Resources Information Center

Zinan, Wen; Sai, George Teoh Boon

2017-01-01

This study investigated foreign language students' perceptions about their Information and Communication Technology (ICT)-based College English Course (CEC) in China. The research used a five-point Likert-scale questionnaire based on Simsek (2008). A factor analysis confirmed the construct validity of the questionnaire and 6 factors were…

Multimodal Navigation in Endoscopic Transsphenoidal Resection of Pituitary Tumors Using Image-Based Vascular and Cranial Nerve Segmentation: A Prospective Validation Study.

PubMed

Dolati, Parviz; Eichberg, Daniel; Golby, Alexandra; Zamani, Amir; Laws, Edward

2016-11-01

Transsphenoidal surgery (TSS) is the most common approach for the treatment of pituitary tumors. However, misdirection, vascular damage, intraoperative cerebrospinal fluid leakage, and optic nerve injuries are all well-known complications, and the risk of adverse events is more likely in less-experienced hands. This prospective study was conducted to validate the accuracy of image-based segmentation coupled with neuronavigation in localizing neurovascular structures during TSS. Twenty-five patients with a pituitary tumor underwent preoperative 3-T magnetic resonance imaging (MRI), and MRI images loaded into the navigation platform were used for segmentation and preoperative planning. After patient registration and subsequent surgical exposure, each segmented neural or vascular element was validated by manual placement of the navigation probe or Doppler probe on or as close as possible to the target. Preoperative segmentation of the internal carotid artery and cavernous sinus matched with the intraoperative endoscopic and micro-Doppler findings in all cases. Excellent correspondence between image-based segmentation and the endoscopic view was also evident at the surface of the tumor and at the tumor-normal gland interfaces. Image guidance assisted the surgeons in localizing the optic nerve and chiasm in 64% of cases. The mean accuracy of the measurements was 1.20 ± 0.21 mm. Image-based preoperative vascular and neural element segmentation, especially with 3-dimensional reconstruction, is highly informative preoperatively and potentially could assist less-experienced neurosurgeons in preventing vascular and neural injury during TSS. In addition, the accuracy found in this study is comparable to previously reported neuronavigation measurements. This preliminary study is encouraging for future prospective intraoperative validation with larger numbers of patients. Copyright © 2016 Elsevier Inc. All rights reserved.

A semi-automatic method for left ventricle volume estimate: an in vivo validation study

NASA Technical Reports Server (NTRS)

Corsi, C.; Lamberti, C.; Sarti, A.; Saracino, G.; Shiota, T.; Thomas, J. D.

2001-01-01

This study aims to the validation of the left ventricular (LV) volume estimates obtained by processing volumetric data utilizing a segmentation model based on level set technique. The validation has been performed by comparing real-time volumetric echo data (RT3DE) and magnetic resonance (MRI) data. A validation protocol has been defined. The validation protocol was applied to twenty-four estimates (range 61-467 ml) obtained from normal and pathologic subjects, which underwent both RT3DE and MRI. A statistical analysis was performed on each estimate and on clinical parameters as stroke volume (SV) and ejection fraction (EF). Assuming MRI estimates (x) as a reference, an excellent correlation was found with volume measured by utilizing the segmentation procedure (y) (y=0.89x + 13.78, r=0.98). The mean error on SV was 8 ml and the mean error on EF was 2%. This study demonstrated that the segmentation technique is reliably applicable on human hearts in clinical practice.

CFD Validation Experiment of a Mach 2.5 Axisymmetric Shock-Wave/Boundary-Layer Interaction

NASA Technical Reports Server (NTRS)

Davis, David O.

2015-01-01

Experimental investigations of specific flow phenomena, e.g., Shock Wave Boundary-Layer Interactions (SWBLI), provide great insight to the flow behavior but often lack the necessary details to be useful as CFD validation experiments. Reasons include: 1.Undefined boundary conditions Inconsistent results 2.Undocumented 3D effects (CL only measurements) 3.Lack of uncertainty analysis While there are a number of good subsonic experimental investigations that are sufficiently documented to be considered test cases for CFD and turbulence model validation, the number of supersonic and hypersonic cases is much less. This was highlighted by Settles and Dodsons [1] comprehensive review of available supersonic and hypersonic experimental studies. In all, several hundred studies were considered for their database.Of these, over a hundred were subjected to rigorous acceptance criteria. Based on their criteria, only 19 (12 supersonic, 7 hypersonic) were considered of sufficient quality to be used for validation purposes. Aeschliman and Oberkampf [2] recognized the need to develop a specific methodology for experimental studies intended specifically for validation purposes.

Development and validation of a melanoma risk score based on pooled data from 16 case-control studies

PubMed Central

Davies, John R; Chang, Yu-mei; Bishop, D Timothy; Armstrong, Bruce K; Bataille, Veronique; Bergman, Wilma; Berwick, Marianne; Bracci, Paige M; Elwood, J Mark; Ernstoff, Marc S; Green, Adele; Gruis, Nelleke A; Holly, Elizabeth A; Ingvar, Christian; Kanetsky, Peter A; Karagas, Margaret R; Lee, Tim K; Le Marchand, Loïc; Mackie, Rona M; Olsson, Håkan; Østerlind, Anne; Rebbeck, Timothy R; Reich, Kristian; Sasieni, Peter; Siskind, Victor; Swerdlow, Anthony J; Titus, Linda; Zens, Michael S; Ziegler, Andreas; Gallagher, Richard P.; Barrett, Jennifer H; Newton-Bishop, Julia

2015-01-01

Background We report the development of a cutaneous melanoma risk algorithm based upon 7 factors; hair colour, skin type, family history, freckling, nevus count, number of large nevi and history of sunburn, intended to form the basis of a self-assessment webtool for the general public. Methods Predicted odds of melanoma were estimated by analysing a pooled dataset from 16 case-control studies using logistic random coefficients models. Risk categories were defined based on the distribution of the predicted odds in the controls from these studies. Imputation was used to estimate missing data in the pooled datasets. The 30th, 60th and 90th centiles were used to distribute individuals into four risk groups for their age, sex and geographic location. Cross-validation was used to test the robustness of the thresholds for each group by leaving out each study one by one. Performance of the model was assessed in an independent UK case-control study dataset. Results Cross-validation confirmed the robustness of the threshold estimates. Cases and controls were well discriminated in the independent dataset (area under the curve 0.75, 95% CI 0.73-0.78). 29% of cases were in the highest risk group compared with 7% of controls, and 43% of controls were in the lowest risk group compared with 13% of cases. Conclusion We have identified a composite score representing an estimate of relative risk and successfully validated this score in an independent dataset. Impact This score may be a useful tool to inform members of the public about their melanoma risk. PMID:25713022

Systematic review of systemic sclerosis-specific instruments for the EULAR Outcome Measures Library: An evolutional database model of validated patient-reported outcomes.

PubMed

Ingegnoli, Francesca; Carmona, Loreto; Castrejon, Isabel

2017-04-01

The EULAR Outcome Measures Library (OML) is a freely available database of validated patient-reported outcomes (PROs). The aim of this study was to provide a comprehensive review of validated PROs specifically developed for systemic sclerosis (SSc) to feed the EULAR OML. A sensitive search was developed in Medline and Embase to identify all validation studies, cohort studies, reviews, or meta-analyses in which the objective were the development or validation of specific PROs evaluating organ involvement, disease activity or damage in SSc. A reviewer screened title and abstracts, selected the studies, and collected data concerning validation using ad hoc forms based on the COSMIN checklist. From 13,140 articles captured, 74 met the predefined criteria. After excluding two instruments as they were unavailable in English the selected 23 studies provided information on seven SSc-specific PROs on different SSc domains: burden of illness (symptom burden index), functional status (Scleroderma Assessment Questionnaire), functional ability (scleroderma Functional Score), Raynaud's phenomenon (Raynaud's condition score), mouth involvement (Mouth Handicap in SSc), gastro-intestinal involvement (University of California Los Angeles-Scleroderma Clinical Trial Consortium Gastro-Intestinal tract 2.0), and skin involvement (skin self-assessment). Each of them is partially validated and has different psychometric requirements. Seven SSc-specific PROs have a minimum validation and were included in the EULAR OML. Further development in the area of disease-specific PROs in SSc is warranted. Copyright © 2017 Elsevier Inc. All rights reserved.

Validation of a web-based questionnaire for pregnant women to assess utilization of internet: survey among an Italian sample.

PubMed

Siliquini, R; Saulle, R; Rabacchi, G; Bert, F; Massimi, A; Bulzomì, V; Boccia, A; La Torre, G

2012-01-01

Objective of this pilot study was to evaluate the reliability and validity of the web-based questionnaire in pregnant women as a tool to examine prevalence, knowledge and attitudes about internet utilization for health-related purposes, in a sample of Italian pregnant women. The questionnaire was composed by 9 sections for a total of 73 items. Reliability analysis was tested and content validity was evaluated using Cronbach's alpha to check internal consistency. Statistical analysis was performed through SPSS 13.0. Questionnaire was administered to 56 pregnant women. The higher value of Cronbach's alpha resulted on 61 items: alpha = 0.786 (n. 73 items: alpha = 0.579). High rate of pregnant women generally utilized internet (87.5%) and the 92.1% confirmed to use internet with the focus to acquire information about pregnancy (p < 0.0001). The questionnaire showed a good reliability property in the pilot study. In terms of internal consistency and validity appeared to have a good performance. Given the high prevalence of pregnant women that use internet to search information about their pregnancy status, professional healthcare workers should give advice regarding official websites where they could retrieve safe information and learn knowledge based on scientific evidence.

A unified approach to validation, reliability, and education study design for surgical technical skills training.

PubMed

Sweet, Robert M; Hananel, David; Lawrenz, Frances

2010-02-01

To present modern educational psychology theory and apply these concepts to validity and reliability of surgical skills training and assessment. In a series of cross-disciplinary meetings, we applied a unified approach of behavioral science principles and theory to medical technical skills education given the recent advances in the theories in the field of behavioral psychology and statistics. While validation of the individual simulation tools is important, it is only one piece of a multimodal curriculum that in and of itself deserves examination and study. We propose concurrent validation throughout the design of simulation-based curriculum rather than once it is complete. We embrace the concept that validity and curriculum development are interdependent, ongoing processes that are never truly complete. Individual predictive, construct, content, and face validity aspects should not be considered separately but as interdependent and complementary toward an end application. Such an approach could help guide our acceptance and appropriate application of these exciting new training and assessment tools for technical skills training in medicine.

Elaboration and Validation of the Medication Prescription Safety Checklist 1

PubMed Central

Pires, Aline de Oliveira Meireles; Ferreira, Maria Beatriz Guimarães; do Nascimento, Kleiton Gonçalves; Felix, Márcia Marques dos Santos; Pires, Patrícia da Silva; Barbosa, Maria Helena

2017-01-01

ABSTRACT Objective: to elaborate and validate a checklist to identify compliance with the recommendations for the structure of medication prescriptions, based on the Protocol of the Ministry of Health and the Brazilian Health Surveillance Agency. Method: methodological research, conducted through the validation and reliability analysis process, using a sample of 27 electronic prescriptions. Results: the analyses confirmed the content validity and reliability of the tool. The content validity, obtained by expert assessment, was considered satisfactory as it covered items that represent the compliance with the recommendations regarding the structure of the medication prescriptions. The reliability, assessed through interrater agreement, was excellent (ICC=1.00) and showed perfect agreement (K=1.00). Conclusion: the Medication Prescription Safety Checklist showed to be a valid and reliable tool for the group studied. We hope that this study can contribute to the prevention of adverse events, as well as to the improvement of care quality and safety in medication use. PMID:28793128

Predicting implementation from organizational readiness for change: a study protocol

PubMed Central

2011-01-01

Background There is widespread interest in measuring organizational readiness to implement evidence-based practices in clinical care. However, there are a number of challenges to validating organizational measures, including inferential bias arising from the halo effect and method bias - two threats to validity that, while well-documented by organizational scholars, are often ignored in health services research. We describe a protocol to comprehensively assess the psychometric properties of a previously developed survey, the Organizational Readiness to Change Assessment. Objectives Our objective is to conduct a comprehensive assessment of the psychometric properties of the Organizational Readiness to Change Assessment incorporating methods specifically to address threats from halo effect and method bias. Methods and Design We will conduct three sets of analyses using longitudinal, secondary data from four partner projects, each testing interventions to improve the implementation of an evidence-based clinical practice. Partner projects field the Organizational Readiness to Change Assessment at baseline (n = 208 respondents; 53 facilities), and prospectively assesses the degree to which the evidence-based practice is implemented. We will conduct predictive and concurrent validities using hierarchical linear modeling and multivariate regression, respectively. For predictive validity, the outcome is the change from baseline to follow-up in the use of the evidence-based practice. We will use intra-class correlations derived from hierarchical linear models to assess inter-rater reliability. Two partner projects will also field measures of job satisfaction for convergent and discriminant validity analyses, and will field Organizational Readiness to Change Assessment measures at follow-up for concurrent validity (n = 158 respondents; 33 facilities). Convergent and discriminant validities will test associations between organizational readiness and different aspects of job satisfaction: satisfaction with leadership, which should be highly correlated with readiness, versus satisfaction with salary, which should be less correlated with readiness. Content validity will be assessed using an expert panel and modified Delphi technique. Discussion We propose a comprehensive protocol for validating a survey instrument for assessing organizational readiness to change that specifically addresses key threats of bias related to halo effect, method bias and questions of construct validity that often go unexplored in research using measures of organizational constructs. PMID:21777479

3D-quantitative structure-activity relationship studies on benzothiadiazepine hydroxamates as inhibitors of tumor necrosis factor-alpha converting enzyme.

PubMed

Murumkar, Prashant R; Giridhar, Rajani; Yadav, Mange Ram

2008-04-01

A set of 29 benzothiadiazepine hydroxamates having selective tumor necrosis factor-alpha converting enzyme inhibitory activity were used to compare the quality and predictive power of 3D-quantitative structure-activity relationship, comparative molecular field analysis, and comparative molecular similarity indices models for the atom-based, centroid/atom-based, data-based, and docked conformer-based alignment. Removal of two outliers from the initial training set of molecules improved the predictivity of models. Among the 3D-quantitative structure-activity relationship models developed using the above four alignments, the database alignment provided the optimal predictive comparative molecular field analysis model for the training set with cross-validated r(2) (q(2)) = 0.510, non-cross-validated r(2) = 0.972, standard error of estimates (s) = 0.098, and F = 215.44 and the optimal comparative molecular similarity indices model with cross-validated r(2) (q(2)) = 0.556, non-cross-validated r(2) = 0.946, standard error of estimates (s) = 0.163, and F = 99.785. These models also showed the best test set prediction for six compounds with predictive r(2) values of 0.460 and 0.535, respectively. The contour maps obtained from 3D-quantitative structure-activity relationship studies were appraised for activity trends for the molecules analyzed. The comparative molecular similarity indices models exhibited good external predictivity as compared with that of comparative molecular field analysis models. The data generated from the present study helped us to further design and report some novel and potent tumor necrosis factor-alpha converting enzyme inhibitors.

Construct Validity of the SF-12v2 for the Homeless Population with Mental Illness: An Instrument to Measure Self-Reported Mental and Physical Health

PubMed Central

Chum, Antony; Skosireva, Anna; Tobon, Juliana; Hwang, Stephen

2016-01-01

Background Self-reported health measures are important indicators used by clinicians and researchers for the evaluation of health interventions, outcome assessment of clinical studies, and identification of health needs to improve resource allocation. However, the application of self-reported health measures relies on developing reliable and valid instruments that are suitable across diverse populations. The main objective of this study is to evaluate the construct validity of the SF-12v.2, an instrument for measuring self-rated physical and mental health, for homeless adults with mental illness. Various interventions have been aimed at improving the health of homeless people with mental illness, and the development of valid instruments to evaluate these interventions is imperative. Study Design We measured self-rated mental and physical health from a quota sample of 575 homeless people with mental illness using the SF-12v2, EQ-5D, Colorado Symptoms Index, and physical/mental health visual analogue scales. We examined the construct validity of the SF-12v2 through confirmatory factor analyses (CFA), and using ANOVA/correlation analyses to compare the SF-12v2 to the other instruments to ascertain discriminant/convergent validity. Results Our CFA showed that the measurement properties of the original SF-12v2 model had a mediocre fit with our empirical data (χ2 = 193.6, df = 43, p < .0001, CFI = 0.85, NFI = 0.83, RMSEA = 0.08). We demonstrate that changes based on theoretical rationale and previous studies can significantly improve the model, achieving an excellent fit in our final model (χ2 = 160.6, df = 48, p < .0001, CFI = 0.95, NFI = 0.95, RMSEA = 0.06). Our CFA results suggest that an alternative scoring method based on the new model may optimize health status measurement of a homeless population. Despite these issues, convergent and discriminant validity of the SF-12v2 (scored based on the original model) was supported through multiple comparisons with other instruments. Conclusion Our study demonstrates for the first time that the SF-12v2 is generally appropriate as a measure of physical and mental health status for a homeless population with mental illness. PMID:26938990

Improving the Performance of the Listening Competency Scale: Revision and Validation

ERIC Educational Resources Information Center

Mickelson, William T.; Welch, S. A.

2013-01-01

Measuring latent traits is central to quantitative listening research and has been the focus of many studies. One such prominent measurement instrument, based on the Wolvin and Coakley (1993) listening taxonomy, was developed by Ford, Wolvin, and Chung (2000). Subsequent validation research (Mickelson & Welch, 2012) called for revisiting and…

Adapting the Media and Technology Usage and Attitudes Scale to Turkish

ERIC Educational Resources Information Center

Özgür, Hasan

2016-01-01

Due to the requirement of a current, valid, and reliable assessment instrument for determining usage frequencies of technology-based media and the attitudes towards these, this study intends to determine the validity and reliability of the Media and Technology Usage and Attitudes Scale, developed by researchers from California State University,…

Training Objectives, Transfer, Validation and Evaluation: A Sri Lankan Study

ERIC Educational Resources Information Center

Wickramasinghe, Vathsala M.

2006-01-01

Using a stratified random sample, this paper examines the training practices of setting objectives, transfer, validation and evaluation in Sri Lanka. The paper further sets out to compare those practices across local, foreign and joint-venture companies based on the assumption that there may be significant differences across companies of different…

Validating Pedometer-Based Physical Activity Time against Accelerometer in Middle School Physical Education

ERIC Educational Resources Information Center

Gao, Zan; Lee, Amelia M.; Solmon, Melinda A.; Kosma, Maria; Carson, Russell L.; Zhang, Tao; Domangue, Elizabeth; Moore, Delilah

2010-01-01

The purpose of this study was to validate physical activity time in middle school physical education as measured by pedometers in relation to a criterion measure, namely, students' accelerometer determined moderate to vigorous physical activity (MVPA). Participants were 155 sixth to eighth graders participating in regularly scheduled physical…

Psychometric Evaluation of the Exercise Identity Scale among Greek Adults and Cross-Cultural Validity

ERIC Educational Resources Information Center

Vlachopoulos, Symeon P.; Kaperoni, Maria; Moustaka, Frederiki C.; Anderson, Dean F.

2008-01-01

The present study reported on translating the Exercise Identity Scale (EIS: Anderson & Cychosz, 1994) into Greek and examining its psychometric properties and cross-cultural validity based on U.S. individuals' EIS responses. Using four samples comprising 33, 103, and 647 Greek individuals, including exercisers and nonexercisers, and a similar…

The Development and Validation of the Intercultural Sensitivity Scale.

ERIC Educational Resources Information Center

Chen, Guo-Ming; Starosta, William J.

The present study developed and assessed reliability and validity of a new instrument, the Intercultural Sensitivity Scale (ISS). Based on a review of the literature, 44 items thought to be important for intercultural sensitivity were generated. A sample of 414 college students rated these items and generated a 24-item final version of the…

Structural Validity of the Life Regard Index

ERIC Educational Resources Information Center

Steger, Michael F.

2007-01-01

Counselors and researchers interested in examining meaning in life often use the Life Regard Index (LRI; J. Battista & R. Almond, 1973). In this study, confirmatory factor analyses (CFAs) of several factor models based on J. Battista & R. Almond's work failed to support the structural validity of the LRI. CFA results suggested an influence of…

The Development and Validation of the Age-Based Rejection Sensitivity Questionnaire

ERIC Educational Resources Information Center

Kang, Sonia K.; Chasteen, Alison L.

2009-01-01

Purpose: There is much evidence suggesting that older adults are often negatively affected by aging stereotypes; however, no method to identify individual differences in vulnerability to these effects has yet been developed. The purpose of this study was to develop a reliable and valid questionnaire to measure individual differences in the…

Validating Work Discrimination and Coping Strategy Models for Sexual Minorities

ERIC Educational Resources Information Center

Chung, Y. Barry; Williams, Wendi; Dispenza, Franco

2009-01-01

The purpose of this study was to validate and expand on Y. B. Chung's (2001) models of work discrimination and coping strategies among lesbian, gay, and bisexual persons. In semistructured individual interviews, 17 lesbians and gay men reported 35 discrimination incidents and their related coping strategies. Responses were coded based on Chung's…

Patterns of Cognitive Strengths and Weaknesses: Identification Rates, Agreement, and Validity for Learning Disabilities Identification

ERIC Educational Resources Information Center

Miciak, Jeremy; Fletcher, Jack M.; Stuebing, Karla K.; Vaughn, Sharon; Tolar, Tammy D.

2014-01-01

Few empirical investigations have evaluated learning disabilities (LD) identification methods based on a pattern of cognitive strengths and weaknesses (PSW). This study investigated the reliability and validity of two proposed PSW methods: the concordance/discordance method (C/DM) and cross battery assessment (XBA) method. Cognitive assessment…

Interpreting Self-Directed Search Profiles: Validity of the "Rule of Eight"

ERIC Educational Resources Information Center

Glavin, Kevin W.; Savickas, Mark L.

2011-01-01

Based on the standard error of measurement, Holland (1985) suggested the "rule of eight" for determining the meaningfulness of differences between two summary scores on the Self Directed Search. The present study empirically examined the rule's validity for practice. The participants were 2397 (1497 females and 900 males) undergraduate…

The Predictive Validity of CBM Writing Indices for Eighth-Grade Students

ERIC Educational Resources Information Center

Amato, Janelle M.; Watkins, Marley W.

2011-01-01

Curriculum-based measurement (CBM) is an alternative to traditional assessment techniques. Technical work has begun to identify CBM writing indices that are psychometrically sound for monitoring older students' writing proficiency. This study examined the predictive validity of CBM writing indices in a sample of 447 eighth-grade students.…

Validation of a web-based questionnaire to assess the dietary intake of Brazilian children aged 7-10 years

USDA-ARS?s Scientific Manuscript database

The Food Intake and Physical Activity of School Children (CAAFE) comprises an online questionnaire to self-report diet and physical activity of Brazilian schoolchildren. The present study aimed to assess the validity (matches, omissions, and intrusions) and moderating factors of the CAAFE. Direct ob...

Validation of a web-based questionnaire to assess the dietary intake of Brazilian children aged 7-10 years

USDA-ARS?s Scientific Manuscript database

The Food Intake and Physical Activity of School Children (CAAFE) comprises an online questionnaire to self-report diet and physical activity of Brazilian schoolchildren. The present study aimed to assess the validity (matches, omissions and intrusions) and moderating factors of the CAAFE. Direct obs...

Validation of Geriatric Depression Scale--5 Scores among Sedentary Older Adults

ERIC Educational Resources Information Center

Marquez, David X.; McAuley, Edward; Motl, Robert W.; Elavsky, Steriani; Konopack, James F.; Jerome, Gerald J.; Kramer, Arthur F.

2006-01-01

This study examined the validity of Geriatric Depression Scale--5 (GDS-5) scores among older sedentary adults based on its structural properties and relationship with external criteria. Participants from two samples (Ns = 185 and 93; M ages = 66 and 67 years) completed baseline assessments as part of randomized controlled exercise trials.…

Establishing the Validity of the Task-Based English Speaking Test (TBEST) for International Teaching Assistants

ERIC Educational Resources Information Center

Witt, Autumn Song

2010-01-01

This dissertation follows an oral language assessment tool from initial design and implementation to validity analysis. The specialized variables of this study are the population: international teaching assistants and the purpose: spoken assessment as a hiring prerequisite. However, the process can easily be applied to other populations and…

Validation of an Adapted Instrument to Measure Students' Attitude towards Science

ERIC Educational Resources Information Center

Chin, Sook Fui; Lim, Hooi Lian

2016-01-01

Attitude towards science (ATS) is a major concern in science education. Although many ATS instruments have been developed, they are based on different cultural systems and having some limitations. This study aims to validate an instrument for measuring students' ATS in Malaysia context. The instrument was adapted from Test of Science-Related…

Questioning the Validity of Inquiry Assessment in a High Stakes Physical Sciences Examination

ERIC Educational Resources Information Center

Ramnarain, Umesh

2014-01-01

The South African science curriculum advocates an inquiry-based approach to practical work. Inquiry is a complex and multifaceted activity involving both cognitive and physical activity; thus, paper-and-pencil items do not provide the authentic context for this assessment. This study investigates the construct validity of inquiry-related questions…

Development and Validation of the Pregnancy and Infant Orientation Questionnaire

PubMed Central

Meyer-Bahlburg, Heino F. L.; Dolezal, Curtis; Johnson, Laurel L.; Kessler, Suzanne J.; Schober, Justine M.; Zucker, Kenneth J.

2009-01-01

The purpose of this current study was the construction, scaling, and scale validation of a self-report questionnaire assessing biographical information and motivation for pregnancy and infant care in men and women with disorders of sex development or other gender variations of potential clinical relevance. The overall design of the questionnaire and the initial item pool were derived from related clinical and research experience. Collection of pilot data and, where appropriate, scale construction (via principal components analyses) were based on Canadian convenience samples of heterosexual (HET) and non-HET men and women (N=414). A sample of gender-dysphoric (GDYS) men and women (n=45) was added for validity analysis. Validation of the resulting scales was based on the demonstration of expected scale differences between HET men and women, as well as between HET, non-HET, and GDYS men and between HET, non-HET, and GDYS women, and was successful with one exception. This study concludes that this new questionnaire, in addition to its descriptive sections, provides several distinct scales related to desires for child bearing and child rearing and has good psychometric properties. PMID:19998066

The development of thematic materials using project based learning for elementary school

NASA Astrophysics Data System (ADS)

Yuliana, M.; Wiryawan, S. A.; Riyadi

2018-05-01

Teaching materials is one of the important factors in supporting on learning process. This paper discussed about developing thematic materials using project based learning. Thematic materials are designed to make students to be active, creative, cooperative, easy in thinking to solve the problem. The purpose of the research was to develop thematic material using project based learning which used valid variables. The method of research which used in this research was four stages of research and development proposed by Thiagarajan consisting of 4 stages, namely: (1) definition stage, (2) design stage, (3) development stage, and (4) stage of dissemination. The first stage was research and information collection, it was in form of need analysis with questionnaire, observation, interview, and document analysis. Design stage was based on the competencies and indicator. The third was development stage, this stage was used to product validation from expert. The validity of research development involved media validator, material validator, and linguistic validator. The result from the validation of thematic material by expert showed that the overall result had a very good rating which ranged from 1 to 5 likert scale, media validation showed a mean score 4,83, the material validation showed mean score 4,68, and the mean of linguistic validation was e 4,74. It showed that the thematic material using project based learning was valid and feasible to be implemented in the context thematic learning.

A Known Group Analysis Validity Study of the Vanderbilt Assessment of Leadership in Education in US Elementary and Secondary Schools

ERIC Educational Resources Information Center

Covay Minor, Elizabeth; Porter, Andrew C.; Murphy, Joseph; Goldring, Ellen B.; Cravens, Xiu; Elloitt, Stephen N.

2014-01-01

The Vanderbilt Assessment of Leadership in Education (VAL-ED) provides educators with a tool for principal evaluation based on principal, teacher, and supervisor reports of principals' learning-centered leadership. In this study, we conduct a known group analysis as part of a larger argument for the validity of the VAL-ED in US elementary and…

Predicting Chinese Children and Youth's Energy Expenditure Using ActiGraph Accelerometers: A Calibration and Cross-Validation Study

ERIC Educational Resources Information Center

Zhu, Zheng; Chen, Peijie; Zhuang, Jie

2013-01-01

Purpose: The purpose of this study was to develop and cross-validate an equation based on ActiGraph accelerometer GT3X output to predict children and youth's energy expenditure (EE) of physical activity (PA). Method: Participants were 367 Chinese children and youth (179 boys and 188 girls, aged 9 to 17 years old) who wore 1 ActiGraph GT3X…

Reliability and validity of the Japanese Migraine Disability Assessment (MIDAS) Questionnaire.

PubMed

Iigaya, Miho; Sakai, Fumihiko; Kolodner, Kenneth B; Lipton, Richard B; Stewart, Walter F

2003-04-01

This study was designed to assess the test-retest reliability, internal consistency, and validity of a Japanese translation of the Migraine Disability Assessment (MIDAS) Questionnaire in a sample of Japanese patients with headache. Previous studies have demonstrated that the English-language version of the MIDAS Questionnaire is a reliable and valid instrument for the assessment of migraine-related disability. Any translations of the MIDAS Questionnaire must also be assessed for reliability and validity. Study participants were recruited from the patient population attending either the Neurology Department of Kitasato University or an affiliated clinic. Participants were eligible for study entry if they had 6 or more primary headaches per year. For reliability testing, participants completed the MIDAS Questionnaire on 2 occasions, exactly 2 weeks apart. To assess validity, patients were also invited to participate in a 90-day daily diary study. Composite measures from the 90-day diaries were compared to equivalent MIDAS measures (ie, 5 questions on headache-related disability and 1 question each on average pain intensity and headache frequency in the last 3 months) and to the total MIDAS score obtained from a third MIDAS Questionnaire completed at the end of this 90-day period. One hundred one patients between the ages of 21 and 77 years were recruited (81 women and 20 men). Ninety-nine patients (80 women and 19 men) participated in the diary study. At baseline, 46.5% of patients were MIDAS grade I or II (minimal, mild, or infrequent disability), 22.2% were MIDAS grade III (moderate disability), and 31.3% were MIDAS grade IV (severe disability). Test-retest Spearman correlations for the 5 disability questions and the questions on average pain intensity and headache frequency ranged from 0.59 to 0.80 (P<.0001). The test-retest Spearman correlation coefficient for the total MIDAS score was 0.83 (P<.0001). The degree to which individual MIDAS questions correlated with the diary-based measures ranged from 0.36 to 0.88. The correlation between the total MIDAS score and the equivalent diary-based measure was 0.66. In general, the mean and median values for the MIDAS items and total MIDAS score were similar to the means and medians for the diary-based measures. However, the mean MIDAS scores for the number of days on which headache was experienced and the number of missed workdays were significantly different compared to the diary-based estimates for these items (P<.05). In addition, the mean MIDAS score for the number of days of missed housework was significantly higher than the corresponding diary-based estimate (P<.01). The results from this study show that the Japanese translation of the MIDAS Questionnaire is comparable with the English-language version in terms of reliability and validity.

Assessing motivation orientations in schizophrenia: Scale development and validation.

PubMed

Cooper, Shanna; Lavaysse, Lindsey M; Gard, David E

2015-01-30

Motivation deficits are common in several disorders including schizophrenia, and are an important factor in both functioning and treatment adherence. Self-Determination Theory (SDT), a leading macro-theory of motivation, has contributed a number of insights into how motivation is impaired in schizophrenia. Nonetheless, self-report measures of motivation appropriate for people with severe mental illness (including those that emphasize SDT) are generally lacking in the literature. To fill this gap, we adapted and abbreviated the well-validated General Causality Orientation Scale for use with people with schizophrenia and with other severe mental disorders (GCOS-clinical populations; GCOS-CP). In Study 1, we tested the similarity of our measure to the existing GCOS (using a college sample) and then validated this new measure in a schizophrenia and healthy control sample (Study 2). Results from Study 1 (N=360) indicated that the GCOS-CP was psychometrically similar to the original GCOS and provided good convergent and discriminant validity. In Study 2, the GCOS-CP was given to individuals with (N=44) and without schizophrenia (N=42). In line with both laboratory-based and observer-based research, people with schizophrenia showed lower motivational autonomy and higher impersonal/amotivated orientations. Additional applications of the GCOS-CP are discussed. Copyright © 2014 Elsevier Ireland Ltd. All rights reserved.

Validity of Students Worksheet Based Problem-Based Learning for 9th Grade Junior High School in living organism Inheritance and Food Biotechnology.

NASA Astrophysics Data System (ADS)

Jefriadi, J.; Ahda, Y.; Sumarmin, R.

2018-04-01

Based on preliminary research of students worksheet used by teachers has several disadvantages such as students worksheet arranged directly drove learners conduct an investigation without preceded by directing learners to a problem or provide stimulation, student's worksheet not provide a concrete imageand presentation activities on the students worksheet not refer to any one learning models curicullum recommended. To address problems Reviews these students then developed a worksheet based on problem-based learning. This is a research development that using Ploom models. The phases are preliminary research, development and assessment. The instruments used in data collection that includes pieces of observation/interviews, instrument self-evaluation, instruments validity. The results of the validation expert on student worksheets get a valid result the average value 80,1%. Validity of students worksheet based problem-based learning for 9th grade junior high school in living organism inheritance and food biotechnology get valid category.

Development and psychometric properties of a Calcium Intake Questionnaire based on the social cognitive theory (CIQ-SCT) for Iranian women.

PubMed

Nematollahi, Mahin; Eslami, Ahmad Ali

2018-01-01

Background: Osteoporosis is common among women which may be mostly due to the low intake of calcium. This article reports the development, cultural adaptation and psychometric properties of a Calcium Intake Questionnaire based on the social cognitive theory (CIQ-SCT)among Iranian women. Methods: In 2016, this cross-sectional study was carried out among 400 younger than 50 years old women in Isfahan, Iran. After literature review, a preliminary 35-item questionnaire was developed. Then, forward-backward translation and cultural adaptation of the tool was conducted. Content Validity Index confirmed by an expert panel and Face Validity was evaluated in a pilot study. Exploratory and confirmatory factor analyses (EFA &CFA) were conducted on the calibration and validation sample, respectively. Reliability was also assessed using internal consistency test. Results: After determining content and face validity, 20 items with 5 factors (self-efficacy,outcome expectations, social support and self-regulation) were obtained. Cronbach alpha for the instrument was found to be 0.901. In EFA, we identified a 4-factor model with a total variance of 72.3%. The results related to CFA (CMIN/DF=1.850, CFI =0.946, TLI=0.938, RMSEA=0.069[90% CI: 0.057-0.081]) indicated that the model was fit to the social cognitive theory. Self regulation was detected as the best predictor for calcium intake. Conclusion: The CIQ-SCT showed acceptable levels of reliability and validity in explaining the calcium intake based on the constructs of social cognitive theory. Further psychometric testing is recommended in different population to approve the external validity of the instrument.

Criterion-Related Validity of the Distance- and Time-Based Walk/Run Field Tests for Estimating Cardiorespiratory Fitness: A Systematic Review and Meta-Analysis.

PubMed

Mayorga-Vega, Daniel; Bocanegra-Parrilla, Raúl; Ornelas, Martha; Viciana, Jesús

2016-01-01

The main purpose of the present meta-analysis was to examine the criterion-related validity of the distance- and time-based walk/run tests for estimating cardiorespiratory fitness among apparently healthy children and adults. Relevant studies were searched from seven electronic bibliographic databases up to August 2015 and through other sources. The Hunter-Schmidt's psychometric meta-analysis approach was conducted to estimate the population criterion-related validity of the following walk/run tests: 5,000 m, 3 miles, 2 miles, 3,000 m, 1.5 miles, 1 mile, 1,000 m, ½ mile, 600 m, 600 yd, ¼ mile, 15 min, 12 min, 9 min, and 6 min. From the 123 included studies, a total of 200 correlation values were analyzed. The overall results showed that the criterion-related validity of the walk/run tests for estimating maximum oxygen uptake ranged from low to moderate (rp = 0.42-0.79), with the 1.5 mile (rp = 0.79, 0.73-0.85) and 12 min walk/run tests (rp = 0.78, 0.72-0.83) having the higher criterion-related validity for distance- and time-based field tests, respectively. The present meta-analysis also showed that sex, age and maximum oxygen uptake level do not seem to affect the criterion-related validity of the walk/run tests. When the evaluation of an individual's maximum oxygen uptake attained during a laboratory test is not feasible, the 1.5 mile and 12 min walk/run tests represent useful alternatives for estimating cardiorespiratory fitness. As in the assessment with any physical fitness field test, evaluators must be aware that the performance score of the walk/run field tests is simply an estimation and not a direct measure of cardiorespiratory fitness.

Is self-reporting workplace activity worthwhile? Validity and reliability of occupational sitting and physical activity questionnaire in desk-based workers.

PubMed

Pedersen, Scott J; Kitic, Cecilia M; Bird, Marie-Louise; Mainsbridge, Casey P; Cooley, P Dean

2016-08-19

With the advent of workplace health and wellbeing programs designed to address prolonged occupational sitting, tools to measure behaviour change within this environment should derive from empirical evidence. In this study we measured aspects of validity and reliability for the Occupational Sitting and Physical Activity Questionnaire that asks employees to recount the percentage of work time they spend in the seated, standing, and walking postures during a typical workday. Three separate cohort samples (N = 236) were drawn from a population of government desk-based employees across several departmental agencies. These volunteers were part of a larger state-wide intervention study. Workplace sitting and physical activity behaviour was measured both subjectively against the International Physical Activity Questionnaire, and objectively against ActivPal accelerometers before the intervention began. Criterion validity and concurrent validity for each of the three posture categories were assessed using Spearman's rank correlation coefficients, and a bias comparison with 95 % limits of agreement. Test-retest reliability of the survey was reported with intraclass correlation coefficients. Criterion validity for this survey was strong for sitting and standing estimates, but weak for walking. Participants significantly overestimated the amount of walking they did at work. Concurrent validity was moderate for sitting and standing, but low for walking. Test-retest reliability of this survey proved to be questionable for our sample. Based on our findings we must caution occupational health and safety professionals about the use of employee self-report data to estimate workplace physical activity. While the survey produced accurate measurements for time spent sitting at work it was more difficult for employees to estimate their workplace physical activity.

Evaluation of passenger health risk assessment of sustainable indoor air quality monitoring in metro systems based on a non-Gaussian dynamic sensor validation method.

PubMed

Kim, MinJeong; Liu, Hongbin; Kim, Jeong Tai; Yoo, ChangKyoo

2014-08-15

Sensor faults in metro systems provide incorrect information to indoor air quality (IAQ) ventilation systems, resulting in the miss-operation of ventilation systems and adverse effects on passenger health. In this study, a new sensor validation method is proposed to (1) detect, identify and repair sensor faults and (2) evaluate the influence of sensor reliability on passenger health risk. To address the dynamic non-Gaussianity problem of IAQ data, dynamic independent component analysis (DICA) is used. To detect and identify sensor faults, the DICA-based squared prediction error and sensor validity index are used, respectively. To restore the faults to normal measurements, a DICA-based iterative reconstruction algorithm is proposed. The comprehensive indoor air-quality index (CIAI) that evaluates the influence of the current IAQ on passenger health is then compared using the faulty and reconstructed IAQ data sets. Experimental results from a metro station showed that the DICA-based method can produce an improved IAQ level in the metro station and reduce passenger health risk since it more accurately validates sensor faults than do conventional methods. Copyright © 2014 Elsevier B.V. All rights reserved.

The development and validation of the core competencies scale (CCS) for the college and university students.

PubMed

Ruan, Bin; Mok, Magdalena Mo Ching; Edginton, Christopher R; Chin, Ming Kai

2012-01-01

This article describes the development and validation of the Core Competencies Scale (CCS) using Bok's (2006) competency framework for undergraduate education. The framework included: communication, critical thinking, character development, citizenship, diversity, global understanding, widening of interest, and career and vocational development. The sample comprised 70 college and university students. Results of analysis using Rasch rating scale modelling showed that there was strong empirical evidence on the validity of the measures in contents, structure, interpretation, generalizability, and response options of the CCS scale. The implication of having developed Rasch-based valid and dependable measures in this study for gauging the value added of college and university education to their students is that the feedback generated from CCS will enable evidence-based decision and policy making to be implemented and strategized. Further, program effectiveness can be measured and thus accountability on the achievement of the program objectives.