Instruments Measuring Integrated Care: A Systematic Review of Measurement Properties.

PubMed

Bautista, Mary Ann C; Nurjono, Milawaty; Lim, Yee Wei; Dessers, Ezra; Vrijhoef, Hubertus Jm

2016-12-01

Policy Points: Investigations on systematic methodologies for measuring integrated care should coincide with the growing interest in this field of research. A systematic review of instruments provides insights into integrated care measurement, including setting the research agenda for validating available instruments and informing the decision to develop new ones. This study is the first systematic review of instruments measuring integrated care with an evidence synthesis of the measurement properties. We found 209 index instruments measuring different constructs related to integrated care; the strength of evidence on the adequacy of the majority of their measurement properties remained largely unassessed. Integrated care is an important strategy for increasing health system performance. Despite its growing significance, detailed evidence on the measurement properties of integrated care instruments remains vague and limited. Our systematic review aims to provide evidence on the state of the art in measuring integrated care. Our comprehensive systematic review framework builds on the Rainbow Model for Integrated Care (RMIC). We searched MEDLINE/PubMed for published articles on the measurement properties of instruments measuring integrated care and identified eligible articles using a standard set of selection criteria. We assessed the methodological quality of every validation study reported using the COSMIN checklist and extracted data on study and instrument characteristics. We also evaluated the measurement properties of each examined instrument per validation study and provided a best evidence synthesis on the adequacy of measurement properties of the index instruments. From the 300 eligible articles, we assessed the methodological quality of 379 validation studies from which we identified 209 index instruments measuring integrated care constructs. The majority of studies reported on instruments measuring constructs related to care integration (33%) and patient-centered care (49%); fewer studies measured care continuity/comprehensive care (15%) and care coordination/case management (3%). We mapped 84% of the measured constructs to the clinical integration domain of the RMIC, with fewer constructs related to the domains of professional (3.7%), organizational (3.4%), and functional (0.5%) integration. Only 8% of the instruments were mapped to a combination of domains; none were mapped exclusively to the system or normative integration domains. The majority of instruments were administered to either patients (60%) or health care providers (20%). Of the measurement properties, responsiveness (4%), measurement error (7%), and criterion (12%) and cross-cultural validity (14%) were less commonly reported. We found <50% of the validation studies to be of good or excellent quality for any of the measurement properties. Only a minority of index instruments showed strong evidence of positive findings for internal consistency (15%), content validity (19%), and structural validity (7%); with moderate evidence of positive findings for internal consistency (14%) and construct validity (14%). Our results suggest that the quality of measurement properties of instruments measuring integrated care is in need of improvement with the less-studied constructs and domains to become part of newly developed instruments. © 2016 Milbank Memorial Fund.

Instruments Measuring Integrated Care: A Systematic Review of Measurement Properties

PubMed Central

BAUTISTA, MARY ANN C.; NURJONO, MILAWATY; DESSERS, EZRA; VRIJHOEF, HUBERTUS JM

2016-01-01

Policy Points: Investigations on systematic methodologies for measuring integrated care should coincide with the growing interest in this field of research.A systematic review of instruments provides insights into integrated care measurement, including setting the research agenda for validating available instruments and informing the decision to develop new ones.This study is the first systematic review of instruments measuring integrated care with an evidence synthesis of the measurement properties.We found 209 index instruments measuring different constructs related to integrated care; the strength of evidence on the adequacy of the majority of their measurement properties remained largely unassessed. Context Integrated care is an important strategy for increasing health system performance. Despite its growing significance, detailed evidence on the measurement properties of integrated care instruments remains vague and limited. Our systematic review aims to provide evidence on the state of the art in measuring integrated care. Methods Our comprehensive systematic review framework builds on the Rainbow Model for Integrated Care (RMIC). We searched MEDLINE/PubMed for published articles on the measurement properties of instruments measuring integrated care and identified eligible articles using a standard set of selection criteria. We assessed the methodological quality of every validation study reported using the COSMIN checklist and extracted data on study and instrument characteristics. We also evaluated the measurement properties of each examined instrument per validation study and provided a best evidence synthesis on the adequacy of measurement properties of the index instruments. Findings From the 300 eligible articles, we assessed the methodological quality of 379 validation studies from which we identified 209 index instruments measuring integrated care constructs. The majority of studies reported on instruments measuring constructs related to care integration (33%) and patient‐centered care (49%); fewer studies measured care continuity/comprehensive care (15%) and care coordination/case management (3%). We mapped 84% of the measured constructs to the clinical integration domain of the RMIC, with fewer constructs related to the domains of professional (3.7%), organizational (3.4%), and functional (0.5%) integration. Only 8% of the instruments were mapped to a combination of domains; none were mapped exclusively to the system or normative integration domains. The majority of instruments were administered to either patients (60%) or health care providers (20%). Of the measurement properties, responsiveness (4%), measurement error (7%), and criterion (12%) and cross‐cultural validity (14%) were less commonly reported. We found <50% of the validation studies to be of good or excellent quality for any of the measurement properties. Only a minority of index instruments showed strong evidence of positive findings for internal consistency (15%), content validity (19%), and structural validity (7%); with moderate evidence of positive findings for internal consistency (14%) and construct validity (14%). Conclusions Our results suggest that the quality of measurement properties of instruments measuring integrated care is in need of improvement with the less‐studied constructs and domains to become part of newly developed instruments. PMID:27995711

In Search of Validity Evidence in Support of the Interpretation and Use of Assessments of Complex Constructs: Discussion of Research on Assessing 21st Century Skills

ERIC Educational Resources Information Center

Ercikan, Kadriye; Oliveri, María Elena

2016-01-01

Assessing complex constructs such as those discussed under the umbrella of 21st century constructs highlights the need for a principled assessment design and validation approach. In our discussion, we made a case for three considerations: (a) taking construct complexity into account across various stages of assessment development such as the…

Measurement properties of patient-reported outcome measures (PROMs) used in adult patients with chronic kidney disease: A systematic review

PubMed Central

Kyte, Derek; Cockwell, Paul; Marshall, Tom; Gheorghe, Adrian; Keeley, Thomas; Slade, Anita; Calvert, Melanie

2017-01-01

Background Patient-reported outcome measures (PROMs) can provide valuable information which may assist with the care of patients with chronic kidney disease (CKD). However, given the large number of measures available, it is unclear which PROMs are suitable for use in research or clinical practice. To address this we comprehensively evaluated studies that assessed the measurement properties of PROMs in adults with CKD. Methods Four databases were searched; reference list and citation searching of included studies was also conducted. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist was used to appraise the methodological quality of the included studies and to inform a best evidence synthesis for each PROM. Results The search strategy retrieved 3,702 titles/abstracts. After 288 duplicates were removed, 3,414 abstracts were screened and 71 full-text articles were retrieved for further review. Of these, 24 full-text articles were excluded as they did not meet the eligibility criteria. Following reference list and citation searching, 19 articles were retrieved bringing the total number of papers included in the final analysis to 66. There was strong evidence supporting internal consistency and moderate evidence supporting construct validity for the Kidney Disease Quality of Life-36 (KDQOL-36) in pre-dialysis patients. In the dialysis population, the KDQOL-Short Form (KDQOL-SF) had strong evidence for internal consistency and structural validity and moderate evidence for test-retest reliability and construct validity while the KDQOL-36 had moderate evidence of internal consistency, test-retest reliability and construct validity. The End Stage Renal Disease-Symptom Checklist Transplantation Module (ESRD-SCLTM) demonstrated strong evidence for internal consistency and moderate evidence for test-retest reliability, structural and construct validity in renal transplant recipients. Conclusions We suggest considering the KDQOL-36 for use in pre-dialysis patients; the KDQOL-SF or KDQOL-36 for dialysis patients and the ESRD-SCLTM for use in transplant recipients. However, further research is required to evaluate the measurement error, structural validity, responsiveness and patient acceptability of PROMs used in CKD. PMID:28636678

Constructing a Validity Argument for the Objective Structured Assessment of Technical Skills (OSATS): A Systematic Review of Validity Evidence

ERIC Educational Resources Information Center

Hatala, Rose; Cook, David A.; Brydges, Ryan; Hawkins, Richard

2015-01-01

In order to construct and evaluate the validity argument for the Objective Structured Assessment of Technical Skills (OSATS), based on Kane's framework, we conducted a systematic review. We searched MEDLINE, EMBASE, CINAHL, PsycINFO, ERIC, Web of Science, Scopus, and selected reference lists through February 2013. Working in duplicate, we selected…

Non-Technical Skills for Surgeons (NOTSS): Critical appraisal of its measurement properties.

PubMed

Jung, James J; Borkhoff, Cornelia M; Jüni, Peter; Grantcharov, Teodor P

2018-02-17

To critically appraise the development and measurement properties, including sensibility, reliability, and validity of the Non-Technical Skills of Surgeons (NOTSS) system. Articles that described development process of the NOTSS system were identified. Relevant primary studies that presented evidence of reliability and validity were identified through a comprehensive literature review. NOTSS was developed through robust item generation and reduction strategies. It was shown to have good content validity, acceptability, and feasibility. Inter-rater reliability increased with greater expertise and number of assessors. Studies demonstrated evidence of cross-sectional construct validity, in that the tool was able to differentiate known groups of varied non-technical skill levels. Evidence of longitudinal construct validity also existed to demonstrate that NOTSS detected changes in non-technical skills before and after targeted training. In populations and settings presented in our critical appraisal, NOTSS provided reliable and valid measurements of intraoperative non-technical skills of surgeons. Copyright © 2018 Elsevier Inc. All rights reserved.

Are we really measuring what we say we're measuring? Using video techniques to supplement traditional construct validation procedures.

PubMed

Podsakoff, Nathan P; Podsakoff, Philip M; Mackenzie, Scott B; Klinger, Ryan L

2013-01-01

Several researchers have persuasively argued that the most important evidence to consider when assessing construct validity is whether variations in the construct of interest cause corresponding variations in the measures of the focal construct. Unfortunately, the literature provides little practical guidance on how researchers can go about testing this. Therefore, the purpose of this article is to describe how researchers can use video techniques to test whether their scales measure what they purport to measure. First, we discuss how researchers can develop valid manipulations of the focal construct that they hope to measure. Next, we explain how to design a study to use this manipulation to test the validity of the scale. Finally, comparing and contrasting traditional and contemporary perspectives on validation, we discuss the advantages and limitations of video-based validation procedures. PsycINFO Database Record (c) 2013 APA, all rights reserved.

A Validation and Reliability Study of the Physical Activity and Healthy Food Efficacy Scale for Children (PAHFE)

ERIC Educational Resources Information Center

Perry, Christina M.; De Ayala, R. J.; Lebow, Ryan; Hayden, Emily

2008-01-01

The purpose of this study was to obtain validity evidence for the Physical Activity and Healthy Food Efficacy Scale for Children (PAHFE). Construct validity evidence identifies four subscales: Goal-Setting for Physical Activity, Goal-Setting for Healthy Food Choices, Decision-Making for Physical Activity, and Decision-Making for Healthy Food…

Development and Validation of a Measure of Elementary Teachers' Science Content Knowledge in Two Multiyear Teacher Professional Development Intervention Projects

ERIC Educational Resources Information Center

Maerten-Rivera, Jaime Lynn; Huggins-Manley, Anne Corinne; Adamson, Karen; Lee, Okhee; Llosa, Lorena

2015-01-01

Using data collected from two multiyear teacher professional development projects employing randomized control trials, this study describes the development and validation of a paper-based test of elementary teachers' science content knowledge (SCK). Evidence of construct validity is presented, including evidence on internal structural…

Educational testing validity and reliability in pharmacy and medical education literature.

PubMed

Hoover, Matthew J; Jung, Rose; Jacobs, David M; Peeters, Michael J

2013-12-16

To evaluate and compare the reliability and validity of educational testing reported in pharmacy education journals to medical education literature. Descriptions of validity evidence sources (content, construct, criterion, and reliability) were extracted from articles that reported educational testing of learners' knowledge, skills, and/or abilities. Using educational testing, the findings of 108 pharmacy education articles were compared to the findings of 198 medical education articles. For pharmacy educational testing, 14 articles (13%) reported more than 1 validity evidence source while 83 articles (77%) reported 1 validity evidence source and 11 articles (10%) did not have evidence. Among validity evidence sources, content validity was reported most frequently. Compared with pharmacy education literature, more medical education articles reported both validity and reliability (59%; p<0.001). While there were more scholarship of teaching and learning (SoTL) articles in pharmacy education compared to medical education, validity, and reliability reporting were limited in the pharmacy education literature.

Adaptation and validation of the Evidence-Based Practice Belief and Implementation scales for French-speaking Swiss nurses and allied healthcare providers.

PubMed

Verloo, Henk; Desmedt, Mario; Morin, Diane

2017-09-01

To evaluate two psychometric properties of the French versions of the Evidence-Based Practice Beliefs and Evidence-Based Practice Implementation scales, namely their internal consistency and construct validity. The Evidence-Based Practice Beliefs and Evidence-Based Practice Implementation scales developed by Melnyk et al. are recognised as valid, reliable instruments in English. However, no psychometric validation for their French versions existed. Secondary analysis of a cross sectional survey. Source data came from a cross-sectional descriptive study sample of 382 nurses and other allied healthcare providers. Cronbach's alpha was used to evaluate internal consistency, and principal axis factor analysis and varimax rotation were computed to determine construct validity. The French Evidence-Based Practice Beliefs and Evidence-Based Practice Implementation scales showed excellent reliability, with Cronbach's alphas close to the scores established by Melnyk et al.'s original versions. Principal axis factor analysis showed medium-to-high factor loading scores without obtaining collinearity. Principal axis factor analysis with varimax rotation of the 16-item Evidence-Based Practice Beliefs scale resulted in a four-factor loading structure. Principal axis factor analysis with varimax rotation of the 17-item Evidence-Based Practice Implementation scale revealed a two-factor loading structure. Further research should attempt to understand why the French Evidence-Based Practice Implementation scale showed a two-factor loading structure but Melnyk et al.'s original has only one. The French versions of the Evidence-Based Practice Beliefs and Evidence-Based Practice Implementation scales can both be considered valid and reliable instruments for measuring Evidence-Based Practice beliefs and implementation. The results suggest that the French Evidence-Based Practice Beliefs and Evidence-Based Practice Implementation scales are valid and reliable and can therefore be used to evaluate the effectiveness of organisational strategies aimed at increasing professionals' confidence in Evidence-Based Practice, supporting its use and implementation. © 2017 John Wiley & Sons Ltd.

Development of a proficiency-based virtual reality simulation training curriculum for laparoscopic appendicectomy.

PubMed

Sirimanna, Pramudith; Gladman, Marc A

2017-10-01

Proficiency-based virtual reality (VR) training curricula improve intraoperative performance, but have not been developed for laparoscopic appendicectomy (LA). This study aimed to develop an evidence-based training curriculum for LA. A total of 10 experienced (>50 LAs), eight intermediate (10-30 LAs) and 20 inexperienced (<10 LAs) operators performed guided and unguided LA tasks on a high-fidelity VR simulator using internationally relevant techniques. The ability to differentiate levels of experience (construct validity) was measured using simulator-derived metrics. Learning curves were analysed. Proficiency benchmarks were defined by the performance of the experienced group. Intermediate and experienced participants completed a questionnaire to evaluate the realism (face validity) and relevance (content validity). Of 18 surgeons, 16 (89%) considered the VR model to be visually realistic and 17 (95%) believed that it was representative of actual practice. All 'guided' modules demonstrated construct validity (P < 0.05), with learning curves that plateaued between sessions 6 and 9 (P < 0.01). When comparing inexperienced to intermediates to experienced, the 'unguided' LA module demonstrated construct validity for economy of motion (5.00 versus 7.17 versus 7.84, respectively; P < 0.01) and task time (864.5 s versus 477.2 s versus 352.1 s, respectively, P < 0.01). Construct validity was also confirmed for number of movements, path length and idle time. Validated modules were used for curriculum construction, with proficiency benchmarks used as performance goals. A VR LA model was realistic and representative of actual practice and was validated as a training and assessment tool. Consequently, the first evidence-based internationally applicable training curriculum for LA was constructed, which facilitates skill acquisition to proficiency. © 2017 Royal Australasian College of Surgeons.

Examining construct and predictive validity of the Health-IT Usability Evaluation Scale: confirmatory factor analysis and structural equation modeling results

PubMed Central

Yen, Po-Yin; Sousa, Karen H; Bakken, Suzanne

2014-01-01

Background In a previous study, we developed the Health Information Technology Usability Evaluation Scale (Health-ITUES), which is designed to support customization at the item level. Such customization matches the specific tasks/expectations of a health IT system while retaining comparability at the construct level, and provides evidence of its factorial validity and internal consistency reliability through exploratory factor analysis. Objective In this study, we advanced the development of Health-ITUES to examine its construct validity and predictive validity. Methods The health IT system studied was a web-based communication system that supported nurse staffing and scheduling. Using Health-ITUES, we conducted a cross-sectional study to evaluate users’ perception toward the web-based communication system after system implementation. We examined Health-ITUES's construct validity through first and second order confirmatory factor analysis (CFA), and its predictive validity via structural equation modeling (SEM). Results The sample comprised 541 staff nurses in two healthcare organizations. The CFA (n=165) showed that a general usability factor accounted for 78.1%, 93.4%, 51.0%, and 39.9% of the explained variance in ‘Quality of Work Life’, ‘Perceived Usefulness’, ‘Perceived Ease of Use’, and ‘User Control’, respectively. The SEM (n=541) supported the predictive validity of Health-ITUES, explaining 64% of the variance in intention for system use. Conclusions The results of CFA and SEM provide additional evidence for the construct and predictive validity of Health-ITUES. The customizability of Health-ITUES has the potential to support comparisons at the construct level, while allowing variation at the item level. We also illustrate application of Health-ITUES across stages of system development. PMID:24567081

A Construct Validity Study of Clinical Competence: A Multitrait Multimethod Matrix Approach

ERIC Educational Resources Information Center

Baig, Lubna; Violato, Claudio; Crutcher, Rodney

2010-01-01

Introduction: The purpose of the study was to adduce evidence for estimating the construct validity of clinical competence measured through assessment instruments used for high-stakes examinations. Methods: Thirty-nine international physicians (mean age = 41 + 6.5 y) participated in high-stakes examination and 3-month supervised clinical practice…

Transtheoretical Model Constructs for Physical Activity Behavior are Invariant across Time among Ethnically Diverse Adults in Hawaii

PubMed Central

Nigg, Claudio R; Motl, Robert W; Horwath, Caroline; Dishman, Rod K

2012-01-01

Objectives Physical activity (PA) research applying the Transtheoretical Model (TTM) to examine group differences and/or change over time requires preliminary evidence of factorial validity and invariance. The current study examined the factorial validity and longitudinal invariance of TTM constructs recently revised for PA. Method Participants from an ethnically diverse sample in Hawaii (N=700) completed questionnaires capturing each TTM construct. Results Factorial validity was confirmed for each construct using confirmatory factor analysis with full-information maximum likelihood. Longitudinal invariance was evidenced across a shorter (3-month) and longer (6-month) time period via nested model comparisons. Conclusions The questionnaires for each validated TTM construct are provided, and can now be generalized across similar subgroups and time points. Further validation of the provided measures is suggested in additional populations and across extended time points. PMID:22778669

Technical Adequacy of the easyCBM Primary-Level Mathematics Measures (Grades K-2), 2009-2010 Version. Technical Report #1006

ERIC Educational Resources Information Center

Anderson, Daniel; Lai, Cheng-Fei; Nese, Joseph F. T.; Park, Bitnara Jasmine; Saez, Leilani; Jamgochian, Elisa; Alonzo, Julie; Tindal, Gerald

2010-01-01

In the following technical report, we present evidence of the technical adequacy of the easyCBM[R] math measures in grades K-2. In addition to reliability information, we present criterion-related validity evidence, both concurrent and predictive, and construct validity evidence. The results represent data gathered throughout the 2009/2010 school…

Reliability and Validity of the Evidence-Based Practice Confidence (EPIC) Scale

ERIC Educational Resources Information Center

Salbach, Nancy M.; Jaglal, Susan B.; Williams, Jack I.

2013-01-01

Introduction: The reliability, minimal detectable change (MDC), and construct validity of the evidence-based practice confidence (EPIC) scale were evaluated among physical therapists (PTs) in clinical practice. Methods: A longitudinal mail survey was conducted. Internal consistency and test-retest reliability were estimated using Cronbach's alpha…

Applying the transtheoretical model to tobacco cessation and prevention: a review of literature.

PubMed

Spencer, Leslie; Pagell, Francie; Hallion, Maria Elena; Adams, Troy B

2002-01-01

To comprehensively review all published, peer-reviewed research on the Transtheoretical Model (TTM) and tobacco cessation and prevention by exploring the validity of its constructs, the evidence for use of interventions based on the TTM, the description of populations using TTM constructs, and the identification of areas for further research. The three research questions answered were: "How is the validity of the TTM as applied to tobacco supported by research?" "How does the TTM describe special populations regarding tobacco use?" "What is the nature of evidence supporting the use of stage-matched tobacco interventions?" Computer Database search (PsychInfo, Medline, Current Contents, ERIC, CINAHL-Allied Health, and Pro-Quest Nursing) and manual journal search. INCLUSION/EXCLUSION CRITERIA: All English, original, research articles on the TTM as it relates to tobacco use published in peer-reviewed journals prior to March 1, 2001, were included. Commentaries, editorials, and books were not included. Articles were categorized as TTM construct validation, population descriptions using TTM constructs, or intervention evaluation using TTM constructs. Summary tables including study design, research rating, purpose, methods, findings, and implications were created. Articles were further divided into groups according to their purpose. Considering both the findings and research quality of each, the three research questions were addressed. The 148 articles reviewed included 54 validation studies, 73 population studies, and 37 interventions (some articles fit two categories). Overall, the evidence in support of the TTM as applied to tobacco use was strong, with supportive studies being more numerous and of a better design than nonsupportive studies. Using established criteria, we rated the construct validity of the entire body of literature as good; however, notable concerns exist about the staging construct. A majority of stage-matched intervention studies provided positive results and were of a better quality than those not supportive of stage-matched interventions; thus, we rated the body of literature using stage-matched tobacco interventions as acceptable and the body of literature using non-stage-matched interventions as suggestive. Population studies indicated that TTM constructs are applicable to a wide variety of general and special populations both in and outside of the United States, although a few exceptions exist. Evidence for the validity of the TTM as it applies to tobacco use is strong and growing; however, it is not conclusive. Eight different staging mechanisms were identified, raising the question of which are most valid and reliable. Interventions tailored to a smoker's stage were successful more often than nontailored interventions in promoting forward stage movement. Stage distribution is well-documented for U.S. populations; however, more research is needed for non-U.S. populations, for special populations, and on other TTM constructs.

The Construct Validity of HPAT-Ireland for the Selection of Medical Students: Unresolved Issues and Future Research Implications

ERIC Educational Resources Information Center

Kelly, Maureen E.; O'Flynn, Siun

2017-01-01

Aptitude tests are widely used in selection. However, despite certain advantages their use remains controversial. This paper aims to critically appraise five sources of evidence for the construct validity of the Health Professions Admission Test (HPAT)-Ireland, an aptitude test used for selecting undergraduate medical students. The objectives are…

Voices from Test-Takers: Further Evidence for Language Assessment Validation and Use

ERIC Educational Resources Information Center

Cheng, Liying; DeLuca, Christopher

2011-01-01

Test-takers' interpretations of validity as related to test constructs and test use have been widely debated in large-scale language assessment. This study contributes further evidence to this debate by examining 59 test-takers' perspectives in writing large-scale English language tests. Participants wrote about their test-taking experiences in…

The Online Student Connectedness Survey: Evidence of Initial Construct Validity

ERIC Educational Resources Information Center

Zimmerman, Tekeisha; Nimon, Kim

2017-01-01

The Online Student Connectedness Survey (OSCS) was introduced to the academic community in 2012 as an instrument designed to measure feelings of connectedness between students participating in online degree and certification programs. The purpose of this study was to examine data from the instrument for initial evidence of validity and reliability…

Validity: Applying Current Concepts and Standards to Gynecologic Surgery Performance Assessments

ERIC Educational Resources Information Center

LeClaire, Edgar L.; Nihira, Mikio A.; Hardré, Patricia L.

2015-01-01

Validity is critical for meaningful assessment of surgical competency. According to the Standards for Educational and Psychological Testing, validation involves the integration of data from well-defined classifications of evidence. In the authoritative framework, data from all classifications support construct validity claims. The two aims of this…

Application of the OMERACT filter to measures of core outcome domains in recent clinical studies of acute gout.

PubMed

Taylor, William J; Redden, David; Dalbeth, Nicola; Schumacher, H Ralph; Edwards, N Lawrence; Simon, Lee S; John, Markus R; Essex, Margaret N; Watson, Douglas J; Evans, Robert; Rome, Keith; Singh, Jasvinder A

2014-03-01

To determine the extent to which instruments that measure core outcome domains in acute gout fulfill the Outcome Measures in Rheumatology (OMERACT) filter requirements of truth, discrimination, and feasibility. Patient-level data from 4 randomized controlled trials of agents designed to treat acute gout and 1 observational study of acute gout were analyzed. For each available measure, construct validity, test-retest reliability, within-group change using effect size, between-group change using the Kruskall-Wallis statistic, and repeated measures generalized estimating equations were assessed. Floor and ceiling effects were also assessed and minimal clinically important difference was estimated. These analyses were presented to participants at OMERACT 11 to help inform voting for possible endorsement. There was evidence for construct validity and discriminative ability for 3 measures of pain [0 to 4 Likert, 0 to 10 numeric rating scale (NRS), 0 to 100 mm visual analog scale (VAS)]. Likewise, there appears to be sufficient evidence for a 4-point Likert scale to possess construct validity and discriminative ability for physician assessment of joint swelling and joint tenderness. There was some evidence for construct validity and within-group discriminative ability for the Health Assessment Questionnaire as a measure of activity limitations, but not for discrimination between groups allocated to different treatment. There is sufficient evidence to support measures of pain (using Likert, NRS, or VAS), joint tenderness, and swelling (using Likert scale) as fulfilling the requirements of the OMERACT filter. Further research on a measure of activity limitations in acute gout clinical trials is required.

Application of the OMERACT filter to measures of core outcome domains in recent clinical studies of acute gout

PubMed Central

Taylor, William J; Redden, David; Dalbeth, Nicola; Schumacher, H Ralph; Edwards, N Lawrence; Simon, Lee S; John, Markus R; Essex, Margaret N; Watson, Douglas J; Evans, Robert; Rome, Keith; Singh, Jasvinder A

2014-01-01

Objective To determine the extent to which instruments that measure core outcome domains in acute gout fulfil the OMERACT filter requirements of truth, discrimination and feasibility. Methods Patient-level data from four randomised controlled trials of agents designed to treat acute gout and one observational study of acute gout were analysed. For each available measure construct validity, test-retest reliability, within-group change using effect size, between-group change using the Kruskall-Wallis statistic and repeated measures generalised estimating equations were assessed. Floor and ceiling effects were also assessed and MCID was estimated. These analyses were presented to participants at OMERACT 11 to help inform voting for possible endorsement. Results There was evidence for construct validity and discriminative ability for 3 measures of pain (0 to 4 Likert, 0 to 10 numeric rating scale, 0 to 100 mm visual analogue scale). Likewise, there appears to be sufficient evidence for a 4-point Likert scale to possess construct validity and discriminative ability for physician assessment of joint swelling and joint tenderness. There was some evidence for construct validity and within-group discriminative ability for the Health Assessment Questionnaire as a measure of activity limitations, but not for discrimination between groups allocated to different treatment. Conclusions There is sufficient evidence to support measures of pain (using Likert, numeric rating scale or visual analogue scales), joint tenderness and swelling (using Likert scale) as fulfilling the requirements of the OMERACT filter. Further research on a measure of activity limitations in acute gout clinical trials is required. PMID:24429178

Construction and Validation of the Clinical Judgment Skill Inventory: Clinical Judgment Skill Competencies That Measure Counselor Debiasing Techniques

ERIC Educational Resources Information Center

Austin, Bryan S.; Leahy, Michael J.

2015-01-01

Purpose: To construct and validate a new self-report instrument, the Clinical Judgment Skill Inventory (CJSI), inclusive of clinical judgment skill competencies that address counselor biases and evidence-based strategies. Method: An Internet-based survey design was used and an exploratory factor analysis was performed on a sample of rehabilitation…

Construct Validity of the Behavior Assessment System for Children (BASC) Self-Report of Personality: Evidence from Adolescents Referred to Residential Treatment

ERIC Educational Resources Information Center

Weis, Robert; Smenner, Lindsey

2007-01-01

The authors investigate the construct validity of the Behavior Assessment System for Children Self-Report of Personality (BASC-SRP; Reynolds & Kamphaus, 1998). A sample of 970 adolescents (16-18 years) with histories of disruptive behavior problems and truancy complete the SRP; a subsample of 290 adolescents also completed the Minnesota…

Sixteen-Item Anxiety Sensitivity Index: Confirmatory Factor Analytic Evidence, Internal Consistency, and Construct Validity in a Young Adult Sample from the Netherlands

ERIC Educational Resources Information Center

Vujanovic, Anka A.; Arrindell, Willem A.; Bernstein, Amit; Norton, Peter J.; Zvolensky, Michael J.

2007-01-01

The present investigation examined the factor structure, internal consistency, and construct validity of the 16-item Anxiety Sensitivity Index (ASI; Reiss Peterson, Gursky, & McNally 1986) in a young adult sample (n = 420) from the Netherlands. Confirmatory factor analysis was used to comparatively evaluate two-factor, three-factor, and…

The Utrecht questionnaire (U-CEP) measuring knowledge on clinical epidemiology proved to be valid.

PubMed

Kortekaas, Marlous F; Bartelink, Marie-Louise E L; de Groot, Esther; Korving, Helen; de Wit, Niek J; Grobbee, Diederick E; Hoes, Arno W

2017-02-01

Knowledge on clinical epidemiology is crucial to practice evidence-based medicine. We describe the development and validation of the Utrecht questionnaire on knowledge on Clinical epidemiology for Evidence-based Practice (U-CEP); an assessment tool to be used in the training of clinicians. The U-CEP was developed in two formats: two sets of 25 questions and a combined set of 50. The validation was performed among postgraduate general practice (GP) trainees, hospital trainees, GP supervisors, and experts. Internal consistency, internal reliability (item-total correlation), item discrimination index, item difficulty, content validity, construct validity, responsiveness, test-retest reliability, and feasibility were assessed. The questionnaire was externally validated. Internal consistency was good with a Cronbach alpha of 0.8. The median item-total correlation and mean item discrimination index were satisfactory. Both sets were perceived as relevant to clinical practice. Construct validity was good. Both sets were responsive but failed on test-retest reliability. One set took 24 minutes and the other 33 minutes to complete, on average. External GP trainees had comparable results. The U-CEP is a valid questionnaire to assess knowledge on clinical epidemiology, which is a prerequisite for practicing evidence-based medicine in daily clinical practice. Copyright © 2016 Elsevier Inc. All rights reserved.

Development and Feasibility of a Virtual Reality Task for the Cognitive Assessment of Older Adults: The ECO-VR.

PubMed

Oliveira, Camila R; Lopes Filho, Brandel José P; Sugarman, Michael A; Esteves, Cristiane S; Lima, Margarida Maria B M P; Moret-Tatay, Carmen; Irigaray, Tatiana Q; Argimon, Irani Iracema L

2016-12-13

Cognitive assessment with virtual reality (VR) may have superior ecological validity for older adults compared to traditional pencil-and-paper cognitive assessment. However, few studies have reported the development of VR tasks. The aim of this study was to present the development, feasibility, content validity, and preliminary evidence of construct validity of an ecological task of cognitive assessment for older adults in VR (ECO-VR). The tasks were prepared based on theoretical and clinical backgrounds. We had 29 non-expert judges identify virtual visual stimuli and three-dimensional scenarios, and five expert judges assisted with content analysis and developing instructions. Finally, six older persons participated in three pilot studies and thirty older persons participated in the preliminary study to identify construct validity evidence. Data were analyzed by descriptive statistics and partial correlation. Target stimuli and three-dimensional scenarios were judged adequate and the content analysis demonstrated that ECO-VR evaluates temporo-spatial orientation, memory, language and executive functioning. We made significant changes to the instructions after the pilot studies to increase comprehensibility and reduce the completion time. The total score of ECO-VR was positively correlated mainly with performance in executive function (r = .172, p < .05) and memory tests (r = .488, p ≤ .01). The ECO-VR demonstrated feasibility for cognitive assessment in older adults, as well as content and construct validity evidences.

The Impact of Model Parameterization and Estimation Methods on Tests of Measurement Invariance with Ordered Polytomous Data

ERIC Educational Resources Information Center

Koziol, Natalie A.; Bovaird, James A.

2018-01-01

Evaluations of measurement invariance provide essential construct validity evidence--a prerequisite for seeking meaning in psychological and educational research and ensuring fair testing procedures in high-stakes settings. However, the quality of such evidence is partly dependent on the validity of the resulting statistical conclusions. Type I or…

Examining Evidence for the Validity of PISA Learning Strategy Scales Based on Student Response Processes

ERIC Educational Resources Information Center

Hopfenbeck, Therese N.; Maul, Andrew

2011-01-01

The aim of this study was to investigate response-process based evidence for the validity of the Programme for International Student Assessment's (PISA) self-report questionnaire scales as measures of specific psychological constructs, with a focus on scales meant to measure inclination toward specific learning strategies. Cognitive interviews (N…

Construct Validity Evidence for Bracken School Readiness Assessment, Third Edition, Spanish Form Scores

ERIC Educational Resources Information Center

Ortiz, Arlene; Clinton, Amanda; Schaefer, Barbara A.

2015-01-01

Convergent and discriminant validity evidence was examined for scores on the Spanish Record Form of the Bracken School Readiness Assessment, Third Edition (BSRA-3). Participants included a sample of 68 Hispanic, Spanish-speaking children ages 4 to 5 years enrolled in preschool programs in Puerto Rico. Scores obtained from the BSRA-3 Spanish Record…

Examining construct and predictive validity of the Health-IT Usability Evaluation Scale: confirmatory factor analysis and structural equation modeling results.

PubMed

Yen, Po-Yin; Sousa, Karen H; Bakken, Suzanne

2014-10-01

In a previous study, we developed the Health Information Technology Usability Evaluation Scale (Health-ITUES), which is designed to support customization at the item level. Such customization matches the specific tasks/expectations of a health IT system while retaining comparability at the construct level, and provides evidence of its factorial validity and internal consistency reliability through exploratory factor analysis. In this study, we advanced the development of Health-ITUES to examine its construct validity and predictive validity. The health IT system studied was a web-based communication system that supported nurse staffing and scheduling. Using Health-ITUES, we conducted a cross-sectional study to evaluate users' perception toward the web-based communication system after system implementation. We examined Health-ITUES's construct validity through first and second order confirmatory factor analysis (CFA), and its predictive validity via structural equation modeling (SEM). The sample comprised 541 staff nurses in two healthcare organizations. The CFA (n=165) showed that a general usability factor accounted for 78.1%, 93.4%, 51.0%, and 39.9% of the explained variance in 'Quality of Work Life', 'Perceived Usefulness', 'Perceived Ease of Use', and 'User Control', respectively. The SEM (n=541) supported the predictive validity of Health-ITUES, explaining 64% of the variance in intention for system use. The results of CFA and SEM provide additional evidence for the construct and predictive validity of Health-ITUES. The customizability of Health-ITUES has the potential to support comparisons at the construct level, while allowing variation at the item level. We also illustrate application of Health-ITUES across stages of system development. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Construct Maps: A Tool to Organize Validity Evidence

ERIC Educational Resources Information Center

McClarty, Katie Larsen

2013-01-01

The construct map is a promising tool for organizing the data standard-setting panelists interpret. The challenge in applying construct maps to standard-setting procedures will be the judicious selection of data to include within this organizing framework. Therefore, this commentary focuses on decisions about what to include in the construct map.…

A critical evaluation of the validity of episodic future thinking: A clinical neuropsychology perspective.

PubMed

Ward, Amanda M

2016-11-01

Episodic future thinking is defined as the ability to mentally simulate a future event. Although episodic future thinking has been studied extensively in neuroscience, this construct has not been explored in depth from the perspective of clinical neuropsychology. The aim of this critical narrative review is to assess the validity and clinical implications of episodic future thinking. A systematic review of episodic future thinking literature was conducted. PubMed and PsycInfo were searched through July 2015 for review and empirical articles with the following search terms: "episodic future thinking," "future mental simulation," "imagining the future," "imagining new experiences," "future mental time travel," "future autobiographical experience," and "prospection." The review discusses evidence that episodic future thinking is important for adaptive functioning, which has implications for neurological populations. To determine the validity of episodic future thinking, the construct is evaluated with respect to related constructs, such as imagination, episodic memory, autobiographical memory, prospective memory, narrative construction, and working memory. Although it has been minimally investigated, there is evidence of convergent and discriminant validity for episodic future thinking. Research has not addressed the incremental validity of episodic future thinking. Practical considerations of episodic future thinking tasks and related constructs in a clinical neuropsychological setting are considered. The utility of episodic future thinking is currently unknown due to the lack of research investigating the validity of episodic future thinking. Future work is discussed, which could determine whether episodic future thinking is an important missing piece in standard clinical neuropsychological assessment. (PsycINFO Database Record (c) 2016 APA, all rights reserved).

A systematic review of validated sinus surgery simulators.

PubMed

Stew, B; Kao, S S-T; Dharmawardana, N; Ooi, E H

2018-06-01

Simulation provides a safe and effective opportunity to develop surgical skills. A variety of endoscopic sinus surgery (ESS) simulators has been described in the literature. Validation of these simulators allows for effective utilisation in training. To conduct a systematic review of the published literature to analyse the evidence for validated ESS simulation. Pubmed, Embase, Cochrane and Cinahl were searched from inception of the databases to 11 January 2017. Twelve thousand five hundred and sixteen articles were retrieved of which 10 112 were screened following the removal of duplicates. Thirty-eight full-text articles were reviewed after meeting search criteria. Evidence of face, content, construct, discriminant and predictive validity was extracted. Twenty articles were included in the analysis describing 12 ESS simulators. Eleven of these simulators had undergone validation: 3 virtual reality, 7 physical bench models and 1 cadaveric simulator. Seven of the simulators were shown to have face validity, 7 had construct validity and 1 had predictive validity. None of the simulators demonstrated discriminate validity. This systematic review demonstrates that a number of ESS simulators have been comprehensively validated. Many of the validation processes, however, lack standardisation in outcome reporting, thus limiting a meta-analysis comparison between simulators. © 2017 John Wiley & Sons Ltd.

[Do Current German-Language Intelligence Tests Take into Consideration the Special Needs of Children with Disabilities?].

PubMed

Mickley, Manfred; Renner, Gerolf

2015-01-01

Do Current German-Language Intelligence Tests Take into Consideration the Special Needs of Children with Disabilities? A review of 23 German intelligence test manuals shows that test-authors do not exclude the use of their tests for children with disabilities. However, these special groups play a minor role in the construction, standardization, and validation of intelligence tests. There is no sufficient discussion and reflection concerning the issue which construct-irrelevant requirements may reduce the validity of the test or which individual test-adaptations are allowed or recommended. Intelligence testing of children with disabilities needs more empirical evidence on objectivity, reliability, and validity of the assessment-procedures employed. Future test construction and validation should systematically analyze construct-irrelevant variance in item format, the special needs of handicapped children, and should give hints for useful test-adaptations.

Construct Validation in Counseling Psychology Research

ERIC Educational Resources Information Center

Hoyt, William T.; Warbasse, Rosalia E.; Chu, Erica Y.

2006-01-01

Counseling psychology researchers devote little attention to theory-based measurement validation, as evidenced by cursory mention of validity issues in the method and discussion sections of published research reports. Especially, many researchers appear unaware of the limitations of correlations between pairs of self-report measures as evidence of…

Validity of the Microcomputer Evaluation Screening and Assessment Aptitude Scores.

ERIC Educational Resources Information Center

Janikowski, Timothy P.; And Others

1991-01-01

Examined validity of Microcomputer Evaluation Screening and Assessment (MESA) aptitude scores relative to General Aptitude Test Battery (GATB) using multitrait-multimethod correlational analyses. Findings from 54 rehabilitation clients and 29 displaced workers revealed no evidence to support the construct validity of the MESA. (Author/NB)

Convergent and Discriminant Validity of the Microcomputer Evaluation Screening and Assessment (MESA) Interest Survey.

ERIC Educational Resources Information Center

Janikowski, Timothy P.; And Others

1990-01-01

Examined construct validity of Microcomputer Evaluation Screening and Assessment (MESA) Interest Survey. Administered MESA and United States Employment Service (USES) Interest Inventory to 74 volunteer rehabilitation clients. Evidence supported convergent and discriminant validity of MESA. Found fewer significant intercorrelations among MESA…

Validation of the Juhnke-Balkin Life Balance Inventory

ERIC Educational Resources Information Center

Davis, R. J.; Balkin, Richard S.; Juhnke, Gerald A.

2014-01-01

Life balance is an important construct within the counseling profession. A validation study utilizing exploratory factor analysis and multiple regression was conducted on the Juhnke-Balkin Life Balance Inventory. Results from the study serve as evidence of validity for an assessment instrument designed to measure life balance.

Systematic review of measurement properties of questionnaires measuring somatization in primary care patients.

PubMed

Sitnikova, Kate; Dijkstra-Kersten, Sandra M A; Mokkink, Lidwine B; Terluin, Berend; van Marwijk, Harm W J; Leone, Stephanie S; van der Horst, Henriëtte E; van der Wouden, Johannes C

2017-12-01

The aim of this review is to critically appraise the evidence on measurement properties of self-report questionnaires measuring somatization in adult primary care patients and to provide recommendations about which questionnaires are most useful for this purpose. We assessed the methodological quality of included studies using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. To draw overall conclusions about the quality of the questionnaires, we conducted an evidence synthesis using predefined criteria for judging the measurement properties. We found 24 articles on 9 questionnaires. Studies on the Patient Health Questionnaire-15 (PHQ-15) and the Four-Dimensional Symptom Questionnaire (4DSQ) somatization subscale prevailed and covered the broadest range of measurement properties. These questionnaires had the best internal consistency, test-retest reliability, structural validity, and construct validity. The PHQ-15 also had good criterion validity, whereas the 4DSQ somatization subscale was validated in several languages. The Bodily Distress Syndrome (BDS) checklist had good internal consistency and structural validity. Some evidence was found for good construct validity and criterion validity of the Physical Symptom Checklist (PSC-51) and good construct validity of the Symptom Check-List (SCL-90-R) somatization subscale. However, these three questionnaires were only studied in a small number of primary care studies. Based on our findings, we recommend the use of either the PHQ-15 or 4DSQ somatization subscale for somatization in primary care. Other questionnaires, such as the BDS checklist, PSC-51 and the SCL-90-R somatization subscale show promising results but have not been studied extensively in primary care. Copyright © 2017 Elsevier Inc. All rights reserved.

Toward validation of a structural approach to conceptualizing psychopathology: A special section of the Journal of Abnormal Psychology.

PubMed

Krueger, Robert F; Tackett, Jennifer L; MacDonald, Angus

2016-11-01

Traditionally, psychopathology has been conceptualized in terms of polythetic categories derived from committee deliberations and enshrined in authoritative psychiatric nosologies-most notably the Diagnostic and Statistical Manual of Mental Disorders (DSM; American Psychiatric Association [APA], 2013). As the limitations of this form of classification have become evident, empirical data have been increasingly relied upon to investigate the structure of psychopathology. These efforts have borne fruit in terms of an increasingly consistent set of psychopathological constructs closely connected with similar personality constructs. However, the work of validating these constructs using convergent sources of data is an ongoing enterprise. This special section collects several new efforts to use structural approaches to study the validity of this empirically based organizational scheme for psychopathology. Inasmuch as a structural approach reflects the natural organization of psychopathology, it has great potential to facilitate comprehensive organization of information on the correlates of psychopathology, providing evidence for the convergent and discriminant validity of an empirical approach to classification. Here, we highlight several themes that emerge from this burgeoning literature. (PsycINFO Database Record (c) 2016 APA, all rights reserved).

Factor Structure and Validity of Paper-and-Pencil Measures of Mental Speed: Evidence for a Higher-Order Model?

ERIC Educational Resources Information Center

Danthiir, Vanessa; Wilhelm, Oliver; Schulze, Ralf; Roberts, Richard D.

2005-01-01

This study explored the structure of elementary cognitive tasks (ECTs) and relations between the corresponding construct(s) with processing speed (Gs) and fluid intelligence (Gf). Participants (N=321) completed 14 ECTs, 3 Gs, and 6 Gf marker tests, all administered in paper-and-pencil format to reduce potential confounds evident when tasks are…

Evidences of Validity of a Scale for Mapping Professional as Defining Competences and Performance by Brazilian Tutors

ERIC Educational Resources Information Center

Coelho, Francisco Antonio, Jr.; Ferreira, Rodrigo Rezende; Paschoal, Tatiane; Faiad, Cristiane; Meneses, Paulo Murce

2015-01-01

The purpose of this study was twofold: to assess evidences of construct validity of the Brazilian Scale of Tutors Competences in the field of Open and Distance Learning and to examine if variables such as professional experience, perception of the student´s learning performance and prior experience influence the development of technical and…

The Occupational Performance History Interview: Evidence for Three Underlying Constructs of Occupational Adaptation.

ERIC Educational Resources Information Center

Mallinson, Trudy; Mahaffey, Lisa; Kielhofner, Gary

1998-01-01

Data from 20 psychiatric clients were used to test the construct validity of the Occupational Performance History Interview, which gathers information on a person's past and present functioning. The instrument appears to measure three underlying constructs--occupational competence, identity, and environment--rather than occupational adaptation.…

The Validity and Responsiveness of Isometric Lower Body Multi-Joint Tests of Muscular Strength: a Systematic Review.

PubMed

Drake, David; Kennedy, Rodney; Wallace, Eric

2017-12-01

Researchers and practitioners working in sports medicine and science require valid tests to determine the effectiveness of interventions and enhance understanding of mechanisms underpinning adaptation. Such decision making is influenced by the supportive evidence describing the validity of tests within current research. The objective of this study is to review the validity of lower body isometric multi-joint tests ability to assess muscular strength and determine the current level of supporting evidence. Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines were followed in a systematic fashion to search, assess and synthesize existing literature on this topic. Electronic databases such as Web of Science, CINAHL and PubMed were searched up to 18 March 2015. Potential inclusions were screened against eligibility criteria relating to types of test, measurement instrument, properties of validity assessed and population group and were required to be published in English. The Consensus-based Standards for the Selection of health Measurement Instruments (COSMIN) checklist was used to assess methodological quality and measurement property rating of included studies. Studies rated as fair or better in methodological quality were included in the best evidence synthesis. Fifty-nine studies met the eligibility criteria for quality appraisal. The ten studies that rated fair or better in methodological quality were included in the best evidence synthesis. The most frequently investigated lower body isometric multi-joint tests for validity were the isometric mid-thigh pull and isometric squat. The validity of each of these tests was strong in terms of reliability and construct validity. The evidence for responsiveness of tests was found to be moderate for the isometric squat test and unknown for the isometric mid-thigh pull. No tests using the isometric leg press met the criteria for inclusion in the best evidence synthesis. Researchers and practitioners can use the isometric squat and isometric mid-thigh pull with confidence in terms of reliability and construct validity. Further work to investigate other validity components such as criterion validity, smallest detectable change and responsiveness to resistance exercise interventions may be beneficial to the current level of evidence.

An evaluation of the construct of earned security in adolescents: evidence from an inpatient sample.

PubMed

Venta, Amanda; Sharp, Carla; Shmueli-Goetz, Yael; Newlin, Elizabeth

2015-01-01

In adult attachment research, a group of individuals who convey secure attachments despite recalling difficult early caregiver relationships has been identified. The term earned security refers to individuals in this group, whereas continuous security refers to individuals who convey secure attachments and describe caring early relationships. Evidence on the validity of earned security in adults is mixed--with one longitudinal study showing that earned secure adults, despite contrary recollections, are actually more likely to have experienced positive caregiving than continuous secure adults. There is currently no evidence of earned security in adolescence, and exploring it in this age group may help shed light on the overall problem of the validity of this construct. Therefore, the broad aim of this study was to examine the construct of earned security in a group of inpatient adolescents. First, the authors aimed to identify a group of adolescents with secure attachments and memories of difficult caregiver relationships (i.e., proposed earned secure group) in a sample of 240 inpatient adolescents. Next, to explore external validity, the authors examined whether this group differed from others with regard to internalizing distress and emotion regulation. Findings indicated that a subset of secure adolescents recall difficult caregiving, as has been noted in adults, and that they differ from others with regard to emotion regulation. Despite this preliminary evidence that earned security can be identified in adolescents, the authors conclude with a discussion of the caveats of applying this construct in adolescents as well as adults.

Investigating Attitudes toward Physical Education: Validation across Two Instruments

ERIC Educational Resources Information Center

Donovan, Corinne Baron; Mercier, Kevin; Phillips, Sharon R.

2015-01-01

The Centers for Disease Control have suggested that physical education plays a role in promoting healthy lifestyles. Prior research suggests a link between attitudes toward physical education and physical activity outside school. The current study provides additional evidence of construct validity through a validation across two instruments…

Measuring Long-Distance Romantic Relationships: A Validity Study

ERIC Educational Resources Information Center

Pistole, M. Carole; Roberts, Amber

2011-01-01

This study investigated aspects of construct validity for the scores of a new long-distance romantic relationship measure. A single-factor structure of the long-distance romantic relationship index emerged, with convergent and discriminant evidence of external validity, high internal consistency reliability, and applied utility of the scores.…

Validating the Watson Glaser Critical Thinking Appraisal

ERIC Educational Resources Information Center

Hassan, Karma El; Madhum, Ghida

2007-01-01

This study validated the Watson Glaser Critical Thinking Appraisal (WGCTA) on a sample of 273 private university students in Lebanon. For that purpose, evidence for construct validation was investigated through identifying the test's factor structure and subscale total correlations, in addition to differences in scores by gender, different levels,…

Viability of Construct Validity of the Speaking Modules of International Language Examinations (IELTS vs. TOEFL iBT): Evidence from Iranian Test-Takers

ERIC Educational Resources Information Center

Zahedi, Keivan; Shamsaee, Saeedeh

2012-01-01

The aim of the present research is to examine the viability of the construct validity of the speaking modules of two internationally recognized language proficiency examinations, namely IELTS and TOEFL iBT. High-stake standardized tests play a crucial and decisive role in determining the future academic life of many people. Overall obtained scores…

An Attempt to Determine the Construct Validity of Measures Hypothesized to Represent an Orientation to Right, Left, or Integrated Hemispheric Brain Function for a Sample of Primary School Children.

ERIC Educational Resources Information Center

Dumbrower, Jule; And Others

1981-01-01

This study attempts to obtain evidence of the construct validity of pupil ability tests hypothesized to represent orientation to right, left, or integrated hemispheric function, and of teacher observation subscales intended to reveal behaviors in school setting that were hypothesized to portray preference for right or left brain function. (Author)

Evidence of validity of the Stress-Producing Life Events (SPLE) instrument.

PubMed

Rizzini, Marta; Santos, Alcione Miranda Dos; Silva, Antônio Augusto Moura da

2018-01-01

OBJECTIVE Evaluate the construct validity of a list of eight Stressful Life Events in pregnant women. METHODS A cross-sectional study was conducted with 1,446 pregnant women in São Luís, MA, and 1,364 pregnant women in Ribeirão Preto, SP (BRISA cohort), from February 2010 to June 2011. In the exploratory factorial analysis, the promax oblique rotation was used and for the calculation of the internal consistency, we used the compound reliability. The construct validity was determined by means of the confirmatory factorial analysis with the method of estimation of weighted least squares adjusted by the mean and variance. RESULTS The model with the best fit in the exploratory analysis was the one that retained three factors with a cumulative variance of 61.1%. The one-factor model did not obtain a good fit in both samples in the confirmatory analysis. The three-factor model called Stress-Producing Life Events presented a good fit (RMSEA < 0.05; CFI/TLI > 0.90) for both samples. CONCLUSIONS The Stress-Producing Life Events constitute a second order construct with three dimensions related to health, personal and financial aspects and violence. This study found evidence that confirms the construct validity of a list of stressor events, entitled Stress-Producing Life Events Inventory.

Psychometric properties of the Portuguese version of place attachment scale for youth in residential care.

PubMed

Magalhães, Eunice; Calheiros, María M

2015-01-01

Although the significant scientific advances on place attachment literature, no instruments exist specifically developed or adapted to residential care. 410 adolescents (11 - 18 years old) participated in this study. The place attachment scale evaluates five dimensions: Place identity, Place dependence, Institutional bonding, Caregivers bonding and Friend bonding. Data analysis included descriptive statistics, content validity, construct validity (Confirmatory Factor Analysis), concurrent validity with correlations with satisfaction with life and with institution, and reliability evidences. The relationship with individual characteristics and placement length was also verified. Content validity analysis revealed that more than half of the panellists perceive all the items as relevant to assess the construct in residential care. The structure with five dimensions revealed good fit statistics and concurrent validity evidences were found, with significant correlations with satisfaction with life and with the institution. Acceptable values of internal consistence and specific gender differences were found. The preliminary psychometric properties of this scale suggest it potential to be used with youth in care.

Physics Metacognition Inventory Part II: Confirmatory factor analysis and Rasch analysis

NASA Astrophysics Data System (ADS)

Taasoobshirazi, Gita; Bailey, MarLynn; Farley, John

2015-11-01

The Physics Metacognition Inventory was developed to measure physics students' metacognition for problem solving. In one of our earlier studies, an exploratory factor analysis provided evidence of preliminary construct validity, revealing six components of students' metacognition when solving physics problems including knowledge of cognition, planning, monitoring, evaluation, debugging, and information management. The college students' scores on the inventory were found to be reliable and related to students' physics motivation and physics grade. However, the results of the exploratory factor analysis indicated that the questionnaire could be revised to improve its construct validity. The goal of this study was to revise the questionnaire and establish its construct validity through a confirmatory factor analysis. In addition, a Rasch analysis was applied to the data to better understand the psychometric properties of the inventory and to further evaluate the construct validity. Results indicated that the final, revised inventory is a valid, reliable, and efficient tool for assessing student metacognition for physics problem solving.

Validation of the Tuebingen CD-25 Inventory as a Measure of Postoperative Health-Related Quality of Life in Patients Treated for Cushing's Disease.

PubMed

Milian, Monika; Kreitschmann-Andermahr, Ilonka; Siegel, Sonja; Kleist, Bernadette; Führer-Sakel, Dagmar; Honegger, Juergen; Buchfelder, Michael; Psaras, Tsambika

2015-01-01

To evaluate the construct and criterion validity of the Tuebingen Cushing's disease quality of life inventory (Tuebingen CD-25) for application in patients treated for Cushing's disease (CD). A total of 176 patients with adrenocorticotropin hormone-dependent CD (144 of them female, overall mean age 46.1 ± 13.7 years) treated at 3 large tertiary referral centers in Germany were studied. Construct validity was assessed by hypothesis testing (self-perceived symptom reduction assessment) and contrasted groups (patients with vs. without hypercorticolism). For this purpose, already existing data from 55 CD patients was used, representing the hypercortisolemic group. Criterion validity (concurrent validity) was assessed in relation to the Cushing's quality of life questionnaire (CushingQoL), the Short Form 36 health survey (SF-36), and the body mass index (BMI). Patients with self-perceived remarkable symptom reduction had significant lower Tuebingen CD-25 scores (i.e. better health-related quality of life) than patients with self-perceived insufficient symptom reduction (p < 0.05). Similarly, the mean scores of the Tuebingen CD-25 scales were lower in patients without hypercortisolism (total score 27.0 ± 17.2) compared to those with hypercortisolism (total score 45.3 ± 22.1; each p < 0.05), providing evidence for construct validity. Criterion validity was confirmed by the correlations between the Tuebingen CD-25 total score and the CushingQoL (Spearman's coefficient -0.733), as well as all scales of the SF-36 (Spearman's coefficient between -0.447 and -0.700). The analyses presented in this large-sample study provide robust evidence for the construct and criterion validity of the Tuebingen CD-25. © 2015 S. Karger AG, Basel.

Knee Injury and Osteoarthritis Outcome Score (KOOS): systematic review and meta-analysis of measurement properties.

PubMed

Collins, N J; Prinsen, C A C; Christensen, R; Bartels, E M; Terwee, C B; Roos, E M

2016-08-01

To conduct a systematic review and meta-analysis to synthesize evidence regarding measurement properties of the Knee injury and Osteoarthritis Outcome Score (KOOS). A comprehensive literature search identified 37 eligible papers evaluating KOOS measurement properties in participants with knee injuries and/or osteoarthritis (OA). Methodological quality was evaluated using the COSMIN checklist. Where possible, meta-analysis of extracted data was conducted for all studies and stratified by age and knee condition; otherwise narrative synthesis was performed. KOOS has adequate internal consistency, test-retest reliability and construct validity in young and old adults with knee injuries and/or OA. The ADL subscale has better content validity for older patients and Sport/Rec for younger patients with knee injuries, while the Pain subscale is more relevant for painful knee conditions. The five-factor structure of the original KOOS is unclear. There is some evidence that the KOOS subscales demonstrate sufficient unidimensionality, but this requires confirmation. Although measurement error requires further evaluation, the minimal detectable change for KOOS subscales ranges from 14.3 to 19.6 for younger individuals, and ≥20 for older individuals. Evidence of responsiveness comes from larger effect sizes following surgical (especially total knee replacement) than non-surgical interventions. KOOS demonstrates adequate content validity, internal consistency, test-retest reliability, construct validity and responsiveness for age- and condition-relevant subscales. Structural validity, cross-cultural validity and measurement error require further evaluation, as well as construct validity of KOOS Physical function Short form. Suggested order of subscales for different knee conditions can be applied in hierarchical testing of endpoints in clinical trials. PROSPERO (CRD42011001603). Copyright © 2016 Osteoarthritis Research Society International. Published by Elsevier Ltd. All rights reserved.

Reliability and preliminary evidence of validity of a Farsi version of the depression anxiety stress scales.

PubMed

Bayani, Ali Asghar

2010-08-01

The internal consistency, test-retest reliability, and construct validity of the Farsi version of the Depression Anxiety Stress Scales were examined, with a sample of 306 undergraduate students (123 men, 183 women) ranging from 18 to 51 years of age (M age = 25.4, SD = 6.1). Participants completed the Satisfaction with Life Scale, Rosenberg Self-esteem Scale, and the Depression Anxiety Stress Scales. The findings confirmed the preliminary reliabilities and preliminary construct validity of the Farsi translation of the Depression Anxiety Stress Scales.

Testing the Predictive Validity and Construct of Pathological Video Game Use

PubMed Central

Groves, Christopher L.; Gentile, Douglas; Tapscott, Ryan L.; Lynch, Paul J.

2015-01-01

Three studies assessed the construct of pathological video game use and tested its predictive validity. Replicating previous research, Study 1 produced evidence of convergent validity in 8th and 9th graders (N = 607) classified as pathological gamers. Study 2 replicated and extended the findings of Study 1 with college undergraduates (N = 504). Predictive validity was established in Study 3 by measuring cue reactivity to video games in college undergraduates (N = 254), such that pathological gamers were more emotionally reactive to and provided higher subjective appraisals of video games than non-pathological gamers and non-gamers. The three studies converged to show that pathological video game use seems similar to other addictions in its patterns of correlations with other constructs. Conceptual and definitional aspects of Internet Gaming Disorder are discussed. PMID:26694472

The Riso-Hudson Enneagram Type Indicator: Estimates of Reliability and Validity

ERIC Educational Resources Information Center

Newgent, Rebecca A.; Parr, Patricia E.; Newman, Isadore; Higgins, Kristin K.

2004-01-01

This investigation was conducted to estimate the reliability and validity of scores on the Riso-Hudson Enneagram Type Indicator (D. R. Riso & R. Hudson, 1999a). Results of 287 participants were analyzed. Alpha suggests an adequate degree of internal consistency. Evidence provides mixed support for construct validity using correlational and…

Are loss of control while eating and overeating valid constructs? A critical review of the literature

PubMed Central

Goldschmidt, Andrea B.

2017-01-01

Background Binge eating is a marker of weight gain and obesity, and a hallmark feature of eating disorders. Yet, its component constructs—overeating and loss of control (LOC) while eating—are poorly understood and difficult to measure. Objective To critically review the human literature concerning the validity of LOC and overeating across the age and weight spectrum. Data sources English-language articles addressing the face, convergent, discriminant, and predictive validity of LOC and overeating were included. Results LOC and overeating appear to have adequate face validity. Emerging evidence supports the convergent and predictive validity of the LOC construct, given its unique cross-sectional and prospective associations with numerous anthropometric, psychosocial, and eating behavior-related factors. Overeating may be best conceptualized as a marker of excess weight status. Limitations Binge eating constructs, particularly in the context of subjectively large episodes, are challenging to measure reliably. Few studies addressed overeating in the absence of LOC, thereby limiting conclusions about the validity of the overeating construct independent of LOC. Additional studies addressing the discriminant validity of both constructs are warranted. Discussion Suggestions for future weight-related research and for appropriately defining binge eating in the eating disorders diagnostic scheme are presented. PMID:28165655

The development and validation of the Incivility from Customers Scale.

PubMed

Wilson, Nicole L; Holmvall, Camilla M

2013-07-01

Scant research has examined customers as sources of workplace incivility, despite evidence suggesting that mistreatment is more common from organizational outsiders, including customers, than from organizational members (Grandey, Kern, & Frone, 2007; Schat & Kelloway, 2005). As an important step in extending the literature on customer incivility, we conducted two studies to develop and validate a measure of this construct. Study 1 used focus groups of retail and restaurant employees (n = 30) to elicit a list of uncivil customer behaviors, based on which we wrote initial scale items. Study 2 used a correlational survey design (n = 439) to pare down the number of scale items to 10 and to garner reliability and validity evidence for the scale. Exploratory and confirmatory factor analyses show that the scale is unidimensional and distinguishable from measures of the related, but distinct, constructs of interpersonal justice and psychological aggression from customers. Reliability analyses show that the scale is internally consistent. Significant correlations between the scale and individuals' job satisfaction, turnover intentions, and general and job-specific psychological strain provide evidence of criterion-related validity. Hierarchical regression analyses show that the scale significantly predicts three of four organizational and personal strain outcomes over and above a workplace incivility measure adapted for customer incivility, providing some evidence of incremental validity. Limitations and future research directions are discussed. PsycINFO Database Record (c) 2013 APA, all rights reserved.

Validation of the Narrowing Beam Walking Test in Lower Limb Prosthesis Users.

PubMed

Sawers, Andrew; Hafner, Brian

2018-04-11

To evaluate the content, construct, and discriminant validity of the Narrowing Beam Walking Test (NBWT), a performance-based balance test for lower limb prosthesis users. Cross-sectional study. Research laboratory and prosthetics clinic. Unilateral transtibial and transfemoral prosthesis users (N=40). Not applicable. Content validity was examined by quantifying the percentage of participants receiving maximum or minimum scores (ie, ceiling and floor effects). Convergent construct validity was examined using correlations between participants' NBWT scores and scores or times on existing clinical balance tests regularly administered to lower limb prosthesis users. Known-groups construct validity was examined by comparing NBWT scores between groups of participants with different fall histories, amputation levels, amputation etiologies, and functional levels. Discriminant validity was evaluated by analyzing the area under each test's receiver operating characteristic (ROC) curve. No minimum or maximum scores were recorded on the NBWT. NBWT scores demonstrated strong correlations (ρ=.70‒.85) with scores/times on performance-based balance tests (timed Up and Go test, Four Square Step Test, and Berg Balance Scale) and a moderate correlation (ρ=.49) with the self-report Activities-specific Balance Confidence scale. NBWT performance was significantly lower among participants with a history of falls (P=.003), transfemoral amputation (P=.011), and a lower mobility level (P<.001). The NBWT also had the largest area under the ROC curve (.81) and was the only test to exhibit an area that was statistically significantly >.50 (ie, chance). The results provide strong evidence of content, construct, and discriminant validity for the NBWT as a performance-based test of balance ability. The evidence supports its use to assess balance impairments and fall risk in unilateral transtibial and transfemoral prosthesis users. Copyright © 2018 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.

What Counts as Evidence of Educational Achievement? The Role of Constructs in the Pursuit of Equity in Assessment

ERIC Educational Resources Information Center

Wiliam, Dylan

2010-01-01

The idea that validity should be considered a property of inferences, rather than of assessments, has developed slowly over the past century. In early writings about the validity of educational assessments, validity was defined as a property of an assessment. The most common definition was that an assessment was valid to the extent that it…

Extension of the Rejection Sensitivity Construct to the Interpersonal Functioning of Gay Men

ERIC Educational Resources Information Center

Pachankis, John E.; Goldfried, Marvin R.; Ramrattan, Melissa E.

2008-01-01

On the basis of recent evidence suggesting that gay men are particularly likely to fear interpersonal rejection, the authors set out to extend the "rejection sensitivity" construct to the mental health concerns of gay men. After establishing a reliable and valid measure of the gay-related rejection sensitivity construct, the authors use this to…

A systematic review of the measurement properties of the European Organisation for Research and Treatment of Cancer In-patient Satisfaction with Care Questionnaire, the EORTC IN-PATSAT32.

PubMed

Neijenhuijs, Koen I; Jansen, Femke; Aaronson, Neil K; Brédart, Anne; Groenvold, Mogens; Holzner, Bernhard; Terwee, Caroline B; Cuijpers, Pim; Verdonck-de Leeuw, Irma M

2018-05-07

The EORTC IN-PATSAT32 is a patient-reported outcome measure (PROM) to assess cancer patients' satisfaction with in-patient health care. The aim of this study was to investigate whether the initial good measurement properties of the IN-PATSAT32 are confirmed in new studies. Within the scope of a larger systematic review study (Prospero ID 42017057237), a systematic search was performed of Embase, Medline, PsycINFO, and Web of Science for studies that investigated measurement properties of the IN-PATSAT32 up to July 2017. Study quality was assessed, data were extracted, and synthesized according to the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology. Nine studies were included in this review. The evidence on reliability and construct validity were rated as sufficient and of the quality of the evidence as moderate. The evidence on structural validity was rated as insufficient and of low quality. The evidence on internal consistency was indeterminate. Measurement error, responsiveness, criterion validity, and cross-cultural validity were not reported in the included studies. Measurement error could be calculated for two studies and was judged indeterminate. In summary, the IN-PATSAT32 performs as expected with respect to reliability and construct validity. No firm conclusions can be made yet whether the IN-PATSAT32 also performs as well with respect to structural validity and internal consistency. Further research on these measurement properties of the PROM is therefore needed as well as on measurement error, responsiveness, criterion validity, and cross-cultural validity. For future studies, it is recommended to take the COSMIN methodology into account.

Validity of the Mayer-Salovey-Caruso Emotional Intelligence Test: Youth Version-Research Edition

ERIC Educational Resources Information Center

Peters, Christine; Kranzler, John H.; Rossen, Eric

2009-01-01

This study examines the criterion-related validity evidence of scores on the Mayer-Salovey-Caruso Emotional Intelligence Test: Youth Version-Research Version. The authors also investigate the relationship between scores on the MSCEIT-YV and chronological age. Results provide initial support for the construct validity of the MSCEIT-YV but also…

The Prosocial and Antisocial Behaviour in Sport Scale: further evidence for construct validity and reliability.

PubMed

Kavussanu, Maria; Stanger, Nicholas; Boardley, Ian D

2013-01-01

The purpose of this research was to provide further evidence for the construct validity (i.e., convergent, concurrent, and discriminant validity) of the Prosocial and Antisocial Behaviour in Sport Scale (PABSS), an instrument that has four subscales measuring prosocial and antisocial behaviour toward teammates and opponents. We also investigated test-retest reliability and stability of the PABSS. We conducted three studies using athletes from a variety of team sports. In Study 1, participants (N = 129) completed the PABSS and measures of physical and verbal aggression, hostility, anger, moral identity, and empathy; a sub-sample (n = 111) also completed the PABSS one week later. In Study 2, in addition to the PABSS, participants (N = 89) completed measures of competitive aggressiveness and anger, moral attitudes, moral disengagement, goal orientation, and anxiety. In Study 3, participants (N = 307) completed the PABSS and a measure of social goals. Across the three studies, the four subscales evidenced the hypothesised relationships with a number of variables. Correlations were large between the two antisocial behaviours and small between the two prosocial behaviours. Overall, the findings supported the convergent, concurrent, and discriminant validity of the scale, provided evidence for its test-retest reliability and stability, and suggest that the instrument is a valid and reliable measure of prosocial and antisocial behaviour in sport.

On the Value of Homogeneous Constructs for Construct Validation, Theory Testing, and the Description of Psychopathology

PubMed Central

Smith, Gregory T.; McCarthy, Denis M.; Zapolski, Tamika C. B.

2010-01-01

The authors argue for a significant shift in how clinical psychology researchers conduct construct validation and theory validation tests. They argue that sound theory and validation tests can best be conducted on measures of unidimensional or homogeneous constructs. Hierarchical organizations of such constructs are useful descriptively and theoretically, but higher order composites do not refer to definable psychological processes. Application of this perspective to the approach of the Diagnostic and Statistical Manual of Mental Disorders to describing psychopathology calls into doubt the traditional use of the syndromal approach, in which single scores reflect the presence of multidimensional disorders. For many forms of psychological dysfunction, this approach does not appear optimal and may need to be discarded. The authors note that their perspective represents a straightforward application of existing psychometric theory, they demonstrate the practical value of adopting this perspective, and they provide evidence that this shift is already under way among clinical researchers. Description in terms of homogeneous dimensions provides improved validity, utility, and parsimony. In contrast, the use of composite diagnoses can retard scientific progress and hamper clinicians' efforts to understand and treat dysfunction. PMID:19719340

[Construct validity of the Medical Outcomes Study's social support scale adapted to Portuguese in the Pró-Saúde Study].

PubMed

Griep, Rosane Harter; Chor, Dóra; Faerstein, Eduardo; Werneck, Guilherme L; Lopes, Cláudia S

2005-01-01

This paper evaluates the construct validity of the Medical Outcomes Study's social support scale adapted to Portuguese, when utilized in a cohort study among non-faculty civil servants at a university in Rio de Janeiro, Brazil (Pró-Saúde Study). Baseline data were obtained in 1999, when 4,030 participants (92.0% of those eligible) completed a multidimensional self-administered questionnaire at the workplace. From the original scale's five social support dimensions, factor analysis of the data extracted only three dimensions: positive social interaction/affective support; emotional/information support; and material support. We estimated associations between social support dimensions and socio-demographic, health, and well being-related characteristics. We confirmed the hypotheses that less isolated individuals, those with better self-rated health, those who reported more participation in group activities, and those with no evidence of common mental disorders reported better perception of social support. In conclusion, we found good evidence for a high construct validity of this scale, supporting its use in future analyses in the Pró-Saúde Study and in similar population groups.

Factor analysis methods and validity evidence: A systematic review of instrument development across the continuum of medical education

NASA Astrophysics Data System (ADS)

Wetzel, Angela Payne

Previous systematic reviews indicate a lack of reporting of reliability and validity evidence in subsets of the medical education literature. Psychology and general education reviews of factor analysis also indicate gaps between current and best practices; yet, a comprehensive review of exploratory factor analysis in instrument development across the continuum of medical education had not been previously identified. Therefore, the purpose for this study was critical review of instrument development articles employing exploratory factor or principal component analysis published in medical education (2006--2010) to describe and assess the reporting of methods and validity evidence based on the Standards for Educational and Psychological Testing and factor analysis best practices. Data extraction of 64 articles measuring a variety of constructs that have been published throughout the peer-reviewed medical education literature indicate significant errors in the translation of exploratory factor analysis best practices to current practice. Further, techniques for establishing validity evidence tend to derive from a limited scope of methods including reliability statistics to support internal structure and support for test content. Instruments reviewed for this study lacked supporting evidence based on relationships with other variables and response process, and evidence based on consequences of testing was not evident. Findings suggest a need for further professional development within the medical education researcher community related to (1) appropriate factor analysis methodology and reporting and (2) the importance of pursuing multiple sources of reliability and validity evidence to construct a well-supported argument for the inferences made from the instrument. Medical education researchers and educators should be cautious in adopting instruments from the literature and carefully review available evidence. Finally, editors and reviewers are encouraged to recognize this gap in best practices and subsequently to promote instrument development research that is more consistent through the peer-review process.

Reliability and Validity of Ambulatory Cognitive Assessments

PubMed Central

Sliwinski, Martin J.; Mogle, Jacqueline A.; Hyun, Jinshil; Munoz, Elizabeth; Smyth, Joshua M.; Lipton, Richard B.

2017-01-01

Mobile technologies are increasingly used to measure cognitive function outside of traditional clinic and laboratory settings. Although ambulatory assessments of cognitive function conducted in people’s natural environments offer potential advantages over traditional assessment approaches, the psychometrics of cognitive assessment procedures have been understudied. We evaluated the reliability and construct validity of ambulatory assessments of working memory and perceptual speed administered via smartphones as part of an ecological momentary assessment (EMA) protocol in a diverse adult sample (N=219). Results indicated excellent between-person reliability (≥.97) for average scores, and evidence of reliable within-person variability across measurement occasions (.41–.53). The ambulatory tasks also exhibited construct validity, as evidence by their loadings on working memory and perceptual speed factors defined by the in-lab assessments. Our findings demonstrate that averaging across brief cognitive assessments made in uncontrolled naturalistic settings provide measurements that are comparable in reliability to assessments made in controlled laboratory environments. PMID:27084835

Validation of educational assessments: a primer for simulation and beyond.

PubMed

Cook, David A; Hatala, Rose

2016-01-01

Simulation plays a vital role in health professions assessment. This review provides a primer on assessment validation for educators and education researchers. We focus on simulation-based assessment of health professionals, but the principles apply broadly to other assessment approaches and topics. Validation refers to the process of collecting validity evidence to evaluate the appropriateness of the interpretations, uses, and decisions based on assessment results. Contemporary frameworks view validity as a hypothesis, and validity evidence is collected to support or refute the validity hypothesis (i.e., that the proposed interpretations and decisions are defensible). In validation, the educator or researcher defines the proposed interpretations and decisions, identifies and prioritizes the most questionable assumptions in making these interpretations and decisions (the "interpretation-use argument"), empirically tests those assumptions using existing or newly-collected evidence, and then summarizes the evidence as a coherent "validity argument." A framework proposed by Messick identifies potential evidence sources: content, response process, internal structure, relationships with other variables, and consequences. Another framework proposed by Kane identifies key inferences in generating useful interpretations: scoring, generalization, extrapolation, and implications/decision. We propose an eight-step approach to validation that applies to either framework: Define the construct and proposed interpretation, make explicit the intended decision(s), define the interpretation-use argument and prioritize needed validity evidence, identify candidate instruments and/or create/adapt a new instrument, appraise existing evidence and collect new evidence as needed, keep track of practical issues, formulate the validity argument, and make a judgment: does the evidence support the intended use? Rigorous validation first prioritizes and then empirically evaluates key assumptions in the interpretation and use of assessment scores. Validation science would be improved by more explicit articulation and prioritization of the interpretation-use argument, greater use of formal validation frameworks, and more evidence informing the consequences and implications of assessment.

Using Scratch with Primary School Children: An Evaluation of Games Constructed to Gauge Understanding of Programming Concepts

ERIC Educational Resources Information Center

Wilson, Amanda; Hainey, Thomas; Connolly, Thomas M.

2013-01-01

Newer approaches such as games-based learning (GBL) and games-based construction are being adopted to motivate and engage students within the Curriculum for Excellence (CfE) in Scotland. GBL and games-based construction suffer from a dearth of empirical evidence supporting their validity as teaching and learning approaches. To address this issue…

Evidence Regarding the Internal Structure: Confirmatory Factor Analysis

ERIC Educational Resources Information Center

Lewis, Todd F.

2017-01-01

American Educational Research Association (AERA) standards stipulate that researchers show evidence of the internal structure of instruments. Confirmatory factor analysis (CFA) is one structural equation modeling procedure designed to assess construct validity of assessments that has broad applicability for counselors interested in instrument…

Construct validity evidence for the Male Role Norms Inventory-Short Form: A structural equation modeling approach using the bifactor model.

PubMed

Levant, Ronald F; Hall, Rosalie J; Weigold, Ingrid K; McCurdy, Eric R

2016-10-01

The construct validity of the Male Role Norms Inventory-Short Form (MRNI-SF) was assessed using a latent variable approach implemented with structural equation modeling (SEM). The MRNI-SF was specified as having a bifactor structure, and validation scales were also specified as latent variables. The latent variable approach had the advantages of separating effects of general and specific factors and controlling for some sources of measurement error. Data (N = 484) were from a diverse sample (38.8% men of color, 22.3% men of diverse sexualities) of community-dwelling and college men who responded to an online survey. The construct validity of the MRNI-SF General Traditional Masculinity Ideology factor was supported for all 4 of the proposed latent correlations with: (a) Male Role Attitudes Scale; (b) general factor of Conformity to Masculine Norms Inventory-46; (c) higher-order factor of Gender Role Conflict Scale; and (d) Personal Attributes Questionnaire-Masculinity Scale. Significant correlations with relevant other latent factors provided concurrent validity evidence for the MRNI-SF specific factors of Negativity toward Sexual Minorities, Importance of Sex, Restrictive Emotionality, and Toughness, with all 8 of the hypothesized relationships supported. However, 3 relationships concerning Dominance were not supported. (The construct validity of the remaining 2 MRNI-SF specific factors-Avoidance of Femininity and Self-Reliance through Mechanical Skills was not assessed.) Comparisons were made, and meaningful differences noted, between the latent correlations emphasized in this study and their raw variable counterparts. Results are discussed in terms of the advantages of an SEM approach and the unique characteristics of the bifactor model. (PsycINFO Database Record (c) 2016 APA, all rights reserved).

Development and Validation of Scores from an Instrument Measuring Student Test-Taking Motivation

ERIC Educational Resources Information Center

Eklof, Hanna

2006-01-01

Using the expectancy-value model of achievement motivation as a basis, this study's purpose is to develop, apply, and validate scores from a self-report instrument measuring student test-taking motivation. Sampled evidence of construct validity for the present sample indicates that a number of the items in the instrument could be used as an…

Construction and validation of clinical contents for development of learning objects.

PubMed

Hortense, Flávia Tatiana Pedrolo; Bergerot, Cristiane Decat; Domenico, Edvane Birelo Lopes de

2018-01-01

to describe the process of construction and validation of clinical contents for health learning objects, aimed at patients in the treatment of head and neck cancer. descriptive, methodological study. The development of the script and the storyboard were based on scientific evidence and submitted to the appreciation of specialists for validation of content. The agreement index was checked quantitatively and the suggestions were qualitatively evaluated. The items described in the roadmap were approved by 99% of expert experts. The suggestions for adjustments were inserted in their entirety in the final version. The free-marginal kappa statistical test, for multiple evaluators, presented value equal to 0.68%, granting a substantial agreement. The steps taken in the construction and validation of the content for the production of educational material for patients with head and neck cancer were adequate, relevant and suitable for use in other subjects.

Validity threats: overcoming interference with proposed interpretations of assessment data.

PubMed

Downing, Steven M; Haladyna, Thomas M

2004-03-01

Factors that interfere with the ability to interpret assessment scores or ratings in the proposed manner threaten validity. To be interpreted in a meaningful manner, all assessments in medical education require sound, scientific evidence of validity. The purpose of this essay is to discuss 2 major threats to validity: construct under-representation (CU) and construct-irrelevant variance (CIV). Examples of each type of threat for written, performance and clinical performance examinations are provided. The CU threat to validity refers to undersampling the content domain. Using too few items, cases or clinical performance observations to adequately generalise to the domain represents CU. Variables that systematically (rather than randomly) interfere with the ability to meaningfully interpret scores or ratings represent CIV. Issues such as flawed test items written at inappropriate reading levels or statistically biased questions represent CIV in written tests. For performance examinations, such as standardised patient examinations, flawed cases or cases that are too difficult for student ability contribute CIV to the assessment. For clinical performance data, systematic rater error, such as halo or central tendency error, represents CIV. The term face validity is rejected as representative of any type of legitimate validity evidence, although the fact that the appearance of the assessment may be an important characteristic other than validity is acknowledged. There are multiple threats to validity in all types of assessment in medical education. Methods to eliminate or control validity threats are suggested.

How Students Create Motivationally Supportive Learning Environments for Themselves: The Concept of Agentic Engagement

ERIC Educational Resources Information Center

Reeve, Johnmarshall

2013-01-01

The present study introduced "agentic engagement" as a newly proposed student-initiated pathway to greater achievement and greater motivational support. Study 1 developed the brief, construct-congruent, and psychometrically strong Agentic Engagement Scale. Study 2 provided evidence for the scale's construct and predictive validity, as…

Latency-Based and Psychophysiological Measures of Sexual Interest Show Convergent and Concurrent Validity.

PubMed

Ó Ciardha, Caoilte; Attard-Johnson, Janice; Bindemann, Markus

2018-04-01

Latency-based measures of sexual interest require additional evidence of validity, as do newer pupil dilation approaches. A total of 102 community men completed six latency-based measures of sexual interest. Pupillary responses were recorded during three of these tasks and in an additional task where no participant response was required. For adult stimuli, there was a high degree of intercorrelation between measures, suggesting that tasks may be measuring the same underlying construct (convergent validity). In addition to being correlated with one another, measures also predicted participants' self-reported sexual interest, demonstrating concurrent validity (i.e., the ability of a task to predict a more validated, simultaneously recorded, measure). Latency-based and pupillometric approaches also showed preliminary evidence of concurrent validity in predicting both self-reported interest in child molestation and viewing pornographic material containing children. Taken together, the study findings build on the evidence base for the validity of latency-based and pupillometric measures of sexual interest.

Reliability and Validity of Scores on the IFSP Rating Scale

ERIC Educational Resources Information Center

Jung, Lee Ann; McWilliam, R. A.

2005-01-01

Evidence is presented regarding the construct validity and internal consistency reliability of scores for an investigator-developed individualized family service plan (IFSP) rating scale. One hundred and twenty IFSPs were rated using a 12-item instrument, the IFSP Rating Scale (McWilliam & Jung, 2001). Using principal components factor…

The Psychopathy Q-Sort. Construct Validity Evidence in a Nonclinical Sample

ERIC Educational Resources Information Center

Fowler, Katherine A.; Lilienfeld, Scott O.

2007-01-01

Scant research has examined the validity of instruments that permit observer ratings of psychopathy. Using a nonclinical (undergraduate) sample, the authors examined the associations between both self- and observer ratings on a psychopathy prototype (Psychopathy Q-Sort, PQS) and widely used measures of psychopathy, antisocial behavior, and…

Trait-specific dependence in romantic relationships.

PubMed

Ellis, Bruce J; Simpson, Jeffry A; Campbell, Lorne

2002-10-01

Informed by three theoretical frameworks--trait psychology, evolutionary psychology, and interdependence theory--we report four investigations designed to develop and test the reliability and validity of a new construct and accompanying multiscale inventory, the Trait-Specific Dependence Inventory (TSDI). The TSDI assesses comparisons between present and alternative romantic partners on major dimensions of mate value. In Study 1, principal components analyses revealed that the provisional pool of theory-generated TSDI items were represented by six factors: Agreeable/Committed, Resource Accruing Potential, Physical Prowess, Emotional Stability, Surgency, and Physical Attractiveness. In Study 2, confirmatory factor analysis replicated these results on a different sample and tested how well different structural models fit the data. Study 3 provided evidence for the convergent and discriminant validity of the six TSDI scales by correlating each one with a matched personality trait scale that did not explicitly incorporate comparisons between partners. Study 4 provided further validation evidence, revealing that the six TSDI scales successfully predicted three relationship outcome measures--love, time investment, and anger/upset--above and beyond matched sets of traditional personality trait measures. These results suggest that the TSDI is a reliable, valid, and unique construct that represents a new trait-specific method of assessing dependence in romantic relationships. The construct of trait-specific dependence is introduced and linked with other theories of mate value.

Construction and Initial Validation of the Multiracial Experiences Measure (MEM)

PubMed Central

Yoo, Hyung Chol; Jackson, Kelly; Guevarra, Rudy P.; Miller, Matthew J.; Harrington, Blair

2015-01-01

This article describes the development and validation of the Multiracial Experiences Measure (MEM): a new measure that assesses uniquely racialized risks and resiliencies experienced by individuals of mixed racial heritage. Across two studies, there was evidence for the validation of the 25-item MEM with 5 subscales including Shifting Expressions, Perceived Racial Ambiguity, Creating Third Space, Multicultural Engagement, and Multiracial Discrimination. The 5-subscale structure of the MEM was supported by a combination of exploratory and confirmatory factor analyses. Evidence of criterion-related validity was partially supported with MEM subscales correlating with measures of racial diversity in one’s social network, color-blind racial attitude, psychological distress, and identity conflict. Evidence of discriminant validity was supported with MEM subscales not correlating with impression management. Implications for future research and suggestions for utilization of the MEM in clinical practice with multiracial adults are discussed. PMID:26460977

Construction and initial validation of the Multiracial Experiences Measure (MEM).

PubMed

Yoo, Hyung Chol; Jackson, Kelly F; Guevarra, Rudy P; Miller, Matthew J; Harrington, Blair

2016-03-01

This article describes the development and validation of the Multiracial Experiences Measure (MEM): a new measure that assesses uniquely racialized risks and resiliencies experienced by individuals of mixed racial heritage. Across 2 studies, there was evidence for the validation of the 25-item MEM with 5 subscales including Shifting Expressions, Perceived Racial Ambiguity, Creating Third Space, Multicultural Engagement, and Multiracial Discrimination. The 5-subscale structure of the MEM was supported by a combination of exploratory and confirmatory factor analyses. Evidence of criterion-related validity was partially supported with MEM subscales correlating with measures of racial diversity in one's social network, color-blind racial attitude, psychological distress, and identity conflict. Evidence of discriminant validity was supported with MEM subscales not correlating with impression management. Implications for future research and suggestions for utilization of the MEM in clinical practice with multiracial adults are discussed. (c) 2016 APA, all rights reserved).

Psychometric properties of the Late-Life Function and Disability Instrument: a systematic review

PubMed Central

2014-01-01

Background The choice of measure for use as a primary outcome in geriatric research is contingent upon the construct of interest and evidence for its psychometric properties. The Late-Life Function and Disability Instrument (LLFDI) has been widely used to assess functional limitations and disability in studies with older adults. The primary aim of this systematic review was to evaluate the current available evidence for the psychometric properties of the LLFDI. Methods Published studies of any design reporting results based on administration of the original version of the LLFDI in community-dwelling older adults were identified after searches of 9 electronic databases. Data related to construct validity (convergent/divergent and known-groups validity), test-retest reliability and sensitivity to change were extracted. Effect sizes were calculated for within-group changes and summarized graphically. Results Seventy-one studies including 17,301 older adults met inclusion criteria. Data supporting the convergent/divergent and known-groups validity for both the Function and Disability components were extracted from 30 and 18 studies, respectively. High test-retest reliability was found for the Function component, while results for the Disability component were more variable. Sensitivity to change of the LLFDI was confirmed based on findings from 25 studies. The basic lower extremity subscale and overall summary score of the Function component and limitation dimension of the Disability component were associated with the strongest relative effect sizes. Conclusions There is extensive evidence to support the construct validity and sensitivity to change of the LLFDI among various clinical populations of community-dwelling older adults. Further work is needed on predictive validity and values for clinically important change. Findings from this review can be used to guide the selection of the most appropriate LLFDI subscale for use an outcome measure in geriatric research and practice. PMID:24476510

Reliability and validity of advanced theory-of-mind measures in middle childhood and adolescence.

PubMed

Hayward, Elizabeth O; Homer, Bruce D

2017-09-01

Although theory-of-mind (ToM) development is well documented for early childhood, there is increasing research investigating changes in ToM reasoning in middle childhood and adolescence. However, the psychometric properties of most advanced ToM measures for use with older children and adolescents have not been firmly established. We report on the reliability and validity of widely used, conventional measures of advanced ToM with this age group. Notable issues with both reliability and validity of several of the measures were evident in the findings. With regard to construct validity, results do not reveal a clear empirical commonality between tasks, and, after accounting for comprehension, developmental trends were evident in only one of the tasks investigated. Statement of contribution What is already known on this subject? Second-order false belief tasks have acceptable internal consistency. The Eyes Test has poor internal consistency. Validity of advanced theory-of-mind tasks is often based on the ability to distinguish clinical from typical groups. What does this study add? This study examines internal consistency across six widely used advanced theory-of-mind tasks. It investigates validity of tasks based on comprehension of items by typically developing individuals. It further assesses construct validity, or commonality between tasks. © 2017 The British Psychological Society.

Valid and Reliable Science Content Assessments for Science Teachers

NASA Astrophysics Data System (ADS)

Tretter, Thomas R.; Brown, Sherri L.; Bush, William S.; Saderholm, Jon C.; Holmes, Vicki-Lynn

2013-03-01

Science teachers' content knowledge is an important influence on student learning, highlighting an ongoing need for programs, and assessments of those programs, designed to support teacher learning of science. Valid and reliable assessments of teacher science knowledge are needed for direct measurement of this crucial variable. This paper describes multiple sources of validity and reliability (Cronbach's alpha greater than 0.8) evidence for physical, life, and earth/space science assessments—part of the Diagnostic Teacher Assessments of Mathematics and Science (DTAMS) project. Validity was strengthened by systematic synthesis of relevant documents, extensive use of external reviewers, and field tests with 900 teachers during assessment development process. Subsequent results from 4,400 teachers, analyzed with Rasch IRT modeling techniques, offer construct and concurrent validity evidence.

Pathological video-gaming among Singaporean youth.

PubMed

Choo, Hyekyung; Gentile, Douglas A; Sim, Timothy; Li, Dongdong; Khoo, Angeline; Liau, Albert K

2010-11-01

Increase in internet use and video-gaming contributes to public concern on pathological or obsessive play of video games among children and adolescents worldwide. Nevertheless, little is known about the prevalence of pathological symptoms in video-gaming among Singaporean youth and the psychometric properties of instruments measuring pathological symptoms in video-gaming. A total of 2998 children and adolescents from 6 primary and 6 secondary schools in Singapore responded to a comprehensive survey questionnaire on sociodemographic characteristics, video-gaming habits, school performance, somatic symptoms, various psychological traits, social functioning and pathological symptoms of video-gaming. After weighting, the survey data were analysed to determine the prevalence of pathological video-gaming among Singaporean youth and gender differences in the prevalence. The construct validity of instrument used to measure pathological symptoms of video-gaming was tested. Of all the study participants, 8.7% were classified as pathological players with more boys reporting more pathological symptoms than girls. All variables, including impulse control problem, social competence, hostility, academic performance, and damages to social functioning, tested for construct validity, were significantly associated with pathological status, providing good evidence for the construct validity of the instrument used. The prevalence rate of pathological video-gaming among Singaporean youth is comparable with that from other countries studied thus far, and gender differences are also consistent with the findings of prior research. The positive evidence of construct validity supports the potential use of the instrument for future research and clinical screening on Singapore children and adolescents' pathological video-gaming.

Assessing impact of physical activity-based youth development programs: validation of the Life Skills Transfer Survey (LSTS).

PubMed

Weiss, Maureen R; Bolter, Nicole D; Kipp, Lindsay E

2014-09-01

A signature characteristic of positive youth development (PYD) programs is the opportunity to develop life skills, such as social, behavioral, and moral competencies, that can be generalized to domains beyond the immediate activity. Although context-specific instruments are available to assess developmental outcomes, a measure of life skills transfer would enable evaluation of PYD programs in successfully teaching skills that youth report using in other domains. The purpose of our studies was to develop and validate a measure of perceived life skills transfer, based on data collected with The First Tee, a physical activity-based PYD program. In 3 studies, we conducted a series of steps to provide content and construct validity and internal consistency reliability for the Life Skills Transfer Survey (LSTS), a measure of perceived life skills transfer. Study 1 provided content validity for the LSTS that included 8 life skills and 50 items. Study 2 revealed construct validity (structural validity) through a confirmatory factor analysis and convergent validity by correlating scores on the LSTS with scores on an assessment tool that measures a related construct. Study 3 offered additional construct validity by reassessing youth 1 year later and showing that scores during both time periods were invariant in factor pattern, loadings, and variances and covariances. Studies 2 and 3 demonstrated internal consistency reliability of the LSTS. RESULTS from 3 studies provide evidence of content and construct validity and internal consistency reliability for the LSTS, which can be used in evaluation research with youth development programs.

Systematic Review of Methods in Low-Consensus Fields: Supporting Commensuration through `Construct-Centered Methods Aggregation' in the Case of Climate Change Vulnerability Research.

PubMed

Delaney, Aogán; Tamás, Peter A; Crane, Todd A; Chesterman, Sabrina

2016-01-01

There is increasing interest in using systematic review to synthesize evidence on the social and environmental effects of and adaptations to climate change. Use of systematic review for evidence in this field is complicated by the heterogeneity of methods used and by uneven reporting. In order to facilitate synthesis of results and design of subsequent research a method, construct-centered methods aggregation, was designed to 1) provide a transparent, valid and reliable description of research methods, 2) support comparability of primary studies and 3) contribute to a shared empirical basis for improving research practice. Rather than taking research reports at face value, research designs are reviewed through inductive analysis. This involves bottom-up identification of constructs, definitions and operationalizations; assessment of concepts' commensurability through comparison of definitions; identification of theoretical frameworks through patterns of construct use; and integration of transparently reported and valid operationalizations into ideal-type research frameworks. Through the integration of reliable bottom-up inductive coding from operationalizations and top-down coding driven from stated theory with expert interpretation, construct-centered methods aggregation enabled both resolution of heterogeneity within identically named constructs and merging of differently labeled but identical constructs. These two processes allowed transparent, rigorous and contextually sensitive synthesis of the research presented in an uneven set of reports undertaken in a heterogenous field. If adopted more broadly, construct-centered methods aggregation may contribute to the emergence of a valid, empirically-grounded description of methods used in primary research. These descriptions may function as a set of expectations that improves the transparency of reporting and as an evolving comprehensive framework that supports both interpretation of existing and design of future research.

Systematic Review of Methods in Low-Consensus Fields: Supporting Commensuration through `Construct-Centered Methods Aggregation’ in the Case of Climate Change Vulnerability Research

PubMed Central

Crane, Todd A.; Chesterman, Sabrina

2016-01-01

There is increasing interest in using systematic review to synthesize evidence on the social and environmental effects of and adaptations to climate change. Use of systematic review for evidence in this field is complicated by the heterogeneity of methods used and by uneven reporting. In order to facilitate synthesis of results and design of subsequent research a method, construct-centered methods aggregation, was designed to 1) provide a transparent, valid and reliable description of research methods, 2) support comparability of primary studies and 3) contribute to a shared empirical basis for improving research practice. Rather than taking research reports at face value, research designs are reviewed through inductive analysis. This involves bottom-up identification of constructs, definitions and operationalizations; assessment of concepts’ commensurability through comparison of definitions; identification of theoretical frameworks through patterns of construct use; and integration of transparently reported and valid operationalizations into ideal-type research frameworks. Through the integration of reliable bottom-up inductive coding from operationalizations and top-down coding driven from stated theory with expert interpretation, construct-centered methods aggregation enabled both resolution of heterogeneity within identically named constructs and merging of differently labeled but identical constructs. These two processes allowed transparent, rigorous and contextually sensitive synthesis of the research presented in an uneven set of reports undertaken in a heterogenous field. If adopted more broadly, construct-centered methods aggregation may contribute to the emergence of a valid, empirically-grounded description of methods used in primary research. These descriptions may function as a set of expectations that improves the transparency of reporting and as an evolving comprehensive framework that supports both interpretation of existing and design of future research. PMID:26901409

Rater reliability and construct validity of a mobile application for posture analysis

PubMed Central

Szucs, Kimberly A.; Brown, Elena V. Donoso

2018-01-01

[Purpose] Measurement of posture is important for those with a clinical diagnosis as well as researchers aiming to understand the impact of faulty postures on the development of musculoskeletal disorders. A reliable, cost-effective and low tech posture measure may be beneficial for research and clinical applications. The purpose of this study was to determine rater reliability and construct validity of a posture screening mobile application in healthy young adults. [Subjects and Methods] Pictures of subjects were taken in three standing positions. Two raters independently digitized the static standing posture image twice. The app calculated posture variables, including sagittal and coronal plane translations and angulations. Intra- and inter-rater reliability were calculated using the appropriate ICC models for complete agreement. Construct validity was determined through comparison of known groups using repeated measures ANOVA. [Results] Intra-rater reliability ranged from 0.71 to 0.99. Inter-rater reliability was good to excellent for all translations. ICCs were stronger for translations versus angulations. The construct validity analysis found that the app was able to detect the change in the four variables selected. [Conclusion] The posture mobile application has demonstrated strong rater reliability and preliminary evidence of construct validity. This application may have utility in clinical and research settings. PMID:29410561

Rater reliability and construct validity of a mobile application for posture analysis.

PubMed

Szucs, Kimberly A; Brown, Elena V Donoso

2018-01-01

[Purpose] Measurement of posture is important for those with a clinical diagnosis as well as researchers aiming to understand the impact of faulty postures on the development of musculoskeletal disorders. A reliable, cost-effective and low tech posture measure may be beneficial for research and clinical applications. The purpose of this study was to determine rater reliability and construct validity of a posture screening mobile application in healthy young adults. [Subjects and Methods] Pictures of subjects were taken in three standing positions. Two raters independently digitized the static standing posture image twice. The app calculated posture variables, including sagittal and coronal plane translations and angulations. Intra- and inter-rater reliability were calculated using the appropriate ICC models for complete agreement. Construct validity was determined through comparison of known groups using repeated measures ANOVA. [Results] Intra-rater reliability ranged from 0.71 to 0.99. Inter-rater reliability was good to excellent for all translations. ICCs were stronger for translations versus angulations. The construct validity analysis found that the app was able to detect the change in the four variables selected. [Conclusion] The posture mobile application has demonstrated strong rater reliability and preliminary evidence of construct validity. This application may have utility in clinical and research settings.

Validity of the Consensual Assessment Technique--Evidence with Three Groups of Judges and an Elementary School Student Sample

ERIC Educational Resources Information Center

Long, Haiying

2012-01-01

As one of the most widely used creativity assessment tools, the Consensual Assessment Technique (CAT) has been praised as a valid tool to assess creativity. In Amabile's (1982) seminal work, the inter-rater reliability was defined as construct validity of the CAT. During the past three decades, researchers followed this definition and…

Construct, concurrent and discriminant validity of Type D personality in the general population: associations with anxiety, depression, stress and cardiac output.

PubMed

Howard, Siobhán; Hughes, Brian M

2012-01-01

The Type D personality, identified by high negative affectivity paired with high social inhibition, has been associated with a number of health-related outcomes in (mainly) cardiac populations. However, despite its prevalence in the health-related literature, how this personality construct fits within existing personality theory has not been directly tested. Using a sample of 134 healthy university students, this study examined the Type D personality in terms of two well-established personality traits; introversion and neuroticism. Construct, concurrent and discriminant validity of this personality type was established through examination of the associations between the Type D personality and psychometrically assessed anxiety, depression and stress, as well as measurement of resting cardiovascular function. Results showed that while the Type D personality was easily represented using alternative measures of both introversion and neuroticism, associations with anxiety, depression and stress were mainly accounted for by neuroticism. Conversely, however, associations with resting cardiac output were attributable to the negative affectivity-social inhibition synergy, explicit within the Type D construct. Consequently, both the construct and concurrent validity of this personality type were confirmed, with discriminant validity evident on examination of physiological indices of well-being.

An Examination of Construct Validity for the EARLI Numeracy Skill Measures

ERIC Educational Resources Information Center

Cheng, Weiyi; Lei, Pui-Wa; DiPerna, James C.

2017-01-01

The purpose of the current study was to examine dimensionality and concurrent validity evidence of the EARLI numeracy measures (DiPerna, Morgan, & Lei, 2007), which were developed to assess key skills such as number identification, counting, and basic arithmetic. Two methods (NOHARM with approximate chi-square test and DIMTEST with DETECT…

Validating Grammaticality Judgment Tests: Evidence from Two New Psycholinguistic Measures

ERIC Educational Resources Information Center

Vafaee, Payman; Suzuki, Yuichi; Kachisnke, Ilina

2017-01-01

Several previous factor-analytic studies on the construct validity of grammaticality judgment tests (GJTs) concluded that untimed GJTs measure explicit knowledge (EK) and timed GJTs measure implicit knowledge (IK) (Bowles, 2011; R. Ellis, 2005; R. Ellis & Loewen, 2007). It has also been shown that, irrespective of the time condition chosen,…

Reliability and Validity of Finger Strength and Endurance Measurements in Rock Climbing

ERIC Educational Resources Information Center

Michailov, Michail Lubomirov; Baláš, Jirí; Tanev, Stoyan Kolev; Andonov, Hristo Stoyanov; Kodejška, Jan; Brown, Lee

2018-01-01

Purpose: An advanced system for the assessment of climbing-specific performance was developed and used to: (a) investigate the effect of arm fixation (AF) on construct validity evidence and reliability of climbing-specific finger-strength measurement; (b) assess reliability of finger-strength and endurance measurements; and (c) evaluate the…

[Job stress and quality of life of primary care health-workers: evidence of validity of the PECVEC questionnaire].

PubMed

Fernández-López, Juan Antonio; Fernández-Fidalgo, María; Martín-Payo, Rubén; Rödel, Andreas

2007-08-01

To evaluate the relationship between Health-Related Quality of Life (HRQL) and stress at work among Primary Care workers, as evidence of the construct validity of the Spanish version (PECVEC) of the profile of quality of life in the chronically ill (PLC) questionnaire. In addition, to check its other psychometric properties. Cross-sectional study. Eighteen primary care centres in Health Area IV, Asturias (Oviedo), Spain, sharing similar socio-demographic conditions. Two hundred and thirty-three primary care nurses and physicians. HRQL was evaluated by the 6 general dimensions of the Spanish version of the PLC. Stress at work was evaluated by the three scales of the Effort-Reward Imbalance (ERI) questionnaire. The construct validity of the PECVEC was assessed by testing the inverse associations of QoL dimensions and job stress ones, when the most important confuser variables were monitored. The non-response rate was low (<3%), and no floor effects and only small ceiling effects were observed. Internal consistency analysis and exploratory and confirmatory factor analysis demonstrated high reliability, factorial validity and convergent/divergent validity of the PECVEC. The PECVEC demonstrates adequate psychometric properties for evaluating HRQL in healthy subjects.

Exploring the reliability and validity of the social-moral awareness test.

PubMed

Livesey, Alexandra; Dodd, Karen; Pote, Helen; Marlow, Elizabeth

2012-11-01

The aim of the study was to explore the validity of the social-moral awareness test (SMAT) a measure designed for assessing socio-moral rule knowledge and reasoning in people with learning disabilities. Comparisons between Theory of Mind and socio-moral reasoning allowed the exploration of construct validity of the tool. Factor structure, reliability and discriminant validity were also assessed. Seventy-one participants with mild-moderate learning disabilities completed the two scales of the SMAT and two False Belief Tasks for Theory of Mind. Reliability of the SMAT was very good, and the scales were shown to be uni-dimensional in factor structure. There was a significant positive relationship between Theory of Mind and both SMAT scales. There is early evidence of the construct validity and reliability of the SMAT. Further assessment of the validity of the SMAT will be required. © 2012 Blackwell Publishing Ltd.

Assessing Knowledge Structures.

ERIC Educational Resources Information Center

Yacci, Michael

This paper presents two general approaches to the assessment of knowledge structures, the first of which entails the building of empirical evidence to support cognitive theory. This type of assessment is concerned with attempting to prove the existence of various knowledge structures; that is, evidence that leads to the construct validity of these…

Dimensionality and construct validity of an instrument designed to measure the metacognitive orientation of science classroom learning environments.

PubMed

Thomas, Gregory P

2004-01-01

The purpose of this study was to establish the factorial construct validity and dimensionality of the Metacognitive Orientation Learning Environment Scale-Science (MOLES-S) which was designed to measure the metacognitive orientation of science classroom learning environments. The metacognitive orientation of a science classroom learning environment is the extent to which psychosocial conditions that are known to enhance students' metacognition are evident within that classroom. The development of items comprising this scale was based on a theoretical understanding of metacognition, learning environments and the development of previous learning environments instruments. Four possible hypothesized structure models, each consistent with the literature, were reviewed and their merits were compared on the basis of empirical data drawn from two populations of 1026 and 1223 Hong Kong secondary school students using confirmatory factor analysis procedures. The scale was calibrated using the Rasch rating scale model using data from the 1223 student sample. The results suggest that there is strong evidence to support the factorial construct validity of the MOLES-S but that, on the basis of the Rasch analysis, there are still suggestions for further refinement and improvement of the MOLES-S.

A test of the validity of the motivational interviewing treatment integrity code.

PubMed

Forsberg, Lars; Berman, Anne H; Kallmén, Håkan; Hermansson, Ulric; Helgason, Asgeir R

2008-01-01

To evaluate the Swedish version of the Motivational Interviewing Treatment Code (MITI), MITI coding was applied to tape-recorded counseling sessions. Construct validity was assessed using factor analysis on 120 MITI-coded sessions. Discriminant validity was assessed by comparing MITI coding of motivational interviewing (MI) sessions with information- and advice-giving sessions as well as by comparing MI-trained practitioners with untrained practitioners. A principal-axis factoring analysis yielded some evidence for MITI construct validity. MITI differentiated between practitioners with different levels of MI training as well as between MI practitioners and advice-giving counselors, thus supporting discriminant validity. MITI may be used as a training tool together with supervision to confirm and enhance MI practice in clinical settings. MITI can also serve as a tool for evaluating MI integrity in clinical research.

Validation of persuasive messages for the promotion of physical activity among people with coronary heart disease.

PubMed

Mendez, Roberto Della Rosa; Rodrigues, Roberta Cunha Matheus; Spana, Thaís Moreira; Cornélio, Marília Estevam; Gallani, Maria Cecília Bueno Jayme; Pérez-Nebra, Amalia Raquel

2012-01-01

to validate the content of persuasive messages for promoting walking among patients with coronary heart disease (CHD). The messages were constructed to strengthen or change patients' attitudes to walking. the selection of persuasive arguments was based on behavioral beliefs (determinants of attitude) related to walking. The messages were constructed based in the Elaboration Likelihood Model and were submitted to content validation. the data was analyzed with the content validity index and by the importance which the patients attributed to the messages' persuasive arguments. Positive behavioral beliefs (i.e. positive and negative reinforcement) and self-efficacy were the appeals which the patients considered important. The messages with validation evidence will be tested in an intervention study for the promotion of the practice of physical activity among patients with CHD.

Development of the Patient Education Materials Assessment Tool (PEMAT): A new measure of understandability and actionability for print and audiovisual patient information

PubMed Central

Shoemaker, Sarah J.; Wolf, Michael S.; Brach, Cindy

2016-01-01

Objective To develop a reliable and valid instrument to assess the understandability and actionability of print and audiovisual materials. Methods We compiled items from existing instruments/guides that the expert panel assessed for face/content validity. We completed four rounds of reliability testing, and produced evidence of construct validity with consumers and readability assessments. Results The experts deemed the PEMAT items face/content valid. Four rounds of reliability testing and refinement were conducted using raters untrained on the PEMAT. Agreement improved across rounds. The final PEMAT showed moderate agreement per Kappa (Average K = 0.57) and strong agreement per Gwet’s AC1 (Average = 0.74). Internal consistency was strong (α = 0.71; Average Item-Total Correlation = 0.62). For construct validation with consumers (n = 47), we found significant differences between actionable and poorly-actionable materials in comprehension scores (76% vs. 63%, p < 0.05) and ratings (8.9 vs. 7.7, p < 0.05). For understandability, there was a significant difference for only one of two topics on consumer numeric scores. For actionability, there were significant positive correlations between PEMAT scores and consumer-testing results, but no relationship for understandability. There were, however, strong, negative correlations between grade-level and both consumer-testing results and PEMAT scores. Conclusions The PEMAT demonstrated strong internal consistency, reliability, and evidence of construct validity. Practice implications The PEMAT can help professionals judge the quality of materials (available at: http://www.ahrq.gov/pemat). PMID:24973195

Development of the Patient Education Materials Assessment Tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information.

PubMed

Shoemaker, Sarah J; Wolf, Michael S; Brach, Cindy

2014-09-01

To develop a reliable and valid instrument to assess the understandability and actionability of print and audiovisual materials. We compiled items from existing instruments/guides that the expert panel assessed for face/content validity. We completed four rounds of reliability testing, and produced evidence of construct validity with consumers and readability assessments. The experts deemed the PEMAT items face/content valid. Four rounds of reliability testing and refinement were conducted using raters untrained on the PEMAT. Agreement improved across rounds. The final PEMAT showed moderate agreement per Kappa (Average K=0.57) and strong agreement per Gwet's AC1 (Average=0.74). Internal consistency was strong (α=0.71; Average Item-Total Correlation=0.62). For construct validation with consumers (n=47), we found significant differences between actionable and poorly-actionable materials in comprehension scores (76% vs. 63%, p<0.05) and ratings (8.9 vs. 7.7, p<0.05). For understandability, there was a significant difference for only one of two topics on consumer numeric scores. For actionability, there were significant positive correlations between PEMAT scores and consumer-testing results, but no relationship for understandability. There were, however, strong, negative correlations between grade-level and both consumer-testing results and PEMAT scores. The PEMAT demonstrated strong internal consistency, reliability, and evidence of construct validity. The PEMAT can help professionals judge the quality of materials (available at: http://www.ahrq.gov/pemat). Copyright © 2014 Elsevier Ireland Ltd. All rights reserved.

Development and Validation of a Multimedia-based Assessment of Scientific Inquiry Abilities

NASA Astrophysics Data System (ADS)

Kuo, Che-Yu; Wu, Hsin-Kai; Jen, Tsung-Hau; Hsu, Ying-Shao

2015-09-01

The potential of computer-based assessments for capturing complex learning outcomes has been discussed; however, relatively little is understood about how to leverage such potential for summative and accountability purposes. The aim of this study is to develop and validate a multimedia-based assessment of scientific inquiry abilities (MASIA) to cover a more comprehensive construct of inquiry abilities and target secondary school students in different grades while this potential is leveraged. We implemented five steps derived from the construct modeling approach to design MASIA. During the implementation, multiple sources of evidence were collected in the steps of pilot testing and Rasch modeling to support the validity of MASIA. Particularly, through the participation of 1,066 8th and 11th graders, MASIA showed satisfactory psychometric properties to discriminate students with different levels of inquiry abilities in 101 items in 29 tasks when Rasch models were applied. Additionally, the Wright map indicated that MASIA offered accurate information about students' inquiry abilities because of the comparability of the distributions of student abilities and item difficulties. The analysis results also suggested that MASIA offered precise measures of inquiry abilities when the components (questioning, experimenting, analyzing, and explaining) were regarded as a coherent construct. Finally, the increased mean difficulty thresholds of item responses along with three performance levels across all sub-abilities supported the alignment between our scoring rubrics and our inquiry framework. Together with other sources of validity in the pilot testing, the results offered evidence to support the validity of MASIA.

Validity and usability of a professional association's web-based knowledge translation portal: American Physical Therapy Association's PTNow.org.

PubMed

Deutsch, Judith E; Romney, Wendy; Reynolds, Jan; Manal, Tara Jo

2015-10-08

PTNow.org is an evidence-based, on-line portal created by a professional membership association to promote use of evidence in practice and to help decrease unwarranted variation in practice. The site contains synthesis documents designed to promote efficient clinical reasoning. These documents were written and peer-reviewed by teams of content experts and master clinicians. The purpose of this paper is to report on the content and construct validity as well as usability of the site. Physical therapist participants used clinical summaries (available in 3 formats--as a full summary with hyperlinks, "quick takes" with hyperlinks, and a portable two-page version) on the PTNow.org site to answer knowledge acquisition and clinical reasoning questions related to four patient scenarios. They also responded to questions about ease of use related to website navigation and about format and completeness of information using a 1-5 Likert scale. Responses were coded to reflect how participants used the site and then were summarized descriptively. Preferences for clinical summary format were analyzed using an analysis of variance (ANOVA) and a Dunnett T3 post hoc analysis. Seventeen participants completed the study. Clinical relevance and completeness ratings by experienced clinicians, which were used as the measure of content validity, ranged from 3.1 to 4.6 on a 5 point scale. Construct validity based on the information on the PTNow.org site was supported for knowledge acquisition questions 66 % of the time and for clinical reasoning questions 40 % of the time. Usability ratings for the full clinical summary were 4.6 (1.2); for the quick takes, 3.5 (.98); and for the portable clinical summary, 4.0 (.45). Participants preferred the full clinical summary over the other two formats (F = 5.908, P = 0.007). One hundred percent of the participants stated that they would recommend the PTNow site to their colleagues. Prelimary evidence supported both content validity and construct validity of knowledge acquisition, and partially supported construct validity of clinical reasoning for the clinical summaries on the PTNow.org site. Usability was supported, with users preferring the full clinical summary over the other two formats. Iterative design is ongoing.

A systematic review on the quality of measurement techniques for the assessment of burn wound depth or healing potential.

PubMed

Jaspers, Mariëlle E H; van Haasterecht, Ludo; van Zuijlen, Paul P M; Mokkink, Lidwine B

2018-06-22

Reliable and valid assessment of burn wound depth or healing potential is essential to treatment decision-making, to provide a prognosis, and to compare studies evaluating different treatment modalities. The aim of this review was to critically appraise, compare and summarize the quality of relevant measurement properties of techniques that aim to assess burn wound depth or healing potential. A systematic literature search was performed using PubMed, EMBASE and Cochrane Library. Two reviewers independently evaluated the methodological quality of included articles using an adapted version of the Consensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. A synthesis of evidence was performed to rate the measurement properties for each technique and to draw an overall conclusion on quality of the techniques. Thirty-six articles were included, evaluating various techniques, classified as (1) laser Doppler techniques; (2) thermography or thermal imaging; (3) other measurement techniques. Strong evidence was found for adequate construct validity of laser Doppler imaging (LDI). Moderate evidence was found for adequate construct validity of thermography, videomicroscopy, and spatial frequency domain imaging (SFDI). Only two studies reported on the measurement property reliability. Furthermore, considerable variation was observed among comparator instruments. Considering the evidence available, it appears that LDI is currently the most favorable technique; thereby assessing burn wound healing potential. Additional research is needed into thermography, videomicroscopy, and SFDI to evaluate their full potential. Future studies should focus on reliability and measurement error, and provide a precise description of which construct is aimed to measure. Copyright © 2018 Elsevier Ltd and ISBI. All rights reserved.

Innovative learning model for improving students’ argumentation skill and concept understanding on science

NASA Astrophysics Data System (ADS)

Nafsiati Astuti, Rini

2018-04-01

Argumentation skill is the ability to compose and maintain arguments consisting of claims, supports for evidence, and strengthened-reasons. Argumentation is an important skill student needs to face the challenges of globalization in the 21st century. It is not an ability that can be developed by itself along with the physical development of human, but it must be developed under nerve like process, giving stimulus so as to require a person to be able to argue. Therefore, teachers should develop students’ skill of arguing in science learning in the classroom. The purpose of this study is to obtain an innovative learning model that are valid in terms of content and construct in improving the skills of argumentation and concept understanding of junior high school students. The assessment of content validity and construct validity was done through Focus Group Discussion (FGD), using the content and construct validation sheet, book model, learning video, and a set of learning aids for one meeting. Assessment results from 3 (three) experts showed that the learning model developed in the category was valid. The validity itself shows that the developed learning model has met the content requirement, the student needs, state of the art, strong theoretical and empirical foundation and construct validity, which has a connection of syntax stages and components of learning model so that it can be applied in the classroom activities

Validity Evidence for the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT).

PubMed

Breimer, Gerben E; Haji, Faizal A; Cinalli, Giuseppe; Hoving, Eelco W; Drake, James M

2017-02-01

Growing demand for transparent and standardized methods for evaluating surgical competence prompted the construction of the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT). To provide validity evidence of the NEVAT by reporting on the tool's internal structure and its relationship with surgical expertise during simulation-based training. The NEVAT was used to assess performance of trainees and faculty at an international neuroendoscopy workshop. All participants performed an endoscopic third ventriculostomy (ETV) on a synthetic simulator. Participants were simultaneously scored by 2 raters using the NEVAT procedural checklist and global rating scale (GRS). Evidence of internal structure was collected by calculating interrater reliability and internal consistency of raters' scores. Evidence of relationships with other variables was collected by comparing the ETV performance of experts, experienced trainees, and novices using Jonckheere's test (evidence of construct validity). Thirteen experts, 11 experienced trainees, and 10 novices participated. The interrater reliability by the intraclass correlation coefficient for the checklist and GRS was 0.82 and 0.94, respectively. Internal consistency (Cronbach's α) for the checklist and the GRS was 0.74 and 0.97, respectively. Median scores with interquartile range on the checklist and GRS for novices, experienced trainees, and experts were 0.69 (0.58-0.86), 0.85 (0.63-0.89), and 0.85 (0.81-0.91) and 3.1 (2.5-3.8), 3.7 (2.2-4.3) and 4.6 (4.4-4.9), respectively. Jonckheere's test showed that the median checklist and GRS score increased with performer expertise ( P = .04 and .002, respectively). This study provides validity evidence for the NEVAT to support its use as a standardized method of evaluating neuroendoscopic competence during simulation-based training. Copyright © 2016 by the Congress of Neurological Surgeons

Utility of pedometers for assessing physical activity: construct validity.

PubMed

Tudor-Locke, Catrine; Williams, Joel E; Reis, Jared P; Pluto, Delores

2004-01-01

Valid assessment of physical activity is necessary to fully understand this important health-related behaviour for research, surveillance, intervention and evaluation purposes. This article is the second in a companion set exploring the validity of pedometer-assessed physical activity. The previous article published in Sports Medicine dealt with convergent validity (i.e. the extent to which an instrument's output is associated with that of other instruments intended to measure the same exposure of interest). The present focus is on construct validity. Construct validity is the extent to which the measurement corresponds with other measures of theoretically-related parameters. Construct validity is typically evaluated by correlational analysis, that is, the magnitude of concordance between two measures (e.g. pedometer-determined steps/day and a theoretically-related parameter such as age, anthropometric measures and fitness). A systematic literature review produced 29 articles published since > or =1980 directly relevant to construct validity of pedometers in relation to age, anthropometric measures and fitness. Reported correlations were combined and a median r-value was computed. Overall, there was a weak inverse relationship (median r = -0.21) between age and pedometer-determined physical activity. A weak inverse relationship was also apparent with both body mass index and percentage overweight (median r = -0.27 and r = -0.22, respectively). Positive relationships regarding indicators of fitness ranged from weak to moderate depending on the fitness measure utilised: 6-minute walk test (median r = 0.69), timed treadmill test (median r = 0.41) and estimated maximum oxygen uptake (median r = 0.22). Studies are warranted to assess the relationship of pedometer-determined physical activity with other important health-related outcomes including blood pressure and physiological parameters such as blood glucose and lipid profiles. The aggregated evidence of convergent validity (presented in the previous companion article) and construct validity herein provides support for considering simple and inexpensive pedometers in both research and practice.

Development and validation of Dutch version of Lasater Clinical Judgment Rubric in hospital practice: An instrument design study.

PubMed

Vreugdenhil, Jettie; Spek, Bea

2018-03-01

Clinical reasoning in patient care is a skill that cannot be observed directly. So far, no reliable, valid instrument exists for the assessment of nursing students' clinical reasoning skills in hospital practice. Lasater's clinical judgment rubric (LCJR), based on Tanner's model "Thinking like a nurse" has been tested, mainly in academic simulation settings. The aim is to develop a Dutch version of the LCJR (D-LCJR) and to test its psychometric properties when used in a hospital traineeship context. A mixed-model approach was used to develop and to validate the instrument. Ten dedicated educational units in a university hospital. A well-mixed group of 52 nursing students, nurse coaches and nurse educators. A Delphi panel developed the D-LCJR. Students' clinical reasoning skills were assessed "live" by nurse coaches, nurse educators and students who rated themselves. The psychometric properties tested during the assessment process are reliability, reproducibility, content validity and construct validity by testing two hypothesis: 1) a positive correlation between assessed and self-reported sum scores (convergent validity) and 2) a linear relation between experience and sum score (clinical validity). The obtained D-LCJR was found to be internally consistent, Cronbach's alpha 0.93. The rubric is also reproducible with intraclass correlations between 0.69 and 0.78. Experts judged it to be content valid. The two hypothesis were both tested significant, supporting evidence for construct validity. The translated and modified LCJR, is a promising tool for the evaluation of nursing students' development in clinical reasoning in hospital traineeships, by students, nurse coaches and nurse educators. More evidence on construct validity is necessary, in particular for students at the end of their hospital traineeship. Based on our research, the D-LCJR applied in hospital traineeships is a usable and reliable tool. Copyright © 2017 Elsevier Ltd. All rights reserved.

Hypersensitivity to sound in tinnitus patients: an analysis of a construct based on questionnaire and audiological data.

PubMed

Bläsing, Lena; Goebel, Gerhard; Flötzinger, Uta; Berthold, Anke; Kröner-Herwig, Birgit

2010-07-01

The purpose of this study was to analyse the Questionnaire on Hypersensitivity to Sound (GUF; Nelting & Finlayson, 2004 ) and to improve its validity based on the analysis of intercorrelations (single item level) with other methods of assessing hyperacusis (uncomfortable loudness level, individual loudness function, self-rated severity of hyperacusis). Subjects consisted of 91 inpatients with tinnitus and hyperacusis. The GUF showed a good reliability (alpha = .92). The factorial structure of the questionnaire reported by Nelting et al (2002) was not completely supported by the evidence in this study. The total score and the single items showed small to moderate correlations with the other modes of measuring hyperacusis. Evidence for convergent and discriminant validity were found, but overall the results corroborate the conceptual heterogeneity of the construct hyperacusis and its dependency on the assessment method. Four items of the GUF with particularly low correlations were excluded from the questionnaire. The revised GUF total score showed slightly but not statistically significant higher convergent and discriminant validity.

The application of transformational leadership theory to parenting: questionnaire development and implications for adolescent self-regulatory efficacy and life satisfaction.

PubMed

Morton, Katie L; Barling, Julian; Rhodes, Ryan E; Mâsse, Louise C; Zumbo, Bruno D; Beauchamp, Mark R

2011-10-01

We draw upon transformational leadership theory to develop an instrument to measure transformational parenting for use with adolescents. First, potential items were generated that were developmentally appropriate and evidence for content validity was provided through the use of focus groups with parents and adolescents. We subsequently provide evidence for several aspects of construct validity of measures derived from the Transformational Parenting Questionnaire (TPQ). Data were collected from 857 adolescents (M(age) = 14.70 years), who rated the behaviors of their mothers and fathers. The results provided support for a second-order measurement model of transformational parenting. In addition, positive relationships between mothers' and fathers' transformational parenting behaviors, adolescents' self-regulatory efficacy for physical activity and healthy eating, and life satisfaction were found. The results of this research support the application of transformational leadership theory to parenting behaviors, as well as the construct validity of measures derived from the TPQ.

Psychometric performance of the brazilian version of the Mini-cuestionario de calidad de vida en la hipertensión arterial (MINICHAL).

PubMed

Soutello, Ana Lúcia Soares; Rodrigues, Roberta Cunha Matheus; Jannuzzi, Fernanda Freire; Spana, Thaís Moreira; Gallani, Maria Cecília Bueno Jayme; Nadruz Junior, Wilson

2011-01-01

This study aimed to evaluate the feasibility, acceptability, ceiling and floor effects, reliability, and convergent construct validity of the Brazilian version of the Mini Cuestionario de Calidad de Vida en la Hipertensión Arterial (MINICHAL). The study included 200 hypertensive outpatients in a university hospital and a primary healthcare unit. The MINICHAL was applied in 3.0 (± 1.0) minutes with 100% of the items answered. A "ceiling effect" was observed in both dimensions and in the total score, as well as evidence of measurement stability (ICC=0.74). The convergent validity was confirmed by significant positive correlations between similar dimensions of the MINICHAL and the SF-36, and significant negative correlations with the Minnesota Living with Heart Failure Questionnaire - MLHFQ, however, correlations between dissimilar constructs were also observed. It was concluded that the Brazilian version of the MINICHAL presents evidence of reliability and validity when applied to hypertensive outpatients.

Screening tool for oropharyngeal dysphagia in stroke - Part I: evidence of validity based on the content and response processes.

PubMed

Almeida, Tatiana Magalhães de; Cola, Paula Cristina; Pernambuco, Leandro de Araújo; Magalhães, Hipólito Virgílio; Magnoni, Carlos Daniel; Silva, Roberta Gonçalves da

2017-08-17

The aim of the present study was to identify the evidence of validity based on the content and response process of the Rastreamento de Disfagia Orofaríngea no Acidente Vascular Encefálico (RADAVE; "Screening Tool for Oropharyngeal Dysphagia in Stroke"). The criteria used to elaborate the questions were based on a literature review. A group of judges consisting of 19 different health professionals evaluated the relevance and representativeness of the questions, and the results were analyzed using the Content Validity Index. In order to evidence validity based on the response processes, 23 health professionals administered the screening tool and analyzed the questions using a structured scale and cognitive interview. The RADAVE structured to be applied in two stages. The first version consisted of 18 questions in stage I and 11 questions in stage II. Eight questions in stage I and four in stage II did not reach the minimum Content Validity Index, requiring reformulation by the authors. The cognitive interview demonstrated some misconceptions. New adjustments were made and the final version was produced with 12 questions in stage I and six questions in stage II. It was possible to develop a screening tool for dysphagia in stroke with adequate evidence of validity based on content and response processes. Both validity evidences obtained so far allowed to adjust the screening tool in relation to its construct. The next studies will analyze the other evidences of validity and the measures of accuracy.

Criterion and incremental validity of the emotion regulation questionnaire

PubMed Central

Ioannidis, Christos A.; Siegling, A. B.

2015-01-01

Although research on emotion regulation (ER) is developing, little attention has been paid to the predictive power of ER strategies beyond established constructs. The present study examined the incremental validity of the Emotion Regulation Questionnaire (ERQ; Gross and John, 2003), which measures cognitive reappraisal and expressive suppression, over and above the Big Five personality factors. It also extended the evidence for the measure's criterion validity to yet unexamined criteria. A university student sample (N = 203) completed the ERQ, a measure of the Big Five, and relevant cognitive and emotion-laden criteria. Cognitive reappraisal predicted positive affect beyond personality, as well as experiential flexibility and constructive self-assertion beyond personality and affect. Expressive suppression explained incremental variance in negative affect beyond personality and in experiential flexibility beyond personality and general affect. No incremental effects were found for worry, social anxiety, rumination, reflection, and preventing negative emotions. Implications for the construct validity and utility of the ERQ are discussed. PMID:25814967

Construct validity of the Chinese version of the Self-care of Heart Failure Index determined using structural equation modeling.

PubMed

Kang, Xiaofeng; Dennison Himmelfarb, Cheryl R; Li, Zheng; Zhang, Jian; Lv, Rong; Guo, Jinyu

2015-01-01

The Self-care of Heart Failure Index (SCHFI) is an empirically tested instrument for measuring the self-care of patients with heart failure. The aim of this study was to develop a simplified Chinese version of the SCHFI and provide evidence for its construct validity. A total of 182 Chinese with heart failure were surveyed. A 2-step structural equation modeling procedure was applied to test construct validity. Factor analysis showed 3 factors explaining 43% of the variance. Structural equation model confirmed that self-care maintenance, self-care management, and self-care confidence are indeed indicators of self-care, and self-care confidence was a positive and equally strong predictor of self-care maintenance and self-care management. Moreover, self-care scores were correlated with the Partners in Health Scale, indicating satisfactory concurrent validity. The Chinese version of the SCHFI is a theory-based instrument for assessing self-care of Chinese patients with heart failure.

Reliability and construct validity of the Instrument to Measure the Impact of Valve Heart Disease on the Patient's Daily Life

PubMed Central

dos Anjos, Daniela Brianne Martins; Rodrigues, Roberta Cunha Matheus; Padilha, Kátia Melissa; Pedrosa, Rafaela Batista dos Santos; Gallani, Maria Cecília Bueno Jayme

2016-01-01

ABSTRACT Objective: evaluate the practicality, acceptability and the floor and ceiling effects, estimate the reliability and verify the convergent construct's validity with the instrument called the Heart Valve Disease Impact on daily life (IDCV) of the valve disease in patients with mitral and or aortic heart valve disease. Method: data was obtained from 86 heart valve disease patients through 3 phases: a face to face interview for a socio-demographic and clinic characterization and then other two done through phone calls of the interviewed patients for application of the instrument (test and repeat test). Results: as for the practicality and acceptability, the instrument was applied with an average time of 9,9 minutes and with 110% of responses, respectively. Ceiling and floor effects observed for all domains, especially floor effect. Reliability was tested using the test - repeating pattern to give evidence of temporal stability of the measurement. Significant negative correlations with moderate to strong magnitude were found between the score of the generic question about the impact of the disease and the scores of IDCV, which points to the validity of the instrument convergent construct. Conclusion: the instrument to measure the impact of valve heart disease on the patient's daily life showed evidence of reliability and validity when applied to patients with heart valve disease. PMID:27992024

Exploring Asian Students' Citizenship Values and Their Relationship to Civic Knowledge and School Participation

ERIC Educational Resources Information Center

Kennedy, Kerry J.; Kuang, Xiaoxue; Chow, Joseph Kui Foon

2013-01-01

Empirical evidence of Asian students' traditional citizenship values was provided in the Asian Regional Module (ARM) of the International Civic and Citizenship Study. This paper is based on a secondary analysis of the ARM data. Three issues are addressed. First, a theoretical analysis of the ARM constructs contributes to their construct validity.…

Adolescent Psychopathy and Personality Theory--The Interpersonal Circumplex: Expanding Evidence of a Nomological Net

ERIC Educational Resources Information Center

Salekin, Randall T.; Leistico, Anne-Marie R.; Trobst, Krista K.; Schrum, Crystal L.; Lochman, John E.

2005-01-01

The construct validity of psychopathy was examined in a sample of 114 male and female young offenders ([M.sub.age] = 15.16) at a southeastern detention center. The interpersonal circumplex served as a framework of general personality from which to examine the construct of adolescent psychopathy. A supplementary analysis of the psychopathy measures…

Decision-Making Style among Adolescents: Relationship with Sensation Seeking and Locus of Control

ERIC Educational Resources Information Center

Baiocco, Roberto; Laghi, Fiorenzo; D'Alessio, Maria

2009-01-01

The principal aim of the study was to examine the psychometric properties and construct validity of the General Decision-Making Scale (GDMS) in a sample of 700 adolescents (aged 15-19 years). Confirmatory and exploratory factor analyses provide evidence for a solid five-dimension structure reflecting the theorized construct: rational, intuitive,…

A Confirmatory Factor Analysis of the Student Evidence-Based Practice Questionnaire (S-EBPQ) in an Australian sample.

PubMed

Beccaria, Lisa; Beccaria, Gavin; McCosker, Catherine

2018-03-01

It is crucial that nursing students develop skills and confidence in using Evidence-Based Practice principles early in their education. This should be assessed with valid tools however, to date, few measures have been developed and applied to the student population. To examine the structural validity of the Student Evidence-Based Practice Questionnaire (S-EBPQ), with an Australian online nursing student cohort. A cross-sectional study for constructing validity. Three hundred and forty-five undergraduate nursing students from an Australian regional university were recruited across two semesters. Confirmatory Factor Analysis was used to examine the structural validity. Confirmatory Factor Analysis was applied which resulted in a good fitting model, based on a revised 20-item tool. The S-EBPQ tool remains a psychometrically robust measure of evidence-based practice use, attitudes, and knowledge and skills and can be applied in an online Australian student context. The findings of this study provided further evidence of the reliability and four factor structure of the S-EBPQ. Opportunities for further refinement of the tool may result in improvements in structural validity. Copyright © 2018 Elsevier Ltd. All rights reserved.

Testing for Measurement Invariance in the Conformity to Masculine Norms-46 Across White and Asian American College Men: Development and Validity of the CMNI-29

PubMed Central

Hsu, Kean; Iwamoto, Derek Kenji

2014-01-01

The Conformity to Masculine Norms Inventory (CMNI; Mahalik et al., 2003) and revised CMNI-46 (Parent & Moradi, 2009) have received a great deal of empirical attention and support for their strong psychometric properties and evidence of construct validity. However, one important area that remains unexplored is how adherence to these masculine norms may vary across race and ethnicity. The current investigation examines the possible racial measurement noninvariance in the CMNI-46 among Asian American and White American college students (N = 893). The results revealed significant measurement differences across groups; specifically, the CMNI-46 was more theoretically consistent for the White American men than the Asian American men. Through exploratory and multigroup confirmatory factor analysis, an 8-factor, 29-item version of the CMNI emerged, displaying an excellent overall model fit for both racial groups. This study provides strong evidence for the use of a streamlined 29-item version of the CMNI, validated with Asian American and White American men. The findings also lend further empirical and psychometric evidence regarding the variance of masculine norms among ethnic groups as well as the variance of the multidimensional construct of masculinity. PMID:25530724

Testing for Measurement Invariance in the Conformity to Masculine Norms-46 Across White and Asian American College Men: Development and Validity of the CMNI-29.

PubMed

Hsu, Kean; Iwamoto, Derek Kenji

2014-10-01

The Conformity to Masculine Norms Inventory (CMNI; Mahalik et al., 2003) and revised CMNI-46 (Parent & Moradi, 2009) have received a great deal of empirical attention and support for their strong psychometric properties and evidence of construct validity. However, one important area that remains unexplored is how adherence to these masculine norms may vary across race and ethnicity. The current investigation examines the possible racial measurement noninvariance in the CMNI-46 among Asian American and White American college students ( N = 893). The results revealed significant measurement differences across groups; specifically, the CMNI-46 was more theoretically consistent for the White American men than the Asian American men. Through exploratory and multigroup confirmatory factor analysis, an 8-factor, 29-item version of the CMNI emerged, displaying an excellent overall model fit for both racial groups. This study provides strong evidence for the use of a streamlined 29-item version of the CMNI, validated with Asian American and White American men. The findings also lend further empirical and psychometric evidence regarding the variance of masculine norms among ethnic groups as well as the variance of the multidimensional construct of masculinity.

The Control Attitudes Scale-Revised: psychometric evaluation in three groups of patients with cardiac illness.

PubMed

Moser, Debra K; Riegel, Barbara; McKinley, Sharon; Doering, Lynn V; Meischke, Hendrika; Heo, Seongkum; Lennie, Terry A; Dracup, Kathleen

2009-01-01

Perceived control is a construct with important theoretical and clinical implications for healthcare providers, yet practical application of the construct in research and clinical practice awaits development of an easily administered instrument to measure perceived control with evidence of reliability and validity. To test the psychometric properties of the Control Attitudes Scale-Revised (CAS-R) using a sample of 3,396 individuals with coronary heart disease, 513 patients with acute myocardial infarction, and 146 patients with heart failure. Analyses were done separately in each patient group. Reliability was assessed using Cronbach's alpha to determine internal consistency, and item homogeneity was assessed using item-total and interitem correlations. Validity was examined using principal component analysis and testing hypotheses about known associations. Cronbach's alpha values for the CAS-R in patients with coronary heart disease, acute myocardial infarction, and heart failure were all greater than .70. Item-total and interitem correlation coefficients for all items were acceptable in the groups. In factor analyses, the same single factor was extracted in all groups, and all items were loaded moderately or strongly to the factor in each group. As hypothesized in the final construct validity test, in all groups, patients with higher levels of perceived control had less depression and less anxiety compared with those of patients who had lower levels of perceived control. This study provides evidence of the reliability and validity of the 8-item CAS-R as a measure of perceived control in patients with cardiac illness and provides important insight into a key patient construct.

Measurement properties of tools measuring mental health knowledge: a systematic review.

PubMed

Wei, Yifeng; McGrath, Patrick J; Hayden, Jill; Kutcher, Stan

2016-08-23

Mental health literacy has received great attention recently to improve mental health knowledge, decrease stigma and enhance help-seeking behaviors. We conducted a systematic review to critically appraise the qualities of studies evaluating the measurement properties of mental health knowledge tools and the quality of included measurement properties. We searched PubMed, PsycINFO, EMBASE, CINAHL, the Cochrane Library, and ERIC for studies addressing psychometrics of mental health knowledge tools and published in English. We applied the COSMIN checklist to assess the methodological quality of each study as "excellent", "good", "fair", or "indeterminate". We ranked the level of evidence of the overall quality of each measurement property across studies as "strong", "moderate", "limited", "conflicting", or "unknown". We identified 16 mental health knowledge tools in 17 studies, addressing reliability, validity, responsiveness or measurement errors. The methodological quality of included studies ranged from "poor" to "excellent" including 6 studies addressing the content validity, internal consistency or structural validity demonstrating "excellent" quality. We found strong evidence of the content validity or internal consistency of 6 tools; moderate evidence of the internal consistency, the content validity or the reliability of 8 tools; and limited evidence of the reliability, the structural validity, the criterion validity, or the construct validity of 12 tools. Both the methodological qualities of included studies and the overall evidence of measurement properties are mixed. Based on the current evidence, we recommend that researchers consider using tools with measurement properties of strong or moderate evidence that also reached the threshold for positive ratings according to COSMIN checklist.

Measuring Constructs in Family Science: How Can Item Response Theory Improve Precision and Validity?

ERIC Educational Resources Information Center

Gordon, Rachel A.

2015-01-01

This article provides family scientists with an understanding of contemporary measurement perspectives and the ways in which item response theory (IRT) can be used to develop measures with desired evidence of precision and validity for research uses. The article offers a nontechnical introduction to some key features of IRT, including its…

Validity Evidence for the Use of the Preventive Resources Inventory with College Students

ERIC Educational Resources Information Center

Lambert, Richard G.; McCarthy, Christopher J.; Gilbert, Trae; Sebree, Mikaela; Steinley-Bumgarner, Michelle

2006-01-01

Measurement properties of scores from the Preventive Resources Inventory (C. J. McCarthy & R. G. Lambert, 2001), a measure of stress-prevention resources, were evaluated. Sample-specific construct validity of 3 primary scales was supported. A 4th, Self-Acceptance, functioned as a higher order factor. Differences were found between those reporting…

Report on the Validation of the Emotionally Intelligent Leadership for Students Inventory

ERIC Educational Resources Information Center

Miguel, Rosanna F.; Allen, Scott J.

2016-01-01

The present study was designed to examine the measurement of the Emotionally Intelligent Leadership (EIL) construct and to provide evidence of validation for the multidimensional Emotionally Intelligence Leadership for Students: Inventory 2.0 (EILS:I 2.0). The EILS:I 2.0 is a self-report assessment of emotionally intelligent leadership in the…

Predicting implementation from organizational readiness for change: a study protocol

PubMed Central

2011-01-01

Background There is widespread interest in measuring organizational readiness to implement evidence-based practices in clinical care. However, there are a number of challenges to validating organizational measures, including inferential bias arising from the halo effect and method bias - two threats to validity that, while well-documented by organizational scholars, are often ignored in health services research. We describe a protocol to comprehensively assess the psychometric properties of a previously developed survey, the Organizational Readiness to Change Assessment. Objectives Our objective is to conduct a comprehensive assessment of the psychometric properties of the Organizational Readiness to Change Assessment incorporating methods specifically to address threats from halo effect and method bias. Methods and Design We will conduct three sets of analyses using longitudinal, secondary data from four partner projects, each testing interventions to improve the implementation of an evidence-based clinical practice. Partner projects field the Organizational Readiness to Change Assessment at baseline (n = 208 respondents; 53 facilities), and prospectively assesses the degree to which the evidence-based practice is implemented. We will conduct predictive and concurrent validities using hierarchical linear modeling and multivariate regression, respectively. For predictive validity, the outcome is the change from baseline to follow-up in the use of the evidence-based practice. We will use intra-class correlations derived from hierarchical linear models to assess inter-rater reliability. Two partner projects will also field measures of job satisfaction for convergent and discriminant validity analyses, and will field Organizational Readiness to Change Assessment measures at follow-up for concurrent validity (n = 158 respondents; 33 facilities). Convergent and discriminant validities will test associations between organizational readiness and different aspects of job satisfaction: satisfaction with leadership, which should be highly correlated with readiness, versus satisfaction with salary, which should be less correlated with readiness. Content validity will be assessed using an expert panel and modified Delphi technique. Discussion We propose a comprehensive protocol for validating a survey instrument for assessing organizational readiness to change that specifically addresses key threats of bias related to halo effect, method bias and questions of construct validity that often go unexplored in research using measures of organizational constructs. PMID:21777479

Building Evidence of Validity: The Relation between Work Values, Interests, Personality, and Personal Values

ERIC Educational Resources Information Center

Leuty, Melanie E.; Hansen, Jo-Ida C.

2013-01-01

The current study used work values components (WVC) to examine the relationship between work values, vocational interests, personality, and personal values. Most intercorrelations between work values and other constructs were in the small effect range. Overall correlations between scale scores provided evidence of convergent and discriminant…

The ad-libitum alcohol 'taste test': secondary analyses of potential confounds and construct validity.

PubMed

Jones, Andrew; Button, Emily; Rose, Abigail K; Robinson, Eric; Christiansen, Paul; Di Lemma, Lisa; Field, Matt

2016-03-01

Motivation to drink alcohol can be measured in the laboratory using an ad-libitum 'taste test', in which participants rate the taste of alcoholic drinks whilst their intake is covertly monitored. Little is known about the construct validity of this paradigm. The objective of this study was to investigate variables that may compromise the validity of this paradigm and its construct validity. We re-analysed data from 12 studies from our laboratory that incorporated an ad-libitum taste test. We considered time of day and participants' awareness of the purpose of the taste test as potential confounding variables. We examined whether gender, typical alcohol consumption, subjective craving, scores on the Alcohol Use Disorders Identification Test and perceived pleasantness of the drinks predicted ad-libitum consumption (construct validity). We included 762 participants (462 female). Participant awareness and time of day were not related to ad-libitum alcohol consumption. Males drank significantly more alcohol than females (p < 0.001), and individual differences in typical alcohol consumption (p = 0.04), craving (p < 0.001) and perceived pleasantness of the drinks (p = 0.04) were all significant predictors of ad-libitum consumption. We found little evidence that time of day or participant awareness influenced alcohol consumption. The construct validity of the taste test was supported by relationships between ad-libitum consumption and typical alcohol consumption, craving and pleasantness ratings of the drinks. The ad-libitum taste test is a valid method for the assessment of alcohol intake in the laboratory.

Construction and Validation of the Perceived Opportunity to Craft Scale.

PubMed

van Wingerden, Jessica; Niks, Irene M W

2017-01-01

We developed and validated a scale to measure employees' perceived opportunity to craft (POC) in two separate studies conducted in the Netherlands (total N = 2329). POC is defined as employees' perception of their opportunity to craft their job. In Study 1, the perceived opportunity to craft scale (POCS) was developed and tested for its factor structure and reliability in an explorative way. Study 2 consisted of confirmatory analyses of the factor structure and reliability of the scale as well as examination of the discriminant and criterion-related validity of the POCS. The results indicated that the scale consists of one dimension and could be reliably measured with five items. Evidence was found for the discriminant validity of the POCS. The scale also showed criterion-related validity when correlated with job crafting (+), job resources (autonomy +; opportunities for professional development +), work engagement (+), and the inactive construct cynicism (-). We discuss the implications of these findings for theory and practice.

Development and validation of the coping with terror scale.

PubMed

Stein, Nathan R; Schorr, Yonit; Litz, Brett T; King, Lynda A; King, Daniel W; Solomon, Zahava; Horesh, Danny

2013-10-01

Terrorism creates lingering anxiety about future attacks. In prior terror research, the conceptualization and measurement of coping behaviors were constrained by the use of existing coping scales that index reactions to daily hassles and demands. The authors created and validated the Coping with Terror Scale to fill the measurement gap. The authors emphasized content validity, leveraging the knowledge of terror experts and groups of Israelis. A multistep approach involved construct definition and item generation, trimming and refining the measure, exploring the factor structure underlying item responses, and garnering evidence for reliability and validity. The final scale comprised six factors that were generally consistent with the authors' original construct specifications. Scores on items linked to these factors demonstrate good reliability and validity. Future studies using the Coping with Terror Scale with other populations facing terrorist threats are needed to test its ability to predict resilience, functional impairment, and psychological distress.

[Validity evidence of the Health-Related Quality of Life for Drug Abusers Test based on the Biaxial Model of Addiction].

PubMed

Lozano, Oscar M; Rojas, Antonio J; Pérez, Cristino; González-Sáiz, Francisco; Ballesta, Rosario; Izaskun, Bilbao

2008-05-01

The aim of this work is to show evidence of the validity of the Health-Related Quality of Life for Drug Abusers Test (HRQoLDA Test). This test was developed to measure specific HRQoL for drugs abusers, within the theoretical addiction framework of the biaxial model. The sample comprised 138 patients diagnosed with opiate drug dependence. In this study, the following constructs and variables of the biaxial model were measured: severity of dependence, physical health status, psychological adjustment and substance consumption. Results indicate that the HRQoLDA Test scores are related to dependency and consumption-related problems. Multiple regression analysis reveals that HRQoL can be predicted from drug dependence, physical health status and psychological adjustment. These results contribute empirical evidence of the theoretical relationships established between HRQoL and the biaxial model, and they support the interpretation of the HRQoLDA Test to measure HRQoL in drug abusers, thus providing a test to measure this specific construct in this population.

IMatter: validation of the NHS Scotland Employee Engagement Index.

PubMed

Snowden, Austyn; MacArthur, Ewan

2014-11-08

Employee engagement is a fundamental component of quality healthcare. In order to provide empirical data of engagement in NHS Scotland an Employee Engagement Index was co-constructed with staff. 'iMatter' consists of 25 Likert questions developed iteratively from the literature and a series of validation events with NHS Scotland staff. The aim of this study was to test the face, content and construct validity of iMatter. Cross sectional survey of NHS Scotland staff. In January 2013 iMatter was sent to 2300 staff across all disciplines in NHS Scotland. 1280 staff completed it. Demographic data were collected. Internal consistency of the scale was calculated. Construct validity consisted of concurrent application of factor analysis and Rasch analysis. Face and content validity were checked using 3 focus groups. The sample was representative of the NHSScotland population. iMatter showed very strong reliability (α = 0.958). Factor analysis revealed a four-factor structure consistent with the following interpretation: iMatter showed evidence of high reliability and validity. It is a popular measure of staff engagement in NHS Scotland. Implications for practice focus on the importance of coproduction in psychometric development.

Construct Validity of Symptom Checklist-90-Revised (SCL-90-R) and General Health Questionnaire-28 (GHQ-28) in Patients with Drug Addiction and Diabetes, and Normal Population

PubMed Central

ARDAKANI, Abolfazl; SEGHATOLESLAM, Tahereh; HABIL, Hussain; JAMEEI, Fahimeh; RASHID, Rusdi; ZAHIRODIN, Alireza; MOTLAQ, Farid; MASJIDI ARANI, Abbas

2016-01-01

Background: Given that validity is the baseline of psychological assessments, there is a need to provide evidence-based data for construct validity of such scales to advance the clinicians for evaluating psychiatric morbidity in psychiatric and psychosomatic setting. Methods: This comparative cross-sectional study aimed to investigate the construct validity of the Malaysian version of the GHQ-28 and the SCL-90-R. The sample comprised 660 individuals including diabetics, drug dependents, and normal population. The research scales were administered to the participants. Convergent and discriminant validity of both scales were investigated by Confirmatory Factor Analysis (CFA) using AMOS. The Pearson correlation coefficient was utilized to obtain the relationship between the two scales. Results: The internal consistency of the GHQ-28 and SCL-90-R were highly acceptable, and confirmatory factor analysis confirmed the convergent validity of both scales. The results of this study revealed that the construct validity of GHQ-28 was acceptable, whereas discriminant validity of SCL-90-R was not adequate. According to Pearson correlation coefficient the relationships between three common subscales of the GHQ-28 and SCL-90-R were significantly positive; somatization (r=0.671, P<0.01), Anxiety (r=0.728, P<0.01), and Depression (r=0.660, P <0.01). Conclusions: This study replicated the construct of the Malaysian version of GHQ-28, yet failed to support the nine-factor structure of the SCL-90-R. Therefore, multidimensionality of the SCL-90-R as clinical purposes is questionable, and it may be a better unitary measure for assessing and screening mental disorders. Further research need to be carried out to prove this finding. PMID:27252914

What Matters More About the Interpersonal Reactivity Index and the Jefferson Scale of Empathy? Their Underlying Constructs or Their Relationships With Pertinent Measures of Clinical Competence and Patient Outcomes?

PubMed

Hojat, Mohammadreza; Gonnella, Joseph S

2017-06-01

In their study published in this issue of Academic Medicine, Costa and colleagues confirmed the underlying constructs of the Interpersonal Reactivity Index (IRI) and the Jefferson Scale of Empathy (JSE) in medical students. The authors of this Commentary propose that in comparing two instruments that both purport to measure empathy, researchers or test users must pay close attention to the target populations, the conceptualizations of empathy, and the validity evidence in relation to pertinent criterion measures. The Commentary's authors draw attention to the fact that the IRI was developed for administration to the general population, whereas the JSE was developed specifically for administration to students and practitioners of health professions. Also, the author of the IRI conceptualized empathy as a combination of cognitive and emotional attributes, whereas the authors of the JSE defined empathy as a predominantly cognitive attribute. These differences are reflected in the content of the items, which determines the underlying constructs of the two instruments. The Commentary authors suggest that any empathy-measuring instrument in the context of health professions education and patient care requires the crucial evidence of significant relationships with indicators of clinical competence and positive patient outcomes. Such validity evidence is readily available for the JSE, and the Commentary authors recommend that researchers make efforts to provide pertinent validity support for any other instrument measuring empathy in health professionals-in-training and in-practice.

Development and construct validation of the Client-Centredness of Goal Setting (C-COGS) scale.

PubMed

Doig, Emmah; Prescott, Sarah; Fleming, Jennifer; Cornwell, Petrea; Kuipers, Pim

2015-07-01

Client-centred philosophy is integral to occupational therapy practice and client-centred goal planning is considered fundamental to rehabilitation. Evaluation of whether goal-planning practices are client-centred requires an understanding of the client's perspective about goal-planning processes and practices. The Client-Centredness of Goal Setting (C-COGS) was developed for use by practitioners who seek to be more client-centred and who require a scale to guide and evaluate individually orientated practice, especially with adults with cognitive impairment related to acquired brain injury. To describe development of the C-COGS scale and examine its construct validity. The C-COGS was administered to 42 participants with acquired brain injury after multidisciplinary goal planning. C-COGS scores were correlated with the Canadian Occupational Performance Measure (COPM) importance scores, and measures of therapeutic alliance, motivation, and global functioning to establish construct validity. The C-COGS scale has three subscales evaluating goal alignment, goal planning participation, and client-centredness of goals. The C-COGS subscale items demonstrated moderately significant correlations with scales measuring similar constructs. Findings provide preliminary evidence to support the construct validity of the C-COGS scale, which is intended to be used to evaluate and reflect on client-centred goal planning in clinical practice, and to highlight factors contributing to best practice rehabilitation.

On Conducting Construct Validity Meta-Analyses for the Rorschach: A Reply to Tibon Czopp and Zeligman (2016).

PubMed

Mihura, Joni L; Meyer, Gregory J; Dumitrascu, Nicolae; Bombel, George

2016-01-01

We respond to Tibon Czopp and Zeligman's (2016) critique of our systematic reviews and meta-analyses of 65 Rorschach Comprehensive System (CS) variables published in Psychological Bulletin (2013). The authors endorsed our supportive findings but critiqued the same methodology when used for the 13 unsupported variables. Unfortunately, their commentary was based on significant misunderstandings of our meta-analytic method and results, such as thinking we used introspectively assessed criteria in classifying levels of support and reporting only a subset of our externally assessed criteria. We systematically address their arguments that our construct label and criterion variable choices were inaccurate and, therefore, meta-analytic validity for these 13 CS variables was artificially low. For example, the authors created new construct labels for these variables that they called "the customary CS interpretation," but did not describe their methodology nor provide evidence that their labels would result in better validity than ours. They cite studies they believe we should have included; we explain how these studies did not fit our inclusion criteria and that including them would have actually reduced the relevant CS variables' meta-analytic validity. Ultimately, criticisms alone cannot change meta-analytic support from negative to positive; Tibon Czopp and Zeligman would need to conduct their own construct validity meta-analyses.

Initial construct validity evidence of a virtual human application for competency assessment in breaking bad news to a cancer patient.

PubMed

Guetterman, Timothy C; Kron, Frederick W; Campbell, Toby C; Scerbo, Mark W; Zelenski, Amy B; Cleary, James F; Fetters, Michael D

2017-01-01

Despite interest in using virtual humans (VHs) for assessing health care communication, evidence of validity is limited. We evaluated the validity of a VH application, MPathic-VR, for assessing performance-based competence in breaking bad news (BBN) to a VH patient. We used a two-group quasi-experimental design, with residents participating in a 3-hour seminar on BBN. Group A (n=15) completed the VH simulation before and after the seminar, and Group B (n=12) completed the VH simulation only after the BBN seminar to avoid the possibility that testing alone affected performance. Pre- and postseminar differences for Group A were analyzed with a paired t -test, and comparisons between Groups A and B were analyzed with an independent t -test. Compared to the preseminar result, Group A's postseminar scores improved significantly, indicating that the VH program was sensitive to differences in assessing performance-based competence in BBN. Postseminar scores of Group A and Group B were not significantly different, indicating that both groups performed similarly on the VH program. Improved pre-post scores demonstrate acquisition of skills in BBN to a VH patient. Pretest sensitization did not appear to influence posttest assessment. These results provide initial construct validity evidence that the VH program is effective for assessing BBN performance-based communication competence.

Initial construct validity evidence of a virtual human application for competency assessment in breaking bad news to a cancer patient

PubMed Central

Guetterman, Timothy C; Kron, Frederick W; Campbell, Toby C; Scerbo, Mark W; Zelenski, Amy B; Cleary, James F; Fetters, Michael D

2017-01-01

Background Despite interest in using virtual humans (VHs) for assessing health care communication, evidence of validity is limited. We evaluated the validity of a VH application, MPathic-VR, for assessing performance-based competence in breaking bad news (BBN) to a VH patient. Methods We used a two-group quasi-experimental design, with residents participating in a 3-hour seminar on BBN. Group A (n=15) completed the VH simulation before and after the seminar, and Group B (n=12) completed the VH simulation only after the BBN seminar to avoid the possibility that testing alone affected performance. Pre- and postseminar differences for Group A were analyzed with a paired t-test, and comparisons between Groups A and B were analyzed with an independent t-test. Results Compared to the preseminar result, Group A’s postseminar scores improved significantly, indicating that the VH program was sensitive to differences in assessing performance-based competence in BBN. Postseminar scores of Group A and Group B were not significantly different, indicating that both groups performed similarly on the VH program. Conclusion Improved pre–post scores demonstrate acquisition of skills in BBN to a VH patient. Pretest sensitization did not appear to influence posttest assessment. These results provide initial construct validity evidence that the VH program is effective for assessing BBN performance-based communication competence. PMID:28794664

Evidence of Convergent and Discriminant Validity of Child, Teacher, and Peer Reports of Teacher-Student Support

PubMed Central

Li, Yan; Hughes, Jan N.; Kwok, Oi-man; Hsu, Hsien-Yuan

2012-01-01

This study investigated the construct validity of measures of teacher-student support in a sample of 709 ethnically diverse second and third grade academically at-risk students. Confirmatory factor analysis investigated the convergent and discriminant validities of teacher, child, and peer reports of teacher-student support and child conduct problems. Results supported the convergent and discriminant validity of scores on the measures. Peer reports accounted for the largest proportion of trait variance and non-significant method variance. Child reports accounted for the smallest proportion of trait variance and the largest method variance. A model with two latent factors provided a better fit to the data than a model with one factor, providing further evidence of the discriminant validity of measures of teacher-student support. Implications for research, policy, and practice are discussed. PMID:21767024

Validation of the GreenLight™ Simulator and development of a training curriculum for photoselective vaporisation of the prostate.

PubMed

Aydin, Abdullatif; Muir, Gordon H; Graziano, Manuela E; Khan, Muhammad Shamim; Dasgupta, Prokar; Ahmed, Kamran

2015-06-01

To assess face, content and construct validity, and feasibility and acceptability of the GreenLight™ Simulator as a training tool for photoselective vaporisation of the prostate (PVP), and to establish learning curves and develop an evidence-based training curriculum. This prospective, observational and comparative study, recruited novice (25 participants), intermediate (14) and expert-level urologists (seven) from the UK and Europe at the 28th European Association of Urological Surgeons Annual Meeting 2013. A group of novices (12 participants) performed 10 sessions of subtask training modules followed by a long operative case, whereas a second group (13) performed five sessions of a given case module. Intermediate and expert groups performed all training modules once, followed by one operative case. The outcome measures for learning curves and construct validity were time to task, coagulation time, vaporisation time, average sweep speed, average laser distance, blood loss, operative errors, and instrument cost. Face and content validity, feasibility and acceptability were addressed through a quantitative survey. Construct validity was demonstrated in two of five training modules (P = 0.038; P = 0.018) and in a considerable number of case metrics (P = 0.034). Learning curves were seen in all five training modules (P < 0.001) and significant reduction in case operative time (P < 0.001) and error (P = 0.017) were seen. An evidence-based training curriculum, to help trainees acquire transferable skills, was produced using the results. This study has shown the GreenLight Simulator to be a valid and useful training tool for PVP. It is hoped that by using the training curriculum for the GreenLight Simulator, novice trainees can acquire skills and knowledge to a predetermined level of proficiency. © 2014 The Authors. BJU International © 2014 BJU International.

The key-features approach to assess clinical decisions: validity evidence to date.

PubMed

Bordage, G; Page, G

2018-05-17

The key-features (KFs) approach to assessment was initially proposed during the First Cambridge Conference on Medical Education in 1984 as a more efficient and effective means of assessing clinical decision-making skills. Over three decades later, we conducted a comprehensive, systematic review of the validity evidence gathered since then. The evidence was compiled according to the Standards for Educational and Psychological Testing's five sources of validity evidence, namely, Content, Response process, Internal structure, Relations to other variables, and Consequences, to which we added two other types related to Cost-feasibility and Acceptability. Of the 457 publications that referred to the KFs approach between 1984 and October 2017, 164 are cited here; the remaining 293 were either redundant or the authors simply mentioned the KFs concept in relation to their work. While one set of articles reported meeting the validity standards, another set examined KFs test development choices and score interpretation. The accumulated validity evidence for the KFs approach since its inception supports the decision-making construct measured and its use to assess clinical decision-making skills at all levels of training and practice and with various types of exam formats. Recognizing that gathering validity evidence is an ongoing process, areas with limited evidence, such as item factor analyses or consequences of testing, are identified as well as new topics needing further clarification, such as the use of the KFs approach for formative assessment and its place within a program of assessment.

A Reanalysis of the Need to Achieve and Its Relationship to Education. Theoretical Paper No. 42.

ERIC Educational Resources Information Center

Marliave, Richard

A review of the literature indicates that measures of the McClelland-Atkinson need-Achievement (nAch) construct are weak in terms of both reliability and validity. The most serious weakness of the model's validity is the lack of evidence for the hypothesized positive relationship between nAch and performance. In addition, the inverse relationship…

Construct Validation of Physical Activity Surveys in Culturally Diverse Older Adults: A Comparison of Four Commonly Used Questionnaires

ERIC Educational Resources Information Center

Moore, Delilah S.; Ellis, Rebecca; Allen, Priscilla D.; Cherry, Katie E.; Monroe, Pamela A.; O'Neil, Carol E.; Wood, Robert H.

2008-01-01

The purpose of this study was to establish validity evidence of four physical activity (PA) questionnaires in culturally diverse older adults by comparing self-report PA with performance-based physical function. Participants were 54 older adults who completed the Continuous Scale Physical Functional Performance 10-item Test (CS-PFP10), Physical…

Six Years of Comprehensive, Clinical, Performance-Based Assessment Using Standardized Patients at the Southern Illinois University School of Medicine.

ERIC Educational Resources Information Center

Vu, Nu Viet; And Others

1992-01-01

The use of a performance-based assessment of senior medical students' clinical skills utilizing standardized patients was evaluated, with 6,804 student-patient encounters involving 405 students over 6 years. Results provide evidence for test security, content validity, construct validity, reliability, and test ability to discriminate a wide range…

The Reliability and Validity of the Coopersmith Self-Esteem Inventory for a Sample of Filipino High School Girls.

ERIC Educational Resources Information Center

Watkins, David; Astilla, Estela

1980-01-01

Evidence is presented partially supporting the reliability and construct validity of the Coopersmith Self-Esteem Inventory with Filipino adolescent girls. A test-retest coefficient of 0.61 was found over a nine-month period. Self-esteem scores were significantly associated with IQ scores and teacher ratings of pupils' self-esteem. (Author/BW)

Factor structure and construct validity of the Anxiety Sensitivity Index among island Puerto Ricans.

PubMed

Cintrón, Jennifer A; Carter, Michele M; Suchday, Sonia; Sbrocco, Tracy; Gray, James

2005-01-01

The factor structure and convergent and discriminant validity of the Anxiety Sensitivity Index (ASI) were examined among a sample of 275 island Puerto Ricans. Results from a confirmatory factor analysis (CFA) comparing our data to factor solutions commonly reported as representative of European American and Spanish populations indicated a poor fit. A subsequent exploratory factor analysis (EFA) indicated that a two-factor solution (Factor 1, Anxiety Sensitivity; Factor 2, Emotional Concerns) provided the best fit. Correlations between the ASI and anxiety measures were moderately high providing evidence of convergent validity, while correlations between the ASI and BDI were significantly lower providing evidence of discriminant validity. Scores on all measures were positively correlated with acculturation, suggesting that those who ascribe to more traditional Hispanic culture report elevated anxiety.

Why Do Fine Motor Skills Predict Mathematics? Construct Validity of the Design Copying Task

ERIC Educational Resources Information Center

Murrah, William M.; Chen, Wei-Bing; Cameron, Claire E.

2013-01-01

Recent educational studies have found evidence that measures of fine motor skills are predictive of educational outcomes. However, the precise nature of fine motor skills has received little attention in these studies. With evidence mounting that fine motor skills are an important indicator of school readiness, investigating the nature of this…

Measuring cognitive vulnerability to depression: Development and validation of the cognitive style questionnaire

PubMed Central

Haeffel, Gerald J.; Gibb, Brandon E.; Metalsky, Gerald I.; Alloy, Lauren B.; Abramson, Lyn Y.; Hankin, Benjamin L.; Joiner, Thomas E.; Swendsen, Joel D.

2014-01-01

The Cognitive Style Questionnaire (CSQ) measures the cognitive vulnerability factor featured in the hopelessness theory of depression. The CSQ has been used in over 30 published studies since its inception, yet detailed information about the psychometric and validity properties of this instrument has yet to be published. In this article, we describe the development of the CSQ and review reliability and validity evidence. Findings to date using college samples, indicate that the CSQ is a reliable measure of cognitive vulnerability with a high degree of construct validity. PMID:18234405

Measuring cognitive vulnerability to depression: development and validation of the cognitive style questionnaire.

PubMed

Haeffel, Gerald J; Gibb, Brandon E; Metalsky, Gerald I; Alloy, Lauren B; Abramson, Lyn Y; Hankin, Benjamin L; Joiner, Thomas E; Swendsen, Joel D

2008-06-01

The Cognitive Style Questionnaire (CSQ) measures the cognitive vulnerability factor featured in the hopelessness theory of depression. The CSQ has been used in over 30 published studies since its inception, yet detailed information about the psychometric and validity properties of this instrument has yet to be published. In this article, we describe the development of the CSQ and review reliability and validity evidence. Findings to date using college samples, indicate that the CSQ is a reliable measure of cognitive vulnerability with a high degree of construct validity.

Absorption in Sport: A Cross-Validation Study

PubMed Central

Koehn, Stefan; Stavrou, Nektarios A. M.; Cogley, Jeremy; Morris, Tony; Mosek, Erez; Watt, Anthony P.

2017-01-01

Absorption has been identified as readiness for experiences of deep involvement in the task. Conceptually, absorption is a key psychological construct, incorporating experiential, cognitive, and motivational components. Although, no operationalization of the construct has been provided to facilitate research in this area, the purpose of this research was the development and examination of the psychometric properties of a sport-specific measure of absorption that evolved from the use of the modified Tellegen Absorption Scale (MODTAS; Jamieson, 2005) in mainstream psychology. The study aimed to provide evidence of the psychometric properties, reliability, and validity of the Measure of Absorption in Sport Contexts (MASCs). The psychometric examination included a calibration sample from Scotland and a cross-validation sample from Australia using a cross-sectional design. The item pool was developed based on existing items from the modified Tellegen Absorption Scale (Jamieson, 2005). The MODTAS items were reworded and translated into a sport context. The Scottish sample consisted of 292 participants and the Australian sample of 314 participants. Congeneric model testing and confirmatory factor analysis for both samples and multi-group invariance testing across samples was used. In the cross-validation sample the MASC subscales showed acceptable internal consistency and construct reliability (≥0.70). Excellent fit indices were found for the final 18-item, six-factor measure in the cross-validation sample, χ(120)2 = 197.486, p < 0.001; CFI = 0.957; TLI = 0.945; RMSEA = 0.045; SRMR = 0.044. Multi-group invariance testing revealed no differences in item meaning, except for two items. The MASC and the Dispositional Flow Scale-2 showed moderate-to-strong positive correlations in both samples, r = 0.38, p < 0.001 and r = 0.42, p < 0.001, supporting the external validity of the MASC. This article provides initial evidence in support of the psychometric properties, reliability, and validity of the sport-specific measure of absorption. The MASC provides rich research opportunities in sport psychology that can enhance the theoretical understanding between absorption and related constructs and facilitate future intervention studies. PMID:28883802

The construct validity of HPAT-Ireland for the selection of medical students: unresolved issues and future research implications.

PubMed

Kelly, Maureen E; O'Flynn, Siun

2017-05-01

Aptitude tests are widely used in selection. However, despite certain advantages their use remains controversial. This paper aims to critically appraise five sources of evidence for the construct validity of the Health Professions Admission Test (HPAT)-Ireland, an aptitude test used for selecting undergraduate medical students. The objectives are to identify gaps in the evidence, draw comparisons with other aptitude tests and outline future research directions. Our appraisal of the literature found that stakeholder feedback indicates that there is reasonable evidence for test content validity for two of the three sections of HPAT-Ireland. By contrast the Non-Verbal Reasoning section is widely criticised as having limited relevance to medical school performance and future clinical practice. In terms of concurrent validity there is a significant small to medium, negative correlation with school exit examinations, but not consistently so across all studies (r = -0.18, -0.28, 0.017). Likewise predictive validity studies vary, from negative to moderate strength correlations with examination performance during early years at medical school. Five studies indicate that HPAT-Ireland is supported in principle by the majority of stakeholders. While one consequence of its introduction is that successful applicants are now coming from more diverse academic backgrounds, there is no evidence that the socio-economic background of medical school entrants has been altered significantly. Negative perceptions of unfairness relating to gender, coaching and socio-economics remain. The evidence to date suggests that while there are slight gender differences, initially favouring males, these vary year on year. In conclusion, the attitudes towards, and performance of, HPAT-Ireland is not unlike that of other aptitude tests widely used internationally. The main justifications for its introduction have been achieved, in that Ireland no longer relies exclusively on a single measure of academic record for selection to medical school. However a number of areas require further research and exploration.

Improving Escalation of Care: Development and Validation of the Quality of Information Transfer Tool.

PubMed

Johnston, Maximilian J; Arora, Sonal; Pucher, Philip H; Reissis, Yannis; Hull, Louise; Huddy, Jeremy R; King, Dominic; Darzi, Ara

2016-03-01

To develop and provide validity and feasibility evidence for the QUality of Information Transfer (QUIT) tool. Prompt escalation of care in the setting of patient deterioration can prevent further harm. Escalation and information transfer skills are not currently measured in surgery. This study comprised 3 phases: the development (phase 1), validation (phase 2), and feasibility analysis (phase 3) of the QUIT tool. Phase 1 involved identification of core skills needed for successful escalation of care through literature review and 33 semistructured interviews with stakeholders. Phase 2 involved the generation of validity evidence for the tool using a simulated setting. Thirty surgeons assessed a deteriorating postoperative patient in a simulated ward and escalated their care to a senior colleague. The face and content validity were assessed using a survey. Construct and concurrent validity of the tool were determined by comparing performance scores using the QUIT tool with those measured using the Situation-Background-Assessment-Recommendation (SBAR) tool. Phase 3 was conducted using direct observation of escalation scenarios on surgical wards in 2 hospitals. A 7-category assessment tool was developed from phase 1 consisting of 24 items. Twenty-one of 24 items had excellent content validity (content validity index >0.8). All 7 categories and 18 of 24 (P < 0.05) items demonstrated construct validity. The correlation between the QUIT and SBAR tools used was strong indicating concurrent validity (r = 0.694, P < 0.001). Real-time scoring of escalation referrals was feasible and indicated that doctors currently have better information transfer skills than nurses when faced with a deteriorating patient. A validated tool to assess information transfer for deteriorating surgical patients was developed and tested using simulation and real-time clinical scenarios. It may improve the quality and safety of patient care on the surgical ward.

Construct validity of the MMPI-2 College Maladjustment (Mt) Scale.

PubMed

Barthlow, Deanna L; Graham, John R; Ben-Porath, Yossef S; McNulty, John L

2004-09-01

The construct validity of the MMPI-2 (Minnesota Multiphasic Personality Inventory-2) College Maladjustment (Mt) Scale was examined using 376 student clients at a university psychological clinic. A principal components analysis and correlations of Mt scale scores with clients' and therapists' ratings of symptoms and functioning showed that the Mt scale identifies the presence of maladjustment as defined in terms of depressive and anxious symptoms. There is no evidence to show that the scale is specific to college students or that it is sensitive to severe psychological disturbance. The Mt scale does not inform the clinician as to why a person is distressed. In addition, there is no evidence from this study to suggest the superiority of the Mt scale over other MMPI-2 maladjustment measures. Therapists should use the entire MMPI-2 profile, not just the Mt scale, to gain the most comprehensive and specific understanding of clients.

[Reliability and construct validity of an instrument to asses the Self-perception of Family Health Status].

PubMed

Lima Rodríguez, Joaquín Salvador; Lima Serrano, Marta; Jiménez Picón, Nerea; Domínguez Sánchez, Isabel

2012-10-01

Family health determines and it is determined by family´s capacity to function effectively as a biosocial unit in a given culture and society. The main of study has been to test reliability and construct validity of an instrument to asses the Self-perception of Family Health Status. We validated its content by an on-line Dephi panel with experts. We surveyed 258 families in them homes or in primary health centres from Seville, Spain. We administered the instrument that has five Likert scales: Family climate, Family integrity, Family functioning, and Family resistance. We tested reliability by Cronbach Alpha and construct validity by exploratory factor analysis. The five scales obtained values α between 0.73 for the Family Climate and 0.89 for Family Integrity. They showed evidence of one-dimensional interpretation after factor analysis, a) all items got weights r>0.30 in first factor before rotations, b) the first factor explained a significant proportion of variance before rotations, and c) the total variance explained by the main factors extracted was greater than 50%. The scales showed their reliability and validity. They could be employed to assess the self-perception of family health status.

The behavioral regulation in sport questionnaire (BRSQ): instrument development and initial validity evidence.

PubMed

Lonsdale, Chris; Hodge, Ken; Rose, Elaine A

2008-06-01

The purpose of the four studies described in this article was to develop and test a new measure of competitive sport participants' intrinsic motivation, extrinsic motivation, and amotivation (self-determination theory; Deci & Ryan, 1985). The items for the new measure, named the Behavioral Regulation in Sport Questionnaire (BRSQ), were constructed using interviews, expert review, and pilot testing. Analyses supported the internal consistency, test-retest reliability, and factorial validity of the BRSQ scores. Nomological validity evidence was also supportive, as BRSQ subscale scores were correlated in the expected pattern with scores derived from measures of motivational consequences. When directly compared with scores derived from the Sport Motivation Scale (SMS; Pelletier, Fortier, Vallerand, Tuson, & Blais, 1995) and a revised version of that questionnaire (SMS-6; Mallett, Kawabata, Newcombe, Otero-Forero, & Jackson, 2007), BRSQ scores demonstrated equal or superior reliability and factorial validity as well as better nomological validity.

The Career Locus of Control Scale for Adolescents: Further Evidence of Validity in the United States

ERIC Educational Resources Information Center

Perry, Justin C.; Liu, Xiongyi; Griffin, Grant C.

2011-01-01

This study examined the construct validity of the Career Locus of Control Scale (CLCS) among diverse urban youth within the United States (N = 308). Confirmatory factor analyses verified two of the three models as acceptable fits. Two new models were also explored. Model 5 (Internality, Luck, and Non-Control), which was one of the new models, was…

The Teacher Sense of Efficacy Scale: Validation Evidence and Behavioral Prediction. WCER Working Paper No. 2006-7

ERIC Educational Resources Information Center

Heneman, Herbert G., III; Kimball, Steven; Milanowski, Anthony

2006-01-01

The present study contributes to knowledge of the construct validity of the short form of the Teacher Sense of Efficacy Scale (and by extension, given their similar content and psychometric properties, to the long form). The authors' research involves: (1) examining the psychometric properties of the TSES on a large sample of elementary, middle,…

Individual safety performance in the construction industry: development and validation of two short scales.

PubMed

DeArmond, Sarah; Smith, April E; Wilson, Christina L; Chen, Peter Y; Cigularov, Konstantin P

2011-05-01

In the current research a short measure of safety performance is developed for use in the construction industry and the relationships between different components of safety performance and safety outcomes (e.g., occupational injuries and work-related pain) are explored within the construction context. This research consists of two field studies. In the first, comprehensive measures of safety compliance and safety participation were shortened and modified to be appropriate for use in construction. Evidence of reliability and validity is provided. Both safety compliance and safety participation were negatively related to occupational injuries, yet these two correlations were not statistically different. In the second study, we investigated the relationships between these two components of safety performance and work-related pain frequency, in addition to replicating Study 1. Safety compliance had a stronger negative relationship with pain than safety participation. Implications for research are discussed. Copyright © 2010 Elsevier Ltd. All rights reserved.

The Illusory Beliefs Inventory: a new measure of magical thinking and its relationship with obsessive compulsive disorder.

PubMed

Kingdon, Bianca L; Egan, Sarah J; Rees, Clare S

2012-01-01

Magical thinking has been proposed to have an aetiological role in obsessive compulsive disorder (OCD). To address the limitations of existing measures of magical thinking we developed and validated a new 24-item measure of magical thinking, the Illusory Beliefs Inventory (IBI). The validation sample comprised a total of 1194 individuals across two samples recruited via an Internet based survey. Factor analysis identified three subscales representing domains relevant to the construct of magical thinking: Magical Beliefs, Spirituality, and Internal State and Thought Action Fusion. The scale had excellent internal consistency and evidence of convergent and discriminant validity. Evidence of criterion-related concurrent validity confirmed that magical thinking is a cognitive domain associated with OCD and is largely relevant to neutralizing, obsessing and hoarding symptoms. It is important for future studies to extend the evidence of the psychometric properties of the IBI in new populations and to conduct longitudinal studies to examine the aetiological role of magical thinking.

Validity of Factors of the Psychopathy Checklist–Revised in Female Prisoners

PubMed Central

Kennealy, Patrick J.; Hicks, Brian M.; Patrick, Christopher J.

2008-01-01

The validity of the Psychopathy Checklist–Revised (PCL-R) has been examined extensively in men, but its validity for women remains understudied. Specifically, the correlates of the general construct of psychopathy and its components as assessed by PCL-R total, factor, and facet scores have yet to be examined in depth. Based on previous research conducted with male offenders, a large female inmate sample was used to examine the patterns of relations between total, factor, and facet scores on the PCL-R and various criterion variables. These variables include ratings of psychopathy based on Cleckley’s criteria, symptoms of antisocial personality disorder, and measures of substance use and abuse, criminal behavior, institutional misconduct, interpersonal aggression, normal range personality, intellectual functioning, and social background variables. Results were highly consistent with past findings in male samples and provide further evidence for the construct validity of the PCL-R two-factor and four-facet models across genders. PMID:17986651

Reliability and Validity of Instruments for Assessing Perinatal Depression in African Settings: Systematic Review and Meta-Analysis

PubMed Central

Tsai, Alexander C.; Scott, Jennifer A.; Hung, Kristin J.; Zhu, Jennifer Q.; Matthews, Lynn T.; Psaros, Christina; Tomlinson, Mark

2013-01-01

Background A major barrier to improving perinatal mental health in Africa is the lack of locally validated tools for identifying probable cases of perinatal depression or for measuring changes in depression symptom severity. We systematically reviewed the evidence on the reliability and validity of instruments to assess perinatal depression in African settings. Methods and Findings Of 1,027 records identified through searching 7 electronic databases, we reviewed 126 full-text reports. We included 25 unique studies, which were disseminated in 26 journal articles and 1 doctoral dissertation. These enrolled 12,544 women living in nine different North and sub-Saharan African countries. Only three studies (12%) used instruments developed specifically for use in a given cultural setting. Most studies provided evidence of criterion-related validity (20 [80%]) or reliability (15 [60%]), while fewer studies provided evidence of construct validity, content validity, or internal structure. The Edinburgh postnatal depression scale (EPDS), assessed in 16 studies (64%), was the most frequently used instrument in our sample. Ten studies estimated the internal consistency of the EPDS (median estimated coefficient alpha, 0.84; interquartile range, 0.71-0.87). For the 14 studies that estimated sensitivity and specificity for the EPDS, we constructed 2 x 2 tables for each cut-off score. Using a bivariate random-effects model, we estimated a pooled sensitivity of 0.94 (95% confidence interval [CI], 0.68-0.99) and a pooled specificity of 0.77 (95% CI, 0.59-0.88) at a cut-off score of ≥9, with higher cut-off scores yielding greater specificity at the cost of lower sensitivity. Conclusions The EPDS can reliably and validly measure perinatal depression symptom severity or screen for probable postnatal depression in African countries, but more validation studies on other instruments are needed. In addition, more qualitative research is needed to adequately characterize local understandings of perinatal depression-like syndromes in different African contexts. PMID:24340036

Measuring striving for understanding and learning value of geometry: a validity study

NASA Astrophysics Data System (ADS)

Ubuz, Behiye; Aydınyer, Yurdagül

2017-11-01

The current study aimed to construct a questionnaire that measures students' personality traits related to striving for understanding and learning value of geometry and then examine its psychometric properties. Through the use of multiple methods on two independent samples of 402 and 521 middle school students, two studies were performed to address this issue to provide support for its validity. In Study 1, exploratory factor analysis indicated the two-factor model. In Study 2, confirmatory factor analysis indicated the better fit of two-factor model compared to one or three-factor model. Convergent and discriminant validity evidence provided insight into the distinctiveness of the two factors. Subgroup validity evidence revealed gender differences for striving for understanding geometry trait favouring girls and grade level differences for learning value of geometry trait favouring the sixth- and seventh-grade students. Predictive validity evidence demonstrated that the striving for understanding geometry trait but not learning value of geometry trait was significantly correlated with prior mathematics achievement. In both studies, each factor and the entire questionnaire showed satisfactory reliability. In conclusion, the questionnaire was psychometrically sound.

Construct Validation of the Physics Metacognition Inventory

NASA Astrophysics Data System (ADS)

Taasoobshirazi, Gita; Farley, John

2013-02-01

The 24-item Physics Metacognition Inventory was developed to measure physics students' metacognition for problem solving. Items were classified into eight subcomponents subsumed under two broader components: knowledge of cognition and regulation of cognition. The students' scores on the inventory were found to be reliable and related to students' physics motivation and physics grade. An exploratory factor analysis provided evidence of construct validity, revealing six components of students' metacognition when solving physics problems including: knowledge of cognition, planning, monitoring, evaluation, debugging, and information management. Although women and men differed on the components, they had equivalent overall metacognition for problem solving. The implications of these findings for future research are discussed.

Development and Validation of a Multidimensional Measure of Family Supportive Supervisor Behaviors (FSSB)

PubMed Central

Hammer, Leslie B.; Kossek, Ellen Ernst; Yragui, Nanette L.; Bodner, Todd E.; Hanson, Ginger C.

2011-01-01

Due to growing work-family demands, supervisors need to effectively exhibit family supportive supervisor behaviors (FSSB). Drawing on social support theory and using data from two samples of lower wage workers, the authors develop and validate a measure of FSSB, defined as behaviors exhibited by supervisors that are supportive of families. FSSB is conceptualized as a multidimensional superordinate construct with four subordinate dimensions: emotional support, instrumental support, role modeling behaviors, and creative work-family management. Results from multilevel confirmatory factor analyses and multilevel regression analyses provide evidence of construct, criterion-related, and incremental validity. The authors found FSSB to be significantly related to work-family conflict, work-family positive spillover, job satisfaction, and turnover intentions over and above measures of general supervisor support. PMID:21660254

Convergent and divergent validity of the Mullen Scales of Early Learning in young children with and without autism spectrum disorder.

PubMed

Swineford, Lauren B; Guthrie, Whitney; Thurm, Audrey

2015-12-01

The purpose of this study was to report on the construct, convergent, and divergent validity of the Mullen Scales of Early Learning (MSEL), a widely used test of development for young children. The sample consisted of 399 children with a mean age of 3.38 years (SD = 1.14) divided into a group of children with autism spectrum disorder (ASD) and a group of children not on the autism spectrum, with and without developmental delays. The study used the MSEL and several other measures assessing constructs relevant to the age range--including developmental skills, autism symptoms, and psychopathology symptoms--across multiple methods of assessment. Multiple-group confirmatory factor analyses revealed good overall fit and equal form of the MSEL 1-factor model across the ASD and nonspectrum groups, supporting the construct validity of the MSEL. However, neither full nor partial invariance of factor loadings was established because of the lower loadings in the ASD group compared with the nonspectrum group. Exploratory structural equation modeling revealed that other measures of developmental skills loaded together with the MSEL domain scores on a Developmental Functioning factor, supporting convergent validity of the MSEL. Divergent validity was supported by the lack of loading of MSEL domain scores on Autism Symptoms or Emotion/Behavior Problems factors. Although factor structure and loadings varied across groups, convergent and divergent validity findings were similar in the ASD and nonspectrum samples. Together, these results demonstrate evidence for the construct, convergent, and divergent validity of the MSEL using powerful data-analytic techniques. (c) 2015 APA, all rights reserved).

Applying modern psychometric techniques to melodic discrimination testing: Item response theory, computerised adaptive testing, and automatic item generation.

PubMed

Harrison, Peter M C; Collins, Tom; Müllensiefen, Daniel

2017-06-15

Modern psychometric theory provides many useful tools for ability testing, such as item response theory, computerised adaptive testing, and automatic item generation. However, these techniques have yet to be integrated into mainstream psychological practice. This is unfortunate, because modern psychometric techniques can bring many benefits, including sophisticated reliability measures, improved construct validity, avoidance of exposure effects, and improved efficiency. In the present research we therefore use these techniques to develop a new test of a well-studied psychological capacity: melodic discrimination, the ability to detect differences between melodies. We calibrate and validate this test in a series of studies. Studies 1 and 2 respectively calibrate and validate an initial test version, while Studies 3 and 4 calibrate and validate an updated test version incorporating additional easy items. The results support the new test's viability, with evidence for strong reliability and construct validity. We discuss how these modern psychometric techniques may also be profitably applied to other areas of music psychology and psychological science in general.

Validation of the Work-Life Balance Culture Scale (WLBCS).

PubMed

Nitzsche, Anika; Jung, Julia; Kowalski, Christoph; Pfaff, Holger

2014-01-01

The purpose of this paper is to describe the theoretical development and initial validation of the newly developed Work-Life Balance Culture Scale (WLBCS), an instrument for measuring an organizational culture that promotes the work-life balance of employees. In Study 1 (N=498), the scale was developed and its factorial validity tested through exploratory factor analyses. In Study 2 (N=513), confirmatory factor analysis (CFA) was performed to examine model fit and retest the dimensional structure of the instrument. To assess construct validity, a priori hypotheses were formulated and subsequently tested using correlation analyses. Exploratory and confirmatory factor analyses revealed a one-factor model. Results of the bivariate correlation analyses may be interpreted as preliminary evidence of the scale's construct validity. The five-item WLBCS is a new and efficient instrument with good overall quality. Its conciseness makes it particularly suitable for use in employee surveys to gain initial insight into a company's perceived work-life balance culture.

A Chimpanzee (Pan troglodytes) Model of Triarchic Psychopathy Constructs: Development and Initial Validation

PubMed Central

Latzman, Robert D.; Drislane, Laura E.; Hecht, Lisa K.; Brislin, Sarah J.; Patrick, Christopher J.; Lilienfeld, Scott O.; Freeman, Hani J.; Schapiro, Steven J.; Hopkins, William D.

2015-01-01

The current work sought to operationalize constructs of the triarchic model of psychopathy in chimpanzees (Pan troglodytes), a species well-suited for investigations of basic biobehavioral dispositions relevant to psychopathology. Across three studies, we generated validity evidence for scale measures of the triarchic model constructs in a large sample (N=238) of socially-housed chimpanzees. Using a consensus-based rating approach, we first identified candidate items for the chimpanzee triarchic (CHMP-Tri) scales from an existing primate personality instrument and refined these into scales. In Study 2, we collected data for these scales from human informants (N=301), and examined their convergent and divergent relations with scales from another triarchic inventory developed for human use. In Study 3, we undertook validation work examining associations between CHMP-Tri scales and task measures of approach-avoidance behavior (N=73) and ability to delay gratification (N=55). Current findings provide support for a chimpanzee model of core dispositions relevant to psychopathy and other forms of psychopathology. PMID:26779396

An Integrative Analysis of the Narcissistic Personality Inventory and the Hypomanic Personality Scale: Implications for Construct Validity.

PubMed

Stanton, Kasey; Daly, Elizabeth; Stasik-O'Brien, Sara M; Ellickson-Larew, Stephanie; Clark, Lee Anna; Watson, David

2017-09-01

The primary goal of this study was to explicate the construct validity of the Narcissistic Personality Inventory (NPI) and the Hypomanic Personality Scale (HPS) by examining their relations both to each other and to measures of personality and psychopathology in a community sample ( N = 255). Structural evidence indicates that the NPI is defined by Leadership/Authority, Grandiose Exhibitionism, and Entitlement/Exploitativeness factors, whereas the HPS is characterized by specific dimensions reflecting Social Vitality, Mood Volatility, and Excitement. Our results establish that (a) factor-based subscales from these instruments display divergent patterns of relations that are obscured when relying exclusively on total scores and (b) some NPI and HPS subscales more clearly tap content specifically relevant to narcissism and mania, respectively, than others. In particular, our findings challenge the construct validity of the NPI Leadership/Authority and HPS Social Vitality subscales, which appear to assess overlapping assertiveness content that is largely adaptive in nature.

Anticipatory Traumatic Reaction: Outcomes Arising From Secondary Exposure to Disasters and Large-Scale Threats.

PubMed

Hopwood, Tanya L; Schutte, Nicola S; Loi, Natasha M

2017-09-01

Two studies, with a total of 707 participants, developed and examined the reliability and validity of a measure for anticipatory traumatic reaction (ATR), a novel construct describing a form of distress that may occur in response to threat-related media reports and discussions. Exploratory and confirmatory factor analysis resulted in a scale comprising three subscales: feelings related to future threat; preparatory thoughts and actions; and disruption to daily activities. Internal consistency was .93 for the overall ATR scale. The ATR scale demonstrated convergent validity through associations with negative affect, depression, anxiety, stress, neuroticism, and repetitive negative thinking. The scale showed discriminant validity in relationships to Big Five characteristics. The ATR scale had some overlap with a measure of posttraumatic stress disorder, but also showed substantial separate variance. This research provides preliminary evidence for the novel construct of ATR as well as a measure of the construct. The ATR scale will allow researchers to further investigate anticipatory traumatic reaction in the fields of trauma, clinical practice, and social psychology.

Simulation-based training for prostate surgery.

PubMed

Khan, Raheej; Aydin, Abdullatif; Khan, Muhammad Shamim; Dasgupta, Prokar; Ahmed, Kamran

2015-10-01

To identify and review the currently available simulators for prostate surgery and to explore the evidence supporting their validity for training purposes. A review of the literature between 1999 and 2014 was performed. The search terms included a combination of urology, prostate surgery, robotic prostatectomy, laparoscopic prostatectomy, transurethral resection of the prostate (TURP), simulation, virtual reality, animal model, human cadavers, training, assessment, technical skills, validation and learning curves. Furthermore, relevant abstracts from the American Urological Association, European Association of Urology, British Association of Urological Surgeons and World Congress of Endourology meetings, between 1999 and 2013, were included. Only studies related to prostate surgery simulators were included; studies regarding other urological simulators were excluded. A total of 22 studies that carried out a validation study were identified. Five validated models and/or simulators were identified for TURP, one for photoselective vaporisation of the prostate, two for holmium enucleation of the prostate, three for laparoscopic radical prostatectomy (LRP) and four for robot-assisted surgery. Of the TURP simulators, all five have demonstrated content validity, three face validity and four construct validity. The GreenLight laser simulator has demonstrated face, content and construct validities. The Kansai HoLEP Simulator has demonstrated face and content validity whilst the UroSim HoLEP Simulator has demonstrated face, content and construct validity. All three animal models for LRP have been shown to have construct validity whilst the chicken skin model was also content valid. Only two robotic simulators were identified with relevance to robot-assisted laparoscopic prostatectomy, both of which demonstrated construct validity. A wide range of different simulators are available for prostate surgery, including synthetic bench models, virtual-reality platforms, animal models, human cadavers, distributed simulation and advanced training programmes and modules. The currently validated simulators can be used by healthcare organisations to provide supplementary training sessions for trainee surgeons. Further research should be conducted to validate simulated environments, to determine which simulators have greater efficacy than others and to assess the cost-effectiveness of the simulators and the transferability of skills learnt. With surgeons investigating new possibilities for easily reproducible and valid methods of training, simulation offers great scope for implementation alongside traditional methods of training. © 2014 The Authors BJU International © 2014 BJU International Published by John Wiley & Sons Ltd.

Examining construct validity of a new naturalistic observational assessment of hand skills for preschool- and school-age children.

PubMed

Chien, Chi-Wen; Brown, Ted; McDonald, Rachael

2012-04-01

The Assessment of Children's Hand Skills is a new assessment that utilises a naturalistic observational method to capture children's real-life hand skill performance when engaged at various types of daily activities in everyday living contexts. The Assessment of Children's Hand Skills is designed for use with 2- to 12-year-old children with a range of disabilities or health conditions. The study aimed to investigate construct validity of the Assessment of Children's Hand Skills in Australian children. Rasch analysis was used to examine internal construct validity of the Assessment of Children's Hand Skills in a mixed sample of 53 children with disabilities (including autism spectrum disorder, developmental/genetic disorders and physical disabilities) and 85 typically developing children. External construct validity was examined by correlating with three questionnaires evaluating daily living skills and hand skills. Rasch goodness-of-fit analysis suggested that all 22 activity items and 19 of 20 hand skill items in the Assessment of Children's Hand Skills measured a single construct. The Assessment of Children's Hand Skills items were placed in a clinically meaningful hierarchy from easy to hard, and the difficulty range of the items also matched the majority of children with disabilities and typically developing preschool-aged children. Moderate to high correlations (0.59 ≤ Spearman's ρ coefficients ≤ 0.89, P < 0.01) were found with the assessments of daily living and fine motor skills. This study provided preliminary evidence supporting the construct validity of the Assessment of Children's Hand Skills for its clinical application in assessing children's real-life hand skill performance in Australian contexts. © 2012 The Authors Australian Occupational Therapy Journal © 2012 Occupational Therapy Australia.

Effect of response format on cognitive reflection: Validating a two- and four-option multiple choice question version of the Cognitive Reflection Test.

PubMed

Sirota, Miroslav; Juanchich, Marie

2018-03-27

The Cognitive Reflection Test, measuring intuition inhibition and cognitive reflection, has become extremely popular because it reliably predicts reasoning performance, decision-making, and beliefs. Across studies, the response format of CRT items sometimes differs, based on the assumed construct equivalence of tests with open-ended versus multiple-choice items (the equivalence hypothesis). Evidence and theoretical reasons, however, suggest that the cognitive processes measured by these response formats and their associated performances might differ (the nonequivalence hypothesis). We tested the two hypotheses experimentally by assessing the performance in tests with different response formats and by comparing their predictive and construct validity. In a between-subjects experiment (n = 452), participants answered stem-equivalent CRT items in an open-ended, a two-option, or a four-option response format and then completed tasks on belief bias, denominator neglect, and paranormal beliefs (benchmark indicators of predictive validity), as well as on actively open-minded thinking and numeracy (benchmark indicators of construct validity). We found no significant differences between the three response formats in the numbers of correct responses, the numbers of intuitive responses (with the exception of the two-option version, which had a higher number than the other tests), and the correlational patterns of the indicators of predictive and construct validity. All three test versions were similarly reliable, but the multiple-choice formats were completed more quickly. We speculate that the specific nature of the CRT items helps build construct equivalence among the different response formats. We recommend using the validated multiple-choice version of the CRT presented here, particularly the four-option CRT, for practical and methodological reasons. Supplementary materials and data are available at https://osf.io/mzhyc/ .

Preliminary Validity of the Eyberg Child Behavior Inventory With Filipino Immigrant Parents

PubMed Central

Coffey, Dean M.; Javier, Joyce R.; Schrager, Sheree M.

2016-01-01

Filipinos are an understudied minority affected by significant behavioral health disparities. We evaluate evidence for the reliability, construct validity, and convergent validity of the Eyberg Child Behavior Inventory (ECBI) in 6- to 12- year old Filipino children (N = 23). ECBI scores demonstrated high internal consistency, supporting a single-factor model (pre-intervention α =.91; post-intervention α =.95). Results document convergent validity with the Child Behavior Checklist Externalizing scale at pretest (r = .54, p < .01) and posttest (r = .71, p < .001). We conclude that the ECBI is a promising tool to measure behavior problems in Filipino children. PMID:27087739

Preliminary Validity of the Eyberg Child Behavior Inventory With Filipino Immigrant Parents.

PubMed

Coffey, Dean M; Javier, Joyce R; Schrager, Sheree M

Filipinos are an understudied minority affected by significant behavioral health disparities. We evaluate evidence for the reliability, construct validity, and convergent validity of the Eyberg Child Behavior Inventory (ECBI) in 6- to 12- year old Filipino children ( N = 23). ECBI scores demonstrated high internal consistency, supporting a single-factor model (pre-intervention α =.91; post-intervention α =.95). Results document convergent validity with the Child Behavior Checklist Externalizing scale at pretest ( r = .54, p < .01) and posttest ( r = .71, p < .001). We conclude that the ECBI is a promising tool to measure behavior problems in Filipino children.

Validation of Using Fitness Center Attendance Electronic Records to Assess the Frequency of Moderate/Vigorous Leisure-Time Physical Activity among Adults

ERIC Educational Resources Information Center

Amireault, Steve; Godin, Gaston

2014-01-01

The purpose of this study was to provide three construct validity evidence for using fitness center attendance electronic records to objectively assess the frequency of leisure-time physical activity among adults. One hundred members of a fitness center (45 women and 55 men; aged 18 to 64 years) completed a self-report leisure-time physical…

Trauma Coping Self-Efficacy: A Context Specific Self-Efficacy Measure for Traumatic Stress

PubMed Central

Benight, Charles C.; Shoji, Kotaro; James, Lori E.; Waldrep, Edward E.; Delahanty, Douglas L.; Cieslak, Roman

2015-01-01

The psychometric properties of a Trauma Coping Self-Efficacy (CSE-T) scale that assesses general trauma-related coping self-efficacy perceptions were assessed. Measurement equivalence was assessed using several different samples: hospitalized trauma patients (n1 = 74, n2 = 69, n3 = 60), three samples of disaster survivors (n1 = 273, n2 = 227, n3 = 138), and trauma exposed college students (N = 242). This is the first multi-sample evaluation of the psychometric properties for a general trauma-related CSE measure. Results showed that a brief and parsimonious 9-item version of the CSE performed well across the samples with a robust factor structure; factor structure and factor loadings were similar across study samples. The 9-item scale CSE-T demonstrated measurement equivalence across samples indicating that the underlying concept of general post-traumatic CSE is organized in a similar manner in the different trauma-exposed groups. These results offer strong support for cross-event construct validity of the CSE-T scale. Associations of the CSE-T with important expected covariates showed significant evidence for convergent validity. Finally, discriminant validity was also supported. Replication of the factor structure, internal reliability, and other evidence for construct validity is a critical next step for future research. PMID:26524542

Validity evidence for the adaptation of the State Mindfulness Scale for Physical Activity (SMS-PA) in Spanish youth.

PubMed

Ullrich-French, Sarah; González Hernández, Juan; Hidalgo Montesinos, María D

2017-02-01

Mindfulness is an increasingly popular construct with promise in enhancing multiple positive health outcomes. Physical activity is an important behavior for enhancing overall health, but no Spanish language scale exists to test how mindfulness during physical activity may facilitate physical activity motivation or behavior. This study examined the validity of a Spanish adaption of a new scale, the State Mindfulness Scale for Physical Activity, to assess mindfulness during a specific experience of physical activity. Spanish youths (N = 502) completed a cross-sectional survey of state mindfulness during physical activity and physical activity motivation regulations based on Self-Determination Theory. A high-order model fit the data well and supports the use of one general state mindfulness factor or the use of separate subscales of mindfulness of mental (e.g., thoughts, emotions) and body (physical movement, muscles) aspects of the experience. Internal consistency reliability was good for the general scale and both sub-scales. The pattern of correlations with motivation regulations provides further support for construct validity with significant and positive correlations with self-determined forms of motivation and significant and negative correlations with external regulation and amotivation. Initial validity evidence is promising for the use of the adapted measure.

The development and validation of an instrument to measure the quality of health research reports in the lay media.

PubMed

Zeraatkar, Dena; Obeda, Michael; Ginsberg, Jeffrey S; Hirsh, Jack

2017-04-20

The media serves as an important link between medical research, as reported in scholarly sources, and the public and has the potential to act as a powerful tool to improve public health. However, concerns about the reliability of health research reports have been raised. Tools to monitor the quality of health research reporting in the media are needed to identify areas of weakness in health research reporting and to subsequently work towards the efficient use of the lay media as a public health tool through which the public's health behaviors can be improved. We developed the Quality Index for health-related Media Reports (QIMR) as a tool to monitor the quality of health research reports in the lay media. The tool was developed according to themes generated from interviews with health journalists and researchers. Item and domain characteristics and scale reliability were assessed. The scale was correlated with a global quality assessment score and media report word count to provide evidence towards its construct validity. The items and domains of the QIMR demonstrated acceptable validity and reliability. Items from the 'validity' domain were negatively skewed, suggesting possible floor effect. These items were not eliminated due to acceptable content and face validity. QIMR total scores produced a strong correlation with raters' global assessment and a moderate correlation with media report word count, providing evidence towards the construct validity of the instrument. The results of this investigation indicate that QIMR can adequately measure the quality of health research reports, with acceptable reliability and validity.

The Need for Instructional Sensitivity and Construct Clarity in Pact: A Commentary on "Examining the Internal Structure Evidence for the Performance Assessment for California Teachers"

ERIC Educational Resources Information Center

Wilkerson, Judy R.

2015-01-01

This commentary on the article titled "Examining the Internal Structure Evidence for the Performance Assessment for California Teachers: A Validation Study of the Elementary Literacy Teaching Event for Tier 1 Teacher Licensure" provides an overview of Performance Assessment for California Teachers (PACT), its relationship to edTPA and…

Validity of CBCL-derived PTSD and dissociation scales: further evidence in a sample of neglected children and adolescents.

PubMed

Milot, Tristan; Plamondon, André; Ethier, Louise S; Lemelin, Jean-Pascal; St-Laurent, Diane; Rousseau, Michel

2013-05-01

There is growing evidence that child neglect is an important risk factor for posttraumatic stress disorder (PTSD) and dissociation. Considering that the Child Behavior Checklist (CBCL) is a widely used measure, the possibility of using validated CBCL-derived trauma symptoms scales could be particularly useful to better understand how trauma symptoms develop among neglected children and adolescents. This study examined the factor structure of three CBCL-derived measures of PTSD and dissociation (namely, PTSD scale, Dissociation scale, and PTSD/Dissociation scale) in a sample of 239 neglected children and adolescents aged 6 to 18 years using the latest version of CBCL (CBCL 6-18). Evidence of convergent validity of these scales was also examined for participants aged 12 and under using two well-validated measures of PTSD and Dissociation: the Trauma Symptoms Checklist for Young Children and the Child Dissociation Checklist. Findings suggest that CBCL-derived measures of trauma symptoms, especially PTSD and Dissociations scales, may be of heuristic value in the study of trauma symptomatology in neglected samples. Factor structure and evidence of convergent validity were supported for these two scales. Results also provide further support to the well-established assumption that PTSD and dissociation are two related but different constructs.

Crafting practice guidelines in the world of evidence-based medicine.

PubMed

Chung, Kevin C; Shauver, Melissa J

2009-10-01

In the era of exponential increase in the medical literature, physicians and health policy-makers are relying on well-constructed, evidence-based practice guidelines to help ensure that the care given to patients is based on valid, scientific data. The construction of practice guidelines, however, may not always adhere to accepted research protocol. In this article, the authors detail the steps required to produce effective, evidence-based practice guidelines. The seven essential steps in crafting a practice guideline are presented: (1) defining a topic, (2) selecting a work group, (3) performing a literature review, (4) writing the guideline, (5) peer review, (6) making plans for review and revision, and (7) dissemination. Given the importance of practice guidelines in supporting everyday practice, this article strives to provide a practical guide in the development of this key component of evidence-based medicine.

Assessing young children's intention-reading in authentic communicative contexts: preliminary evidence and clinical utility.

PubMed

Greenslade, Kathryn J; Coggins, Truman E

2014-01-01

Identifying what a communication partner is looking at (referential intention) and why (social intention) is essential to successful social communication, and may be challenging for children with social communication deficits. This study explores a clinical task that assesses these intention-reading abilities within an authentic context. To gather evidence of the task's reliability and validity, and to discuss its clinical utility. The intention-reading task was administered to twenty 4-7-year-olds with typical development (TD) and ten with autism spectrum disorder (ASD). Task items were embedded in an authentic activity, and they targeted the child's ability to identify the examiner's referential and social intentions, which were communicated through joint attention behaviours. Reliability and construct validity evidence were addressed using established psychometric methods. Reliability and validity evidence supported the use of task scores for identifying children whose intention-reading warranted concern. Evidence supported the reliability of task administration and coding, and item-level codes were highly consistent with overall task performance. Supporting task validity, group differences aligned with predictions, with children with ASD exhibiting poorer and more variable task scores than children with TD. Also, as predicted, task scores correlated significantly with verbal mental age and ratings of parental concerns regarding social communication abilities. The evidence provides preliminary support for the reliability and validity of the clinical task's scores in assessing young children's real-time intention-reading abilities, which are essential for successful interactions in school and beyond. © 2014 Royal College of Speech and Language Therapists.

Noncognitive constructs in graduate admissions: an integrative review of available instruments.

PubMed

Megginson, Lucy

2009-01-01

In the graduate admission process, both cognitive and noncognitive instruments evaluate a candidate's potential success in a program of study. Traditional cognitive measures include the Graduate Record Examination or graduate grade point average, while noncognitive constructs such as personality, attitude, and motivation are generally measured through letters of recommendation, interviews, or personality inventories. Little consensus exists as to what criteria constitute valid and effective measurements of graduate student potential. This integrative review of available tools to measure noncognitive constructs will assist graduate faculty in identifying valid and reliable instruments that will enhance a more holistic assessment of nursing graduate candidates. Finally, as evidence-based practice begins to penetrate academic processes and as graduate faculty realize the predictive significance of noncognitive attributes, faculty can use the information in this integrative review to guide future research.

Psychometric Properties of the Perceived Wellness Culture and Environment Support Scale.

PubMed

Melnyk, Bernadette Mazurek; Szalacha, Laura A; Amaya, Megan

2018-05-01

This study reports on the psychometric properties of the 11-item Perceived Wellness Culture and Environment Support Scale (PWCESS) and its relationship with employee healthy lifestyle beliefs and behaviors. Faculty and staff (N = 3959) at a large public university in the United States mid-west completed the PWCESS along with healthy lifestyle beliefs and behaviors scales. Data were randomly split into 2 halves to explore the PWCESS' validity and reliability and the second half to confirm findings. Principal components analysis indicated a unidimensional construct. The PWCESS was positively related to healthy lifestyle beliefs and behaviors supporting the scale's validity. Confirmatory factor analysis supported the unidimensional construct (Cronbach's α = .92). Strong evidence supports the validity and reliability of the PWCESS. Future use of this scale could guide workplace intervention strategies to improve organizational wellness culture and employee health outcomes.

Virtual reality simulator training for laparoscopic colectomy: what metrics have construct validity?

PubMed

Shanmugan, Skandan; Leblanc, Fabien; Senagore, Anthony J; Ellis, C Neal; Stein, Sharon L; Khan, Sadaf; Delaney, Conor P; Champagne, Bradley J

2014-02-01

Virtual reality simulation for laparoscopic colectomy has been used for training of surgical residents and has been considered as a model for technical skills assessment of board-eligible colorectal surgeons. However, construct validity (the ability to distinguish between skill levels) must be confirmed before widespread implementation. This study was designed to specifically determine which metrics for laparoscopic sigmoid colectomy have evidence of construct validity. General surgeons that had performed fewer than 30 laparoscopic colon resections and laparoscopic colorectal experts (>200 laparoscopic colon resections) performed laparoscopic sigmoid colectomy on the LAP Mentor model. All participants received a 15-minute instructional warm-up and had never used the simulator before the study. Performance was then compared between each group for 21 metrics (procedural, 14; intraoperative errors, 7) to determine specifically which measurements demonstrate construct validity. Performance was compared with the Mann-Whitney U-test (p < 0.05 was significant). Fifty-three surgeons; 29 general surgeons, and 24 colorectal surgeons enrolled in the study. The virtual reality simulators for laparoscopic sigmoid colectomy demonstrated construct validity for 8 of 14 procedural metrics by distinguishing levels of surgical experience (p < 0.05). The most discriminatory procedural metrics (p < 0.01) favoring experts were reduced instrument path length, accuracy of the peritoneal/medial mobilization, and dissection of the inferior mesenteric artery. Intraoperative errors were not discriminatory for most metrics and favored general surgeons for colonic wall injury (general surgeons, 0.7; colorectal surgeons, 3.5; p = 0.045). Individual variability within the general surgeon and colorectal surgeon groups was not accounted for. The virtual reality simulators for laparoscopic sigmoid colectomy demonstrated construct validity for 8 procedure-specific metrics. However, using virtual reality simulator metrics to detect intraoperative errors did not discriminate between groups. If the virtual reality simulator continues to be used for the technical assessment of trainees and board-eligible surgeons, the evaluation of performance should be limited to procedural metrics.

Validity of the Thai EQ-5D in an occupational population in Thailand.

PubMed

Kimman, Merel; Vathesatogkit, Prin; Woodward, Mark; Tai, E-Shyong; Thumboo, Julian; Yamwong, Sukit; Ratanachaiwong, Wipa; Wee, Hwee-Lin; Sritara, Piyamitr

2013-08-01

To assess the construct validity of the Thai EuroQoL (EQ-5D) among an occupational population in Thailand. Data were derived from a large cohort study among employees of the Electricity Generating Authority of Thailand. In 2008 and 2009, 4,850 participants completed the Thai EQ-5D and Short-Form 36 version 2 (SF-36v2). Thai preferences weights were used to convert EQ-5D health states into EQ-5D index scores. Construct validity of the Thai EQ-5D was examined by specifying and testing hypotheses about the relationships between the EQ-5D, SF-36v2, and participants' demographic and medical characteristics. Construct validity of the Thai EQ-5D was supported by expected relationships with SF-36v2 scale and summary scores. For example, SF-36v2 scores on the mental health scale were much lower for participants who reported having problems on the EQ-5D anxiety/depression dimension compared to those reporting no problems (mean norm-based SF-36v2 scores: 52.9 vs. 41.8, p < 0.001). Additionally, reporting a problem in a given EQ-5D dimension was generally associated with lower SF-36v2 summary scores. The EQ-5D index score distinguished between groups of participants in the expected manner, on the basis of sex, age, education and self-reported health, thus providing evidence of known-groups validity. The study demonstrated good construct validity of the Thai EQ-5D in a large occupational population in Thailand.

An instrument to assess subjective task value beliefs regarding the decision to pursue postgraduate training.

PubMed

Hagemeier, Nicholas E; Murawski, Matthew M

2014-02-12

To develop and validate an instrument to assess subjective ratings of the perceived value of various postgraduate training paths followed using expectancy-value as a theoretical framework; and to explore differences in value beliefs across type of postgraduate training pursued and type of pharmacy training completed prior to postgraduate training. A survey instrument was developed to sample 4 theoretical domains of subjective task value: intrinsic value, attainment value, utility value, and perceived cost. Retrospective self-report methodology was employed to examine respondents' (N=1,148) subjective task value beliefs specific to their highest level of postgraduate training completed. Exploratory and confirmatory factor analytic techniques were used to evaluate and validate value belief constructs. Intrinsic, attainment, utility, cost, and financial value constructs resulted from exploratory factor analysis. Cross-validation resulted in a 26-item instrument that demonstrated good model fit. Differences in value beliefs were noted across type of postgraduate training pursued and pharmacy training characteristics. The Postgraduate Training Value Instrument demonstrated evidence of reliability and construct validity. The survey instrument can be used to assess value beliefs regarding multiple postgraduate training options in pharmacy and potentially inform targeted recruiting of individuals to those paths best matching their own value beliefs.

Psychometric Performance of the Arabic Versions of the Cancer Behavior Inventory-Brief and the Functional Assessment of Cancer Therapy-Breast.

PubMed

Algamdi, Maaidah M; Hanneman, Sandra K

2018-02-14

Valid and reliable instruments in Arabic are needed to measure self-efficacy and quality of life for Arabic patients with cancer. The aim of this study was to test the psychometric performance of the Cancer Behavior Inventory-Brief Arabic (CBI-BA), including participant understanding of items, and the Functional Assessment of Cancer Therapy-Breast Arabic (FACT-BA). Using a cross-sectional design, 438 cancer patients completed the CBI-BA, 30 of whom completed cognitive interviews. A subsample 167 women with breast cancer also completed the FACT-BA. Internal consistency evidence was assessed with Cronbach's α and construct validity with principal axis factoring. Internal consistency estimates were acceptable for the total CBI-BA (α = .81) and FACT-BA (α = .88) scales. Exploratory factor analyses showed evidence of construct validity for the CBI-BA; 1 factor was derived, compared with four in the original English version. Cognitive interviews indicated satisfactory patient understanding of CBI-BA items. The Arabic version of the general FACT-General scale had 4 factors according to expectation. The CBI-BA has adequate psychometric performance for the measurement of self-efficacy for coping with cancer in Arabic patients. The FACT-General Arabic has adequate evidence of reliability and validity for the measurement of quality of life in Arabic women with breast cancer. The availability of culturally sensitive and psychometrically sound instruments for Arabic patients diagnosed with cancer should be valuable for healthcare clinicians and researchers to assess self-efficacy for coping with cancer and quality of life.

Assessment of a condition-specific quality-of-life measure for patients with developmentally absent teeth: validity and reliability testing.

PubMed

Akram, A J; Ireland, A J; Postlethwaite, K C; Sandy, J R; Jerreat, A S

2013-11-01

This article describes the process of validity and reliability testing of a condition-specific quality-of-life measure for patients with hypodontia presenting for orthodontic treatment. The development of the instrument is described in a previous article. Royal Devon and Exeter NHS Foundation Trust & Musgrove Park Hospital, Taunton. The child perception questionnaire was used as a standard against which to test criterion validity. The Bland and Altman method was used to check agreement between the two questionnaires. Construct validity was tested using principal component analysis on the four sections of the questionnaire. Test-retest reliability was tested using intraclass correlation coefficient and Bland and Altman method. Cronbach's alpha was used to test internal consistency reliability. Overall the questionnaire showed good reliability, criterion and construct validity. This together with previous evidence of good face and content validity suggests that the instrument may prove useful in clinical practice and further research. This study has demonstrated that the newly developed condition-specific quality-of-life questionnaire is both valid and reliable for use in young patients with hypodontia. © 2013 John Wiley & Sons A/S. Published by Blackwell Publishing Ltd.

Measuring Constructs in Family Science: How Can Item Response Theory Improve Precision and Validity?

PubMed Central

Gordon, Rachel A.

2014-01-01

This article provides family scientists with an understanding of contemporary measurement perspectives and the ways in which item response theory (IRT) can be used to develop measures with desired evidence of precision and validity for research uses. The article offers a nontechnical introduction to some key features of IRT, including its orientation toward locating items along an underlying dimension and toward estimating precision of measurement for persons with different levels of that same construct. It also offers a didactic example of how the approach can be used to refine conceptualization and operationalization of constructs in the family sciences, using data from the National Longitudinal Survey of Youth 1979 (n = 2,732). Three basic models are considered: (a) the Rasch and (b) two-parameter logistic models for dichotomous items and (c) the Rating Scale Model for multicategory items. Throughout, the author highlights the potential for researchers to elevate measurement to a level on par with theorizing and testing about relationships among constructs. PMID:25663714

Validating the Construct of Coercion in Family Routines: Expanding the Unit of Analysis in Behavioral Assessment with Families of Children with Developmental Disabilities

PubMed Central

Lucyshyn, Joseph M.; Irvin, Larry K.; Blumberg, E. Richard; Laverty, Robelyn; Horner, Robert H.; Sprague, Jeffrey R.

2015-01-01

We conducted an observational study of parent-child interaction in home activity settings (routines) of families raising young children with developmental disabilities and problem behavior. Our aim was to empirically investigate the construct validity of coercion in typical but unsuccessful family routines. The long-term goal was to develop an expanded ecological unit of analysis that may contribute to sustainable behavioral family intervention. Ten children with autism and/or mental retardation and their families participated. Videotaped observations were conducted in typical but unsuccessful home routines. Parent-child interaction in routines was coded in real time and sequential analyses were conducted to test hypotheses about coercive processes. Following observation, families were interviewed about the social validity of the construct. Results confirmed the presence of statistically significant, attention-driven coercive processes in routines in which parents were occupied with non-child centered tasks. Results partially confirmed the presence of escape-driven coercive processes in routines in which parent demands are common. Additional analysis revealed an alternative pattern with greater magnitude. Family perspectives suggested the social validity of the construct. Results are discussed in terms of preliminary, partial evidence for coercive processes in routines of families of children with developmental disabilities. Implications for behavioral assessment and intervention design are discussed. PMID:26321883

Expanding the Nomological Net of the Pathological Narcissism Inventory: German Validation and Extension in a Clinical Inpatient Sample.

PubMed

Morf, Carolyn C; Schürch, Eva; Küfner, Albrecht; Siegrist, Philip; Vater, Aline; Back, Mitja; Mestel, Robert; Schröder-Abé, Michela

2017-06-01

The Pathological Narcissism Inventory (PNI) is a multidimensional measure for assessing grandiose and vulnerable features in narcissistic pathology. The aim of the present research was to construct and validate a German translation of the PNI and to provide further information on the PNI's nomological net. Findings from a first study confirm the psychometric soundness of the PNI and replicate its seven-factor first-order structure. A second-order structure was also supported but with several equivalent models. A second study investigating associations with a broad range of measures ( DSM Axis I and II constructs, emotions, personality traits, interpersonal and dysfunctional behaviors, and well-being) supported the concurrent validity of the PNI. Discriminant validity with the Narcissistic Personality Inventory was also shown. Finally, in a third study an extension in a clinical inpatient sample provided further evidence that the PNI is a useful tool to assess the more pathological end of narcissism.

Validity of the AUDIT-C screen for at-risk drinking among students utilizing university primary care.

PubMed

Campbell, Clare E; Maisto, Stephen A

2018-03-22

Research is needed to establish the psychometric properties of brief screens in university primary care settings. This study aimed to assess the construct validity of one such screen, the Alcohol Use Disorders Identification Test-Consumption (AUDIT-C), for detecting at-risk drinking among students who have utilized on-campus primary care. 389 students recently seen in university primary care completed a confidential online survey in December 2014. Bivariate correlations between the AUDIT-C and measures of alcohol consumption and negative drinking consequences provided concurrent evidence for construct validity. Receiver Operating Characteristic curve analyses determined optimal cut-off scores for at-risk drinking. The AUDIT-C significantly correlated with measures of alcohol consumption and negative drinking consequences (p < .001). Analyses support optimal AUDIT-C cut-off scores of 5 for females and 7 for males. The AUDIT-C is a valid screen for at-risk drinking among students who utilize university primary care.

Construct validity and reliability of a real-time multidimensional smartphone app to assess pain in children and adolescents with cancer.

PubMed

Stinson, Jennifer N; Jibb, Lindsay A; Nguyen, Cynthia; Nathan, Paul C; Maloney, Anne Marie; Dupuis, L Lee; Gerstle, J Ted; Hopyan, Sevan; Alman, Benjamin A; Strahlendorf, Caron; Portwine, Carol; Johnston, Donna L

2015-12-01

We evaluated the construct validity (including responsiveness), reliability, and feasibility of the Pain Squad multidimensional smartphone-based pain assessment application (app) in children and adolescents with cancer, using 2 descriptive studies with repeated measures. Participants (8-18 years) undergoing cancer treatment were drawn from 4 pediatric cancer centers. In study 1, 92 participants self-reported their level of pain twice daily for 2 weeks using the Pain Squad app to assess app construct validity and reliability. In study 2, 14 participants recorded their level of pain twice a day for 1 week before and 2 weeks after cancer-related surgery to determine app responsiveness. Participants in both studies completed multiple measures to determine the construct validity and feasibility of the Pain Squad app. Correlations between average weekly pain ratings on the Pain Squad app and recalled least, average, and worst weekly pain were moderate to high (0.43-0.68). Correlations with health-related quality of life and pain coping (measured with PedsQL Inventory 4.0, PedsQL Cancer Module, and Pain Coping Questionnaire) were -0.46 to 0.29. The app showed excellent internal consistency (α = 0.96). Pain ratings changed because of surgery with large effect sizes between baseline and the first week postsurgery (>0.85) and small effect sizes between baseline and the second week postsurgery (0.13-0.32). These findings provide evidence of the construct validity, reliability, and feasibility of the Pain Squad app in children and adolescents with cancer. Use of real-time data capture approaches should be considered in future studies of childhood cancer pain. A video accompanying this abstract is available online as Supplemental Digital Content at http://links.lww.com/PAIN/A169.

Capability beliefs regarding evidence-based practice are associated with application of EBP and research use: validation of a new measure.

PubMed

Wallin, Lars; Boström, Anne-Marie; Gustavsson, J Petter

2012-08-01

Beliefs about capabilities, or self-efficacy, is a construct originating in social cognitive psychology. Capability beliefs have been found to be positively associated with intention and healthcare practice behaviour. A measure of an individual's beliefs about his/her capability to apply the components of evidence-based practice (EBP) has potential to be useful in implementation research. To evaluate the concurrent validity and internal structure of a new scale measuring nurses' capability beliefs regarding EBP. Data were taken from a prospective longitudinal study in Sweden (the Longitudinal Analyses of Nursing Education and Entry in Worklife [LANE]). A cohort of nursing students who graduated in the autumn of 2004 that was followed up 2 years after their graduation was used (n= 1,256). Concurrent validity was tested relating different levels of capability beliefs to extent of research use and application of EBP. An item-response approach was applied in the evaluation of internal structure of the proposed scale (six items). The psychometric analyses indicated that the six items could be summed to reflect a one-dimensional scale. Nurses with the highest level of capability beliefs reported that they used research findings in clinical practice more than twice as often as those with lower levels of capability beliefs. They also participated in the implementation of evidence seven times more often. There is a need for further studies of the construct and predictive validity of the scale. It should also be validated in other groups of health professionals. Learning including mastery experiences, role modelling, social persuasion, and manageable stress could be used in undergraduate education as well as practice development to increase beliefs about capabilities which might open the way to increased application of EBP in healthcare practice. This new measure is well grounded in social cognitive theory, functions as a one-dimensional scale and possesses promising properties of concurrent validity. ©2012 Sigma Theta Tau International.

The reliability and validity of the SF-8 with a conflict-affected population in northern Uganda.

PubMed

Roberts, Bayard; Browne, John; Ocaka, Kaducu Felix; Oyok, Thomas; Sondorp, Egbert

2008-12-02

The SF-8 is a health-related quality of life instrument that could provide a useful means of assessing general physical and mental health amongst populations affected by conflict. The purpose of this study was to test the validity and reliability of the SF-8 with a conflict-affected population in northern Uganda. A cross-sectional multi-staged, random cluster survey was conducted with 1206 adults in camps for internally displaced persons in Gulu and Amuru districts of northern Uganda. Data quality was assessed by analysing the number of incomplete responses to SF-8 items. Response distribution was analysed using aggregate endorsement frequency. Test-retest reliability was assessed in a separate smaller survey using the intraclass correlation test. Construct validity was measured using principal component analysis, and the Pearson Correlation test for item-summary score correlation and inter-instrument correlations. Known groups validity was assessed using a two sample t-test to evaluates the ability of the SF-8 to discriminate between groups known to have, and not have, physical and mental health problems. The SF-8 showed excellent data quality. It showed acceptable item response distribution based upon analysis of aggregate endorsement frequencies. Test-retest showed a good intraclass correlation of 0.61 for PCS and 0.68 for MCS. The principal component analysis indicated strong construct validity and concurred with the results of the validity tests by the SF-8 developers. The SF-8 also showed strong construct validity between the 8 items and PCS and MCS summary score, moderate inter-instrument validity, and strong known groups validity. This study provides evidence on the reliability and validity of the SF-8 amongst IDPs in northern Uganda.

The reliability and validity of the SF-8 with a conflict-affected population in northern Uganda

PubMed Central

Roberts, Bayard; Browne, John; Ocaka, Kaducu Felix; Oyok, Thomas; Sondorp, Egbert

2008-01-01

Background The SF-8 is a health-related quality of life instrument that could provide a useful means of assessing general physical and mental health amongst populations affected by conflict. The purpose of this study was to test the validity and reliability of the SF-8 with a conflict-affected population in northern Uganda. Methods A cross-sectional multi-staged, random cluster survey was conducted with 1206 adults in camps for internally displaced persons in Gulu and Amuru districts of northern Uganda. Data quality was assessed by analysing the number of incomplete responses to SF-8 items. Response distribution was analysed using aggregate endorsement frequency. Test-retest reliability was assessed in a separate smaller survey using the intraclass correlation test. Construct validity was measured using principal component analysis, and the Pearson Correlation test for item-summary score correlation and inter-instrument correlations. Known groups validity was assessed using a two sample t-test to evaluates the ability of the SF-8 to discriminate between groups known to have, and not have, physical and mental health problems. Results The SF-8 showed excellent data quality. It showed acceptable item response distribution based upon analysis of aggregate endorsement frequencies. Test-retest showed a good intraclass correlation of 0.61 for PCS and 0.68 for MCS. The principal component analysis indicated strong construct validity and concurred with the results of the validity tests by the SF-8 developers. The SF-8 also showed strong construct validity between the 8 items and PCS and MCS summary score, moderate inter-instrument validity, and strong known groups validity. Conclusion This study provides evidence on the reliability and validity of the SF-8 amongst IDPs in northern Uganda. PMID:19055716

Development and validation of a reading-related assessment battery in Malay for the purpose of dyslexia assessment.

PubMed

Lee, Lay Wah

2008-06-01

Malay is an alphabetic language with transparent orthography. A Malay reading-related assessment battery which was conceptualised based on the International Dyslexia Association definition of dyslexia was developed and validated for the purpose of dyslexia assessment. The battery consisted of ten tests: Letter Naming, Word Reading, Non-word Reading, Spelling, Passage Reading, Reading Comprehension, Listening Comprehension, Elision, Rapid Letter Naming and Digit Span. Content validity was established by expert judgment. Concurrent validity was obtained using the schools' language tests as criterion. Evidence of predictive and construct validity was obtained through regression analyses and factor analyses. Phonological awareness was the most significant predictor of word-level literacy skills in Malay, with rapid naming making independent secondary contributions. Decoding and listening comprehension made separate contributions to reading comprehension, with decoding as the more prominent predictor. Factor analysis revealed four factors: phonological decoding, phonological naming, comprehension and verbal short-term memory. In conclusion, despite differences in orthography, there are striking similarities in the theoretical constructs of reading-related tasks in Malay and in English.

Psychometric instrumentation: reliability and validity of instruments used for clinical practice, evidence-based practice projects and research studies.

PubMed

Mayo, Ann M

2015-01-01

It is important for CNSs and other APNs to consider the reliability and validity of instruments chosen for clinical practice, evidence-based practice projects, or research studies. Psychometric testing uses specific research methods to evaluate the amount of error associated with any particular instrument. Reliability estimates explain more about how well the instrument is designed, whereas validity estimates explain more about scores that are produced by the instrument. An instrument may be architecturally sound overall (reliable), but the same instrument may not be valid. For example, if a specific group does not understand certain well-constructed items, then the instrument does not produce valid scores when used with that group. Many instrument developers may conduct reliability testing only once, yet continue validity testing in different populations over many years. All CNSs should be advocating for the use of reliable instruments that produce valid results. Clinical nurse specialists may find themselves in situations where reliability and validity estimates for some instruments that are being utilized are unknown. In such cases, CNSs should engage key stakeholders to sponsor nursing researchers to pursue this most important work.

The Self-Presentation Motives for Physical Activity Questionnaire: Instrument Development and Preliminary Construct Validity Evidence.

PubMed

Howle, Timothy C; Dimmock, James A; Whipp, Peter R; Jackson, Ben

2015-06-01

With the aim of advancing the literature on impression management in physical activity settings, we developed a theoretically derived 2 by 2 instrument that was designed to measure different types of context-specific self-presentation motives. Following item generation and expert review (Study 1), the instrument was completed by 206 group exercise class attendees (Study 2) and 463 high school physical education students (Study 3). Our analyses supported the intended factor structure (i.e., reflecting acquisitive-agentic, acquisitive-communal, protective-agentic, and protective-communal motives). We found some support for construct validity, and the self-presentation motives were associated with variables of theoretical and applied interest (e.g., impression motivation and construction, social anxiety, social and achievement goals, efficacy beliefs, engagement). Taken together, the results indicate that the Self-presentation Motives for Physical Activity Questionnaire (SMPAQ) may be useful for measuring various types of self-presentation motives in physical activity settings.

Relationship status: Scales for assessing the vitality of late adolescents' relationships with their parents.

PubMed

Klos, D S; Paddock, J R

1978-12-01

Three criteria for assessing relationship status were proposed: self-disclosure despite the risk of parental disapproval; openness to critical feedback from parents; constructive confrontation when angry with parents. These concepts were operationalized as narratives of nine interpersonal dilemmas, to which late adolescents responded by indicating "What would you do if you were in this situation?" Reliable example-anchored scales were constructed from the responses of one sample of college students and then cross-validated with two other samples. Social class had a significant but small effect on the relationship status scores; but age and sex of adolescent and sex of parent did not. The patterns of correlations of the Relationship Status Scales among themselves and with the Parent-Child Relations Questionnaire, the College Self-Expression Scale, the Fear of Negative Evaluation Scale, and Hogan's Empathy Scale were interpreted as evidence of construct validity.

Measuring In-Hospital Recovery After Colorectal Surgery Within a Well-Established Enhanced Recovery Pathway: A Comparison Between Hospital Length of Stay and Time to Readiness for Discharge.

PubMed

Balvardi, Saba; Pecorelli, Nicolò; Castelino, Tanya; Niculiseanu, Petru; Liberman, A Sender; Charlebois, Patrick; Stein, Barry; Carli, Franco; Mayo, Nancy E; Feldman, Liane S; Fiore, Julio F

2018-05-15

Hospital length of stay is often used as a measure of in-hospital recovery but may be confounded by organizational factors. Time to readiness for discharge may provide a superior index of recovery. The purpose of this study was to contribute evidence for the construct validity of time to readiness for discharge and length of stay as measures of in-hospital recovery after colorectal surgery in the context of a well-established enhanced recovery pathway. This was an observational validation study designed according to the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist. The study was conducted at a university-affiliated tertiary hospital. A total of 100 consecutive patients undergoing elective colorectal resection (mean age = 65 y; 57% men; 81% laparoscopic) who participated in a randomized controlled trial were included. We tested a priori hypotheses that length of stay and time-to-readiness for discharge are longer in patients undergoing open surgery, with lower physical status, with severe comorbidities, with postoperative complications, undergoing rectal surgery, who are older (≥75 y), who have a new stoma, and who have inflammatory bowel disease. Median time-to-readiness for discharge and length of stay were both 3 days. For both measures, 6 of 8 construct validity hypotheses were supported (hypotheses 1 and 4-8). The use of secondary data from a randomized controlled trial (risk of selection bias) was a limitation. Results may not be generalizable to institutions where patient care is not equally structured. This study contributes evidence to the construct validity of time-to-readiness for discharge and length of stay as measures of in-hospital recovery within enhanced recovery pathways. Our findings suggest that length of stay can be a less resource-intensive and equally construct-valid index of in-hospital recovery compared with time-to-readiness for discharge. Enhanced recovery pathways may decrease process-of-care variances that impact length of stay, allowing more timely discharge once discharge criteria are achieved. See Video Abstract at http://links.lww.com/DCR/A564.

Rasch Validation of a Measure of Reform-Oriented Science Teaching Practices

ERIC Educational Resources Information Center

You, Hye Sun

2016-01-01

Growing evidence from recent curriculum documents and previous research suggests that reform-oriented science teaching practices promote students' conceptual understanding, levels of achievement, and motivation to learn, especially when students are actively engaged in constructing their ideas through scientific inquiries. However, it is difficult…

Psychopathy in Bulgaria: The cross-cultural generalizability of the Hare Psychopathy Checklist

PubMed Central

Wilson, Michael J.; Abramowitz, Carolyn; Vasilev, Georgi; Bozgunov, Kiril; Vassileva, Jasmin

2014-01-01

The generalizability of the psychopathy construct to Eastern European cultures has not been well-studied, and no prior studies have evaluated psychopathy in non-offender samples from this population. The current validation study examines the factor structure, internal consistency, and external validity of the Bulgarian translation of the Hare Psychopathy Checklist: Screening Version. Two hundred sixty-two Bulgarian adults from the general community were assessed, of which 185 had a history of substance dependence. Confirmatory factor analysis indicated good fit for the two-, three-, and four-factor models of psychopathy. Zero-order and partial correlation analyses were conducted between the two factors of psychopathy and criterion measures of antisocial behavior, internalizing and externalizing psychopathology, personality traits, addictive disorders and demographic characteristics. Relationships to external variables provided evidence for the convergent and discriminant validity of the psychopathy construct in a Bulgarian community sample. PMID:25313268

Construct validity of the Iowa Gambling Task.

PubMed

Buelow, Melissa T; Suhr, Julie A

2009-03-01

The Iowa Gambling Task (IGT) was created to assess real-world decision making in a laboratory setting and has been applied to various clinical populations (i.e., substance abuse, schizophrenia, pathological gamblers) outside those with orbitofrontal cortex damage, for whom it was originally developed. The current review provides a critical examination of lesion, functional neuroimaging, developmental, and clinical studies in order to examine the construct validity of the IGT. The preponderance of evidence provides support for the use of the IGT to detect decision making deficits in clinical populations, in the context of a more comprehensive evaluation. The review includes a discussion of three critical issues affecting the validity of the IGT, as it has recently become available as a clinical instrument: the lack of a concise definition as to what aspect of decision making the IGT measures, the lack of data regarding reliability of the IGT, and the influence of personality and state mood on IGT performance.

Beliefs about language development: construct validity evidence.

PubMed

Donahue, Mavis L; Fu, Qiong; Smith, Everett V

2012-01-01

Understanding language development is incomplete without recognizing children's sociocultural environments, including adult beliefs about language development. Yet there is a need for data supporting valid inferences to assess these beliefs. The current study investigated the psychometric properties of data from a survey (MODeL) designed to explore beliefs in the popular culture, and their alignment with more formal theories. Support for the content, substantive, structural, generalizability, and external aspects of construct validity of the data were investigated. Subscales representing Behaviorist, Cognitive, Nativist, and Sociolinguistic models were identified as dimensions of beliefs. More than half of the items showed a high degree of consensus, suggesting culturally-transmitted beliefs. Behaviorist ideas were most popular. Bilingualism and ethnicity were related to Cognitive and Sociolinguistic beliefs. Identifying these beliefs may clarify the nature of child-directed speech, and enable the design of language intervention programs that are congruent with family and cultural expectations.

Testing a self-determination theory model of children's physical activity motivation: a cross-sectional study.

PubMed

Sebire, Simon J; Jago, Russell; Fox, Kenneth R; Edwards, Mark J; Thompson, Janice L

2013-09-26

Understanding children's physical activity motivation, its antecedents and associations with behavior is important and can be advanced by using self-determination theory. However, research among youth is largely restricted to adolescents and studies of motivation within certain contexts (e.g., physical education). There are no measures of self-determination theory constructs (physical activity motivation or psychological need satisfaction) for use among children and no previous studies have tested a self-determination theory-based model of children's physical activity motivation. The purpose of this study was to test the reliability and validity of scores derived from scales adapted to measure self-determination theory constructs among children and test a motivational model predicting accelerometer-derived physical activity. Cross-sectional data from 462 children aged 7 to 11 years from 20 primary schools in Bristol, UK were analysed. Confirmatory factor analysis was used to examine the construct validity of adapted behavioral regulation and psychological need satisfaction scales. Structural equation modelling was used to test cross-sectional associations between psychological need satisfaction, motivation types and physical activity assessed by accelerometer. The construct validity and reliability of the motivation and psychological need satisfaction measures were supported. Structural equation modelling provided evidence for a motivational model in which psychological need satisfaction was positively associated with intrinsic and identified motivation types and intrinsic motivation was positively associated with children's minutes in moderate-to-vigorous physical activity. The study provides evidence for the psychometric properties of measures of motivation aligned with self-determination theory among children. Children's motivation that is based on enjoyment and inherent satisfaction of physical activity is associated with their objectively-assessed physical activity and such motivation is positively associated with perceptions of psychological need satisfaction. These psychological factors represent potential malleable targets for interventions to increase children's physical activity.

A new test for the assessment of working memory in clinical settings: Validation and norming of a month ordering task.

PubMed

Buekenhout, Imke; Leitão, José; Gomes, Ana A

2018-05-24

Month ordering tasks have been used in experimental settings to obtain measures of working memory (WM) capacity in older/clinical groups based solely on their face validity. We sought to assess the appropriateness of using a month ordering task in other contexts, including clinical settings, as a psychometrically sound WM assessment. To this end, we constructed a month ordering task (ucMOT), studied its reliability (internal consistency and temporal stability), and gathered construct-related and criterion-related validity evidence for its use as a WM assessment. The ucMOT proved to be internally consistent and temporally stable, and analyses of the criterion-related validity evidence revealed that its scores predicted the efficiency of language comprehension processes known to depend crucially on WM resources, namely, processes involved in pronoun interpretation. Furthermore, all ucMOT items discriminated between younger and older age groups; the global scores were significantly correlated with scores on well-established WM tasks and presented lower correlations with instruments that evaluate different (although related) processes, namely, inhibition and processing speed. We conclude that the ucMOT possesses solid psychometric properties. Accordingly, we acquired normative data for the Portuguese population, which we present as a regression-based algorithm that yields z scores adjusted for age, gender, and years of formal education. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

The Diabetes Evaluation Framework for Innovative National Evaluations (DEFINE): Construct and Content Validation Using a Modified Delphi Method.

PubMed

Paquette-Warren, Jann; Tyler, Marie; Fournie, Meghan; Harris, Stewart B

2017-06-01

In order to scale-up successful innovations, more evidence is needed to evaluate programs that attempt to address the rising prevalence of diabetes and the associated burdens on patients and the healthcare system. This study aimed to assess the construct and content validity of the Diabetes Evaluation Framework for Innovative National Evaluations (DEFINE), a tool developed to guide the evaluation, design and implementation with built-in knowledge translation principles. A modified Delphi method, including 3 individual rounds (questionnaire with 7-point agreement/importance Likert scales and/or open-ended questions) and 1 group round (open discussion) were conducted. Twelve experts in diabetes, research, knowledge translation, evaluation and policy from Canada (Ontario, Quebec and British Columbia) and Australia participated. Quantitative consensus criteria were an interquartile range of ≤1. Qualitative data were analyzed thematically and confirmed by participants. An importance scale was used to determine a priority multi-level indicator set. Items rated very or extremely important by 80% or more of the experts were reviewed in the final group round to build the final set. Participants reached consensus on the content and construct validity of DEFINE, including its title, overall goal, 5-step evaluation approach, medical and nonmedical determinants of health schematics, full list of indicators and associated measurement tools, priority multi-level indicator set and next steps in DEFINE's development. Validated by experts, DEFINE has the right theoretic components to evaluate comprehensively diabetes prevention and management programs and to support acquisition of evidence that could influence the knowledge translation of innovations to reduce the burden of diabetes. Copyright © 2017 The Authors. Published by Elsevier Inc. All rights reserved.

A mixed methods approach to adapting and evaluating the functional assessment of HIV infection (FAHI), Swahili version, for use with low literacy populations.

PubMed

Nyongesa, Moses K; Sigilai, Antipa; Hassan, Amin S; Thoya, Janet; Odhiambo, Rachael; Van de Vijver, Fons J R; Newton, Charles R J C; Abubakar, Amina

2017-01-01

Despite bearing the largest HIV-related burden, little is known of the Health-Related Quality of Life (HRQoL) among people living with HIV in sub-Saharan Africa. One of the factors contributing to this gap in knowledge is the lack of culturally adapted and validated measures of HRQoL that are relevant for this setting. We set out to adapt the Functional Assessment of HIV Infection (FAHI) Questionnaire, an HIV-specific measure of HRQoL, and evaluate its internal consistency and validity. The three phase mixed-methods study took place in a rural setting at the Kenyan Coast. Phase one involved a scoping review to describe the evidence base of the reliability and validity of FAHI as well as the geographical contexts in which it has been administered. Phase two involved in-depth interviews (n = 38) to explore the content validity, and initial piloting for face validation of the adapted FAHI. Phase three was quantitative (n = 103) and evaluated the internal consistency, convergent and construct validities of the adapted interviewer-administered questionnaire. In the first phase of the study, we identified 16 studies that have used the FAHI. Most (82%) were conducted in North America. Only seven (44%) of the reviewed studies reported on the psychometric properties of the FAHI. In the second phase, most of the participants (37 out of 38) reported satisfaction with word clarity and content coverage whereas 34 (89%) reported satisfaction with relevance of the items, confirming the face validity of the adapted questionnaire during initial piloting. Our participants indicated that HIV impacted on their physical, functional, emotional, and social wellbeing. Their responses overlapped with items in four of the five subscales of the FAHI Questionnaire establishing its content validity. In the third phase, the internal consistency of the scale was found to be satisfactory with subscale Cronbach's α ranging from 0.55 to 0.78. The construct and convergent validity of the tool were supported by acceptable factor loadings for most of the items on the respective sub-scales and confirmation of expected significant correlations of the FAHI subscale scores with scores of a measure of common mental disorders. The adapted interviewer-administered Swahili version of FAHI questionnaire showed initial strong evidence of good psychometric properties with satisfactory internal consistency and acceptable validity (content, face, and convergent validity). It gives impetus for further validation work, especially construct validity, in similar settings before it can be used for research and clinical purposes in the entire East African region.

Measurement properties of depression questionnaires in patients with diabetes: a systematic review.

PubMed

van Dijk, Susan E M; Adriaanse, Marcel C; van der Zwaan, Lennart; Bosmans, Judith E; van Marwijk, Harm W J; van Tulder, Maurits W; Terwee, Caroline B

2018-06-01

To conduct a systematic review on measurement properties of questionnaires measuring depressive symptoms in adult patients with type 1 or type 2 diabetes. A systematic review of the literature in MEDLINE, EMbase and PsycINFO was performed. Full text, original articles, published in any language up to October 2016 were included. Eligibility for inclusion was independently assessed by three reviewers who worked in pairs. Methodological quality of the studies was evaluated by two independent reviewers using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. Quality of the questionnaires was rated per measurement property, based on the number and quality of the included studies and the reported results. Of 6286 unique hits, 21 studies met our criteria evaluating nine different questionnaires in multiple settings and languages. The methodological quality of the included studies was variable for the different measurement properties: 9/15 studies scored 'good' or 'excellent' on internal consistency, 2/5 on reliability, 0/1 on content validity, 10/10 on structural validity, 8/11 on hypothesis testing, 1/5 on cross-cultural validity, and 4/9 on criterion validity. For the CES-D, there was strong evidence for good internal consistency, structural validity, and construct validity; moderate evidence for good criterion validity; and limited evidence for good cross-cultural validity. The PHQ-9 and WHO-5 also performed well on several measurement properties. However, the evidence for structural validity of the PHQ-9 was inconclusive. The WHO-5 was less extensively researched and originally not developed to measure depression. Currently, the CES-D is best supported for measuring depressive symptoms in diabetes patients.

The PEDro scale had acceptably high convergent validity, construct validity, and interrater reliability in evaluating methodological quality of pharmaceutical trials.

PubMed

Yamato, Tie Parma; Maher, Chris; Koes, Bart; Moseley, Anne

2017-06-01

The Physiotherapy Evidence Database (PEDro) scale has been widely used to investigate methodological quality in physiotherapy randomized controlled trials; however, its validity has not been tested for pharmaceutical trials. The aim of this study was to investigate the validity and interrater reliability of the PEDro scale for pharmaceutical trials. The reliability was also examined for the Cochrane Back and Neck (CBN) Group risk of bias tool. This is a secondary analysis of data from a previous study. We considered randomized placebo controlled trials evaluating any pain medication for chronic spinal pain or osteoarthritis. Convergent validity was evaluated by correlating the PEDro score with the summary score of the CBN risk of bias tool. The construct validity was tested using a linear regression analysis to determine the degree to which the total PEDro score is associated with treatment effect sizes, journal impact factor, and the summary score for the CBN risk of bias tool. The interrater reliability was estimated using the Prevalence and Bias Adjusted Kappa coefficient and 95% confidence interval (CI) for the PEDro scale and CBN risk of bias tool. Fifty-three trials were included, with 91 treatment effect sizes included in the analyses. The correlation between PEDro scale and CBN risk of bias tool was 0.83 (95% CI 0.76-0.88) after adjusting for reliability, indicating strong convergence. The PEDro score was inversely associated with effect sizes, significantly associated with the summary score for the CBN risk of bias tool, and not associated with the journal impact factor. The interrater reliability for each item of the PEDro scale and CBN risk of bias tool was at least substantial for most items (>0.60). The intraclass correlation coefficient for the PEDro score was 0.80 (95% CI 0.68-0.88), and for the CBN, risk of bias tool was 0.81 (95% CI 0.69-0.88). There was evidence for the convergent and construct validity for the PEDro scale when used to evaluate methodological quality of pharmacological trials. Both risk of bias tools have acceptably high interrater reliability. Copyright © 2017 Elsevier Inc. All rights reserved.

‘Emotional Intelligence’: Lessons from Lesions

PubMed Central

Hogeveen, J.; Salvi, C.; Grafman, J.

2018-01-01

‘Emotional intelligence’ (EI) is one of the most highly used psychological terms in popular nomenclature, yet its construct, divergent, and predictive validities are contentiously debated. Despite this debate, the EI construct is composed of a set of emotional abilities – recognizing emotional states in the self and others, using emotions to guide thought and behavior, understanding how emotions shape behavior, and emotion regulation – that undoubtedly influence important social and personal outcomes. In this review, evidence from human lesion studies is reviewed in order to provide insight into the necessary brain regions for each of these core emotional abilities. Critically, we consider how this neuropsychological evidence might help to guide efforts to define and measure EI. PMID:27647325

'Emotional Intelligence': Lessons from Lesions.

PubMed

Hogeveen, J; Salvi, C; Grafman, J

2016-10-01

'Emotional intelligence' (EI) is one of the most highly used psychological terms in popular nomenclature, yet its construct, divergent, and predictive validities are contentiously debated. Despite this debate, the EI construct is composed of a set of emotional abilities - recognizing emotional states in the self and others, using emotions to guide thought and behavior, understanding how emotions shape behavior, and emotion regulation - that undoubtedly influence important social and personal outcomes. In this review, evidence from human lesion studies is reviewed in order to provide insight into the necessary brain regions for each of these core emotional abilities. Critically, we consider how this neuropsychological evidence might help to guide efforts to define and measure EI. Copyright © 2016 Elsevier Ltd. All rights reserved.

Validation of a global scale to assess the quality of interprofessional teamwork in mental health settings.

PubMed

Tomizawa, Ryoko; Yamano, Mayumi; Osako, Mitue; Hirabayashi, Naotugu; Oshima, Nobuo; Sigeta, Masahiro; Reeves, Scott

2017-12-01

Few scales currently exist to assess the quality of interprofessional teamwork through team members' perceptions of working together in mental health settings. The purpose of this study was to revise and validate an interprofessional scale to assess the quality of teamwork in inpatient psychiatric units and to use it multi-nationally. A literature review was undertaken to identify evaluative teamwork tools and develop an additional 12 items to ensure a broad global focus. Focus group discussions considered adaptation to different care systems using subjective judgements from 11 participants in a pre-test of items. Data quality, construct validity, reproducibility, and internal consistency were investigated in the survey using an international comparative design. Exploratory factor analysis yielded five factors with 21 items: 'patient/community centred care', 'collaborative communication', 'interprofessional conflict', 'role clarification', and 'environment'. High overall internal consistency, reproducibility, adequate face validity, and reasonable construct validity were shown in the USA and Japan. The revised Collaborative Practice Assessment Tool (CPAT) is a valid measure to assess the quality of interprofessional teamwork in psychiatry and identifies the best strategies to improve team performance. Furthermore, the revised scale will generate more rigorous evidence for collaborative practice in psychiatry internationally.

A comparison of the psychometric properties of the psychopathic personality inventory full-length and short-form versions.

PubMed

Kastner, Rebecca M; Sellbom, Martin; Lilienfeld, Scott O

2012-03-01

The Psychopathic Personality Inventory (PPI) has shown promising construct validity as a measure of psychopathy. Because of its relative efficiency, a short-form version of the PPI (PPI-SF) was developed and has proven useful in many psychopathy studies. The validity of the PPI-SF, however, has not been thoroughly examined, and no studies have directly compared the validity of the short form with that of the full-length version. The current study was designed to compare the psychometric properties of both PPI versions, with an emphasis on convergent and discriminant validity in predicting external criteria conceptually relevant to psychopathy. We used both prison (n = 558) and college samples (n = 322) for this investigation. PPI scale scores were more reliable and more strongly correlated with the conceptually relevant criterion measures compared with the PPI-SF, particularly in the prison sample. There were no differences in relative discriminant validity. Thus, overall, the PPI full-length version showed more evidence of construct validity than did the short form, and the consequences of this psychometric difference should be considered when evaluating the clinical utility of each measure.

Perceived visual informativeness (PVI): construct and scale development to assess visual information in printed materials.

PubMed

King, Andy J; Jensen, Jakob D; Davis, LaShara A; Carcioppolo, Nick

2014-01-01

There is a paucity of research on the visual images used in health communication messages and campaign materials. Even though many studies suggest further investigation of these visual messages and their features, few studies provide specific constructs or assessment tools for evaluating the characteristics of visual messages in health communication contexts. The authors conducted 2 studies to validate a measure of perceived visual informativeness (PVI), a message construct assessing visual messages presenting statistical or indexical information. In Study 1, a 7-item scale was created that demonstrated good internal reliability (α = .91), as well as convergent and divergent validity with related message constructs such as perceived message quality, perceived informativeness, and perceived attractiveness. PVI also converged with a preference for visual learning but was unrelated to a person's actual vision ability. In addition, PVI exhibited concurrent validity with a number of important constructs including perceived message effectiveness, decisional satisfaction, and three key public health theory behavior predictors: perceived benefits, perceived barriers, and self-efficacy. Study 2 provided more evidence that PVI is an internally reliable measure and demonstrates that PVI is a modifiable message feature that can be tested in future experimental work. PVI provides an initial step to assist in the evaluation and testing of visual messages in campaign and intervention materials promoting informed decision making and behavior change.

Development and validation of the Delaying Gratification Inventory.

PubMed

Hoerger, Michael; Quirk, Stuart W; Weed, Nathan C

2011-09-01

Deficits in gratification delay are associated with a broad range of public health problems, such as obesity, risky sexual behavior, and substance abuse. However, 6 decades of research on the construct has progressed less quickly than might be hoped, largely because of measurement issues. Although past research has implicated 5 domains of delay behavior, involving food, physical pleasures, social interactions, money, and achievement, no published measure to date has tapped all 5 components of the content domain. Existing measures have been criticized for limitations related to efficiency, reliability, and construct validity. Using an innovative Internet-mediated approach to survey construction, we developed the 35-item 5-factor Delaying Gratification Inventory (DGI). Evidence from 4 studies and a large, diverse sample of respondents (N = 10,741) provided support for the psychometric properties of the measure. Specifically, scores on the DGI demonstrated strong internal consistency and test-retest reliability for the 35-item composite, each of the 5 domains, and a 10-item short form. The 5-factor structure fit the data well and had good measurement invariance across subgroups. Construct validity was supported by correlations with scores on closely related self-control measures, behavioral ratings, Big Five personality trait measures, and measures of adjustment and psychopathology, including those on the Minnesota Multiphasic Personality Inventory-2-Restructured Form. DGI scores also showed incremental validity in accounting for well-being and health-related variables. The present investigation holds implications for improving public health, accelerating future research on gratification delay, and facilitating survey construction research more generally by demonstrating the suitability of an Internet-mediated strategy.

Development and validation of the Delaying Gratification Inventory

PubMed Central

Hoerger, Michael; Quirk, Stuart W.; Weed, Nathan C.

2011-01-01

Deficits in gratification delay are associated with a broad range of public health problems, such as obesity, risky sexual behavior, and substance abuse. However, six decades of research on the construct has progressed less quickly than might be hoped, largely due to measurement issues. Although past research implicates five domains of delay behavior, involving food, physical pleasures, social interactions, money, and achievement, no published measure to date has tapped all five components of the content domain. Existing measures have been criticized for limitations related to efficiency, reliability, and construct validity. Using an innovative Internet-mediated approach to survey construction, we developed the 35-item five-factor Delaying Gratification Inventory (DGI). Evidence from four studies and a large, diverse sample of respondents (N = 10,741) provided support for the psychometric properties of the measure. Specifically, scores on the DGI demonstrated strong internal consistency and test-retest reliability for the 35-item composite, each of the five domains, and a 10-item short-form. The five-factor structure fit the data well and had good measurement invariance across subgroups. Construct validity was supported by correlations with scores on closely-related self-control measures, behavioral ratings, Big Five personality trait measures, and measures of adjustment and psychopathology, including those on the Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF). DGI scores also showed incremental validity in accounting for well-being and health-related variables. The present investigation holds implications for improving public health, accelerating future research on gratification delay, and facilitating survey construction research more generally by demonstrating the suitability of an Internet-mediated strategy. PMID:21480721

A Methodological Review of the Assessment of Humanism in Medical Students.

PubMed

Buck, Era; Holden, Mark; Szauter, Karen

2015-11-01

Humanism is a complex construct that defies simplistic measurement. How educators measure humanism shapes understanding and implications for learners. This systematic review sought to address the following questions: How do medical educators assess humanism in medical students, and how does the measurement impact the understanding of humanism in undergraduate medical education (UME)? Using the IECARES (integrity, excellence, compassion, altruism, respect, empathy, and service) Gold Foundation framework, a search of English literature databases from 2000 to 2013 on assessment of humanism in medical students revealed more than 900 articles, of which 155 met criteria for analysis. Using descriptive statistics, articles and assessments were analyzed for construct measured, study design, assessment method, instrument type, perspective/source of assessment, student level, validity evidence, and national context. Of 202 assessments reported in 155 articles, 162 (80%) used surveys; 164 (81%) used student self-reports. One hundred nine articles (70%) included only one humanism construct. Empathy was the most prevalent construct present in 96 (62%); 49 (51%) of those used a single instrument. One hundred fifteen (74%) used exclusively quantitative data; only 48 (31%) used a longitudinal design. Construct underrepresentation was identified as a threat to validity in half of the assessments. Articles included 34 countries; 87 (56%) were from North America. Assessment of humanism in UME incorporates a limited scope of a complex construct, often relying on single quantitative measures from self-reported survey instruments. This highlights the need for multiple methods, perspectives, and longitudinal designs to strengthen the validity of humanism assessments.

The Chinese version of hospital anxiety and depression scale: Psychometric properties in Chinese cancer patients and their family caregivers.

PubMed

Li, Qiuping; Lin, Yi; Hu, Caiping; Xu, Yinghua; Zhou, Huiya; Yang, Liping; Xu, Yongyong

2016-12-01

The Hospital Anxiety and Depression Scale (HADS) acts as one of the most frequently used self-reported measures in cancer practice. The evidence for construct validity of HADS, however, remains inconclusive. The objective of this study is to evaluate the psychometric properties of the Chinese version HADS (C-HADS) in terms of construct validity, internal consistency reliability, and concurrent validity in dyads of Chinese cancer patients and their family caregivers. This was a cross-sectional study, conducted in multiple centers: one hospital in each of the seven different administrative regions in China from October 2014 to May 2015. A total of 641 dyads, consisting of cancer patients and family caregivers, completed a survey assessing their demographic and background information, anxiety and depression using C-HADS, and quality of life (QOL) using Chinese version SF-12. Data analysis methods included descriptive statistics, confirmatory factor analysis (CFA), and Pearson correlations. Both the two-factor and one-factor models offered the best and adequate fit to the data in cancer patients and family caregivers respectively. The comparison of the two-factor and single-factor models supports the basic assumption of two-factor construct of C-HADS. The overall and two subscales of C-HADS in both cancer patients and family caregivers had good internal consistency and acceptable concurrent validity. The Chinese version of the HADS may be a reliable and valid screening tool, as indicated by its original two-factor structure. The finding supports the basic assumption of two-factor construct of HADS. Copyright © 2016 Elsevier Ltd. All rights reserved.

Methodological Issues in Measuring the Development of Character

ERIC Educational Resources Information Center

Card, Noel A.

2017-01-01

In this article I provide an overview of the methodological issues involved in measuring constructs relevant to character development and education. I begin with a nontechnical overview of the 3 fundamental psychometric properties of measurement: reliability, validity, and equivalence. Developing and evaluating measures to ensure evidence of all 3…

Multidimensional Treatment of Fear of Death.

ERIC Educational Resources Information Center

Hoelter, Jon W.

1979-01-01

Presents a multidimensional conception of fear of death and provides subscales for measuring suggested dimensions (fear of the dying process, of the dead, of being destroyed, for significant others, of the unknown, of conscious death, for body after death, and of premature death). Evidence for construct validity is provided. (Author/BEF)

Physics Metacognition Inventory Part Ii: Confirmatory Factor Analysis and Rasch Analysis

ERIC Educational Resources Information Center

Taasoobshirazi, Gita; Bailey, MarLynn; Farley, John

2015-01-01

The Physics Metacognition Inventory was developed to measure physics students' metacognition for problem solving. In one of our earlier studies, an exploratory factor analysis provided evidence of preliminary construct validity, revealing six components of students' metacognition when solving physics problems including knowledge of cognition,…

41 CFR 60-3.15 - Documentation of impact and validity evidence.

Code of Federal Regulations, 2011 CFR

2011-07-01

... group should be described (essential). (6) Sample description. A description of how the research sample... research sample compares with the relevant labor market or work force, the method by which the relevant... quantitative data which identify or define the job constructs, such as factor analyses, should be provided...

41 CFR 60-3.15 - Documentation of impact and validity evidence.

Code of Federal Regulations, 2013 CFR

2013-07-01

... group should be described (essential). (6) Sample description. A description of how the research sample... research sample compares with the relevant labor market or work force, the method by which the relevant... quantitative data which identify or define the job constructs, such as factor analyses, should be provided...

41 CFR 60-3.15 - Documentation of impact and validity evidence.

Code of Federal Regulations, 2010 CFR

2010-07-01

... group should be described (essential). (6) Sample description. A description of how the research sample... research sample compares with the relevant labor market or work force, the method by which the relevant... quantitative data which identify or define the job constructs, such as factor analyses, should be provided...

29 CFR 1607.15 - Documentation of impact and validity evidence.

Code of Federal Regulations, 2011 CFR

2011-07-01

... (essential). (6) Sample description. A description of how the research sample was identified and selected... the size of each subgroup (essential). A description of how the research sample compares with the...). Any quantitative data which identify or define the job constructs, such as factor analyses, should be...

29 CFR 1607.15 - Documentation of impact and validity evidence.

Code of Federal Regulations, 2012 CFR

2012-07-01

... (essential). (6) Sample description. A description of how the research sample was identified and selected... the size of each subgroup (essential). A description of how the research sample compares with the...). Any quantitative data which identify or define the job constructs, such as factor analyses, should be...

41 CFR 60-3.15 - Documentation of impact and validity evidence.

Code of Federal Regulations, 2014 CFR

2014-07-01

... group should be described (essential). (6) Sample description. A description of how the research sample... research sample compares with the relevant labor market or work force, the method by which the relevant... quantitative data which identify or define the job constructs, such as factor analyses, should be provided...

41 CFR 60-3.15 - Documentation of impact and validity evidence.

Code of Federal Regulations, 2012 CFR

2012-07-01

... group should be described (essential). (6) Sample description. A description of how the research sample... research sample compares with the relevant labor market or work force, the method by which the relevant... quantitative data which identify or define the job constructs, such as factor analyses, should be provided...

29 CFR 1607.15 - Documentation of impact and validity evidence.

Code of Federal Regulations, 2013 CFR

2013-07-01

... (essential). (6) Sample description. A description of how the research sample was identified and selected... the size of each subgroup (essential). A description of how the research sample compares with the...). Any quantitative data which identify or define the job constructs, such as factor analyses, should be...

29 CFR 1607.15 - Documentation of impact and validity evidence.

Code of Federal Regulations, 2014 CFR

2014-07-01

... (essential). (6) Sample description. A description of how the research sample was identified and selected... the size of each subgroup (essential). A description of how the research sample compares with the...). Any quantitative data which identify or define the job constructs, such as factor analyses, should be...

Implicit attitudes towards homosexuality: reliability, validity, and controllability of the IAT.

PubMed

Banse, R; Seise, J; Zerbes, N

2001-01-01

Two experiments were conducted to investigate the psychometric properties of an Implicit Association Test (IAT; Greenwald, McGhee, & Schwartz, 1998) that was adapted to measure implicit attitudes towards homosexuality. In a first experiment, the validity of the Homosexuality-IAT was tested using a known group approach. Implicit and explicit attitudes were assessed in heterosexual and homosexual men and women (N = 101). The results provided compelling evidence for the convergent and discriminant validity of the Homosexuality-IAT as a measure of implicit attitudes. No evidence was found for two alternative explanations of IAT effects (familiarity with stimulus material and stereotype knowledge). The internal consistency of IAT scores was satisfactory (alpha s > .80), but retest correlations were lower. In a second experiment (N = 79) it was shown that uninformed participants were able to fake positive explicit but not implicit attitudes. Discrepancies between implicit and explicit attitudes towards homosexuality could be partially accounted for by individual differences in the motivation to control prejudiced behavior, thus providing independent evidence for the validity of the implicit attitude measure. Neither explicit nor implicit attitudes could be changed by persuasive messages. The results of both experiments are interpreted as evidence for a single construct account of implicit and explicit attitudes towards homosexuality.

The six-minute walk test as a measure of postoperative recovery after colorectal resection: further examination of its measurement properties.

PubMed

Pecorelli, Nicolò; Fiore, Julio F; Gillis, Chelsia; Awasthi, Rashami; Mappin-Kasirer, Benjamin; Niculiseanu, Petru; Fried, Gerald M; Carli, Francesco; Feldman, Liane S

2016-06-01

Patients, clinicians and researchers seek an easy, reproducible and valid measure of postoperative recovery. The six-minute walk test (6MWT) is a low-cost measure of physical function, which is a relevant dimension of recovery. The aim of the present study was to contribute further evidence for the validity of the 6MWT as a measure of postoperative recovery after colorectal surgery. This study involved a sample of 174 patients enrolled in three previous randomized controlled trials. Construct validity was assessed by testing the hypotheses that the distance walked in 6 min (6MWD) at 4 weeks after surgery is greater (1) in younger versus older patients, (2) in patients with higher preoperative physical status versus lower, (3) after laparoscopic versus open surgery, (4) in patients without postoperative complications versus with postoperative complications; and that 6MWD (5) correlates cross-sectionally with self-reported physical activity as measured with a questionnaire (CHAMPS). Statistical analysis was performed using linear regression and Spearman's correlation. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist was used to guide the formulation of hypotheses and reporting of results. One hundred and fifty-one patients who completed the 6MWT at 4 weeks after surgery were included in the analysis. All hypotheses tested for construct validity were supported by the data. Older age, poorer physical status, open surgery and occurrence of postoperative complications were associated with clinically relevant reduction in 6MWD (>19 m). There was a moderate positive correlation between 6MWD and patient-reported physical activity (r = 0.46). This study contributes further evidence for the construct validity of the 6MWT as a measure of postoperative recovery after colorectal surgery. Results from this study support the use of the 6MWT as an outcome measure in studies evaluating interventions aimed to improve postoperative recovery.

Current Status of Simulation-based Training Tools in Orthopedic Surgery: A Systematic Review.

PubMed

Morgan, Michael; Aydin, Abdullatif; Salih, Alan; Robati, Shibby; Ahmed, Kamran

To conduct a systematic review of orthopedic training and assessment simulators with reference to their level of evidence (LoE) and level of recommendation. Medline and EMBASE library databases were searched for English language articles published between 1980 and 2016, describing orthopedic simulators or validation studies of these models. All studies were assessed for LoE, and each model was subsequently awarded a level of recommendation using a modified Oxford Centre for Evidence-Based Medicine classification, adapted for education. A total of 76 articles describing orthopedic simulators met the inclusion criteria, 47 of which described at least 1 validation study. The most commonly identified models (n = 34) and validation studies (n = 26) were for knee arthroscopy. Construct validation was the most frequent validation study attempted by authors. In all, 62% (47 of 76) of the simulator studies described arthroscopy simulators, which also contained validation studies with the highest LoE. Orthopedic simulators are increasingly being subjected to validation studies, although the LoE of such studies generally remain low. There remains a lack of focus on nontechnical skills and on cost analyses of orthopedic simulators. Copyright © 2017 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

An empirical look at the Defense Mechanism Test (DMT): reliability and construct validity.

PubMed

Ekehammar, Bo; Zuber, Irena; Konstenius, Marja-Liisa

2005-07-01

Although the Defense Mechanism Test (DMT) has been in use for almost half a century, there are still quite contradictory views about whether it is a reliable instrument, and if so, what it really measures. Thus, based on data from 39 female students, we first examined DMT inter-coder reliability by analyzing the agreement among trained judges in their coding of the same DMT protocols. Second, we constructed a "parallel" photographic picture that retained all structural characteristic of the original and analyzed DMT parallel-test reliability. Third, we examined the construct validity of the DMT by (a) employing three self-report defense-mechanism inventories and analyzing the intercorrelations between DMT defense scores and corresponding defenses in these instruments, (b) studying the relationships between DMT responses and scores on trait and state anxiety, and (c) relating DMT-defense scores to measures of self-esteem. The main results showed that the DMT can be coded with high reliability by trained coders, that the parallel-test reliability is unsatisfactory compared to traditional psychometric standards, that there is a certain generalizability in the number of perceptual distortions that people display from one picture to another, and that the construct validation provided meager empirical evidence for the conclusion that the DMT measures what it purports to measure, that is, psychological defense mechanisms.

Reliability and validity of the Haitian Creole PHQ-9.

PubMed

Marc, Linda G; Henderson, Whitney R; Desrosiers, Astrid; Testa, Marcia A; Jean, Samuel E; Akom, Eniko Edit

2014-12-01

There is limited information on depression in Haitians and this is partly attributable to the absence of culturally and linguistically adapted measures for depression. To perform a psychometric evaluation of the Haitian-Creole version of the PHQ-9 administered to men who have sex with men (MSM) in the Republic of Haiti. This study uses a cross-sectional design and data are from the Integrated Behavioral and Biological HIV Survey (IBBS) for MSM in Haiti. Inclusion criteria required that participants be male, ≥ 18 years, report sexual relations with a male partner in the last 12 months, and lived in Haiti during the past 3 months. Respondent Driven Sampling was used for participant recruitment. A structured questionnaire was verbally administered in Haitian-Creole capturing information on sociodemographics, sexual behaviors, human immunodeficiency virus (HIV) status and depressive symptomatology using the PHQ-9. Psychometric analyses of the translated PHQ-9 assessed unidimensionality, factor structure, reliability, construct validity, and differential item functioning (DIF) across subgroups (age, educational level, sexual orientation and HIV status). In a study population of 1,028 MSM, the Haitian-Creole version of the PHQ-9 is unidimensional, has moderately high internal consistency reliability (α = 0.78), and shows evidence of construct validity where HIV-positive subjects have greater depression (p = 0.002). There is no evidence of DIF across age, education, sexual orientation or HIV status. HIV-positive MSM are twice as likely to screen positive for moderately severe and severe depressive symptoms compared to their HIV-negative counterparts. There is strong evidence for the psychometric adequacy of the translated PHQ-9 screening tool as a measure of depression with MSM in Haiti. Future research is necessary to examine the predictive validity of depression for subsequent health behaviors or clinical outcomes among Haitian MSM.

A Psychometric Analysis of the Italian Version of the eHealth Literacy Scale Using Item Response and Classical Test Theory Methods

PubMed Central

Dima, Alexandra Lelia; Schulz, Peter Johannes

2017-01-01

Background The eHealth Literacy Scale (eHEALS) is a tool to assess consumers’ comfort and skills in using information technologies for health. Although evidence exists of reliability and construct validity of the scale, less agreement exists on structural validity. Objective The aim of this study was to validate the Italian version of the eHealth Literacy Scale (I-eHEALS) in a community sample with a focus on its structural validity, by applying psychometric techniques that account for item difficulty. Methods Two Web-based surveys were conducted among a total of 296 people living in the Italian-speaking region of Switzerland (Ticino). After examining the latent variables underlying the observed variables of the Italian scale via principal component analysis (PCA), fit indices for two alternative models were calculated using confirmatory factor analysis (CFA). The scale structure was examined via parametric and nonparametric item response theory (IRT) analyses accounting for differences between items regarding the proportion of answers indicating high ability. Convergent validity was assessed by correlations with theoretically related constructs. Results CFA showed a suboptimal model fit for both models. IRT analyses confirmed all items measure a single dimension as intended. Reliability and construct validity of the final scale were also confirmed. The contrasting results of factor analysis (FA) and IRT analyses highlight the importance of considering differences in item difficulty when examining health literacy scales. Conclusions The findings support the reliability and validity of the translated scale and its use for assessing Italian-speaking consumers’ eHealth literacy. PMID:28400356

First evaluation of the Behavioral Addiction Indoor Tanning Screener (BAITS) in a nationwide representative sample.

PubMed

Diehl, K; Görig, T; Breitbart, E W; Greinert, R; Hillhouse, J J; Stapleton, J L; Schneider, S

2018-01-01

Evidence suggests that indoor tanning may have addictive properties. However, many instruments for measuring indoor tanning addiction show poor validity and reliability. Recently, a new instrument, the Behavioral Addiction Indoor Tanning Screener (BAITS), has been developed. To test the validity and reliability of the BAITS by using a multimethod approach. We used data from the first wave of the National Cancer Aid Monitoring on Sunbed Use, which included a cognitive pretest (August 2015) and a Germany-wide representative survey (October to December 2015). In the cognitive pretest 10 users of tanning beds were interviewed and 3000 individuals aged 14-45 years were included in the representative survey. Potential symptoms of indoor tanning addiction were measured using the BAITS, a brief screening survey with seven items (answer categories: yes vs. no). Criterion validity was assessed by comparing the results of BAITS with usage parameters. Additionally, we tested internal consistency and construct validity. A total of 19·7% of current and 1·8% of former indoor tanning users were screened positive for symptoms of a potential indoor tanning addiction. We found significant associations between usage parameters and the BAITS (criterion validity). Internal consistency (reliability) was good (Kuder-Richardson-20, 0·854). The BAITS was shown to be a homogeneous construct (construct validity). Compared with other short instruments measuring symptoms of a potential indoor tanning addiction, the BAITS seems to be a valid and reliable tool. With its short length and the binary items the BAITS is easy to use in large surveys. © 2017 British Association of Dermatologists.

Development, Validation, and Deployment of a Revised Air Traffic Control Color Vision Test: Incorporating Advanced Technologies and Oceanic Procedures and En Route Automation Modernization Systems

DTIC Science & Technology

2013-09-01

through direct sampling of form and content of critical display data. Evidence of construct validity is provided by correlation with the Colour ...measured by the Colour Assessment and Diagnosis (CAD; ARTS Background Colors STARS Background Colors ERAM Background Colors Figure 3...Gelade, G. (1980). A feature-integration theory of attention. Cognitive Psychology , 12, 97–136. Xing, J. & Schroeder, D.J. (2006). Reexamination of

Implicit Sex Guilt Predicts Sexual Behaviors: Evidence for the Validity of the Sex Guilt Implicit Association Test.

PubMed

Totonchi, Delaram A; Derlega, Valerian J; Janda, Louis H

2018-05-14

Self-report measures of sexuality may be influenced by people's conscious concerns about confidentiality and social desirability. Alternatively, non-conscious measures (e.g., implicit association tests; IATs) are designed to minimize these validity concerns. We constructed an IAT measure of sex guilt using 154 male and female university students. The sex guilt IAT demonstrated convergent validity as it correlated with various sexual behaviors and incremental validity as it improved the prediction of several sexual behaviors beyond that provided by the Mosher sex guilt scale. We conclude that a non-conscious measure of sex guilt may complement the use of self-reports in studying sexual behaviors.

Development and implementation of a virtual reality laparoscopic colorectal training curriculum.

PubMed

Wynn, Greg; Lykoudis, Panagis; Berlingieri, Pasquale

2017-12-12

Contemporary surgical training can be compromised by fewer practical opportunities. Simulation can fill this gap to optimize skills' development and progress monitoring. A structured virtual reality (VR) laparoscopic sigmoid colectomy curriculum is constructed and its validity and outcomes assessed. Parameters and thresholds were defined by analysing the performance of six expert surgeons completing the relevant module on the LAP Mentor simulator. Fourteen surgical trainees followed the curriculum, performance being recorded and analysed. Evidence of validity was assessed. Time to complete procedure, number of movements of right and left instrument, and total path length of right and left instrument movements demonstrated evidence of validity and clear learning curves, with a median of 14 attempts needed to complete the curriculum. A structured curriculum is proposed for training in laparoscopic sigmoid colectomy in a VR environment based on objective metrics in addition to expert consensus. Validity has been demonstrated for some key metrics. Copyright © 2017 Elsevier Inc. All rights reserved.

Idiopathic Pulmonary Fibrosis: Clinically Meaningful Primary Endpoints in Phase 3 Clinical Trials

PubMed Central

Collard, Harold R.; Anstrom, Kevin J.; Flaherty, Kevin R.; Fleming, Thomas R.; King, Talmadge E.; Martinez, Fernando J.; Brown, Kevin K.

2012-01-01

Definitive evidence of clinical efficacy in a Phase 3 trial is best shown by a beneficial impact on a clinically meaningful endpoint—that is, an endpoint that directly measures how a patient feels (symptoms), functions (the ability to perform activities in daily life), or survives. In idiopathic pulmonary fibrosis (IPF), we believe the endpoints that best meet these criteria are all-cause mortality and all-cause nonelective hospitalization. There are no validated measures of symptoms or broader constructs such as health status or funtional status in IPF. A surrogate endpoint is defined as an indirect measure that is intended to substitute for a clinically meaningful endpoint. Surrogate endpoints can be appropriate outcome measures if validated. However, validation requires substantial evidence that the effect of an intervention on a clinically meaningful endpoint is reliably predicted by the effect of an intervention on the surrogate endpoint. For patients with IPF, there are currently no validated surrogate endpoints. PMID:22505745

Development and Testing of the Nurse Manager EBP Competency Scale.

PubMed

Shuman, Clayton J; Ploutz-Snyder, Robert J; Titler, Marita G

2018-02-01

The purpose of this study was to develop and evaluate the validity and reliability of an instrument to measure nurse manager competencies regarding evidence-based practice (EBP). The Nurse Manager EBP Competency Scale consists of 16 items for respondents to indicate their perceived level of competency on a 0 to 3 Likert-type scale. Content validity was demonstrated through expert panel review and pilot testing. Principal axis factoring and Cronbach's alpha evaluated construct validity and internal consistency reliability, respectively. Eighty-three nurse managers completed the scale. Exploratory factor analysis resulted in a 16-item scale with two subscales, EBP Knowledge ( n = 6 items, α = .90) and EBP Activity ( n = 10 items, α = .94). Cronbach's alpha for the entire scale was .95. The Nurse Manager EBP Competency Scale is a brief measure of nurse manager EBP competency with evidence of validity and reliability. The scale can enhance our understanding in future studies regarding how nurse manager EBP competency affects implementation.

Inappropriate and Excessive Guilt: Instrument Validation and Developmental Differences in Relation to Depression

PubMed Central

Tilghman-Osborne, Carlos; Felton, Julia W.

2014-01-01

Inappropriate or excessive guilt is listed as a symptom of depression by the American Psychiatric Association (1994). Although many measures of guilt have been developed, definitional and operational problems exist, especially in the application of such measures in childhood and adolescence. To address these problems, the current study introduces the Inappropriate and Excessive Guilt Scale (IEGS), assesses its validity for use with children and adolescents, and tests its relation to depression across development. From a sample of 370 children between 7 and 16 years old, results provided (1) evidence that items designed to assess inappropriate and excessive guilt converged onto a single underlying factor, (2) support for the convergent, discriminant, and construct validity of the IEGS in a general youth population, and (3) evidence of incremental validity of the IEGS over-and-above other measures of guilt. Results also supported the hypothesis that inappropriate and excessive guilt as well as negative cognitive errors become less normative and more depressotypic with age. PMID:22086497

Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures

PubMed Central

2013-01-01

Background Two of the current methodological barriers to implementation science efforts are the lack of agreement regarding constructs hypothesized to affect implementation success and identifiable measures of these constructs. In order to address these gaps, the main goals of this paper were to identify a multi-level framework that captures the predominant factors that impact implementation outcomes, conduct a systematic review of available measures assessing constructs subsumed within these primary factors, and determine the criterion validity of these measures in the search articles. Method We conducted a systematic literature review to identify articles reporting the use or development of measures designed to assess constructs that predict the implementation of evidence-based health innovations. Articles published through 12 August 2012 were identified through MEDLINE, CINAHL, PsycINFO and the journal Implementation Science. We then utilized a modified five-factor framework in order to code whether each measure contained items that assess constructs representing structural, organizational, provider, patient, and innovation level factors. Further, we coded the criterion validity of each measure within the search articles obtained. Results Our review identified 62 measures. Results indicate that organization, provider, and innovation-level constructs have the greatest number of measures available for use, whereas structural and patient-level constructs have the least. Additionally, relatively few measures demonstrated criterion validity, or reliable association with an implementation outcome (e.g., fidelity). Discussion In light of these findings, our discussion centers on strategies that researchers can utilize in order to identify, adapt, and improve extant measures for use in their own implementation research. In total, our literature review and resulting measures compendium increases the capacity of researchers to conceptualize and measure implementation-related constructs in their ongoing and future research. PMID:23414420

Psychometric Evaluation of a Korean Version of the Adherence to Refills and Medications Scale (ARMS) in Adults With Type 2 Diabetes.

PubMed

Kim, Chun-Ja; Park, Eunyoung; Schlenk, Elizabeth A; Kim, Moonsun; Kim, Dae Jung

2016-04-01

The purpose of the study was to examine the reliability and validity of the Adherence to Refills and Medications Scale-Korean (ARMS-K) among Korean adults with type 2 diabetes. The Korean translated ARMS-K was back-translated to ensure translation equivalency. A cross-sectional survey was used to evaluate the psychometric properties with exploratory factor analysis for validity and Cronbach's alpha coefficients for reliability. The factor analysis of construct validity identified 3 dimensions of the ARMS-K, explaining 54.7% of the total variance. The internal consistency reliability for the total instrument was acceptable with a Cronbach's alpha of .801. There was good correlation between the ARMS-K and 8-item Morisky Medication Adherence Scale-Korean version (r = -0.698), indicating that these scales measure theoretically related constructs as evidence of convergent validity. As evidence of known groups validity, there was a significant association between the ARMS-K score and glycemic control (P = .048), indicating that the good glycemic controlled group was more likely to have a higher rate of adherence to refills and medications than the poor glycemic controlled group. These results support the cross-cultural applicability of the concepts underlying the ARMS-K. The ARMS-K can be used not only to assess adherence to refills and medications in Koreans with diabetes but also to examine the potential role of adherence to refills and medications in enhanced glycemic control of people with diabetes in a variety of clinical settings. © 2016 The Author(s).

Development and psychometric evaluation of a cardiovascular risk and disease management knowledge assessment tool.

PubMed

Rosneck, James S; Hughes, Joel; Gunstad, John; Josephson, Richard; Noe, Donald A; Waechter, Donna

2014-01-01

This article describes the systematic construction and psychometric analysis of a knowledge assessment instrument for phase II cardiac rehabilitation (CR) patients measuring risk modification disease management knowledge and behavioral outcomes derived from national standards relevant to secondary prevention and management of cardiovascular disease. First, using adult curriculum based on disease-specific learning outcomes and competencies, a systematic test item development process was completed by clinical staff. Second, a panel of educational and clinical experts used an iterative process to identify test content domain and arrive at consensus in selecting items meeting criteria. Third, the resulting 31-question instrument, the Cardiac Knowledge Assessment Tool (CKAT), was piloted in CR patients to ensure use of application. Validity and reliability analyses were performed on 3638 adults before test administrations with additional focused analyses on 1999 individuals completing both pretreatment and posttreatment administrations within 6 months. Evidence of CKAT content validity was substantiated, with 85% agreement among content experts. Evidence of construct validity was demonstrated via factor analysis identifying key underlying factors. Estimates of internal consistency, for example, Cronbach's α = .852 and Spearman-Brown split-half reliability = 0.817 on pretesting, support test reliability. Item analysis, using point biserial correlation, measured relationships between performance on single items and total score (P < .01). Analyses using item difficulty and item discrimination indices further verified item stability and validity of the CKAT. A knowledge instrument specifically designed for an adult CR population was systematically developed and tested in a large representative patient population, satisfying psychometric parameters, including validity and reliability.

The 11-item Medication Adherence Reasons Scale: reliability and factorial validity among patients with hypertension in Malaysian primary healthcare settings

PubMed Central

Shima, Razatul; Farizah, Hairi; Majid, Hazreen Abdul

2015-01-01

INTRODUCTION The aim of this study was to assess the reliability and validity of a modified Malaysian version of the Medication Adherence Reasons Scale (MAR-Scale). METHODS In this cross-sectional study, the 15-item MAR-Scale was administered to 665 patients with hypertension who attended one of the four government primary healthcare clinics in the Hulu Langat and Klang districts of Selangor, Malaysia, between early December 2012 and end-March 2013. The construct validity was examined in two phases. Phase I consisted of translation of the MAR-Scale from English to Malay, a content validity check by an expert panel, a face validity check via a small preliminary test among patients with hypertension, and exploratory factor analysis (EFA). Phase II involved internal consistency reliability calculations and confirmatory factor analysis (CFA). RESULTS EFA verified five existing factors that were previously identified (i.e. issues with medication management, multiple medications, belief in medication, medication availability, and the patient’s forgetfulness and convenience), while CFA extracted four factors (medication availability issues were not extracted). The final modified MAR-Scale model, which had 11 items and a four-factor structure, provided good evidence of convergent and discriminant validities. Cronbach’s alpha coefficient was > 0.7, indicating good internal consistency of the items in the construct. The results suggest that the modified MAR-Scale has good internal consistencies and construct validity. CONCLUSION The validated modified MAR-Scale (Malaysian version) was found to be suitable for use among patients with hypertension receiving treatment in primary healthcare settings. However, the comprehensive measurement of other factors that can also lead to non-adherence requires further exploration. PMID:25902719

The 11-item Medication Adherence Reasons Scale: reliability and factorial validity among patients with hypertension in Malaysian primary healthcare settings.

PubMed

Shima, Razatul; Farizah, Hairi; Majid, Hazreen Abdul

2015-08-01

The aim of this study was to assess the reliability and validity of a modified Malaysian version of the Medication Adherence Reasons Scale (MAR-Scale). In this cross-sectional study, the 15-item MAR-Scale was administered to 665 patients with hypertension who attended one of the four government primary healthcare clinics in the Hulu Langat and Klang districts of Selangor, Malaysia, between early December 2012 and end-March 2013. The construct validity was examined in two phases. Phase I consisted of translation of the MAR-Scale from English to Malay, a content validity check by an expert panel, a face validity check via a small preliminary test among patients with hypertension, and exploratory factor analysis (EFA). Phase II involved internal consistency reliability calculations and confirmatory factor analysis (CFA). EFA verified five existing factors that were previously identified (i.e. issues with medication management, multiple medications, belief in medication, medication availability, and the patient's forgetfulness and convenience), while CFA extracted four factors (medication availability issues were not extracted). The final modified MAR-Scale model, which had 11 items and a four-factor structure, provided good evidence of convergent and discriminant validities. Cronbach's alpha coefficient was > 0.7, indicating good internal consistency of the items in the construct. The results suggest that the modified MAR-Scale has good internal consistencies and construct validity. The validated modified MAR-Scale (Malaysian version) was found to be suitable for use among patients with hypertension receiving treatment in primary healthcare settings. However, the comprehensive measurement of other factors that can also lead to non-adherence requires further exploration.

Mental Help Seeking Attitudes Scale (MHSAS): Development, reliability, validity, and comparison with the ATSPPH-SF and IASMHS-PO.

PubMed

Hammer, Joseph H; Parent, Mike C; Spiker, Douglas A

2018-01-01

Attitudes is a key help-seeking construct that influences treatment seeking behavior via intention to seek help, per the theory of planned behavior (TPB). This article presents the development and psychometric evaluation of the Mental Help Seeking Attitudes Scale (MHSAS), designed to measure respondents' overall evaluation (unfavorable vs. favorable) of their seeking help from a mental health professional. In Study 1 (N = 857 United States adults), exploratory factor analysis (EFA), confirmatory factor analysis (CFA), and item response theory (IRT) analysis were used to identify an optimal set of 9 items that demonstrated initial evidence of internal consistency, unidimensionality, and strong measurement equivalence/invariance (ME/I) across gender, past help-seeking experience, and psychological distress. Initial convergent evidence of validity was demonstrated via theoretically anticipated relationships between the MHSAS and key variables in the help-seeking nomological network (e.g., subjective norms, perceived behavioral control, intention, public stigma, self-stigma, anticipated risks and benefits, gender, previous help seeking). Initial incremental evidence of validity was demonstrated when the MHSAS demonstrated the ability to account for unique variance in help-seeking intention, beyond that accounted for by the Attitudes Toward Seeking Professional Psychological Help-Short Form scale (ATSPPH-SF) and the Psychological Openness subscale of the Inventory of Attitudes Toward Seeking Mental Health Services (IASMHS-PO). Study 2 (N = 207 United States adults at Times 1 and 2) provided initial evidence of test-retest reliability over a 3-week period. The MHSAS offers mental health professionals a new tool for measuring attitudes that may avoid limitations of current help seeking-attitudes measures (e.g., construct-irrelevant variance). (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Assessing Students' Understanding of Macroevolution: Concerns regarding the validity of the MUM

NASA Astrophysics Data System (ADS)

Novick, Laura R.; Catley, Kefyn M.

2012-11-01

In a recent article, Nadelson and Southerland (2010. Development and preliminary evaluation of the Measure of Understanding of Macroevolution: Introducing the MUM. The Journal of Experimental Education, 78, 151-190) reported on their development of a multiple-choice concept inventory intended to assess college students' understanding of macroevolutionary concepts, the Measure of Understanding Macroevolution (MUM). Given that the only existing evolution inventories assess understanding of natural selection, a microevolutionary concept, a valid assessment of students' understanding of macroevolution would be a welcome and necessary addition to the field of science education. Although the conceptual framework underlying Nadelson and Southerland's test is promising, we believe the test has serious shortcomings with respect to validity evidence for the construct being tested. We argue and provide evidence that these problems are serious enough that the MUM should not be used in its current form to measure students' understanding of macroevolution.

Scale of attitudes toward alcohol - Spanish version: evidences of validity and reliability 1

PubMed Central

Ramírez, Erika Gisseth León; de Vargas, Divane

2017-01-01

ABSTRACT Objective: validate the Scale of attitudes toward alcohol, alcoholism and individuals with alcohol use disorders in its Spanish version. Method: methodological study, involving 300 Colombian nurses. Adopting the classical theory, confirmatory factor analysis was applied without prior examination, based on the strong historical evidence of the factorial structure of the original scale to determine the construct validity of this Spanish version. To assess the reliability, Cronbach’s Alpha and Mc Donalid’s Omega coefficients were used. Results: the confirmatory factor analysis indicated the good fit of the scale model in a four-factor distribution, with a cut-off point at 3.2, demonstrating 66.7% of sensitivity. Conclusions: the Scale of attitudes toward alcohol, alcoholism and individuals with alcohol use disorders in Spanish presented robust psychometric qualities, affirming that the instrument possesses a solid factorial structure and reliability and is capable of precisely measuring the nurses’ atittudes towards the phenomenon proposed. PMID:28793126

Development and validation of a new instrument to evaluate the ease of use of patient-controlled analgesic modalities for postoperative patients.

PubMed

Harding, Gale; Schein, Jeff R; Nelson, Winnie W; Vallow, Sue; Olson, William H; Hewitt, David J; Polomano, Rosemary C

2010-03-01

To describe the development and psychometric evaluation of a questionnaire assessing the ease of use that patients associate with patient-controlled analgesia (PCA) modalities. Qualitative interviews were conducted with patients who had experience with intravenous (IV) PCA for postoperative pain management to generate items relevant to the ease of using PCA modalities. The content validity of the resulting questionnaire was examined through follow-up patient interviews, and an expert panel reviewed the questionnaire. Cognitive debriefing interviews were conducted with patients to determine the clarity and content of the instructions, items, and response scales, and the ease of completing the instrument. Psychometric evaluation was performed with patients who had undergone surgery and received IV PCA for postoperative pain management. Item and scale quality and the internal consistency reliability of the questionnaire were assessed. Construct validity was evaluated by examining the relationship between subscales of the questionnaire with patient-reported outcome measures. Known-groups validity was determined by assessing the instrument's ability to differentiate between patients with versus without an IV PCA problem. A potential limitation of this study was the exclusive sampling of patients who had experience with IV PCA. The Patient Ease-of-Care (EOC) Questionnaire included 23 items in the following subscales: Confidence with Device, Comfort with Device, Movement, Dosing Confidence, Pain Control, Knowledge/Understanding, and Satisfaction. Coefficient alpha reliability estimates were ≥ 0.66 for Overall EOC (includes all subscales except Satisfaction) and all EOC subscales. Construct validity was supported by the moderate relationship between the Pain Control subscale and measures of pain severity and pain interference; additional evidence of construct validity was provided by correlations of the Confidence with Device subscale, the Satisfaction subscale, and Overall EOC with measures of pain severity, pain interference, and satisfaction. Significant mean score differences were reported between participants with and without IV PCA problems for Overall EOC and for the Comfort with Device, Confidence with Device, Movement, Pain Control, and Satisfaction subscales indicating known-groups validity. Results provide evidence for the reliability and validity of the Patient EOC Questionnaire as a measure of the ease of use that patients associate with PCA systems and may be useful for evaluating emerging PCA modalities.

Mindfulness: A systematic review of instruments to measure an emergent patientreported outcome (PRO)

PubMed Central

Park, Taehwan; Reilly-Spong, Maryanne

2013-01-01

Purpose Mindfulness has emerged as an important health concept based on evidence that mindfulness interventions reduce symptoms and improve health-related quality of life. The objectives of this study were to systematically assess and compare the properties of instruments to measure self-reported mindfulness. Methods Ovid Medline®, CINAHL®, and PsycINFO® were searched through May 2012, and articles were selected if their primary purpose was development or evaluation of the measurement properties (validity, reliability, responsiveness) of a self-report mindfulness scale. Two reviewers independently evaluated the methodological quality of the selected studies using the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist. Discrepancies were discussed with a third reviewer, and scored by consensus. Finally, a level of evidence approach was used to synthesize results and study quality. Results Our search strategy identified a total of 2,588 articles. Forty-six articles, reporting 79 unique studies, met inclusion criteria. Ten instruments quantifying mindfulness as a unidimensional scale (n=5) or as a set of 2 to 5 subscales (n=5) were reviewed. The Mindful Attention Awareness Scale (MAAS) was evaluated by the most studies (n=27), and had positive overall quality ratings for most of the psychometric properties reviewed. The Five Facet Mindfulness Questionnaire (FFMQ) received the highest possible rating (“consistent findings in multiple studies of good methodological quality”) for two properties, internal consistency and construct validation by hypothesis testing. However, none of the instruments had sufficient evidence of content validity. Comprehensiveness of construct coverage had not been assessed; qualitative methods to confirm understanding and relevance were absent. In addition, estimates of test-retest reliability, responsiveness, or measurement error to guide users in protocol development or interpretation of scores were lacking. Conclusions Current mindfulness scales have important conceptual differences, and none can be strongly recommended based solely on superior psychometric properties. Important limitations in the field are the absence of qualitative evaluations and accepted external referents to support construct validity. Investigators need to proceed cautiously before optimizing any mindfulness intervention based on the existing scales. PMID:23539467

Reliability and validity of the de Morton Mobility Index in individuals with sub-acute stroke.

PubMed

Braun, Tobias; Marks, Detlef; Thiel, Christian; Grüneberg, Christian

2018-02-04

To establish the validity and reliability of the de Morton Mobility Index (DEMMI) in patients with sub-acute stroke. This cross-sectional study was performed in a neurological rehabilitation hospital. We assessed unidimensionality, construct validity, internal consistency reliability, inter-rater reliability, minimal detectable change and possible floor and ceiling effects of the DEMMI in adult patients with sub-acute stroke. The study included a total sample of 121 patients with sub-acute stroke. We analysed validity (n = 109) and reliability (n = 51) in two sub-samples. Rasch analysis indicated unidimensionality with an overall fit to the model (chi-square = 12.37, p = 0.577). All hypotheses on construct validity were confirmed. Internal consistency reliability (Cronbach's alpha = 0.94) and inter-rater reliability (intraclass correlation coefficient = 0.95; 95% confidence interval: 0.92-0.97) were excellent. The minimal detectable change with 90% confidence was 13 points. No floor or ceiling effects were evident. These results indicate unidimensionality, sufficient internal consistency reliability, inter-rater reliability, and construct validity of the DEMMI in patients with a sub-acute stroke. Advantages of the DEMMI in clinical application are the short administration time, no need for special equipment and interval level data. The de Morton Mobility Index, therefore, may be a useful performance-based bedside test to measure mobility in individuals with a sub-acute stroke across the whole mobility spectrum. Implications for Rehabilitation The de Morton Mobility Index (DEMMI) is an unidimensional measurement instrument of mobility in individuals with sub-acute stroke. The DEMMI has excellent internal consistency and inter-rater reliability, and sufficient construct validity. The minimal detectable change of the DEMMI with 90% confidence in stroke rehabilitation is 13 points. The lack of any floor or ceiling effects on hospital admission indicates applicability across the whole mobility spectrum of patients with sub-acute stroke.

Construction and Validation of Textbook Analysis Grids for Ecology and Environmental Education

ERIC Educational Resources Information Center

Caravita, Silvia; Valente, Adriana; Luzi, Daniela; Pace, Paul; Valanides, Nicos; Khalil, Iman; Berthou, Guillemette; Kozan-Naumescu, Adrienne; Clement, Pierre

2008-01-01

Knowledge about ecology and environmental education (EE) constitutes a basic tool for promoting a sustainable future, and was a target area of the BIOHEAD-Citizen Project. School textbooks were considered as representative sources of evidence in terms of ecology and environmental education, and were used for comparison among the countries…

A Psychometric Investigation of the Academic Motivation Scale Using a United States Sample.

ERIC Educational Resources Information Center

Cokley, Kevin O.; Bernard, Naijean; Cunningham, Dana; Motoike, Janice

2001-01-01

Examines the factor structure of the Academic Motivation Scale with a United States student population. There was some support for a 7-factor structure. Evidence of construct validity examining the relationship with academic self concept and academic achievement is mixed. Discusses ethnic and gender differences in motivation. (Contains 37…

Assessing Teamwork and Collaboration in High School Students: A Multimethod Approach

ERIC Educational Resources Information Center

Wang, Lijuan; MacCann, Carolyn; Zhuang, Xiaohua; Liu, Ou Lydia; Roberts, Richard D.

2009-01-01

Various policy papers assert that teamwork is an essential skill for the 21st-century workforce. However, outside of organizational psychology research with adult populations, there are few reliable assessments of this construct with suitable validity evidence for test scores. To redress this issue, self-report, situational judgment, and…

The Perceived Stress Reactivity Scale: Measurement Invariance, Stability, and Validity in Three Countries

ERIC Educational Resources Information Center

Schlotz, Wolff; Yim, Ilona S.; Zoccola, Peggy M.; Jansen, Lars; Schulz, Peter

2011-01-01

There is accumulating evidence that individual differences in stress reactivity contribute to the risk for stress-related disease. However, the assessment of stress reactivity remains challenging, and there is a relative lack of questionnaires reliably assessing this construct. We here present the Perceived Stress Reactivity Scale (PSRS), a…

Work Values, Cognitive Strategies, and Applicant Reactions in a Structured Pre-Employment Interview for Ethical Integrity.

ERIC Educational Resources Information Center

Pawlowski, Donna R.; Hollwitz, John

2000-01-01

Notes that companies emphasize ethical behavior, and schools and professional groups devote many resources to applied ethics training. Describes initial construct validation of a structured ethical integrity pre-employment interview. Reviews evidence relating to cognitive and impression management strategies used when college students encounter an…

Assessing Measurement Invariance for Spanish Sentence Repetition and Morphology Elicitation Tasks

ERIC Educational Resources Information Center

Kapantzoglou, Maria; Thompson, Marilyn S.; Gray, Shelley; Restrepo, M. Adelaida

2016-01-01

Purpose: The purpose of this study was to evaluate evidence supporting the construct validity of two grammatical tasks (sentence repetition, morphology elicitation) included in the Spanish Screener for Language Impairment in Children (Restrepo, Gorin, & Gray, 2013). We evaluated if the tasks measured the targeted grammatical skills in the same…

Cognitive Complexity in the Remote Association Test--Chinese Version

ERIC Educational Resources Information Center

Hung, Su-Pin; Huang, Po-Sheng; Chen, Hsueh-Chih

2016-01-01

The remote association test (RAT) has been applied in various fields; however, evidence of construct validity for the original version and subsequent extensions of the RAT remains limited. This study aimed to elucidate the dimensionality and the relationship between item features and item difficulties for the RAT--Chinese Version (RAT-C) using the…

Trait Sources of Spirituality Scale: Assessing Trait Spirituality More Inclusively

ERIC Educational Resources Information Center

Westbrook, Charles J.; Davis, Don E.; McElroy, Stacey E.; Brubaker, Kacy; Choe, Elise; Karaga, Sara; Dooley, Matt; O'Bryant, Brittany L.; Van Tongeren, Daryl R.; Hook, Joshua

2018-01-01

We develop the Trait Sources of Spirituality Scale (TSSS), which assesses experiences of closeness to the sacred, within and outside a religious tradition. After using factor analysis to finalize the scale, we examine evidence of construct validity, including latent profile analysis that reveals 5 patterns of how spirituality is experienced.

Invited Reaction: The Work Cognition Inventory--Initial Evidence of Construct Validity

ERIC Educational Resources Information Center

Newman, Daniel A.; Joseph, Dana L.; Sparkman, Torrence E.; Carpenter, Nichelle C.

2011-01-01

Employee engagement research is typified by the relabeling and reinvention of classic job attitude concepts. In this article, the authors comment on the development of the Work Cognition Inventory (WCI), an instrument designed to assess eight major antecedents of employee engagement/work passion. The antecedents measured by the WCI include job…

A Multi-Method Multi-Analytic Approach to Establishing Internal Construct Validity Evidence: The Sport Multidimensional Perfectionism Scale 2

ERIC Educational Resources Information Center

Gotwals, John K.; Dunn, John G. H.

2009-01-01

This article presents a chronology of three empirical studies that outline the measurement process by which two new subscales ("Doubts about Actions" and "Organization") were developed and integrated into a revised version of Dunn, Causgrove Dunn, and Syrotuik's (2002) "Sport Multidimensional Perfectionism Scale"…

Life Style Assessment: So What!

ERIC Educational Resources Information Center

Aubry, William E.

The construct life style was used by Alfred Adler to describe the characteristic way in which individuals act and think. Followers of his theories are now collecting evidence to support or validate his contentions. The assessment of client life styles serves: (1) to make the client aware of his misconceptions, (2) as a reference point for therapy,…

Related Critical Psychometric Issues and Their Resolutions during Development of PE Metrics

ERIC Educational Resources Information Center

Fox, Connie; Zhu, Weimo; Park, Youngsik; Fisette, Jennifer L.; Graber, Kim C.; Dyson, Ben; Avery, Marybell; Franck, Marian; Placek, Judith H.; Rink, Judy; Raynes, De

2011-01-01

In addition to validity and reliability evidence, other psychometric qualities of the PE Metrics assessments needed to be examined. This article describes how those critical psychometric issues were addressed during the PE Metrics assessment bank construction. Specifically, issues included (a) number of items or assessments needed, (b) training…

Measures of Instruction for Creative Engagement: Making Metacognition, Modeling and Creative Thinking Visible

ERIC Educational Resources Information Center

Pitts, Christine; Anderson, Ross; Haney, Michele

2018-01-01

The purpose of the current study was to estimate reliability, internal consistency and construct validity of the Measure of Instruction for Creative Engagement (MICE) instrument. The MICE uses an iterative process of evidence collection and scoring through teacher observations to determine instructional domain ratings and overall scores. The…

[A Validation Study of the Modified Korean Version of Ethical Leadership at Work Questionnaire (K-ELW)].

PubMed

Kim, Jeong-Eon; Park, Eun-Jun

2015-04-01

The purpose of this study was to validate the Korean version of the Ethical Leadership at Work questionnaire (K-ELW) that measures RNs' perceived ethical leadership of their nurse managers. The strong validation process suggested by Benson (1998), including translation and cultural adaptation stage, structural stage, and external stage, was used. Participants were 241 RNs who reported their perceived ethical leadership using both the pre-version of K-ELW and a previously known Ethical Leadership Scale, and interactional justice of their managers, as well as their own demographics, organizational commitment and organizational citizenship behavior. Data analyses included descriptive statistics, Pearson correlation coefficients, reliability coefficients, exploratory factor analysis, and confirmatory factor analysis. SPSS 19.0 and Amos 18.0 versions were used. A modified K-ELW was developed from construct validity evidence and included 31 items in 7 domains: People orientation, task responsibility fairness, relationship fairness, power sharing, concern for sustainability, ethical guidance, and integrity. Convergent validity, discriminant validity, and concurrent validity were supported according to the correlation coefficients of the 7 domains with other measures. The results of this study provide preliminary evidence that the modified K-ELW can be adopted in Korean nursing organizations, and reliable and valid ethical leadership scores can be expected.

Problem-solving style and multicultural personality dispositions: a study of construct validity.

PubMed

Houtz, John C; Ponterotto, Joseph G; Burger, Claudia; Marino, Cherylynn

2010-06-01

This exploratory study examined the relationship between problem-solving styles and multicultural personality dispositions among 91 graduate students enrolled in an urban university located in the northeast United States. Problem-solving style was assessed with the three dimensions of the VIEW: an Assessment of Problem Solving Style. Multicultural personality was assessed with the five-factor Multicultural Personality Questionnaire (MPQ); its factors of Cultural Empathy, Open-mindedness, Social Initiative, and Flexibility correlated significantly with Explorer and External problem-solving styles, as predicted. The Emotional Stability subscale also correlated significantly with scores on Explorer style, suggesting that individuals who prefer "thinking in new directions" in problem solving are more likely to report remaining calm under stressful situations. Collectively, study results provided additional evidence of construct validity for the VIEW.

Cultural differences are not always reducible to individual differences.

PubMed

Na, Jinkyung; Grossmann, Igor; Varnum, Michael E W; Kitayama, Shinobu; Gonzalez, Richard; Nisbett, Richard E

2010-04-06

We show that differences in social orientation and in cognition that exist between cultures and social classes do not necessarily have counterparts in individual differences within those groups. Evidence comes from a large-scale study conducted with 10 measures of independent vs. interdependent social orientation and 10 measures of analytic vs. holistic cognitive style. The social measures successfully distinguish between interdependence (viewing oneself as embedded in relations with others) and independence (viewing oneself as disconnected from others) at the group level. However, the correlations among the measures were negligible. Similar results were obtained for the cognitive measures, for which there are no coherent individual differences despite the validity of the construct at the group level. We conclude that behavioral constructs that distinguish among groups need not be valid as measures of individual differences.

A Meta-Analysis of the Convergent Validity of Self-Control Measures

PubMed Central

Duckworth, Angela Lee; Kern, Margaret L.

2011-01-01

There is extraordinary diversity in how the construct of self-control is operationalized in research studies. We meta-analytically examined evidence of convergent validity among executive function, delay of gratification, and self- and informant-report questionnaire measures of self-control. Overall, measures demonstrated moderate convergence (rrandom = .27 [95% CI = .24, .30]; rfixed = .34 [.33, .35], k = 282 samples, N = 33,564 participants), although there was substantial heterogeneity in the observed correlations. Correlations within and across types of self-control measures were strongest for informant-report questionnaires and weakest for executive function tasks. Questionnaires assessing sensation seeking impulses could be distinguished from questionnaires assessing processes of impulse regulation. We conclude that self-control is a coherent but multidimensional construct best assessed using multiple methods. PMID:21643479

Development and validation of a new assessment tool for suturing skills in medical students.

PubMed

Sundhagen, Henriette Pisani; Almeland, Stian Kreken; Hansson, Emma

2018-01-01

In recent years, emphasis has been put on that medical student should demonstrate pre-practice/pre-registration core procedural skills to ensure patient safety. Nonetheless, the formal teaching and training of basic suturing skills to medical students have received relatively little attention and there is no standard for what should be tested and how. The aim of this study was to develop and validate, using scientific methods, a tool for assessment of medical students' suturing skills, measuring both micro- and macrosurgical qualities. A tool was constructed and content, construct, concurrent validity, and inter-rater, inter-item, inter-test reliability were tested. Three groups were included: students with no training in suturing skills, students who have had training, plastic surgery. The results show promising reliability and validity when assessing novice medical students' suturing skills. Further studies are needed on implementation of the instrument. Moreover, how the instrument can be used to give formative feedback, evaluate if a required standard is met and for curriculum development needs further investigation.Level of Evidence: Not ratable.

A Critical Analysis of Assessment Quality in Genomics and Bioinformatics Education Research

PubMed Central

Campbell, Chad E.; Nehm, Ross H.

2013-01-01

The growing importance of genomics and bioinformatics methods and paradigms in biology has been accompanied by an explosion of new curricula and pedagogies. An important question to ask about these educational innovations is whether they are having a meaningful impact on students’ knowledge, attitudes, or skills. Although assessments are necessary tools for answering this question, their outputs are dependent on their quality. Our study 1) reviews the central importance of reliability and construct validity evidence in the development and evaluation of science assessments and 2) examines the extent to which published assessments in genomics and bioinformatics education (GBE) have been developed using such evidence. We identified 95 GBE articles (out of 226) that contained claims of knowledge increases, affective changes, or skill acquisition. We found that 1) the purpose of most of these studies was to assess summative learning gains associated with curricular change at the undergraduate level, and 2) a minority (<10%) of studies provided any reliability or validity evidence, and only one study out of the 95 sampled mentioned both validity and reliability. Our findings raise concerns about the quality of evidence derived from these instruments. We end with recommendations for improving assessment quality in GBE. PMID:24006400

Interest in Aesthetic Rhinoplasty Scale.

PubMed

Naraghi, Mohsen; Atari, Mohammad

2017-04-01

Interest in cosmetic surgery is increasing, with rhinoplasty being one of the most popular surgical procedures. It is essential that surgeons identify patients with existing psychological conditions before any procedure. This study aimed to develop and validate the Interest in Aesthetic Rhinoplasty Scale (IARS). Four studies were conducted to develop the IARS and to evaluate different indices of validity (face, content, construct, criterion, and concurrent validities) and reliability (internal consistency, split-half coefficient, and temporal stability) of the scale. The four study samples included a total of 463 participants. Statistical analysis revealed satisfactory psychometric properties in all samples. Scores on the IARS were negatively correlated with self-esteem scores ( r  = -0.296; p  < 0.01) and positively associated with scores for psychopathologic symptoms ( r  = 0.164; p  < 0.05), social dysfunction ( r  = 0.268; p  < 0.01), and depression ( r  = 0.308; p  < 0.01). The internal and test-retest coefficients of consistency were found to be high (α = 0.93; intraclass coefficient = 0.94). Rhinoplasty patients were found to have significantly higher IARS scores than nonpatients ( p  < 0.001). Findings of the present studies provided evidence for face, content, construct, criterion, and concurrent validities and internal and test-retest reliability of the IARS. This evidence supports the use of the scale in clinical and research settings. Thieme Medical Publishers 333 Seventh Avenue, New York, NY 10001, USA.

Measuring attitudes towards suicide: Preliminary evaluation of an attitude towards suicide scale.

PubMed

Cwik, Jan Christopher; Till, Benedikt; Bieda, Angela; Blackwell, Simon E; Walter, Carolin; Teismann, Tobias

2017-01-01

Our study aimed to validate a previously published scale assessing attitudes towards suicide. Factor structure, convergent and discriminant validity, and predictive validity were investigated. Adult German participants (N=503; mean age=24.74years; age range=18-67years) anonymously completed a set of questionnaires. An exploratory factor analysis was conducted, and incongruous items were deleted. Subsequently, scale properties of the reduced scale and its construct validity were analyzed. A confirmatory factor analysis was then conducted in an independent sample (N=266; mean age=28.77years; age range=18-88years) to further confirm the factor structure of the questionnaire. Parallel analysis indicated a three-factor solution, which was also supported by confirmatory factor analysis: right to commit suicide, interpersonal gesture and resilience. The subscales demonstrated acceptable construct and discriminant validity. Cronbach's α for the subscales ranged from 0.67 to 0.83, explaining 49.70% of the total variance. Positive attitudes towards suicide proved to be predictive of suicide risk status, providing preliminary evidence for the utility of the scale. Future studies aiming to reproduce the factor structure in a more heterogeneous sample are warranted. Copyright © 2016 Elsevier Inc. All rights reserved.

Reliability and Construct Validity of the Portuguese Version of the Psychological Capital Questionnaire.

PubMed

Antunes, Ana Cristina; Caetano, António; Pina E Cunha, Miguel

2017-06-01

The Psychological Capital Questionnaire (PCQ) is the most commonly used measure for assessing psychological capital in work settings. Although several studies confirmed its factorial validity, most validation studies only examined the four-factor structure preconized by Luthans, Youssef, and Avolio, not attending to empirical evidence on alternative factorial structures. The present study aimed to test the psychometric properties of the Portuguese version of the PCQ, by using two independent samples (NS1 = 542; NS2 = 115) of Portuguese employees. We conducted a series of confirmatory factor analyses and found that, unlike previous findings, a five-factor solution of the PCQ best fitted the data. The evidence obtained also supported the existence of a second-order factor, psychological capital. The coefficients of internal consistency, as measured by Cronbach's alpha, were adequate and test-retest reliability suggested that the PCQ presented a lower stability than personality factors. Convergent validity, assessed with average variance extracted, revealed problems in the optimism subscale. The discriminant validity of the PCQ was confirmed by its correlations with Positive and Negative Affect and Big Five personality factors. Hierarchical regression analyses showed that this measure has incremental validity over personality and affect when predicting job performance.

Translations of Developmental Screening Instruments: An Evidence Map of Available Research.

PubMed

El-Behadli, Ana F; Neger, Emily N; Perrin, Ellen C; Sheldrick, R Christopher

2015-01-01

Children whose parents do not speak English experience significant disparities in the identification of developmental delays and disorders; however, little is known about the availability and validity of translations of developmental screeners. The goal was to create a map of the scientific evidence regarding translations of the 9 Academy of Pediatrics-recommended screening instruments into languages other than English. The authors conducted a systematic search of Medline and PsycINFO, references of identified articles, publishers' Web sites, and official manuals. Through evidence mapping, a new methodology supported by AHRQ and the Cochrane Collaboration, the authors documented the extent and distribution of published evidence supporting translations of developmental screeners. Data extraction focused on 3 steps of the translation and validation process: (1) translation methods used, (2) collection of normative data in the target language, and (3) evidence for reliability and validity. The authors identified 63 distinct translations among the 9 screeners, of which 44 had supporting evidence published in peer-reviewed sources. Of the 63 translations, 35 had at least some published evidence regarding translation methods used, 28 involving normative data, and 32 regarding reliability and/or construct validity. One-third of the translations found were of the Denver Developmental Screening Test. Specific methods used varied greatly across screeners, as did the level of detail with which results were reported. Few developmental screeners have been translated into many languages. Evidence map of the authors demonstrates considerable variation in both the amount and the comprehensiveness of information available about translated instruments. Informal guidelines exist for conducting translation of psychometric instruments but not for documentation of this process. The authors propose that uniform guidelines be established for reporting translation research in peer-reviewed journals, similar to those for clinical trials and studies of diagnostic accuracy.

Construct validity of the Health Science Reasoning Test.

PubMed

Huhn, Karen; Black, Lisa; Jensen, Gail M; Deutsch, Judith E

2011-01-01

The aim of this study was to evaluate the construct validity of the Health Science Reasoning Test (HSRT) by determining if the test could discriminate between expert and novice physical therapists' critical-thinking skills. Experts identified from a random list of certified clinical specialists and students in the first year of their physical therapy education from two physical therapy programs completed the HSRT. Experts (n = 73) had a higher total HSRT score (mean 24.06, SD 3.92) than the novices (n = 79) (mean 22.49, SD 3.2), with the difference being statistically significant t (148) = 2.67, p = 0.008. The HSRT total score discriminated between expert and novice critical-thinking skills, therefore establishing construct validity. To our knowledge, this is the first study to compare expert and novice performance on a standardized test. The opportunity to have a tool that provides evidence of students' critical thinking skills could be helpful for educators and students. The test results could aid in identifying areas of students' strengths and weaknesses, thereby enabling targeted remediation to improve critical thinking skills, which are key factors in clinical reasoning, a necessary skill for effective physical therapy practice.

The Individualized Classroom Assessment Scoring System (inCLASS): Preliminary Reliability and Validity of a System for Observing Preschoolers’ Competence in Classroom Interactions

PubMed Central

Downer, Jason T.; Booren, Leslie M.; Lima, Olivia K.; Luckner, Amy E.; Pianta, Robert C.

2012-01-01

This paper introduces the Individualized Classroom Assessment Scoring System (inCLASS), an observation tool that targets children’s interactions in preschool classrooms with teachers, peers, and tasks. In particular, initial evidence is reported of the extent to which the inCLASS meets the following psychometric criteria: inter-rater reliability, normal distributions and adequate range, construct validity, and criterion-related validity. These initial findings suggest that the inCLASS has the potential to provide an authentic, contextualized assessment of young children’s classroom behaviors. Future directions for research with the inCLASS are discussed. PMID:23175598

Using Hospital Anxiety and Depression Scale (HADS) on patients with epilepsy: Confirmatory factor analysis and Rasch models.

PubMed

Lin, Chung-Ying; Pakpour, Amir H

2017-02-01

The problems of mood disorders are critical in people with epilepsy. Therefore, there is a need to validate a useful tool for the population. The Hospital Anxiety and Depression Scale (HADS) has been used on the population, and showed that it is a satisfactory screening tool. However, more evidence on its construct validity is needed. A total of 1041 people with epilepsy were recruited in this study, and each completed the HADS. Confirmatory factor analysis (CFA) and Rasch analysis were used to understand the construct validity of the HADS. In addition, internal consistency was tested using Cronbachs' α, person separation reliability, and item separation reliability. Ordering of the response descriptors and the differential item functioning (DIF) were examined using the Rasch models. The HADS showed that 55.3% of our participants had anxiety; 56.0% had depression based on its cutoffs. CFA and Rasch analyses both showed the satisfactory construct validity of the HADS; the internal consistency was also acceptable (α=0.82 in anxiety and 0.79 in depression; person separation reliability=0.82 in anxiety and 0.73 in depression; item separation reliability=0.98 in anxiety and 0.91 in depression). The difficulties of the four-point Likert scale used in the HADS were monotonically increased, which indicates no disordering response categories. No DIF items across male and female patients and across types of epilepsy were displayed in the HADS. The HADS has promising psychometric properties on construct validity in people with epilepsy. Moreover, the additive item score is supported for calculating the cutoff. Copyright © 2016 British Epilepsy Association. Published by Elsevier Ltd. All rights reserved.

FG7142, yohimbine, and βCCE produce anxiogenic-like effects in the elevated plus-maze but do not affect brainstem activated hippocampal theta.

PubMed

Yeung, Michelle; Lu, Lily; Hughes, Adam M; Treit, Dallas; Dickson, Clayton T

2013-12-01

The neurobiological underpinnings of anxiety are of paramount importance to selective and efficacious pharmaceutical intervention. Hippocampal theta frequency in urethane anaesthetized rats is suppressed by all known (and some previously unknown) anti-anxiety (anxiolytic) drugs. Although these findings support the predictive validity of this assay, its construct validity (i.e., whether theta frequency actually indexes anxiety per se) has not been a subject of systematic investigation. We reasoned that if anxiolytic drugs suppress hippocampal theta frequency, then drugs that increase anxiety (i.e., anxiogenic agents) should increase theta frequency, thus providing evidence of construct validity. We used three proven anxiogenic drugs--two benzodiazepine receptor inverse agonists, N-methyl-β-carboline-3-carboxamide (FG7142) and β-carboline-3-carboxylate ethyl ester (βCCE), and one α2 noradrenergic receptor antagonist, 17α-hydroxy-yohimban-16α-carboxylic acid methyl ester (yohimbine) as pharmacological probes to assess the construct validity of the theta model. Although all three anxiogenic drugs significantly increased behavioural measures of anxiety in the elevated plus-maze, none of the three increased the frequency of hippocampal theta oscillations in the neurophysiological model. As a positive control, we demonstrated that diazepam, a proven anxiolytic drug, decreased the frequency of hippocampal theta, as in all other studies using this model. Given this discrepancy between the significant effects of anxiogenic drugs in the behavioural model and the null effects of these drugs in the neurophysiological model, we conclude that the construct validity of the hippocampal theta model of anxiety is questionable. Copyright © 2013 Elsevier Ltd. All rights reserved.

Validation of GC and HPLC systems for residue studies

DOE Office of Scientific and Technical Information (OSTI.GOV)

Williams, M.

1995-12-01

For residue studies, GC and HPLC system performance must be validated prior to and during use. One excellent measure of system performance is the standard curve and associated chromatograms used to construct that curve. The standard curve is a model of system response to an analyte over a specific time period, and is prima facia evidence of system performance beginning at the auto sampler and proceeding through the injector, column, detector, electronics, data-capture device, and printer/plotter. This tool measures the performance of the entire chromatographic system; its power negates most of the benefits associated with costly and time-consuming validation ofmore » individual system components. Other measures of instrument and method validation will be discussed, including quality control charts and experimental designs for method validation.« less

A mixed methods approach to adapting and evaluating the functional assessment of HIV infection (FAHI), Swahili version, for use with low literacy populations

PubMed Central

Sigilai, Antipa; Hassan, Amin S.; Thoya, Janet; Odhiambo, Rachael; Van de Vijver, Fons J. R.; Newton, Charles R. J. C.; Abubakar, Amina

2017-01-01

Background Despite bearing the largest HIV-related burden, little is known of the Health-Related Quality of Life (HRQoL) among people living with HIV in sub-Saharan Africa. One of the factors contributing to this gap in knowledge is the lack of culturally adapted and validated measures of HRQoL that are relevant for this setting. Aims We set out to adapt the Functional Assessment of HIV Infection (FAHI) Questionnaire, an HIV-specific measure of HRQoL, and evaluate its internal consistency and validity. Methods The three phase mixed-methods study took place in a rural setting at the Kenyan Coast. Phase one involved a scoping review to describe the evidence base of the reliability and validity of FAHI as well as the geographical contexts in which it has been administered. Phase two involved in-depth interviews (n = 38) to explore the content validity, and initial piloting for face validation of the adapted FAHI. Phase three was quantitative (n = 103) and evaluated the internal consistency, convergent and construct validities of the adapted interviewer-administered questionnaire. Results In the first phase of the study, we identified 16 studies that have used the FAHI. Most (82%) were conducted in North America. Only seven (44%) of the reviewed studies reported on the psychometric properties of the FAHI. In the second phase, most of the participants (37 out of 38) reported satisfaction with word clarity and content coverage whereas 34 (89%) reported satisfaction with relevance of the items, confirming the face validity of the adapted questionnaire during initial piloting. Our participants indicated that HIV impacted on their physical, functional, emotional, and social wellbeing. Their responses overlapped with items in four of the five subscales of the FAHI Questionnaire establishing its content validity. In the third phase, the internal consistency of the scale was found to be satisfactory with subscale Cronbach’s α ranging from 0.55 to 0.78. The construct and convergent validity of the tool were supported by acceptable factor loadings for most of the items on the respective sub-scales and confirmation of expected significant correlations of the FAHI subscale scores with scores of a measure of common mental disorders. Conclusion The adapted interviewer-administered Swahili version of FAHI questionnaire showed initial strong evidence of good psychometric properties with satisfactory internal consistency and acceptable validity (content, face, and convergent validity). It gives impetus for further validation work, especially construct validity, in similar settings before it can be used for research and clinical purposes in the entire East African region. PMID:28380073

Theoretically informed correlates of hepatitis B knowledge among four Asian groups: the health behavior framework.

PubMed

Maxwell, Annette E; Stewart, Susan L; Glenn, Beth A; Wong, Weng Kee; Yasui, Yutaka; Chang, L Cindy; Taylor, Victoria M; Nguyen, Tung T; Chen, Moon S; Bastani, Roshan

2012-01-01

Few studies have examined theoretically informed constructs related to hepatitis B (HBV) testing, and comparisons across studies are challenging due to lack of uniformity in constructs assessed. The present analysis examined relationships among Health Behavior Framework factors across four Asian American groups to advance the development of theory-based interventions for HBV testing in at-risk populations. Data were collected from 2007-2010 as part of baseline surveys during four intervention trials promoting HBV testing among Vietnamese-, Hmong-, Korean- and Cambodian-Americans (n = 1,735). Health Behavior Framework constructs assessed included: awareness of HBV, knowledge of transmission routes, perceived susceptibility, perceived severity, doctor recommendation, stigma of HBV infection, and perceived efficacy of testing. Within each group we assessed associations between our intermediate outcome of knowledge of HBV transmission and other constructs, to assess the concurrent validity of our model and instruments. While the absolute levels for Health Behavior Framework factors varied across groups, relationships between knowledge and other factors were generally consistent. This suggests similarities rather than differences with respect to posited drivers of HBV-related behavior. Our findings indicate that Health Behavior Framework constructs are applicable to diverse ethnic groups and provide preliminary evidence for the construct validity of the Health Behavior Framework.

Theoretically Informed Correlates of Hepatitis B Knowledge among Four Asian Groups: The Health Behavior Framework

PubMed Central

Maxwell, AE; Stewart, SL; Glenn, BA; Wong, WK; Yasui, Y; Chang, LC; Taylor, VM; Nguyen, TT; Chen, MS; Bastani, R

2012-01-01

Background Few studies have examined theoretically informed constructs related to hepatitis B (HBV) testing, and comparisons across studies is challenging due to lack of uniformity in constructs assessed. This analysis examines relationships among Health Behavior Framework factors across four Asian American groups to advance the development of theory-based interventions for HBV testing in at-risk populations. Methods Data were collected from 2007–2010 as part of baseline surveys during four intervention trials promoting HBV testing among Vietnamese-, Hmong-, Korean- and Cambodian-Americans (n = 1,735). Health Behavior Framework constructs assessed included: awareness of HBV, knowledge of transmission routes, perceived susceptibility, perceived severity, doctor recommendation, stigma of HBV infection, and perceived efficacy of testing. Within each group we assessed associations between our intermediate outcome of knowledge of HBV transmission and other constructs, to assess the concurrent validity of our model and instruments. Results While the absolute levels for Health Behavior Framework factors varied across groups, relationships between knowledge and other factors were generally consistent. This suggests similarities rather than differences with respect to posited drivers of HBV-related behavior. Discussion Our findings indicate that Health Behavior Framework constructs are applicable to diverse ethnic groups and provide preliminary evidence for the construct validity of the Health Behavior Framework. PMID:22799389

Reliable and valid assessment of point-of-care ultrasonography.

PubMed

Todsen, Tobias; Tolsgaard, Martin Grønnebæk; Olsen, Beth Härstedt; Henriksen, Birthe Merete; Hillingsø, Jens Georg; Konge, Lars; Jensen, Morten Lind; Ringsted, Charlotte

2015-02-01

To explore the reliability and validity of the Objective Structured Assessment of Ultrasound Skills (OSAUS) scale for point-of-care ultrasonography (POC US) performance. POC US is increasingly used by clinicians and is an essential part of the management of acute surgical conditions. However, the quality of performance is highly operator-dependent. Therefore, reliable and valid assessment of trainees' ultrasonography competence is needed to ensure patient safety. Twenty-four physicians, representing novices, intermediates, and experts in POC US, scanned 4 different surgical patient cases in a controlled set-up. All ultrasound examinations were video-recorded and assessed by 2 blinded radiologists using OSAUS. Reliability was examined using generalizability theory. Construct validity was examined by comparing performance scores between the groups and by correlating physicians' OSAUS scores with diagnostic accuracy. The generalizability coefficient was high (0.81) and a D-study demonstrated that 1 assessor and 5 cases would result in similar reliability. The construct validity of the OSAUS scale was supported by a significant difference in the mean scores between the novice group (17.0; SD 8.4) and the intermediate group (30.0; SD 10.1), P = 0.007, as well as between the intermediate group and the expert group (72.9; SD 4.4), P = 0.04, and by a high correlation between OSAUS scores and diagnostic accuracy (Spearman ρ correlation coefficient = 0.76; P < 0.001). This study demonstrates high reliability as well as evidence of construct validity of the OSAUS scale for assessment of POC US competence. Hence, the OSAUS scale may be suitable for both in-training as well as end-of-training assessment.

Construct validity of the Nutrition and Activity Knowledge Scale in a French sample of adolescents with mild to moderate intellectual disability.

PubMed

Maïano, Christophe; Bégarie, Jérôme; Morin, Alexandre J S; Garbarino, Jean-Marie; Ninot, Grégory

2010-01-01

The purpose of this study was to test the reliability (i.e. internal consistency and test-retest reliability) and construct validity (i.e. content validity, factor validity, measurement invariance, and latent mean invariance) of the Nutrition and Activity Knowledge Scale (NAKS) in a sample of French adolescents with mild to moderate Intellectual Disability (ID). A total sample of 260 adolescents (144 boys and 116 girls), aged between 12 and 18 years old, with mild to moderate ID was involved in two studies. In the first study, analysis of items' content reveals that many words from the original version were not understood or induced confusion. These items were reworded and simplified while retaining their original meaning. In the second study, results provided support for: (i) the factor validity and reliability of a 15-item French version of the NAKS; (ii) the measurement invariance of the resulting NAKS across genders and ID levels; (iii) the partial measurement invariance of the resulting NAKS across age groups and type of school placement. In addition, the latent means of the 15-item French version of the NAKS proved to be invariant across gender, age categories, and ID levels, but to vary across type of school placement (with adolescents schooled in self-contained classes from regular schools presenting higher levels of NAK than adolescents placed in specialized establishments). The present results thus provide preliminary evidence regarding the construct validity of a 15-item French version of the NAKS in a sample of adolescents with ID.

Center for Epidemiologic Studies Depression Scale for Children: psychometric testing of the Chinese version.

PubMed

Li, Ho Cheung William; Chung, Oi Kwan Joyce; Ho, Ka Yan

2010-11-01

This paper is a report of psychometric testing of the Chinese version of the Center for Epidemiologic Studies Depression Scale for Children. The availability of a valid and reliable instrument that accurately detects depressive symptoms in children is crucial before any psychological intervention can be appropriately planned and evaluated. There is no such an instrument for Chinese children. A test-retest, within-subjects design was used. A total of 313 primary school students between the ages of 8 and 12 years were invited to participate in the study in 2009. Participants were asked to respond to the Chinese version of the Center for Epidemiologic Studies Depression Scale for Children, short form of the State Anxiety Scale for Children and Rosenberg's Self-Esteem Scale. The internal consistency, content validity and construct validity and test-retest reliability of the Chinese version of the Center for Epidemiologic Studies Depression Scale for Children were assessed. The newly-translated scale demonstrated adequate internal consistency, good content validity and appropriate convergent and discriminant validity. Confirmatory factor analysis added further evidence of the construct validity of the scale. Results suggest that the newly-translated scale can be used as a self-report assessment tool in detecting depressive symptoms of Chinese children aged between 8 and 12 years. © 2010 Blackwell Publishing Ltd.

Herth hope index: psychometric testing of the Chinese version.

PubMed

Chan, Keung Sum; Li, Ho Cheung William; Chan, Sally Wai-Chi; Lopez, Violeta

2012-09-01

This article is a report on psychometric testing of the Chinese version of the herth hope index. The availability of a valid and reliable instrument that accurately measures the level of hope in patients with heart failure is crucial before any hope-enhancing interventions can be appropriately planned and evaluated. There is no such instrument for Chinese people. A test-retest, within-subjects design was used. A purposive sample of 120 Hong Kong Chinese patients with heart failure between the ages of 60 and 80 years admitted to two medical wards was recruited during an 8-month period in 2009. Participants were asked to respond to the Chinese version of the herth hope index, Hamilton depression rating scale and Rosenberg's self-esteem scale. The internal consistency, content validity and construct validity and test-retest reliability of the Chinese version of the herth hope index were assessed. The newly translated scale demonstrated adequate internal consistency, good content validity and appropriate convergent and discriminant validity. Confirmatory factor analysis added further evidence of the construct validity of the scale. Results suggest that the newly translated scale can be used as a self-report assessment tool in assessing the level of hope in Hong Kong Chinese patients with heart failure. © 2011 Blackwell Publishing Ltd.

A Psychometric Analysis of the Italian Version of the eHealth Literacy Scale Using Item Response and Classical Test Theory Methods.

PubMed

Diviani, Nicola; Dima, Alexandra Lelia; Schulz, Peter Johannes

2017-04-11

The eHealth Literacy Scale (eHEALS) is a tool to assess consumers' comfort and skills in using information technologies for health. Although evidence exists of reliability and construct validity of the scale, less agreement exists on structural validity. The aim of this study was to validate the Italian version of the eHealth Literacy Scale (I-eHEALS) in a community sample with a focus on its structural validity, by applying psychometric techniques that account for item difficulty. Two Web-based surveys were conducted among a total of 296 people living in the Italian-speaking region of Switzerland (Ticino). After examining the latent variables underlying the observed variables of the Italian scale via principal component analysis (PCA), fit indices for two alternative models were calculated using confirmatory factor analysis (CFA). The scale structure was examined via parametric and nonparametric item response theory (IRT) analyses accounting for differences between items regarding the proportion of answers indicating high ability. Convergent validity was assessed by correlations with theoretically related constructs. CFA showed a suboptimal model fit for both models. IRT analyses confirmed all items measure a single dimension as intended. Reliability and construct validity of the final scale were also confirmed. The contrasting results of factor analysis (FA) and IRT analyses highlight the importance of considering differences in item difficulty when examining health literacy scales. The findings support the reliability and validity of the translated scale and its use for assessing Italian-speaking consumers' eHealth literacy. ©Nicola Diviani, Alexandra Lelia Dima, Peter Johannes Schulz. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 11.04.2017.

The development and validation of the client expectations of massage scale.

PubMed

Boulanger, Karen T; Campo, Shelly; Glanville, Jennifer L; Lowe, John B; Yang, Jingzhen

2012-01-01

Although there is evidence that client expectations influence client outcomes, a valid and reliable scale for measuring the range of client expectations for both massage therapy and the behaviors of their massage therapists does not exist. Understanding how client expectations influence client outcomes would provide insight into how massage achieves its reported effects. To develop and validate the Client Expectations of Massage Scale (CEMS), a measure of clients' clinical, educational, interpersonal, and outcome expectations. Offices of licensed massage therapists in Iowa. A practice-based research methodology was used to collect data from two samples of massage therapy clients. For Sample 1, 21 volunteer massage therapists collected data from their clients before the massage. Factor analysis was conducted to test construct validity and coefficient alpha was used to assess reliability. Correlational analyses with the CEMS, previous measures of client expectations, and the Life Orientation Test-Revised were examined to test the convergent and discriminant validity of the CEMS. For Sample 2, 24 massage therapists distributed study materials for clients to complete before and after a massage therapy session. Structural equation modeling was used to assess the construct, discriminant, and predictive validity of the CEMS. Sample 1 involved 320 and Sample 2 involved 321 adult massage clients. Standard care provided by licensed massage therapists. Numeric Rating Scale for pain and Positive and Negative Affect Schedule-Revised (including the Serenity subscale). The CEMS demonstrated good construct, convergent, discriminant and predictive validity, and adequate reliability. Client expectations were generally positive toward massage and their massage therapists. Positive outcome expectations had a positive effect on clients' changes in pain and serenity. High interpersonal expectations had a negative effect on clients' changes in serenity. Client expectations contribute to the nonspecific effects of massage therapy.

Validation of the Impostor Phenomenon among Managers

PubMed Central

Rohrmann, Sonja; Bechtoldt, Myriam N.; Leonhardt, Mona

2016-01-01

Following up on earlier investigations, the present research aims at validating the construct impostor phenomenon by taking other personality correlates into account and to examine whether the impostor phenomenon is a construct in its own right. In addition, gender effects as well as associations with dispositional working styles and strain are examined. In an online study we surveyed a sample of N = 242 individuals occupying leadership positions in different sectors. Confirmatory factor analyses provide empirical evidence for the discriminant validity of the impostor phenomenon. In accord with earlier studies we show that the impostor phenomenon is accompanied by higher levels of anxiety, dysphoric moods, emotional instability, a generally negative self-evaluation, and perfectionism. The study does not reveal any gender differences concerning the impostor phenomenon. With respect to working styles, persons with an impostor self-concept tend to show perfectionist as well as procrastinating behaviors. Moreover, they report being more stressed and strained by their work. In sum, the findings show that the impostor phenomenon constitutes a dysfunctional personality style. Practical implications are discussed. PMID:27313554

Development and Psychometric Validation of the Dementia Attitudes Scale

PubMed Central

O'Connor, Melissa L.; McFadden, Susan H.

2010-01-01

This study employed qualitative construct mapping and factor analysis to construct a scale to measure attitudes toward dementia. Five family caregivers, five professionals, and five college students participated in structured interviews. Qualitative analysis of the interviews led to a 46-item scale, which was reduced to 20 items following principal axis factoring with two different samples: college students (N = 302) and certified nursing assistant students (N = 145). Confirmatory factor analysis was then conducted with another sample of college students (N = 157). The final scale, titled the Dementia Attitudes Scale (DAS), essentially had a two-factor structure; the factors were labeled “dementia knowledge” and “social comfort.” Total-scale Cronbach's alphas ranged 0.83–0.85. Evidence for convergent validity was promising, as the DAS correlated significantly with scales that measured ageism and attitudes toward disabilities (range of correlations = 0.44–0.55; mean correlation = 0.50). These findings demonstrate the reliability and validity of the DAS, supporting its use as a research tool.

Diagnosis of borderline personality disorder in China: current status and future directions.

PubMed

Zhong, Jie; Leung, Freedom

2009-02-01

This paper reviews the current status and future directions of borderline personality disorder (BPD) research in China. Although the committee of the third version of the Chinese Classification of Mental Disorders (CCMD-3) rejected BPD as a valid diagnostic category and instead adopted the term impulsive personality disorder (IPD), our literature review on personality disorders from 1979 to 2008 in China indicated that BPD was the most popular research topic among researchers and clinicians. Available empiric evidence on BPD in China provided preliminary support for the construct validity and clinical utility of BPD in clinical and nonclinical Chinese samples. Future studies in the following areas are suggested: 1) developing reliable assessment instruments for measuring BPD pathology in China, 2) comparing the construct validity and phenomenology of CCMD IPD and DSM BPD among Chinese patients, 3) examining potential cultural differences in symptom expression of BPD pathology among the Chinese, and 4) exploring indigenous and imported methods for treating BPD patients in China.

Development and validation of the Body, Eating, and Exercise Comparison Orientation Measure (BEECOM) among college women.

PubMed

Fitzsimmons-Craft, Ellen E; Bardone-Cone, Anna M; Harney, Megan B

2012-09-01

We constructed and validated a measure of comparison dimensions associated with eating pathology, namely, the body, eating, and exercise comparison orientation measure (BEECOM). Participants were 441 undergraduate women. In Study 1, items were generated and refined via exploratory factor analysis, yielding three interpretable factors (i.e., body, eating, and exercise comparison orientation). Confirmatory factor analysis was then used to confirm the three-factor structure of the BEECOM and to investigate the potential presence of a higher-order factor. Given that the lower-order factors loaded strongly onto a higher-order factor, it is appropriate to use a total BEECOM score, in addition to subscale scores. Further, the BEECOM's scores yielded evidence of internal consistency and construct validity in this sample. Study 2 demonstrated two-week test-retest reliability of the BEECOM among college women. Overall, the BEECOM demonstrated good psychometric properties and may be useful for more comprehensively assessing eating disorder-related social comparison behavior. Copyright © 2012 Elsevier Ltd. All rights reserved.

Evidence of Construct Validity in the Assessment of Hebephilia.

PubMed

Stephens, Skye; Seto, Michael C; Goodwill, Alasdair M; Cantor, James M

2017-01-01

Hebephilia refers to a persistent intense sexual interest in pubescent children. Although not as widely studied as pedophilia, studies of hebephilia have indicated convergence in self-report and sexual arousal. The present study expanded on previous work by examining convergent and divergent validity across indicators of hebephilia that included self-report, sexual behavior, and sexual arousal in a sample of 2238 men who had sexually offended. We included men who denied such interest and specifically examined the overlap between hebephilia and pedophilia and examined pedohebephilia (i.e., sexual interests in both prepubescent and pubescent children). Results indicated that there was considerable convergence across indicators of hebephilia. The results suggested poor divergent validity between hebephilia and pedophilia, as there was substantial overlap between the two constructs across analyses. Finally, a distinct pattern of sexual arousal was found in offenders with pedohebephilia. The results of the present study were discussed with a focus on implications for the assessment of sexual interest in children and the conceptualization of pedohebephilia.

Attitudes toward science: measurement and psychometric properties of the Test of Science-Related Attitudes for its use in Spanish-speaking classrooms

NASA Astrophysics Data System (ADS)

Navarro, Marianela; Förster, Carla; González, Caterina; González-Pose, Paulina

2016-06-01

Understanding attitudes toward science and measuring them remain two major challenges for science teaching. This article reviews the concept of attitudes toward science and their measurement. It subsequently analyzes the psychometric properties of the Test of Science-Related Attitudes (TOSRA), such as its construct validity, its discriminant and concurrent validity, and its reliability. The evidence presented suggests that TOSRA, in its Spanish-adapted version, has adequate construct validity regarding its theoretical referents, as well as good indexes of reliability. In addition, it determines the attitudes toward science of secondary school students in Santiago de Chile (n = 664) and analyzes the sex variable as a differentiating factor in such attitudes. The analysis by sex revealed low-relevance gender difference. The results are contrasted with those obtained in English-speaking countries. This TOSRA sample showed good psychometric parameters for measuring and evaluating attitudes toward science, which can be used in classrooms of Spanish-speaking countries or with immigrant populations with limited English proficiency.

Cross-Cultural Adaptation and Validation of the SWAL-QoL Questionnaire in Greek.

PubMed

Georgopoulos, Voula C; Perdikogianni, Myrto; Mouskenteri, Myrto; Psychogiou, Loukia; Oikonomou, Maria; Malandraki, Georgia A

2018-02-01

The purpose of this study was to translate and adapt the 44-item SWAL-QoL into Greek and examine its internal consistency, test-retest reliability, external construct validity, and discriminant validity in order to provide a validated dysphagia-specific QoL instrument in the Greek language. The instrument was translated into Greek using the back translation to ensure linguistic validity and was culturally adapted resulting in the SWAL-QoL-GR. Two groups of participants were included: a patient group of 86 adults (48 males; age range: 18-87 years) diagnosed with oropharyngeal dysphagia, and an age-matched healthy control group (39 adults; 19 males; age range: 18-84 years). The Greek 30-item version of the WHOQOL-BREF was used for assessment of construct validity. Overall, the questionnaire achieved good to excellent psychometric values. Internal consistency of all 10 subscales and the physical symptoms scale of the SWAL-QoL-GR assessed by Cronbach's α was good to excellent (0.811 < α < 0.940). Test-retest validity was found to be good to excellent as well. In addition, moderate to strong correlations were found between seven of the ten subscales of the SWAL-QoL-GR with limited items of the WHOQΟL-BREF (0.401 < ρ < 0.65), supporting good construct validity of the SWAL-QoL-GR. The SWAL-QoL-GR also correctly differentiated between patients with dysphagia and age-matched healthy controls (p < 0.001) on all 11 scales, further indicating excellent discriminant validity. Finally, no significant differences were found between the two sexes. This cultural adaptation and validation allows the use of this tool in Greece, further enhancing our clinical and scientific efforts to increase the evidence-based practice resources for dysphagia rehabilitation in Greece.

Construct Validity: Advances in Theory and Methodology

PubMed Central

Strauss, Milton E.; Smith, Gregory T.

2008-01-01

Measures of psychological constructs are validated by testing whether they relate to measures of other constructs as specified by theory. Each test of relations between measures reflects on the validity of both the measures and the theory driving the test. Construct validation concerns the simultaneous process of measure and theory validation. In this chapter, we review the recent history of validation efforts in clinical psychological science that has led to this perspective, and we review five recent advances in validation theory and methodology of importance for clinical researchers. These are: the emergence of nonjustificationist philosophy of science; an increasing appreciation for theory and the need for informative tests of construct validity; valid construct representation in experimental psychopathology; the need to avoid representing multidimensional constructs with a single score; and the emergence of effective new statistical tools for the evaluation of convergent and discriminant validity. PMID:19086835

Patients' and observers' perceptions of involvement differ. Validation study on inter-relating measures for shared decision making.

PubMed

Kasper, Jürgen; Heesen, Christoph; Köpke, Sascha; Fulcher, Gary; Geiger, Friedemann

2011-01-01

Patient involvement into medical decisions as conceived in the shared decision making method (SDM) is essential in evidence based medicine. However, it is not conclusively evident how best to define, realize and evaluate involvement to enable patients making informed choices. We aimed at investigating the ability of four measures to indicate patient involvement. While use and reporting of these instruments might imply wide overlap regarding the addressed constructs this assumption seems questionable with respect to the diversity of the perspectives from which the assessments are administered. The study investigated a nested cohort (N = 79) of a randomized trial evaluating a patient decision aid on immunotherapy for multiple sclerosis. Convergent validities were calculated between observer ratings of videotaped physician-patient consultations (OPTION) and patients' perceptions of the communication (Shared Decision Making Questionnaire, Control Preference Scale & Decisional Conflict Scale). OPTION reliability was high to excellent. Communication performance was low according to OPTION and high according to the three patient administered measures. No correlations were found between observer and patient judges, neither for means nor for single items. Patient report measures showed some moderate correlations. Existing SDM measures do not refer to a single construct. A gold standard is missing to decide whether any of these measures has the potential to indicate patient involvement. Pronounced heterogeneity of the underpinning constructs implies difficulties regarding the interpretation of existing evidence on the efficacy of SDM. Consideration of communication theory and basic definitions of SDM would recommend an inter-subjective focus of measurement. Controlled-Trials.com ISRCTN25267500.

Psychometric properties of a short form of the Center for Epidemiologic Studies Depression (CES-D-10) scale for screening depressive symptoms in healthy community dwelling older adults.

PubMed

Mohebbi, Mohammadreza; Nguyen, Van; McNeil, John J; Woods, Robyn L; Nelson, Mark R; Shah, Raj C; Storey, Elsdon; Murray, Anne M; Reid, Christopher M; Kirpach, Brenda; Wolfe, Rory; Lockery, Jessica E; Berk, Michael

The 10-item Center for the Epidemiological Studies of Depression Short Form (CES-D-10) is a widely used self-report measure of depression symptomatology. The aim of this study is to investigate the psychometric properties of the CES-D-10 in healthy community dwelling older adults. The sample consists of 19,114 community-based individuals residing in Australia and the United States who participated in the ASPREE trial baseline assessment. All individuals were free of any major illness at the time. We evaluated construct validity by performing confirmatory factor analysis, examined measurement invariance across country and gender followed by evaluating item discrimination bias in age, gender, race, ethnicity and education level, and assessing internal consistency. High item-total correlations and Cronbach's alpha indicated high internal consistency. The factor analyses suggested a unidimensional factor structure. Construct validity was supported in the overall sample, and by country and gender sub-groups. The CES-D-10 was invariant across countries, and although evidence of marginal gender non-invariance was observed there was no evidence of notable gender specific item discrimination bias. No notable differences in discrimination parameters or group membership measurement non-invariance were detected by gender, age, race, ethnicity, and education level. These findings suggest the CES-D-10 is a reliable and valid measure of depression in a volunteer sample. No noteworthy evidence of invariance and/or item discrimination bias is observed across gender, age, race, language and ethnic groups. Copyright © 2017 Elsevier Inc. All rights reserved.

Should essays and other "open-ended"-type questions retain a place in written summative assessment in clinical medicine?

PubMed

Hift, Richard J

2014-11-28

Written assessments fall into two classes: constructed-response or open-ended questions, such as the essay and a number of variants of the short-answer question, and selected-response or closed-ended questions; typically in the form of multiple-choice. It is widely believed that constructed response written questions test higher order cognitive processes in a manner that multiple-choice questions cannot, and consequently have higher validity. An extensive review of the literature suggests that in summative assessment neither premise is evidence-based. Well-structured open-ended and multiple-choice questions appear equivalent in their ability to assess higher cognitive functions, and performance in multiple-choice assessments may correlate more highly than the open-ended format with competence demonstrated in clinical practice following graduation. Studies of construct validity suggest that both formats measure essentially the same dimension, at least in mathematics, the physical sciences, biology and medicine. The persistence of the open-ended format in summative assessment may be due to the intuitive appeal of the belief that synthesising an answer to an open-ended question must be both more cognitively taxing and similar to actual experience than is selecting a correct response. I suggest that cognitive-constructivist learning theory would predict that a well-constructed context-rich multiple-choice item represents a complex problem-solving exercise which activates a sequence of cognitive processes which closely parallel those required in clinical practice, hence explaining the high validity of the multiple-choice format. The evidence does not support the proposition that the open-ended assessment format is superior to the multiple-choice format, at least in exit-level summative assessment, in terms of either its ability to test higher-order cognitive functioning or its validity. This is explicable using a theory of mental models, which might predict that the multiple-choice format will have higher validity, a statement for which some empiric support exists. Given the superior reliability and cost-effectiveness of the multiple-choice format consideration should be given to phasing out open-ended format questions in summative assessment. Whether the same applies to non-exit-level assessment and formative assessment is a question which remains to be answered; particularly in terms of the educational effect of testing, an area which deserves intensive study.

The development and validation of the Self-Efficacy Beliefs about Equitable Science Teaching and learning instrument for prospective elementary teachers

NASA Astrophysics Data System (ADS)

Ritter, Jennifer M.

1999-12-01

The purpose of this study was to develop, validate and establish the reliability of an instrument to assess the self-efficacy beliefs of prospective elementary teachers with regards to science teaching and learning for diverse learners. The study used Bandura's theoretical framework, in that the instrument would use the self-efficacy construct to explore the beliefs of prospective elementary science teachers with regards to science teaching and learning to diverse learners: specifically the two dimensions of self-efficacy beliefs defined by Bandura (1977): personal self-efficacy and outcome expectancy. A seven step plan was designed and followed in the process of developing the instrument, which was titled the Self-Efficacy Beliefs about Equitable Science Teaching or SEBEST. Diverse learners as recognized by Science for All Americans (1989) are "those who in the past who have largely been bypassed in science and mathematics education: ethnic and language minorities and girls" (p. xviii). That definition was extended by this researcher to include children from low socioeconomic backgrounds based on the research by Gomez and Tabachnick (1992). The SEBEST was administered to 226 prospective elementary teachers at The Pennsylvania State University. Using the results from factor analyses, Coefficient Alpha, and Chi-Square a 34 item instrument was found to achieve the greatest balance across the construct validity, reliability and item balance with the content matrix. The 34 item SEBEST was found to load purely on four factors across the content matrix thus providing evidence construct validity. The Coefficient Alpha reliability for the 34 item SEBEST was .90 and .82 for the PSE sub-scale and .78 for the OE sub-scale. A Chi-Square test (X2 = 2.7 1, df = 7, p > .05) was used to confirm that the 34 items were balanced across the Personal Self-Efficacy/Outcome Expectancy and Ethnicity/LanguageMinority/Gender Socioeconomic Status/dimensions of the content matrix. Based on the standardized development procedures used and the associated evidence, the SEBEST appears to be a content and construct valid instrument, with high internal reliability and moderate test-retest reliability qualities, for use with prospective elementary teachers to assess self-efficacy beliefs for teaching and learning science for diverse learners.

Relationships among Safety Climate, Safety Behavior, and Safety Outcomes for Ethnic Minority Construction Workers.

PubMed

Lyu, Sainan; Hon, Carol K H; Chan, Albert P C; Wong, Francis K W; Javed, Arshad Ali

2018-03-09

In many countries, it is common practice to attract and employ ethnic minority (EM) or migrant workers in the construction industry. This primarily occurs in order to alleviate the labor shortage caused by an aging workforce with a lack of new entrants. Statistics show that EM construction workers are more likely to have occupational fatal and nonfatal injuries than their local counterparts; however, the mechanism underlying accidents and injuries in this vulnerable population has been rarely examined. This study aims to investigate relationships among safety climate, safety behavior, and safety outcomes for EM construction workers. To this end, a theoretical research model was developed based on a comprehensive review of the current literature. In total, 289 valid questionnaires were collected face-to-face from 223 Nepalese construction workers and 56 Pakistani construction workers working on 15 construction sites in Hong Kong. Structural equation modelling was employed to validate the constructs and test the hypothesized model. Results show that there were significant positive relationships between safety climate and safety behaviors, and significant negative relationships between safety behaviors and safety outcomes for EM construction workers. This research contributes to the literature regarding EM workers by providing empirical evidence of the mechanisms by which safety climate affects safety behaviors and outcomes. It also provides insights in order to help the key stakeholders formulate safety strategies for EM workers in many areas where numerous EM workers are employed, such as in the U.S., the UK, Australia, Singapore, Malaysia, and the Middle East.

Development of a Short Questionnaire to Measure an Extended Set of Job Demands, Job Resources, and Positive Health Outcomes: The New Brief Job Stress Questionnaire

PubMed Central

INOUE, Akiomi; KAWAKAMI, Norito; SHIMOMITSU, Teruichi; TSUTSUMI, Akizumi; HARATANI, Takashi; YOSHIKAWA, Toru; SHIMAZU, Akihito; ODAGIRI, Yuko

2014-01-01

This study aimed to investigate the reliability and construct validity of a new version of the Brief Job Stress Questionnaire (New BJSQ), which measures an extended set of psychosocial factors at work by adding new scales/items to the current version of the BJSQ. Additional scales/items were extensively collected from theoretical job stress models and similar questionnaires in several countries. Scales/items were field-tested and refined through a pilot internet survey. Finally, an 84-item questionnaire (141 items in total when combined with the current BJSQ) was developed. A nationally representative survey was administered to employees in Japan (n=1,633) to examine the reliability and construct validity. Most scales showed acceptable levels of internal consistency and test-retest reliability. Principal component analyses showed that the first factor explained 50% or greater proportion of the variance in most scales. A scale factor analysis and a correlation analysis showed that these scales fit the theoretical expectations. These findings provided a piece of evidence that the New BJSQ scales are reliable and valid. Although more detailed content and construct validity should be examined in future study, the New BJSQ is a useful instrument to evaluate psychosocial work environment and positive mental health outcomes in the current workplace. PMID:24492763

Developing a Self-Scoring Comprehensive Instrument to Measure Rest's Four-Component Model of Moral Behavior: The Moral Skills Inventory.

PubMed

Chambers, David W

2011-01-01

One of the most extensively studied constructs in dental education is the four-component model of moral behavior proposed by James Rest and the set of instruments for measuring it developed by Rest, Muriel Bebeau, and others. Although significant associations have been identified between the four components Rest proposed (called here Moral Sensitivity, Moral Reasoning, Moral Integrity, and Moral Courage) and dental ethics courses and practitioners with disciplined licenses, there is no single instrument that measures all four components, and existing single component instruments require professional scoring. This article describes the development and validation of a short, self-scoring instrument, the Moral Skills Inventory, that measures all four components. Evidence of face validity, test/retest reliability, and concurrent convergent and divergent predictive validity are demonstrated in three populations: dental students, clinical dental faculty members, and regents and officers of the American College of Dentists. Significant issues remain in developing the Rest four-component model for use in dental education and practice. Specifically, further construct validation research is needed to understand the nature of the components. In particular, it remains undetermined whether moral constructs are characteristics of individuals that drive behavior in specific situations or whether particular patterns of moral behavior learned and used in response to individual circumstances are summarized by researchers and then imputed to practitioners.

Development of a short questionnaire to measure an extended set of job demands, job resources, and positive health outcomes: the new brief job stress questionnaire.

PubMed

Inoue, Akiomi; Kawakami, Norito; Shimomitsu, Teruichi; Tsutsumi, Akizumi; Haratani, Takashi; Yoshikawa, Toru; Shimazu, Akihito; Odagiri, Yuko

2014-01-01

This study aimed to investigate the reliability and construct validity of a new version of the Brief Job Stress Questionnaire (New BJSQ), which measures an extended set of psychosocial factors at work by adding new scales/items to the current version of the BJSQ. Additional scales/items were extensively collected from theoretical job stress models and similar questionnaires in several countries. Scales/items were field-tested and refined through a pilot internet survey. Finally, an 84-item questionnaire (141 items in total when combined with the current BJSQ) was developed. A nationally representative survey was administered to employees in Japan (n=1,633) to examine the reliability and construct validity. Most scales showed acceptable levels of internal consistency and test-retest reliability. Principal component analyses showed that the first factor explained 50% or greater proportion of the variance in most scales. A scale factor analysis and a correlation analysis showed that these scales fit the theoretical expectations. These findings provided a piece of evidence that the New BJSQ scales are reliable and valid. Although more detailed content and construct validity should be examined in future study, the New BJSQ is a useful instrument to evaluate psychosocial work environment and positive mental health outcomes in the current workplace.

Self- and Peer-Assessment: Evidence from the Accounting and Finance Discipline

ERIC Educational Resources Information Center

Hassan, Omaima A. G.; Fox, Alison; Hannah, Gwen

2014-01-01

Self- and peer-assessment of student work is an area that is under-researched in the accounting education literature, although the subject area of study seems to influence the results obtained in prior studies. The current study contributes to the literature by examining the accuracy and construct validity of self- and peer-assessment by…

Designs for Operationalizing Collaborative Problem Solving for Automated Assessment

ERIC Educational Resources Information Center

Scoular, Claire; Care, Esther; Hesse, Friedrich W.

2017-01-01

Collaborative problem solving is a complex skill set that draws on social and cognitive factors. The construct remains in its infancy due to lack of empirical evidence that can be drawn upon for validation. The differences and similarities between two large-scale initiatives that reflect this state of the art, in terms of underlying assumptions…

Middle School Students' Learning about Genetic Inheritance through On-Line Scaffolding Supports

ERIC Educational Resources Information Center

Manokore, Viola

2010-01-01

The main goal of school science is to enable learners to become scientifically literate through their participation in scientific discourses (McNeill & Krajcik, 2009). One of the key elements of scientific discourses is the ability to construct scientific explanations that consist of valid claims supported by appropriate evidence (e.g., McNeill &…

Exploratory Factor Analysis as a Construct Validation Tool: (Mis)applications in Applied Linguistics Research

ERIC Educational Resources Information Center

Karami, Hossein

2015-01-01

Factor analysis has been frequently exploited in applied research to provide evidence about the underlying factors in various measurement instruments. A close inspection of a large number of studies published in leading applied linguistic journals shows that there is a misconception among applied linguists as to the relative merits of exploratory…

The Convergent and Concurrent Validity of Trait-Based Prototype Assessment of Personality Disorder Categories in Homeless Persons

ERIC Educational Resources Information Center

Samuel, Douglas B.; Connolly, Adrian J.; Ball, Samuel A.

2012-01-01

The "DSM-5" proposal indicates that personality disorders (PDs) be defined as collections of maladaptive traits but does not provide a specific diagnostic method. However, researchers have previously suggested that PD constructs can be assessed by comparing individuals' trait profiles with those prototypic of PDs and evidence from the…

Approaching Sign Language Test Construction: Adaptation of the German Sign Language Receptive Skills Test

ERIC Educational Resources Information Center

Haug, Tobias

2011-01-01

There is a current need for reliable and valid test instruments in different countries in order to monitor deaf children's sign language acquisition. However, very few tests are commercially available that offer strong evidence for their psychometric properties. A German Sign Language (DGS) test focusing on linguistic structures that are acquired…

Workplace Social Self-Efficacy: Concept, Measure, and Initial Validity Evidence

ERIC Educational Resources Information Center

Fan, Jinyan; Litchfield, Robert C.; Islam, Sayeed; Weiner, Brianne; Alexander, Monique; Liu, Cong; Kulviwat, Songpol

2013-01-01

The authors proposed the construct of workplace social self-efficacy (WSSE) and developed an inventory to measure it. Two empirical studies were conducted to examine the psychometric properties of this new measure. In Study 1, we described the development of the WSSE inventory and explored its factor structure in a sample of 304 full-time…

Measuring Physician Cognitive Load: Validity Evidence for a Physiologic and a Psychometric Tool

ERIC Educational Resources Information Center

Szulewski, Adam; Gegenfurtner, Andreas; Howes, Daniel W.; Sivilotti, Marco L. A.; van Merriënboer, Jeroen J. G.

2017-01-01

In general, researchers attempt to quantify cognitive load using physiologic and psychometric measures. Although the construct measured by both of these metrics is thought to represent overall cognitive load, there is a paucity of studies that compares these techniques to one another. The authors compared data obtained from one physiologic tool…

Construct and Predictive Validity Evidence for Curriculum-Based Measures of Early Literacy and Numeracy Skills in Kindergarten

ERIC Educational Resources Information Center

Betts, Joseph; Pickart, Mary; Heistad, Dave

2009-01-01

The assessment of early literacy and numeracy skills can provide useful and important information in pursuance of the goal to increase student academic achievement. At present, there have been promising results using curriculum-based measurement (CBM) for evaluating early literacy and early numeracy. There has been little research investigating…

Constructing and Evaluating a Validity Argument for the Final-Year Ward Simulation Exercise

ERIC Educational Resources Information Center

Till, Hettie; Ker, Jean; Myford, Carol; Stirling, Kevin; Mires, Gary

2015-01-01

The authors report final-year ward simulation data from the University of Dundee Medical School. Faculty who designed this assessment intend for the final score to represent an individual senior medical student's level of clinical performance. The results are included in each student's portfolio as one source of evidence of the student's…

The Hip Sports Activity Scale (HSAS) for patients with femoroacetabular impingement.

PubMed

Naal, Florian D; Miozzari, Hermes H; Kelly, Bryan T; Magennis, Erin M; Leunig, Michael; Noetzli, Hubert P

2013-01-01

To develop and validate a sports activity scale for patients with a diagnosis of femoroacetabular impingement (FAI).
 A nine level Hip Sports Activity Scale (HSAS) was constructed both in German and English languages. Fifty-nine consecutive patients undergoing surgical treatment for FAI at two centers in Switzerland and in the US completed a questionnaire set consisting of the HSAS, the University of California at Los Angeles (UCLA) activity scale and different hip joint-specific and generic outcome tools. For reliability assessment, the HSAS was completed twice about nine days apart. Evidence of reliability, validity and responsiveness was investigated by classical psychometric analyses.
 Reliability was excellent for both the German and the English versions with intraclass correlation coefficients of 0.94 and 0.96, respectively. Evidence of convergent validity was supported by moderate to high correlations with the UCLA activity scale and with the joint-specific measures used. Evidence of divergent validity was supported by low correlations with the SF-12 Mental Component Scale and the WOMAC stiffness subscale. The standardised response mean was 0.69.
 The HSAS is a reliable and valid tool to determine sports levels in patients suffering from FAI. Its use in future studies investigating outcomes in young patients with hip disease can be recommended.
 Level III, Diagnostic Studies - An independent, masked comparison with an appropriate population of patients, but reference standard not applied to all study patients.

Development and Psychometric Properties of the Nursing Critical Thinking in Clinical Practice Questionnaire.

PubMed

Zuriguel-Pérez, Esperanza; Falcó-Pegueroles, Anna; Roldán-Merino, Juan; Agustino-Rodriguez, Sandra; Gómez-Martín, Maria Del Carmen; Lluch-Canut, Maria Teresa

2017-08-01

A complex healthcare environment, with greater need for care based on the patient and evidence-based practice, are factors that have contributed to the increased need for critical thinking in professional competence. At the theoretical level, Alfaro-LeFevre () put forward a model of critical thinking made up of four components. And although these explain the construct, instruments for their empirical measurement are lacking. The purpose of the study was to develop and validate the psychometric properties of an instrument, the Nursing Critical Thinking in Clinical Practice Questionnaire (N-CT-4 Practice), designed to evaluate the critical thinking abilities of nurses in the clinical setting. A cross-sectional survey design was used. A pool of items was generated for evaluation by a panel of experts who considered their validity for the new instrument, which was finally made up of 109 items. Following this, validation was carried out using a sample of 339 nurses at a hospital in Barcelona, Spain. Reliability was determined by means of internal consistency and test-retest stability over time, although the validity of the construct was assessed by means of confirmatory factor analysis. The content validity index of the N-CT-4 Practice was .85. Cronbach's alpha coefficient for the whole instrument was .96. The intraclass correlation coefficient was .77. Confirmatory factor analysis showed that the instrument was in line with the four-dimensional model proposed by Alfaro-LeFevre (). The psychometric properties of theN-CT-4 Practice uphold its potential for use in measuring critical thinking and in future research related with the examination of critical thinking. © 2017 The Authors Worldviews on Evidence-Based Nursing published by Wiley Periodicals, Inc. on behalf of Sigma Theta Tau International The Honor Society of Nursing.

Development and evaluation of the Expressions of Moral Injury Scale-Military Version.

PubMed

Currier, Joseph M; Farnsworth, Jacob K; Drescher, Kent D; McDermott, Ryon C; Sims, Brook M; Albright, David L

2018-05-01

There is consensus that military personnel can encounter a far more diverse set of challenges than researchers and clinicians have historically appreciated. Moral injury (MI) represents an emerging construct to capture behavioural, social, and spiritual suffering that may transcend and overlap with mental health diagnoses (e.g., post-traumatic stress disorder and major depressive disorder). The Expressions of Moral Injury Scale-Military Version (EMIS-M) was developed to provide a reliable and valid means for assessing the warning signs of a MI in military populations. Drawing on independent samples of veterans who had served in a war-zone environment, factor analytic results revealed 2 distinct factors related to MI expressions directed at both self (9 items) and others (8 items). These subscales generated excellent internal consistency and temporal stability over a 6-month period. When compared to measures of post-traumatic stress disorder, major depressive disorder, and other theoretically relevant constructs (e.g., forgiveness, social support, moral emotions, and combat exposure), EMIS-M scores demonstrated strong convergent, divergent, and incremental validity. In addition, although structural equation modelling findings supported a possible general MI factor in Study 2, the patterns of associations for self- and other-directed expressions yielded evidence for differential validity with varying forms of forgiveness and combat exposure. As such, the EMIS-M provides a face valid, psychometrically validated tool for assessing expressions of apparent MI subtypes in research and clinical settings. Looking ahead, the EMIS-M will hopefully advance the scientific understanding of MI while supporting innovation for clinicians to tailor evidence-based treatments and/or develop novel approaches for addressing MI in their work. Copyright © 2017 John Wiley & Sons, Ltd.

Psychometric evaluation of the pediatric and parent-proxy Patient-Reported Outcomes Measurement Information System and the Neurology and Traumatic Brain Injury Quality of Life measurement item banks in pediatric traumatic brain injury.

PubMed

Bertisch, Hilary; Rivara, Frederick P; Kisala, Pamela A; Wang, Jin; Yeates, Keith Owen; Durbin, Dennis; Zonfrillo, Mark R; Bell, Michael J; Temkin, Nancy; Tulsky, David S

2017-07-01

The primary objective is to provide evidence of convergent and discriminant validity for the pediatric and parent-proxy versions of the Patient-Reported Outcomes Measurement Information System (PROMIS) Anxiety, Depression, Anger, Peer Relations, Mobility, Pain Interference, and Fatigue item banks, the Neurology Quality of Life measurement system (Neuro-QOL) Cognition-General Concerns and Stigma item banks, and the Traumatic Brain Injury Quality of Life (TBI-QOL) Executive Function and Headache item banks in a pediatric traumatic brain injury (TBI) sample. Participants were 134 parent-child (ages 8-18 years) days. Children all sustained TBI and the dyads completed outcome ratings 6 months after injury at one of six medical centers across the United States. Ratings included PROMIS, Neuro-QOL, and TBI-QOL item banks, as well as the Pediatric Quality of Life inventory (PedsQL), the Health Behavior Inventory (HBI), and the Strengths and Difficulties Questionnaire (SDQ) as legacy criterion measures against which these item banks were validated. The PROMIS, Neuro-QOL, and TBI-QOL item banks demonstrated good convergent validity, as evidenced by moderate to strong correlations with comparable scales on the legacy measures. PROMIS, Neuro-QOL, and TBI-QOL item banks showed weaker correlations with ratings of unrelated constructs on legacy measures, providing evidence of discriminant validity. Our results indicate that the constructs measured by the PROMIS, Neuro-QOL, and TBI-QOL item banks are valid in our pediatric TBI sample and that it is appropriate to use these standardized scores for our primary study analyses.

Testing a self-determination theory model of children’s physical activity motivation: a cross-sectional study

PubMed Central

2013-01-01

Background Understanding children’s physical activity motivation, its antecedents and associations with behavior is important and can be advanced by using self-determination theory. However, research among youth is largely restricted to adolescents and studies of motivation within certain contexts (e.g., physical education). There are no measures of self-determination theory constructs (physical activity motivation or psychological need satisfaction) for use among children and no previous studies have tested a self-determination theory-based model of children’s physical activity motivation. The purpose of this study was to test the reliability and validity of scores derived from scales adapted to measure self-determination theory constructs among children and test a motivational model predicting accelerometer-derived physical activity. Methods Cross-sectional data from 462 children aged 7 to 11 years from 20 primary schools in Bristol, UK were analysed. Confirmatory factor analysis was used to examine the construct validity of adapted behavioral regulation and psychological need satisfaction scales. Structural equation modelling was used to test cross-sectional associations between psychological need satisfaction, motivation types and physical activity assessed by accelerometer. Results The construct validity and reliability of the motivation and psychological need satisfaction measures were supported. Structural equation modelling provided evidence for a motivational model in which psychological need satisfaction was positively associated with intrinsic and identified motivation types and intrinsic motivation was positively associated with children’s minutes in moderate-to-vigorous physical activity. Conclusions The study provides evidence for the psychometric properties of measures of motivation aligned with self-determination theory among children. Children’s motivation that is based on enjoyment and inherent satisfaction of physical activity is associated with their objectively-assessed physical activity and such motivation is positively associated with perceptions of psychological need satisfaction. These psychological factors represent potential malleable targets for interventions to increase children’s physical activity. PMID:24067078

Instruments assessing attitudes toward or capability regarding self-management of osteoarthritis: a systematic review of measurement properties.

PubMed

Eyles, J P; Hunter, D J; Meneses, S R F; Collins, N J; Dobson, F; Lucas, B R; Mills, K

2017-08-01

To make a recommendation on the "best" instrument to assess attitudes toward and/or capabilities regarding self-management of osteoarthritis (OA) based on available measurement property evidence. Electronic searches were performed in MEDLINE, EMBASE, CINAHL and PsychINFO (inception to 27 December 2016). Two reviewers independently rated measurement properties using the Consensus-based Standards for the selection of Health Measurement Instruments (COSMIN) 4-point scale. Best evidence synthesis was determined by considering COSMIN ratings for measurement property results and the level of evidence available for each measurement property of each instrument. Eight studies out of 5653 publications met the inclusion criteria, with eight instruments identified for evaluation: Multidimensional Health Locus of Control (MHLC), Perceived Behavioural Control (PBC), Patient Activation Measure (PAM), Educational Needs Assessment (ENAT), Stages of Change Questionnaire in Osteoarthritis (SCQOA), Effective Consumer Scale (EC-17) and Perceived Efficacy in Patient-Physician Interactions five item (PEPPI-5) and ten item scales. Measurement properties assessed for these instruments included internal consistency (k = 8), structural validity (k = 8), test-retest reliability (k = 2), measurement error (k = 1), hypothesis testing (k = 3) and cross-cultural validity (k = 3). No information was available for content validity, responsiveness or minimal important change (MIC)/minimal important difference (MID). The Dutch PEPPI-5 demonstrated the best measurement property evidence; strong evidence for internal consistency and structural validity but limited evidence for reliability and construct validity. Although PEPPI-5 was identified as having the best measurement properties, overall there is a poor level of evidence currently available concerning measurement properties of instruments to assess attitudes toward and/or capabilities regarding osteoarthritis self-management. Further well-designed studies investigating measurement properties of existing instruments are required. Copyright © 2017 Osteoarthritis Research Society International. All rights reserved.

Developing nurse and physician questionnaires to assess primary work areas in intensive care units.

PubMed

Rashid, Mahbub; Boyle, Diane K; Crosser, Michael

2014-01-01

The objective of the study was to develop instruments for describing and assessing some aspects of design of the primary work areas of nurses and physicians in intensive care units (ICUs). Separate questionnaires for ICU physicians and nurses were developed. Items related to individual- and unit-level design features of the primary work areas of nurses and physicians were organized using constructs found in the literature. Items related to staff satisfaction and staff use of time in relation to primary work area design were also included. All items and constructs were reviewed by experts for content validity and were modified as needed before use. The final questionnaires were administered to a convenience sample of 4 ICUs in 2 large urban hospitals. A total of 55 nurses and 29 physicians completed the survey. The Cronbach α was used to measure internal consistency, and factor analysis was used to provide construct-related validity. Convergent and discriminant validity were assessed through examining bivariate correlations between relevant scales/items. Analysis of variance was used to identify whether the between-group member responses were significant among the 4 units. The Cronbach α values for all except 3 preliminary scales indicated acceptable reliability. Factor analysis indicated that some preliminary scales could be partitioned into subscales for finer descriptions of the primary work areas. Correlational analysis provided strong evidence of convergent and discriminant validity of all the scales and subscales. The significance level of F-statistics showed that the units were significantly different from each other, providing evidence of more between-unit variance than within-unit variance. Therefore, the questionnaires developed in the study offer a promising departure point for rigorous description and evaluation of the primary work areas in relation to staff satisfaction and use of time in ICUs at a time when the importance of such studies is growing.

From primed construct to motivated behavior: validation processes in goal pursuit.

PubMed

Demarree, Kenneth G; Loersch, Chris; Briñol, Pablo; Petty, Richard E; Payne, B Keith; Rucker, Derek D

2012-12-01

Past research has found that primes can automatically initiate unconscious goal striving. Recent models of priming have suggested that this effect can be moderated by validation processes. According to a goal-validation perspective, primes should cause changes in one's motivational state to the extent people have confidence in the prime-related mental content. Across three experiments, we provided the first direct empirical evidence for this goal-validation account. Using a variety of goal priming manipulations (cooperation vs. competition, achievement, and self-improvement vs. saving money) and validity inductions (power, ease, and writing about confidence), we demonstrated that the impact of goal primes on behavior occurs to a greater extent when conditions foster confidence (vs. doubt) in mental contents. Indeed, when conditions foster doubt, goal priming effects are eliminated or counter to the implications of the prime. The implications of these findings for research on goal priming and validation processes are discussed.

Competency-Based Training and Simulation: Making a "Valid" Argument.

PubMed

Noureldin, Yasser A; Lee, Jason Y; McDougall, Elspeth M; Sweet, Robert M

2018-02-01

The use of simulation as an assessment tool is much more controversial than is its utility as an educational tool. However, without valid simulation-based assessment tools, the ability to objectively assess technical skill competencies in a competency-based medical education framework will remain challenging. The current literature in urologic simulation-based training and assessment uses a definition and framework of validity that is now outdated. This is probably due to the absence of awareness rather than an absence of comprehension. The following review article provides the urologic community an updated taxonomy on validity theory as it relates to simulation-based training and assessments and translates our simulation literature to date into this framework. While the old taxonomy considered validity as distinct subcategories and focused on the simulator itself, the modern taxonomy, for which we translate the literature evidence, considers validity as a unitary construct with a focus on interpretation of simulator data/scores.

A motor speech assessment for children with severe speech disorders: reliability and validity evidence.

PubMed

Strand, Edythe A; McCauley, Rebecca J; Weigand, Stephen D; Stoeckel, Ruth E; Baas, Becky S

2013-04-01

In this article, the authors report reliability and validity evidence for the Dynamic Evaluation of Motor Speech Skill (DEMSS), a new test that uses dynamic assessment to aid in the differential diagnosis of childhood apraxia of speech (CAS). Participants were 81 children between 36 and 79 months of age who were referred to the Mayo Clinic for diagnosis of speech sound disorders. Children were given the DEMSS and a standard speech and language test battery as part of routine evaluations. Subsequently, intrajudge, interjudge, and test-retest reliability were evaluated for a subset of participants. Construct validity was explored for all 81 participants through the use of agglomerative cluster analysis, sensitivity measures, and likelihood ratios. The mean percentage of agreement for 171 judgments was 89% for test-retest reliability, 89% for intrajudge reliability, and 91% for interjudge reliability. Agglomerative hierarchical cluster analysis showed that total DEMSS scores largely differentiated clusters of children with CAS vs. mild CAS vs. other speech disorders. Positive and negative likelihood ratios and measures of sensitivity and specificity suggested that the DEMSS does not overdiagnose CAS but sometimes fails to identify children with CAS. The value of the DEMSS in differential diagnosis of severe speech impairments was supported on the basis of evidence of reliability and validity.

Further Validation of the Multidimensional Fatigue Symptom Inventory-Short Form

PubMed Central

Stein, Kevin D.; Jacobsen, Paul B.; Blanchard, Chris M.; Thors, Christina

2008-01-01

A growing body of evidence is documenting the multidimensional nature of cancer-related fatigue. Although several multidimensional measures of fatigue have been developed, further validation of these scales is needed. To this end, the current study sought to evaluate the factorial and construct validity of the 30-item Multidimensional Fatigue Symptom Inventory-Short Form (MFSI-SF). A heterogeneous sample of 304 cancer patients (mean age 55 years) completed the MFSI-SF, along with several other measures of psychosocial functioning including the MOS-SF-36 and Fatigue Symptom Inventory, following the fourth cycle of chemotherapy treatment. The results of a confirmatory factor analysis indicated the 5-factor model provided a good fit to the data as evidenced by commonly used goodness of fit indices (CFI 0.90 and IFI 0.90). Additional evidence for the validity of the MFSI-SF was provided via correlations with other relevant instruments (range −0.21 to 0.82). In sum, the current study provides support for the MFSI-SF as a valuable tool for the multidimensional assessment of cancer-related fatigue. PMID:14711465

Relevance of Hypersexual Disorder to Family Medicine and Primary Care as a Complex Multidimensional Chronic Disease Construct

PubMed Central

Vrijhoef, Bert; De Maeseneer, Jan; Vansintejan, Johan; Devroey, Dirk

2013-01-01

Hypersexual disorder (HD) is not defined in a uniform way in the psychiatric literature. In the absence of solid evidence on prevalence, causes, empirically validated diagnostic criteria, instruments for diagnosis, consistent guidelines on treatment options, medical and psychosocial consequences, and type of caregivers that need to be involved, HD remains a controversial and relatively poorly understood chronic disease construct. The role of family medicine in the detection, treatment, and followup of HD is not well studied. The purpose of this paper is to describe the complexity of HD as a multidimensional chronic disease construct and its relevance to family medicine and primary care. PMID:24066230

Why does self-reported emotional intelligence predict job performance? A meta-analytic investigation of mixed EI.

PubMed

Joseph, Dana L; Jin, Jing; Newman, Daniel A; O'Boyle, Ernest H

2015-03-01

Recent empirical reviews have claimed a surprisingly strong relationship between job performance and self-reported emotional intelligence (also commonly called trait EI or mixed EI), suggesting self-reported/mixed EI is one of the best known predictors of job performance (e.g., ρ = .47; Joseph & Newman, 2010b). Results further suggest mixed EI can robustly predict job performance beyond cognitive ability and Big Five personality traits (Joseph & Newman, 2010b; O'Boyle, Humphrey, Pollack, Hawver, & Story, 2011). These criterion-related validity results are problematic, given the paucity of evidence and the questionable construct validity of mixed EI measures themselves. In the current research, we update and reevaluate existing evidence for mixed EI, in light of prior work regarding the content of mixed EI measures. Results of the current meta-analysis demonstrate that (a) the content of mixed EI measures strongly overlaps with a set of well-known psychological constructs (i.e., ability EI, self-efficacy, and self-rated performance, in addition to Conscientiousness, Emotional Stability, Extraversion, and general mental ability; multiple R = .79), (b) an updated estimate of the meta-analytic correlation between mixed EI and supervisor-rated job performance is ρ = .29, and (c) the mixed EI-job performance relationship becomes nil (β = -.02) after controlling for the set of covariates listed above. Findings help to establish the construct validity of mixed EI measures and further support an intuitive theoretical explanation for the uncommonly high association between mixed EI and job performance--mixed EI instruments assess a combination of ability EI and self-perceptions, in addition to personality and cognitive ability. PsycINFO Database Record (c) 2015 APA, all rights reserved.

A systematic review of the measurement properties of the Body Image Scale (BIS) in cancer patients.

PubMed

Melissant, Heleen C; Neijenhuijs, Koen I; Jansen, Femke; Aaronson, Neil K; Groenvold, Mogens; Holzner, Bernhard; Terwee, Caroline B; van Uden-Kraan, Cornelia F; Cuijpers, Pim; Verdonck-de Leeuw, Irma M

2018-06-01

Body image is acknowledged as an important aspect of health-related quality of life in cancer patients. The Body Image Scale (BIS) is a patient-reported outcome measure (PROM) to evaluate body image in cancer patients. The aim of this study was to systematically review measurement properties of the BIS among cancer patients. A search in Embase, MEDLINE, PsycINFO, and Web of Science was performed to identify studies that investigated measurement properties of the BIS (Prospero ID 42017057237). Study quality was assessed (excellent, good, fair, poor), and data were extracted and analyzed according to the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology on structural validity, internal consistency, reliability, measurement error, hypothesis testing for construct validity, and responsiveness. Evidence was categorized into sufficient, insufficient, inconsistent, or indeterminate. Nine studies were included. Evidence was sufficient for structural validity (one factor solution), internal consistency (α = 0.86-0.96), and reliability (r > 0.70); indeterminate for measurement error (information on minimal important change lacked) and responsiveness (increasing body image disturbance in only one study); and inconsistent for hypothesis testing (conflicting results). Quality of the evidence was moderate to low. No studies reported on cross-cultural validity. The BIS is a PROM with good structural validity, internal consistency, and test-retest reliability, but good quality studies on the other measurement properties are needed to optimize evidence. It is recommended to include a wider variety of cancer diagnoses and treatment modalities in these future studies.

Construct Validity of the Nepalese School Leaving English Reading Test

ERIC Educational Resources Information Center

Dawadi, Saraswati; Shrestha, Prithvi N.

2018-01-01

There has been a steady interest in investigating the validity of language tests in the last decades. Despite numerous studies on construct validity in language testing, there are not many studies examining the construct validity of a reading test. This paper reports on a study that explored the construct validity of the English reading test in…

The Nature of Science Instrument-Elementary (NOSI-E): Using Rasch principles to develop a theoretically grounded scale to measure elementary student understanding of the nature of science

NASA Astrophysics Data System (ADS)

Peoples, Shelagh

The purpose of this study was to determine which of three competing models will provide, reliable, interpretable, and responsive measures of elementary students' understanding of the nature of science (NOS). The Nature of Science Instrument-Elementary (NOSI-E), a 28-item Rasch-based instrument, was used to assess students' NOS understanding. The NOS construct was conceptualized using five construct dimensions (Empirical, Inventive, Theory-laden, Certainty and Socially & Culturally Embedded). The competing models represent three internal models for the NOS construct. One postulate is that the NOS construct is unidimensional where one latent construct explains the relationship between the 28 items of the NOSI-E. Alternatively, the NOS construct is composed of five independent unidimensional constructs (the consecutive approach). Lastly, the NOS construct is multidimensional and composed of five inter-related but separate dimensions. A validity argument was developed that hypothesized that the internal structure of the NOS construct is best represented by the multidimensional Rasch model. Four sets of analyses were performed in which the three representations were compared. These analyses addressed five validity aspects (content, substantive, generalizability, structural and external) of construct validity. The vast body of evidence supported the claim that the NOS construct is composed of five separate but inter-related dimensions that is best represented by the multidimensional Rasch model. The results of the multidimensional analyses indicated that the items of the five subscales were of excellent technical quality, exhibited no differential item functioning (based on gender), had an item hierarchy that conformed to theoretical expectations; and together formed subscales of reasonable reliability (> 0.7 on each subscale) that were responsive to change in the construct. Theory-laden scores from the multidimensional model predicted students' science achievement with scores from all five NOS dimensions significantly predicting students' perceptions of the constructivist nature of their classroom learning environment. The NOSI-E instrument is a theoretically grounded scale that can measure elementary students' NOS understanding and appears suitable for use in science education research.

Continued Validation of the O-SCORE (Ottawa Surgical Competency Operating Room Evaluation): Use in the Simulated Environment.

PubMed

MacEwan, Matthew J; Dudek, Nancy L; Wood, Timothy J; Gofton, Wade T

2016-01-01

CONSTRUCT: The Ottawa Surgical Competency Operating Room Evaluation (O-SCORE) is a 9-item surgical evaluation tool designed to assess technical competence in surgical trainees using behavioral anchors. The initial development of the O-SCORE produced evidence for valid results. Further work is required to determine if the use of a single surgeon or an unblinded rater introduces bias. In addition, the relationship of the O-SCORE to other currently used technical assessment tools should be explored to provide validity evidence related to the relationship to other measures. We have designed this project to provide continued validity evidence for the O-SCORE related to these two issues. Nineteen residents and 2 staff Orthopedic Surgeons from the University of Ottawa volunteered to participate in a 2-part OSCE style station. Participants completed a written questionnaire followed by a videotaped 10-minute simulated open reduction and internal fixation of a midshaft radius fracture. Videos were rated individually by 2 blinded staff orthopedic surgeons using an Objective Structured Assessment of Technical Skills (OSATS) global rating scale, an OSATS checklist, and the O-SCORE in random order. O-SCORE results appeared sensitive to surgical training level even when raters were blinded. In addition, strong agreement between two independent observers using the O-SCORE suggests that the measure captures a performance easily recognized by surgical observers. Ratings on the O-SCORE also were strongly associated with global ratings on the currently most validated technical evaluation tool (OSATS). Collectively, these results suggest that the O-SCORE generates accurate, reproducible, and meaningful results when used in a randomized and blinded fashion, providing continued validity evidence for using this tool to evaluate surgical trainee competence. The O-SCORE was able to differentiate surgical trainee level using blinded raters providing further evidence of validity for the O-SCORE. There was strong agreement between two independent observers using the O-SCORE. Ratings on the O-SCORE also demonstrated equivalence to scores on the most validated technical evaluation tool (OSATS). These results suggest that the O-SCORE demonstrates accurate and reproducible results when used in a randomized and blinded fashion providing continued validity evidence for this tool in the evaluation of surgical competence in the trainees.

Actual curriculum development practices instrument: Testing for factorial validity

NASA Astrophysics Data System (ADS)

Foi, Liew Yon; Bakar, Kamariah Abu; Hamzah, Mohd Sahandri Gani; Alwi, Nor Hayati

2014-09-01

The Actual Curriculum Development Practices Instrument (ACDP-I) was developed and the factorial validity of the ACDP-I was tested (n = 107) using exploratory factor analysis procedures in the earlier work of [1]. Despite the ACDP-I appears to be content and construct valid instrument with very high internal reliability qualities for using in Malaysia, the accumulated evidences are still needed to provide a sound scientific basis for the proposed score interpretations. Therefore, the present study addresses this concern by utilising the confirmatory factor analysis to further confirm the theoretical structure of the variable Actual Curriculum Development Practices (ACDP) and enrich the psychometrical properties of ACDP-I. Results of this study have practical implication to both researchers and educators whose concerns focus on teachers' classroom practices and the instrument development and validation process.

Validity and reliability testing of the Prenatal Psychosocial Profile.

PubMed

Curry, M A; Campbell, R A; Christian, M

1994-04-01

Two studies of low-income pregnant women (N = 179) were done to examine the validity and reliability of the Prenatal Psychosocial Profile (PPP). The PPP, a composite of the Rosenberg Self-Esteem Scale, the Support Behaviors Inventory, and a newly developed measure of stress, is a brief, comprehensive clinical assessment of psychosocial risk during pregnancy. Construct validity of the stress scale was supported by theoretically predicted negative correlations with self-esteem, partner support, and support from others (N = 91). Convergent validity of the stress scale was demonstrated by a correlation of .71 with the Difficult Life Circumstances Scale. Adequate levels of internal consistency were found. Interrelationships between the four subscales were consistent with the underlying conceptualization, and there was beginning evidence of the factorial independence of the subscales.

Developing and investigating the use of single-item measures in organizational research.

PubMed

Fisher, Gwenith G; Matthews, Russell A; Gibbons, Alyssa Mitchell

2016-01-01

The validity of organizational research relies on strong research methods, which include effective measurement of psychological constructs. The general consensus is that multiple item measures have better psychometric properties than single-item measures. However, due to practical constraints (e.g., survey length, respondent burden) there are situations in which certain single items may be useful for capturing information about constructs that might otherwise go unmeasured. We evaluated 37 items, including 18 newly developed items as well as 19 single items selected from existing multiple-item scales based on psychometric characteristics, to assess 18 constructs frequently measured in organizational and occupational health psychology research. We examined evidence of reliability; convergent, discriminant, and content validity assessments; and test-retest reliabilities at 1- and 3-month time lags for single-item measures using a multistage and multisource validation strategy across 3 studies, including data from N = 17 occupational health subject matter experts and N = 1,634 survey respondents across 2 samples. Items selected from existing scales generally demonstrated better internal consistency reliability and convergent validity, whereas these particular new items generally had higher levels of content validity. We offer recommendations regarding when use of single items may be more or less appropriate, as well as 11 items that seem acceptable, 14 items with mixed results that might be used with caution due to mixed results, and 12 items we do not recommend using as single-item measures. Although multiple-item measures are preferable from a psychometric standpoint, in some circumstances single-item measures can provide useful information. (c) 2016 APA, all rights reserved).

Psychometric properties of the Portuguese version of the Depressive Cognition Scale in Brazilian adults with diabetes mellitus.

PubMed

Sousa, Valmi D; Zanetti, Maria L; Zauszniewski, Jaclene A; Mendes, Isabel A C; Daguano, Michelle O

2008-01-01

Identifying depressive cognitions in Brazilians with diabetes can be important step to prevent the development of clinical depression, which is negatively associated with diabetes self-management. This study focused on the psychometric testing of the Portuguese version of the Depressive Cognition Scale, the Escala Cognitiva de Depressão (ECD), among 82 Brazilian adults with diabetes mellitus. The questionnaire was assessed for internal consistency, homogeneity, and construct validity using factor analysis and convergent validity assessment with the Portuguese version of the Beck Depression Inventory, the Inventário de Depressão Beck (IDB). Cronbach's alpha for the ECD was .88. The homogeneity of the instrument was supported by item-to-total correlations between .30 and .70. Factor extraction generated only one factor with eigenvalues greater than 1, which is consistent with the English version. The ECD's total score had a weak but significant correlation with the IDB's total score (r = .24, p < .05), indicating convergent validity. Evidence for the reliability and construct validity of the ECD was provided by this study. This scale has the potential to become a useful screening tool for depressive cognitions among Brazilians with diabetes.

Development and validation of the Measure of Indigenous Racism Experiences (MIRE)

PubMed Central

Paradies, Yin C; Cunningham, Joan

2008-01-01

Background In recent decades there has been increasing evidence of a relationship between self-reported racism and health. Although a plethora of instruments to measure racism have been developed, very few have been described conceptually or psychometrically Furthermore, this research field has been limited by a dearth of instruments that examine reactions/responses to racism and by a restricted focus on African American populations. Methods In response to these limitations, the 31-item Measure of Indigenous Racism Experiences (MIRE) was developed to assess self-reported racism for Indigenous Australians. This paper describes the development of the MIRE together with an opportunistic examination of its content, construct and convergent validity in a population health study involving 312 Indigenous Australians. Results Focus group research supported the content validity of the MIRE, and inter-item/scale correlations suggested good construct validity. A good fit with a priori conceptual dimensions was demonstrated in factor analysis, and convergence with a separate item on discrimination was satisfactory. Conclusion The MIRE has considerable utility as an instrument that can assess multiple facets of racism together with responses/reactions to racism among indigenous populations and, potentially, among other ethnic/racial groups. PMID:18426602

Validation of the Mobile Information Software Evaluation Tool (MISET) With Nursing Students.

PubMed

Secco, M Loretta; Furlong, Karen E; Doyle, Glynda; Bailey, Judy

2016-07-01

This study evaluated the Mobile Information Software Evaluation Tool (MISET) with a sample of Canadian undergraduate nursing students (N = 240). Psychometric analyses determined how well the MISET assessed the extent that nursing students find mobile device-based information resources useful and supportive of learning in the clinical and classroom settings. The MISET has a valid three-factor structure with high explained variance (74.7%). Internal consistency reliabilities were high for the MISET total (.90) and three subscales: Usefulness/Helpfulness, Information Literacy Support, and Use of Evidence-Based Sources (.87 to .94). Construct validity evidence included significantly higher mean total MISET, Helpfulness/Usefulness, and Information Literacy Support scores for senior students and those with higher computer competence. The MISET is a promising tool to evaluate mobile information technologies and information literacy support; however, longitudinal assessment of changes in scores over time would determine scale sensitivity and responsiveness. [J Nurs Educ. 2016;55(7):385-390.]. Copyright 2016, SLACK Incorporated.

How to construct and implement script concordance tests: insights from a systematic review.

PubMed

Dory, Valérie; Gagnon, Robert; Vanpee, Dominique; Charlin, Bernard

2012-06-01

Programmes of assessment should measure the various components of clinical competence. Clinical reasoning has been traditionally assessed using written tests and performance-based tests. The script concordance test (SCT) was developed to assess clinical data interpretation skills. A recent review of the literature examined the validity argument concerning the SCT. Our aim was to provide potential users with evidence-based recommendations on how to construct and implement an SCT. A systematic review of relevant databases (MEDLINE, ERIC [Education Resources Information Centre], PsycINFO, the Research and Development Resource Base [RDRB, University of Toronto]) and Google Scholar, medical education journals and conference proceedings was conducted for references in English or French. It was supplemented by ancestry searching and by additional references provided by experts. The search yielded 848 references, of which 80 were analysed. Studies suggest that tests with around 100 items (25-30 cases), of which 25% are discarded after item analysis, should provide reliable scores. Panels with 10-20 members are needed to reach adequate precision in terms of estimated reliability. Panellists' responses can be analysed by checking for moderate variability among responses. Studies of alternative scoring methods are inconclusive, but the traditional scoring method is satisfactory. There is little evidence on how best to determine a pass/fail threshold for high-stakes examinations. Our literature search was broad and included references from medical education journals not indexed in the usual databases, conference abstracts and dissertations. There is good evidence on how to construct and implement an SCT for formative purposes or medium-stakes course evaluations. Further avenues for research include examining the impact of various aspects of SCT construction and implementation on issues such as educational impact, correlations with other assessments, and validity of pass/fail decisions, particularly for high-stakes examinations. © Blackwell Publishing Ltd 2012.

The Employment Precariousness Scale (EPRES): psychometric properties of a new tool for epidemiological studies among waged and salaried workers.

PubMed

Vives, Alejandra; Amable, Marcelo; Ferrer, Montserrat; Moncada, Salvador; Llorens, Clara; Muntaner, Carles; Benavides, Fernando G; Benach, Joan

2010-08-01

Despite the fact that labour market flexibility has resulted in an expansion of precarious employment in industrialised countries, to date there is limited empirical evidence concerning its health consequences. The Employment Precariousness Scale (EPRES) is a newly developed, theory-based, multidimensional questionnaire specifically devised for epidemiological studies among waged and salaried workers. To assess the acceptability, reliability and construct validity of EPRES in a sample of waged and salaried workers in Spain. A sample of 6968 temporary and permanent workers from a population-based survey carried out in 2004-2005 was analysed. The survey questionnaire was interviewer administered and included the six EPRES subscales, and measures of the psychosocial work environment (COPSOQ ISTAS21) and perceived general and mental health (SF-36). A high response rate to all EPRES items indicated good acceptability; Cronbach's alpha coefficients, over 0.70 for all subscales and the global score, demonstrated good internal consistency reliability; exploratory factor analysis using principal axis analysis and varimax rotation confirmed the six-subscale structure and the theoretical allocation of all items. Patterns across known groups and correlation coefficients with psychosocial work environment measures and perceived health demonstrated the expected relations, providing evidence of construct validity. Our results provide evidence in support of the psychometric properties of EPRES, which appears to be a promising tool for the measurement of employment precariousness in public health research.

Development of a Self-Determination Measure for College Students: Validity Evidence for the Basic Needs Satisfaction at College Scale

ERIC Educational Resources Information Center

Jenkins-Guarnieri, Michael A.; Vaughan, Angela L.; Wright, Stephen L.

2015-01-01

We adapted a work self-determination measure to create the Basic Needs Satisfaction at College Scale. Confirmatory factor analysis and item response theory analyses with data from 525 adults supported a 3-factor model with 13 items most sensitive for lower to middle range levels of the autonomy, competence, and relatedness constructs.

Development of the SIT, an Instrument to Evaluate the Transfer Effects of Adult Education Programs for Social Inclusion

ERIC Educational Resources Information Center

de Greef, Maurice; Segers, Mien; Verte, Dominique

2010-01-01

To date, hardly any evidence is available on the quality of adult education programs for vulnerable adults. Evaluation instruments or models mostly focussed on regular education and less on programs of adult education aiming to enhance social inclusion. This study presents a first exploration of the construct validity of a newly developed…

Validating the Learning Cycle Models of Business Simulation Games via Student Perceived Gains in Skills and Knowledge

ERIC Educational Resources Information Center

Tao, Yu-Hui; Yeh, C. Rosa; Hung, Kung Chin

2015-01-01

Several theoretical models have been constructed to determine the effects of buisness simulation games (BSGs) on learning performance. Although these models agree on the concept of learning-cycle effect, no empirical evidence supports the claim that the use of learning cycle activities with BSGs produces an effect on incremental gains in knowledge…

Development and Validation of MMPI-2-RF Scales for Indexing Triarchic Psychopathy Constructs.

PubMed

Sellbom, Martin; Drislane, Laura E; Johnson, Alexandria K; Goodwin, Brandee E; Phillips, Tasha R; Patrick, Christopher J

2016-10-01

The triarchic model characterizes psychopathy in terms of three distinct dispositional constructs of boldness, meanness, and disinhibition. The model can be operationalized through scales designed specifically to index these domains or by using items from other inventories that provide coverage of related constructs. The present study sought to develop and validate scales for assessing the triarchic model domains using items from the Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF). A consensus rating approach was used to identify items relevant to each triarchic domain, and following psychometric refinement, the resulting MMPI-2-RF-based triarchic scales were evaluated for convergent and discriminant validity in relation to multiple psychopathy-relevant criterion variables in offender and nonoffender samples. Expected convergent and discriminant associations were evident very clearly for the Boldness and Disinhibition scales and somewhat less clearly for the Meanness scale. Moreover, hierarchical regression analyses indicated that all MMPI-2-RF triarchic scales incremented standard MMPI-2-RF scale scores in predicting extant triarchic model scale scores. The widespread use of MMPI-2-RF in clinical and forensic settings provides avenues for both clinical and research applications in contexts where traditional psychopathy measures are less likely to be administered. © The Author(s) 2015.

A systematic review of a functional assessment Tool: UCSD Performance-based skill assessment (UPSA).

PubMed

Becattini-Oliveira, Ana Claudia; Dutra, Douglas de Farias; Spenciere de Oliveira Campos, Bárbara; de Araujo, Verônica Carvalho; Charchat-Fichman, Helenice

2018-05-18

Performance based assessment instruments have been employed in functional capacity measurement of mental disorders. The aim of this systematic review was to identify the psychometric properties of the UCSD Performance-based Skill Assessment (UPSA). A search was conducted using the PRISMA protocol and 'UPSA' as key word term on electronic databases, with a date range for articles published from 2001-2017. Published studies involving community-dwelling adults were included. Pharmacological and/or clinical interventions involving clinical outcomes and/or institutionalized samples were excluded. Data related to construct validity, test-retest reliability and sensitivity/specificity were extracted, summarized and analyzed according to UPSA versions and psychiatric disorders. Fifty-eight studies including 8782 Community-dwelling adults met selection criteria. Data supporting the construct and known-groups validity were extracted from 41 studies involving Schizophrenia and schizoaffective disorders and 17 studies involving other metal illness. The UPSA was culturally adapted to 8 different languages and employed in 17 countries. Few studies reported sensitivity and specificity and the cut-off points could not be generalized. Moderate to strong evidence of construct validity and test-retest reliability was found. Few studies proposed cut-off points. The UPSA showed good psychometric properties in different versions including those culturally adapted. Copyright © 2018 Elsevier B.V. All rights reserved.

Assessing Measurement Invariance for Spanish Sentence Repetition and Morphology Elicitation Tasks.

PubMed

Kapantzoglou, Maria; Thompson, Marilyn S; Gray, Shelley; Restrepo, M Adelaida

2016-04-01

The purpose of this study was to evaluate evidence supporting the construct validity of two grammatical tasks (sentence repetition, morphology elicitation) included in the Spanish Screener for Language Impairment in Children (Restrepo, Gorin, & Gray, 2013). We evaluated if the tasks measured the targeted grammatical skills in the same way across predominantly Spanish-speaking children with typical language development and those with primary language impairment. A multiple-group, confirmatory factor analytic approach was applied to examine factorial invariance in a sample of 307 predominantly Spanish-speaking children (177 with typical language development; 130 with primary language impairment). The 2 newly developed grammatical tasks were modeled as measures in a unidimensional confirmatory factor analytic model along with 3 well-established grammatical measures from the Clinical Evaluation of Language Fundamentals-Fourth Edition, Spanish (Wiig, Semel, & Secord, 2006). Results suggest that both new tasks measured the construct of grammatical skills for both language-ability groups in an equivalent manner. There was no evidence of bias related to children's language status for the Spanish Screener for Language Impairment in Children Sentence Repetition or Morphology Elicitation tasks. Results provide support for the validity of the new tasks as measures of grammatical skills.

Exploring the Validity of Proposed Transgenic Animal Models of Attention-Deficit Hyperactivity Disorder (ADHD).

PubMed

de la Peña, June Bryan; Dela Peña, Irene Joy; Custodio, Raly James; Botanas, Chrislean Jun; Kim, Hee Jin; Cheong, Jae Hoon

2018-05-01

Attention-deficit/hyperactivity disorder (ADHD) is a common, behavioral, and heterogeneous neurodevelopmental condition characterized by hyperactivity, impulsivity, and inattention. Symptoms of this disorder are managed by treatment with methylphenidate, amphetamine, and/or atomoxetine. The cause of ADHD is unknown, but substantial evidence indicates that this disorder has a significant genetic component. Transgenic animals have become an essential tool in uncovering the genetic factors underlying ADHD. Although they cannot accurately reflect the human condition, they can provide insights into the disorder that cannot be obtained from human studies due to various limitations. An ideal animal model of ADHD must have face (similarity in symptoms), predictive (similarity in response to treatment or medications), and construct (similarity in etiology or underlying pathophysiological mechanism) validity. As the exact etiology of ADHD remains unclear, the construct validity of animal models of ADHD would always be limited. The proposed transgenic animal models of ADHD have substantially increased and diversified over the years. In this paper, we compiled and explored the validity of proposed transgenic animal models of ADHD. Each of the reviewed transgenic animal models has strengths and limitations. Some fulfill most of the validity criteria of an animal model of ADHD and have been extensively used, while there are others that require further validation. Nevertheless, these transgenic animal models of ADHD have provided and will continue to provide valuable insights into the genetic underpinnings of this complex disorder.

Full immersion simulation: validation of a distributed simulation environment for technical and non-technical skills training in Urology.

PubMed

Brewin, James; Tang, Jessica; Dasgupta, Prokar; Khan, Muhammad S; Ahmed, Kamran; Bello, Fernando; Kneebone, Roger; Jaye, Peter

2015-07-01

To evaluate the face, content and construct validity of the distributed simulation (DS) environment for technical and non-technical skills training in endourology. To evaluate the educational impact of DS for urology training. DS offers a portable, low-cost simulated operating room environment that can be set up in any open space. A prospective mixed methods design using established validation methodology was conducted in this simulated environment with 10 experienced and 10 trainee urologists. All participants performed a simulated prostate resection in the DS environment. Outcome measures included surveys to evaluate the DS, as well as comparative analyses of experienced and trainee urologist's performance using real-time and 'blinded' video analysis and validated performance metrics. Non-parametric statistical methods were used to compare differences between groups. The DS environment demonstrated face, content and construct validity for both non-technical and technical skills. Kirkpatrick level 1 evidence for the educational impact of the DS environment was shown. Further studies are needed to evaluate the effect of simulated operating room training on real operating room performance. This study has shown the validity of the DS environment for non-technical, as well as technical skills training. DS-based simulation appears to be a valuable addition to traditional classroom-based simulation training. © 2014 The Authors BJU International © 2014 BJU International Published by John Wiley & Sons Ltd.

Development and testing of the CALDs and CLES+T scales for international nursing students' clinical learning environments.

PubMed

Mikkonen, Kristina; Elo, Satu; Miettunen, Jouko; Saarikoski, Mikko; Kääriäinen, Maria

2017-08-01

The purpose of this study was to develop and test the psychometric properties of the new Cultural and Linguistic Diversity scale, which is designed to be used with the newly validated Clinical Learning Environment, Supervision and Nurse Teacher scale for assessing international nursing students' clinical learning environments. In various developed countries, clinical placements are known to present challenges in the professional development of international nursing students. A cross-sectional survey. Data were collected from eight Finnish universities of applied sciences offering nursing degree courses taught in English during 2015-2016. All the relevant students (N = 664) were invited and 50% chose to participate. Of the total data submitted by the participants, 28% were used for scale validation. The construct validity of the two scales was tested by exploratory factor analysis, while their validity with respect to convergence and discriminability was assessed using Spearman's correlation. Construct validation of the Clinical Learning Environment, Supervision and Nurse Teacher scale yielded an eight-factor model with 34 items, while validation of the Cultural and Linguistic Diversity scale yielded a five-factor model with 21 items. A new scale was developed to improve evidence-based mentorship of international nursing students in clinical learning environments. The instrument will be useful to educators seeking to identify factors that affect the learning of international students. © 2017 John Wiley & Sons Ltd.

Cross-cultural adaptation and validation of the Chinese Comfort, Afford, Respect, and Expect scale of caring nurse-patient interaction competence.

PubMed

Chung, Hui-Chun; Hsieh, Tsung-Cheng; Chen, Yueh-Chih; Chang, Shu-Chuan; Hsu, Wen-Lin

2017-11-29

To investigate the construct validity and reliability of the Chinese Comfort, Afford, Respect, and Expect scale, which can be used to determine clinical nurses' competence. The results can also serve to promote nursing competence and improve patient satisfaction. Nurse-patient interaction is critical for improving nursing care quality. However, to date, no relevant validated instrument has been proposed for assessing caring nurse-patient interaction competence in clinical practice. This study adapted and validated the Chinese version of the caring nurse-patient interaction scale. A cross-cultural adaptation and validation study. A psychometric analysis of the four major constructs of the Chinese Comfort, Afford, Respect, and Expect scale was conducted on a sample of 356 nurses from a medical centre in China. Item analysis and exploratory factor analysis were adopted to extract the main components, both the internal consistency and correlation coefficients were used to examine reliability and a confirmatory factor analysis was adopted to verify the construct validity. The goodness-of-fit results of the model were strong. The standardised factor loadings of the Chinese Comfort, Afford, Respect, and Expect scale ranged from 0.73-0.95, indicating that the validity and reliability of this instrument were favourable. Moreover, the 12 extracted items explained 95.9% of the measured content of the Chinese Comfort, Afford, Respect, and Expect scale. The results serve as empirical evidence regarding the validity and reliability of the Chinese Comfort, Afford, Respect, and Expect scale. Hospital nurses increasingly demand help from patients and their family members in identifying health problems and assisting with medical decision-making. Therefore, enhancing nurses' competence in nurse-patient interactions is crucial for nursing and hospital managers to improve nursing care quality. The Chinese caring nurse-patient interaction scale can serve as an effective tool for nursing and hospital managers to evaluate the caring nurse-patient interaction confidence of nurses and improve inpatient satisfaction and quality of care. © 2017 John Wiley & Sons Ltd.

Relationships among Safety Climate, Safety Behavior, and Safety Outcomes for Ethnic Minority Construction Workers

PubMed Central

Lyu, Sainan; Chan, Albert P. C.; Wong, Francis K. W.

2018-01-01

In many countries, it is common practice to attract and employ ethnic minority (EM) or migrant workers in the construction industry. This primarily occurs in order to alleviate the labor shortage caused by an aging workforce with a lack of new entrants. Statistics show that EM construction workers are more likely to have occupational fatal and nonfatal injuries than their local counterparts; however, the mechanism underlying accidents and injuries in this vulnerable population has been rarely examined. This study aims to investigate relationships among safety climate, safety behavior, and safety outcomes for EM construction workers. To this end, a theoretical research model was developed based on a comprehensive review of the current literature. In total, 289 valid questionnaires were collected face-to-face from 223 Nepalese construction workers and 56 Pakistani construction workers working on 15 construction sites in Hong Kong. Structural equation modelling was employed to validate the constructs and test the hypothesized model. Results show that there were significant positive relationships between safety climate and safety behaviors, and significant negative relationships between safety behaviors and safety outcomes for EM construction workers. This research contributes to the literature regarding EM workers by providing empirical evidence of the mechanisms by which safety climate affects safety behaviors and outcomes. It also provides insights in order to help the key stakeholders formulate safety strategies for EM workers in many areas where numerous EM workers are employed, such as in the U.S., the UK, Australia, Singapore, Malaysia, and the Middle East. PMID:29522503

Discriminant content validity: a quantitative methodology for assessing content of theory-based measures, with illustrative applications.

PubMed

Johnston, Marie; Dixon, Diane; Hart, Jo; Glidewell, Liz; Schröder, Carin; Pollard, Beth

2014-05-01

In studies involving theoretical constructs, it is important that measures have good content validity and that there is not contamination of measures by content from other constructs. While reliability and construct validity are routinely reported, to date, there has not been a satisfactory, transparent, and systematic method of assessing and reporting content validity. In this paper, we describe a methodology of discriminant content validity (DCV) and illustrate its application in three studies. Discriminant content validity involves six steps: construct definition, item selection, judge identification, judgement format, single-sample test of content validity, and assessment of discriminant items. In three studies, these steps were applied to a measure of illness perceptions (IPQ-R) and control cognitions. The IPQ-R performed well with most items being purely related to their target construct, although timeline and consequences had small problems. By contrast, the study of control cognitions identified problems in measuring constructs independently. In the final study, direct estimation response formats for theory of planned behaviour constructs were found to have as good DCV as Likert format. The DCV method allowed quantitative assessment of each item and can therefore inform the content validity of the measures assessed. The methods can be applied to assess content validity before or after collecting data to select the appropriate items to measure theoretical constructs. Further, the data reported for each item in Appendix S1 can be used in item or measure selection. Statement of contribution What is already known on this subject? There are agreed methods of assessing and reporting construct validity of measures of theoretical constructs, but not their content validity. Content validity is rarely reported in a systematic and transparent manner. What does this study add? The paper proposes discriminant content validity (DCV), a systematic and transparent method of assessing and reporting whether items assess the intended theoretical construct and only that construct. In three studies, DCV was applied to measures of illness perceptions, control cognitions, and theory of planned behaviour response formats. Appendix S1 gives content validity indices for each item of each questionnaire investigated. Discriminant content validity is ideally applied while the measure is being developed, before using to measure the construct(s), but can also be applied after using a measure. © 2014 The British Psychological Society.

Validation of the Mayo Hip Score: construct validity, reliability and responsiveness to change.

PubMed

Singh, Jasvinder A; Schleck, Cathy; Harmsen, W Scott; Lewallen, David G

2016-01-19

Previous studies have provided the initial evidence for construct validity and test-retest reliability of the Mayo Hip Score. Instruments used for Total Hip Arthroplasty (THA) outcomes assessment should be valid, reliable and responsive to change. Our main objective was to examine the responsiveness to change, association with subsequent revision and the construct validity of the Mayo hip score. Discriminant ability was assessed by calculating effect size (ES), standardized response mean (SRM) and Guyatt's responsiveness index (GRI). Minimal clinically important difference (MCII) and moderate improvement thresholds were calculated. We assessed construct validity by examining association of scores with preoperative patient characteristics and correlation with Harris hip score, and assessed association of scores with the risk of subsequent revision. Five thousand three hundred seven provided baseline data; of those with baseline data, 2,278 and 2,089 (39%) provided 2- and 5-year data, respectively. Large ES, SRM and GRI ranging 2.66-2.78, 2.42-2.61 and 1.67-1.88 were noted for Mayo hip scores with THA, respectively. The MCII and moderate improvement thresholds were 22.4-22.7 and 39.4-40.5 respectively. Hazard ratios of revision surgery were higher with lower final score or less improvement in Mayo hip score at 2-years and borderline significant/non-significant at 5-years, respectively: (1) score ≤55 with hazard ratios of 2.24 (95% CI, 1.45, 3.46; p = 0.0003) and 1.70 (95% CI, 1.00, 2.92; p = 0.05) of implant revision subsequently, compared to 72-80 points; (2) no improvement or worsening score with hazard ratios 3.94 (95% CI, 1.50, 10.30; p = 0.005) and 2.72 (95% CI, 0.85,8.70; p = 0.09), compared to improvement >50-points. Mayo hip score had significant positive correlation with younger age, male gender, lower BMI, lower ASA class and lower Deyo-Charlson index (p ≤ 0.003 for each) and with Harris hip scores (p < 0.001). Mayo Hip Score is valid, sensitive to change and associated with future risk of revision surgery in patients with primary THA.

The Mastery Rubric for Evidence-Based Medicine: Institutional Validation via Multidimensional Scaling.

PubMed

Tractenberg, Rochelle E; Gushta, Matthew M; Weinfeld, Jeffrey M

2016-01-01

CONSTRUCT: In this study we describe a multidimensional scaling (MDS) exercise to validate the curricular elements composing a new Mastery Rubric (MR) for a curriculum in evidence-based medicine (EBM). This MR-EBM comprises 10 elements of knowledge, skills, and abilities (KSAs) representing our institutional learning goals of career-spanning engagement with EBM. An MR also includes developmental trajectories for each KSA, beginning with medical school coursework, including residency training, and outlining the qualifications of individuals to teach and mentor in EBM. The development was not part of the validation effort, as our curriculum is focused at a single stage (undergraduate medical students). An MR comprises the desired KSAs for an entire curriculum, together with descriptions of a learner's performance and/or capabilities as they develop from novice to proficiency of the curricular target(s). The MR construct is intended to support curriculum development or refinement by capturing the KSAs that support the articulation of concrete learning goals; it also promotes assessment that demonstrates development in the target KSAs and encourages reflection and self-directed learning throughout the learner's career. Two other MRs have been published, and this is the first one specific to teaching and learning in medicine; this is also the first one created specifically to evaluate an existing curriculum. To validate the dispersion of the elements of the EBM curriculum, the nine clinical instructors in the EBM two-course curriculum completed an MDS exercise, rating the similarities of the 10 curricular elements. MDS is a mathematical approach to understanding relationships among concepts/objects when these relationships are difficult to quantify. Eliciting similarity ratings biased the responses toward the null hypothesis (that the elements are not different). MDS results suggested that the MR represents 10 different, although related, facets of the construct "evidence-based medicine." The results support the makeup of the MR-EBM, and its use to revise our EBM curriculum so that it is more closely aligned with this MR. An MR is a tool, and the MR-EBM that we describe can be useful to develop or evaluate a curriculum in EBM. The MR tool is particularly compatible with the objectives of training for EBM and practice and can be applied to create or evaluate a curriculum using any topical KSA framework. The MR-EBM we describe could be adopted or adapted to represent other institutional objectives for EBM training.

Standard Setting Methods for Pass/Fail Decisions on High-Stakes Objective Structured Clinical Examinations: A Validity Study.

PubMed

Yousuf, Naveed; Violato, Claudio; Zuberi, Rukhsana W

2015-01-01

CONSTRUCT: Authentic standard setting methods will demonstrate high convergent validity evidence of their outcomes, that is, cutoff scores and pass/fail decisions, with most other methods when compared with each other. The objective structured clinical examination (OSCE) was established for valid, reliable, and objective assessment of clinical skills in health professions education. Various standard setting methods have been proposed to identify objective, reliable, and valid cutoff scores on OSCEs. These methods may identify different cutoff scores for the same examinations. Identification of valid and reliable cutoff scores for OSCEs remains an important issue and a challenge. Thirty OSCE stations administered at least twice in the years 2010-2012 to 393 medical students in Years 2 and 3 at Aga Khan University are included. Psychometric properties of the scores are determined. Cutoff scores and pass/fail decisions of Wijnen, Cohen, Mean-1.5SD, Mean-1SD, Angoff, borderline group and borderline regression (BL-R) methods are compared with each other and with three variants of cluster analysis using repeated measures analysis of variance and Cohen's kappa. The mean psychometric indices on the 30 OSCE stations are reliability coefficient = 0.76 (SD = 0.12); standard error of measurement = 5.66 (SD = 1.38); coefficient of determination = 0.47 (SD = 0.19), and intergrade discrimination = 7.19 (SD = 1.89). BL-R and Wijnen methods show the highest convergent validity evidence among other methods on the defined criteria. Angoff and Mean-1.5SD demonstrated least convergent validity evidence. The three cluster variants showed substantial convergent validity with borderline methods. Although there was a high level of convergent validity of Wijnen method, it lacks the theoretical strength to be used for competency-based assessments. The BL-R method is found to show the highest convergent validity evidences for OSCEs with other standard setting methods used in the present study. We also found that cluster analysis using mean method can be used for quality assurance of borderline methods. These findings should be further confirmed by studies in other settings.

The Nordic Musculoskeletal Questionnaire: cross-cultural adaptation into Turkish assessing its psychometric properties.

PubMed

Kahraman, Turhan; Genç, Arzu; Göz, Evrim

2016-10-01

The purpose of this study was to linguistically and culturally adapt the Nordic Musculoskeletal Questionnaire (NMQ) for use in Turkey, and to examine the psychometric properties of this adapted version. The cross-cultural adaptation was achieved by translating the items from the original version, with back-translation performed by independent mother-tongue translators, followed by committee review. Reliability (internal consistency and test-retest) was examined for 198 participants who completed the NMQ twice (with a 1 week interval). Construct validity was examined with data from 126 participants from the same population, who completed further four questionnaires related to the body regions described in the NMQ. The internal consistency was excellent (Cronbach's alpha = 0.896). The test-retest reliability was examined with the prevalence-adjusted bias-adjusted kappa (PABAK) and all items showed moderate to almost perfect reliability (PABAK = 0.57-0.90). Participants with a musculoskeletal problem in a related region had significantly more disability/pain, as assessed by the relevant questionnaires (p < 0.001), indicating that the NMQ had a good construct validity. This study provided considerable evidence that the Turkish version of the NMQ has appropriate psychometric properties, including good test-retest reliability, internal consistency and construct validity. It can be used for screening and epidemiological investigations of musculoskeletal symptoms. Implications for Rehabilitation The Nordic Musculoskeletal Questionnaire (NMQ) can be used for the screening of musculoskeletal problems. The NMQ allows comparison of musculoskeletal problems in different body regions in epidemiological studies with large numbers of participants. The Turkish version of the NMQ can be used for rehabilitation due to its appropriate psychometric properties, including good test-retest reliability, internal consistency and construct validity.

Face and construct validity of a computer-based virtual reality simulator for ERCP.

PubMed

Bittner, James G; Mellinger, John D; Imam, Toufic; Schade, Robert R; Macfadyen, Bruce V

2010-02-01

Currently, little evidence supports computer-based simulation for ERCP training. To determine face and construct validity of a computer-based simulator for ERCP and assess its perceived utility as a training tool. Novice and expert endoscopists completed 2 simulated ERCP cases by using the GI Mentor II. Virtual Education and Surgical Simulation Laboratory, Medical College of Georgia. Outcomes included times to complete the procedure, reach the papilla, and use fluoroscopy; attempts to cannulate the papilla, pancreatic duct, and common bile duct; and number of contrast injections and complications. Subjects assessed simulator graphics, procedural accuracy, difficulty, haptics, overall realism, and training potential. Only when performance data from cases A and B were combined did the GI Mentor II differentiate novices and experts based on times to complete the procedure, reach the papilla, and use fluoroscopy. Across skill levels, overall opinions were similar regarding graphics (moderately realistic), accuracy (similar to clinical ERCP), difficulty (similar to clinical ERCP), overall realism (moderately realistic), and haptics. Most participants (92%) claimed that the simulator has definite training potential or should be required for training. Small sample size, single institution. The GI Mentor II demonstrated construct validity for ERCP based on select metrics. Most subjects thought that the simulated graphics, procedural accuracy, and overall realism exhibit face validity. Subjects deemed it a useful training tool. Study repetition involving more participants and cases may help confirm results and establish the simulator's ability to differentiate skill levels based on ERCP-specific metrics.

The development and validation of an instrument to measure preservice teachers' self-efficacy in regard to the teaching of science as inquiry

NASA Astrophysics Data System (ADS)

Dira-Smolleck, Lori

The purpose of this study was to develop, validate and establish the reliability of an instrument that measures preservice teachers' self-efficacy in regard to the teaching of science as inquiry. The instrument (TSI) is based upon the work of Bandura, Riggs, and Enochs & Riggs (1990). The study used Bandura's theoretical framework in that the instrument uses the self-efficacy construct to explore the beliefs of prospective elementary science teachers with regards to the teaching of science through inquiry: specifically, the two dimensions of self-efficacy beliefs defined by Bandura: personal self-efficacy and outcome expectancy. Self-efficacy in regard to the teaching of science as inquiry was measured through the use of a 69-item Likert scale instrument designed by the author of the study. A 13-step plan was designed and followed in the process of developing the instrument. Using the results from Chronbach Alpha and Analysis of Variance, a 69-item instrument was found to achieve the greatest balance across the construct validity, reliability and item balance with the Essential Elements of Classroom Inquiry content matrix. Based on the standardized development processes used and the associated evidence, the TSI appears to be a content and construct valid instrument, with high internal reliability for use with prospective elementary teachers to assess self-efficacy beliefs in regard to the teaching of science as inquiry. Implications for research, policy and practice are also discussed.

Psychometric Validation of the Academic Motivation Scale in a Dental Student Sample.

PubMed

Orsini, Cesar; Binnie, Vivian; Evans, Phillip; Ledezma, Priscilla; Fuentes, Fernando; Villegas, Maria J

2015-08-01

The Academic Motivation Scale is one of the most frequently used instruments to assess academic motivation. It relies on the self-determination theory of human motivation. However, motivation has been understudied in dental education. Therefore, to address the lack of valid instruments to assess academic motivation in dental education and contribute to future research in the field, the aim of this study was to analyze the psychometric properties of this instrument in a sample of dental students. Participants were 989 Chilean undergraduate dental students (86% response rate) who completed a survey containing a Chilean face-valid version of the Spanish Academic Motivation Scale and three other motivation-related instruments to assess the survey's construct and criterion validity. Later, 76 of the students (out of 100 invited) took the survey again to assess its test-retest stability. The instrument's construct validity was supported by the superior goodness of fit of the seven-subscale Academic Motivation Scale over competing models through confirmatory factor analysis and by the expected correlations among its subscales. The concurrent criterion validity was supported by the confirmation of correlations between its subscales and external criteria. Adequate internal consistency and test-retest correlations were also found. The evidence from this study suggests that the Academic Motivation Scale is a preliminarily valid and reliable instrument to assess motivation in the predoctoral dental context. Future research in this area is needed to confirm or refute these results.

Validation of the PedsQL Epilepsy Module: A pediatric epilepsy-specific health-related quality of life measure.

PubMed

Modi, Avani C; Junger, Katherine F; Mara, Constance A; Kellermann, Tanja; Barrett, Lauren; Wagner, Janelle; Mucci, Grace A; Bailey, Laurie; Almane, Dace; Guilfoyle, Shanna M; Urso, Lauryn; Hater, Brooke; Hustzi, Heather; Smith, Gigi; Herrmann, Bruce; Perry, M Scott; Zupanc, Mary; Varni, James W

2017-11-01

To validate a brief and reliable epilepsy-specific, health-related quality of life (HRQOL) measure in children with various seizure types, treatments, and demographic characteristics. This national validation study was conducted across five epilepsy centers in the United States. Youth 5-18 years and caregivers of youth 2-18 years diagnosed with epilepsy completed the PedsQL Epilepsy Module and additional questionnaires to establish reliability and validity of the epilepsy-specific HRQOL instrument. Demographic and medical data were collected through chart reviews. Factor analysis was conducted, and internal consistency (Cronbach's alphas), test-retest reliability, and construct validity were assessed. Questionnaires were analyzed from 430 children with epilepsy (M age = 9.9 years; range 2-18 years; 46% female; 62% white: non-Hispanic; 76% monotherapy, 54% active seizures) and their caregivers. The final PedsQL Epilepsy Module is a 29-item measure with five subscales (i.e., Impact, Cognitive, Sleep, Executive Functioning, and Mood/Behavior) with parallel child and caregiver reports. Internal consistency coefficients ranged from 0.70-0.94. Construct validity and convergence was demonstrated in several ways, including strong relationships with seizure outcomes, antiepileptic drug (AED) side effects, and well-established measures of executive, cognitive, and emotional/behavioral functioning. The PedsQL Epilepsy Module is a reliable measure of HRQOL with strong evidence of its validity across the epilepsy spectrum in both clinical and research settings. Wiley Periodicals, Inc. © 2017 International League Against Epilepsy.

Further evidence for the reliability and validity of the Modified Dental Anxiety Scale.

PubMed

Humphris, G M; Freeman, R; Campbell, J; Tuutti, H; D'Souza, V

2000-12-01

To gain further evidence of the psychometric properties of the Modified Dental Anxiety Scale. Dental admission clinics. Consecutive sampling, cross-sectional survey. Patients (n = 800) in four cities (Belfast, Northern Ireland; Helsinki, Finland; Jyväskylä, Finland and Dubai, UAE). Questionnaire booklet handed to patients, attending clinics, for completion following an invitation by the researcher to be included in the study. Modified Dental Anxiety Scale (MDAS), together with further questions concerning dental attendance and nervousness about dental procedures. Overall 9.3 per cent of patients indicated high dental anxiety. MDAS showed high levels of internal consistency, and good construct validity. The relationship of dental anxiety with age was similar to previous reports and showed lowered anxiety levels in older patients. Data from three countries has supported the psychometric properties of this modified and brief dental anxiety scale.

Assessing fidelity in individual and family therapy for adolescent substance abuse.

PubMed

Hogue, Aaron; Dauber, Sarah; Chinchilla, Priscilla; Fried, Adam; Henderson, Craig; Inclan, Jaime; Reiner, Robert H; Liddle, Howard A

2008-09-01

This study introduces an observational measure of fidelity in evidence-based practices for adolescent substance abuse treatment. The Therapist Behavior Rating Scale-Competence (TBRS-C) measures adherence and competence in individual cognitive-behavioral therapy and multidimensional family therapy for adolescent substance abuse. The TBRS-C assesses fidelity to the core therapeutic goals of each approach and also contains global ratings of therapist competence. Study participants were 136 clinically referred adolescents and their families observed in 437 treatment sessions. The TBRS-C demonstrated strong interrater reliability for goal-specific ratings of treatment adherence, and modest reliability for goal-specific and global ratings of therapist competence, evidence of construct validity, and discriminant validity with an observational measure of therapeutic alliance. The utility of the TBRS-C for evaluating treatment fidelity in field settings is discussed.

Self-Reported Emotion Reactivity Among Early-Adolescent Girls: Evidence for Convergent and Discriminant Validity in an Urban Community Sample.

PubMed

Evans, Spencer C; Blossom, Jennifer B; Canter, Kimberly S; Poppert-Cordts, Katrina; Kanine, Rebecca; Garcia, Andrea; Roberts, Michael C

2016-05-01

Emotion reactivity, measured via the self-report Emotion Reactivity Scale (ERS), has shown unique associations with different forms of psychopathology and suicidal thoughts and behaviors; however, this limited body of research has been conducted among adults and older adolescents of predominantly White/European ethnic backgrounds. The present study investigated the validity of ERS scores for measuring emotion reactivity among an urban community sample of middle-school-age girls. Participants (N = 93, ages 11-15, 76% African-American, 18% Latina) completed the ERS and measures of emotion coping, internalizing problems, proactive and reactive aggression, negative life events, and lifetime suicidal ideation and substance use. As hypothesized, ERS scores were significantly associated with internalizing problems, poor emotion coping, negative life events, reactive aggression, and suicidal ideation (evidence for convergent validity), but showed little to no association with proactive aggression or lifetime substance use (evidence for discriminant validity). A series of logistic regressions were conducted to further explore the associations among internalizing problems, emotion reactivity, and suicidal ideation. With depressive symptoms included in the model, emotion reactivity was no longer uniquely predictive of lifetime suicidal ideation, nor did it serve as a moderator of other associations. In conjunction with previous research, these findings offer further support for the construct validity and research utility of the ERS as a self-report measure of emotion reactivity in adolescents. Copyright © 2016. Published by Elsevier Ltd.

Therapist self-report of evidence-based practices in usual care for adolescent behavior problems: factor and construct validity.

PubMed

Hogue, Aaron; Dauber, Sarah; Henderson, Craig E

2014-01-01

This study introduces a therapist-report measure of evidence-based practices for adolescent conduct and substance use problems. The Inventory of Therapy Techniques-Adolescent Behavior Problems (ITT-ABP) is a post-session measure of 27 techniques representing four approaches: cognitive-behavioral therapy (CBT), family therapy (FT), motivational interviewing (MI), and drug counseling (DC). A total of 822 protocols were collected from 32 therapists treating 71 adolescents in six usual care sites. Factor analyses identified three clinically coherent scales with strong internal consistency across the full sample: FT (8 items; α = .79), MI/CBT (8 items; α = .87), and DC (9 items, α = .90). The scales discriminated between therapists working in a family-oriented site versus other sites and showed moderate convergent validity with therapist reports of allegiance and skill in each approach. The ITT-ABP holds promise as a cost-efficient quality assurance tool for supporting high-fidelity delivery of evidence-based practices in usual care.

Psychometric properties of the Leisure Time Satisfaction Scale in family caregivers.

PubMed

Martínez-Rodríguez, Silvia; Iraurgi, Ioseba; Gómez-Marroquin, Ignacio; Carrasco, María; Ortiz-Marqués, Nuria; Stevens, Alan B

2016-05-01

Despite evidence of the numerous benefits of leisure to health and well-being appropriate tools to assess this construct are lacking. The purpose of this work was to analyse the psychometric properties of the Spanish version of the Leisure Time Satisfaction (LTS). The sample was made up of 1048 primary family caregivers of dependent people. Scale structure was subjected to exploratory and confirmatory factor analysis. Concurrent and convergent validity were assessed by correlation with validated questionnaires for measuring burden (Zarit Burden Inventory - ZBI) and health (SF-36 Health Survey). The results show a high level of internal consistency (Cronbach’s alpha = .938) suitable fit of the dimensional model tested via confirmatory factor analysis (GFI = .925, BBNNFI= .996; IFI= .998, RMSEA= .043), and appropriate convergent validity with similar constructs (r = -.44 with ZBI; and r-values between .226 and .440 with SF-36 dimensions). Psychometric results obtained from the LTS are promising and the results enable us to draw the conclusion that it is a suitable tool for assessing caregivers’ leisure time satisfaction.

Mapping the nomological network of employee self-determined safety motivation: A preliminary measure in China.

PubMed

Jiang, Li; Tetrick, Lois E

2016-09-01

The present study introduced a preliminary measure of employee safety motivation based on the definition of self-determination theory from Fleming (2012) research and validated the structure of self-determined safety motivation (SDSM) by surveying 375 employees in a Chinese high-risk organization. First, confirmatory factor analysis (CFA) was used to examine the factor structure of SDSM, and indices of five-factor model CFA met the requirements. Second, a nomological network was examined to provide evidence of the construct validity of SDSM. Beyond construct validity, the analysis also produced some interesting results concerning the relationship between leadership antecedents and safety motivation, and between safety motivation and safety behavior. Autonomous motivation was positively related to transformational leadership, negatively related to abusive supervision, and positively related to safety behavior. Controlled motivation with the exception of introjected regulation was negatively related to transformational leadership, positively related to abusive supervision, and negatively related to safety behavior. The unique role of introjected regulation and future research based on self-determination theory were discussed. Copyright © 2016 Elsevier Ltd. All rights reserved.

An overview of self-administered health literacy instruments.

PubMed

O Neill, Braden; Gonçalves, Daniela; Ricci-Cabello, Ignacio; Ziebland, Sue; Valderas, Jose

2014-01-01

With the increasing recognition of health literacy as a worldwide research priority, the development and refinement of indices to measure the construct is an important area of inquiry. Furthermore, the proliferation of online resources and research means that there is a growing need for self-administered instruments. We undertook a systematic overview to identify all published self-administered health literacy assessment indices to report their content and considerations associated with their administration. A primary aim of this study was to assist those seeking to employ a self-reported health literacy index to select one that has been developed and validated for an appropriate context, as well as with desired administration characteristics. Systematic searches were carried out in four electronic databases, and studies were included if they reported the development and/or validation of a novel health literacy assessment measure. Data were systematically extracted on key characteristics of the instruments: breadth of construct ("generic" vs. "content- or context- specific" health literacy), whether it was an original instrument or a derivative, country of origin, administration characteristics, age of target population (adult vs. pediatric), and evidence for validity. 35 articles met the inclusion criteria. There were 27 original instruments (27/35; 77.1%) and 8 derivative instruments (8/35; 22.9%). 22 indices measured "general" health literacy (22/35; 62.9%) while the remainder measured condition- or context- specific health literacy (13/35; 37.1%). Most health literacy measures were developed in the United States (22/35; 62.9%), and about half had adequate face, content, and construct validity (16/35; 45.7%). Given the number of measures available for many specific conditions and contexts, and that several have acceptable validity, our findings suggest that the research agenda should shift towards the investigation and elaboration of health literacy as a construct itself, in order for research in health literacy measurement to progress.

Cataract surgeons outperform medical students in Eyesi virtual reality cataract surgery: evidence for construct validity.

PubMed

Selvander, Madeleine; Asman, Peter

2013-08-01

To investigate construct validity for modules hydromaneuvers and phaco on the Eyesi surgical simulator. Seven cataract surgeons and 17 medical students performed capsulorhexis, hydromaneuvers, phaco, navigation, forceps, cracking and chopping modules in a standardized manner. Three trials were performed on each module (two on phaco) in the above order. Performance parameters as calculated by the simulator for each trial were saved. Video recordings of the second trial of the modules capsulorhexis, hydromaneuvers and phaco were evaluated with the modified Objective Structured Assessment of Surgical Skill (OSATS) and Objective Structured Assessment of Cataract Surgical Skill (OSACSS) tools. Cataract surgeons outperformed medical students with regard to overall score on capsulorhexis (p < 0.001, p = 0.035, p = 0.010 for the tree iterations, respectively), navigation (p = 0.024, p = 0.307, p = 0.007), forceps (p = 0.017, p = 0.03, p = 0.028). Less obvious differences in overall score were found for modules cracking and chopping (p = 0.266, p = 0.022, p = 0.324) and phaco (p = 0.011, p = 0.081 for the two iterations, respectively). No differences in overall score were found on hydromaneuvers (p = 0.588, p = 0.503, p = 0.773), but surgeons received better scores from the evaluations of the modified OSATS (p = 0.001) and OSACSS (capsulorhexis, p = 0.003; hydromaneuvers, p = 0.017; phaco, p = 0.001). Construct validity was found on several modules previously not investigated (phaco, hydromaneuvers, cracking and chopping, navigation), and our results confirm previously demonstrated construct validity for capsulorhexis and forceps modules. Interestingly, validation of the hydromaneuvers module required OSACSS video evaluation tool. A further development of the scoring system in the simulator for the hydromaneuvers module would be advantageous and make training and evaluation of progress more accessible and immediate. © 2012 The Authors. Acta Ophthalmologica © 2012 Acta Ophthalmologica Scandinavica Foundation.

[Development of an instrument to measure psychosocial determinants of physical activity behavior among coronary heart disease patients].

PubMed

Mendez, Roberto Della Rosa; Rodrigues, Roberta Cunha Matheus; Cornélio, Marilia Estevam; Gallani, Maria Cecília Bueno Jayme; Godin, Gaston

2010-09-01

The aim of this study was to report the development and the analysis of content validity and reliability of the Psychosocial Determinants of Physical Activity among Coronary Heart Disease Patients Questionnaire, based on an extension of the Theory of Planned Behavior. In the content validity step, three experts evaluated the instrument which was, afterwards, pre-tested with five subjects in order to obtain a conceptually appropriate and easily understood instrument. Fifty-one patients participated in the evaluation of internal consistency of the reviewed instrument. Cronbach's alpha coefficients above 0.75 were observed for the constructs: Intention, Attitude, Subjective Norm, Self-efficacy and Habit. The new instrument demonstrated acceptable evidence of content validity and reliability.

First-in-human Phase 1 CRISPR Gene Editing Cancer Trials: Are We Ready?

PubMed

Baylis, Francoise; McLeod, Marcus

2017-01-01

A prospective first-in-human Phase 1 CRISPR gene editing trial in the United States for patients with melanoma, synovial sarcoma, and multiple myeloma offers hope that gene editing tools may usefully treat human disease. An overarching ethical challenge with first-in-human Phase 1 clinical trials, however, is knowing when it is ethically acceptable to initiate such trials on the basis of safety and efficacy data obtained from pre-clinical studies. If the pre-clinical studies that inform trial design are themselves poorly designed - as a result of which the quality of pre-clinical evidence is deficient - then the ethical requirement of scientific validity for clinical research may not be satisfied. In turn, this could mean that the Phase 1 clinical trial will be unsafe and that trial participants will be exposed to risk for no potential benefit. To assist sponsors, researchers, clinical investigators and reviewers in deciding when it is ethically acceptable to initiate first-in-human Phase 1 CRISPR gene editing clinical trials, structured processes have been developed to assess and minimize translational distance between pre-clinical and clinical research. These processes draw attention to various features of internal validity, construct validity, and external validity. As well, the credibility of supporting evidence is to be critically assessed with particular attention to optimism bias, financial conflicts of interest and publication bias. We critically examine the pre-clinical evidence used to justify the first-inhuman Phase 1 CRISPR gene editing cancer trial in the United States using these tools. We conclude that the proposed trial cannot satisfy the ethical requirement of scientific validity because the supporting pre-clinical evidence used to inform trial design is deficient. Copyright© Bentham Science Publishers; For any queries, please email at epub@benthamscience.org.

Why the Major Field Test in Business Does Not Report Subscores: Reliability and Construct Validity Evidence. Research Report. ETS RR-12-11

ERIC Educational Resources Information Center

Ling, Guangming

2012-01-01

To assess the value of individual students' subscores on the Major Field Test in Business (MFT Business), I examined the test's internal structure with factor analysis and structural equation model methods, and analyzed the subscore reliabilities using the augmented scores method. Analyses of the internal structure suggested that the MFT Business…

Examining an Executive Function Rating Scale as a Predictor of Achievement in Children at Risk for Behavior Problems

ERIC Educational Resources Information Center

Sadeh, Shanna S.; Burns, Matthew K.; Sullivan, Amanda L.

2012-01-01

Evidence suggests that executive function (EF) may be a potent and malleable predictor of academic achievement in children. Schools may be able to use this predictive power if researchers develop EF measures that not only have ecological and construct validity, but also are also efficient and affordable. To this end, Garcia-Barrera and colleagues…

Confirmatory Factor Structure of the Kaufman Assessment Battery for Children-Second Edition with Preschool Children: Too Young for Differentiation?

ERIC Educational Resources Information Center

Potvin, Deborah C. H.; Keith, Timothy Z.; Caemmerer, Jacqueline M.; Trundt, Katherine M.

2015-01-01

With an age range from 3 to 13 years, the Kaufman Assessment Battery for Children-Second Edition (KABC-II) offers an appealing option for the assessment of cognitive abilities for children. Although independent research has provided evidence of the construct validity of the KABC-II for school-age children, previous studies have rarely included an…

Reliability and validity of the test of incremental respiratory endurance measures of inspiratory muscle performance in COPD.

PubMed

Formiga, Magno F; Roach, Kathryn E; Vital, Isabel; Urdaneta, Gisel; Balestrini, Kira; Calderon-Candelario, Rafael A; Campos, Michael A; Cahalin, Lawrence P

2018-01-01

The Test of Incremental Respiratory Endurance (TIRE) provides a comprehensive assessment of inspiratory muscle performance by measuring maximal inspiratory pressure (MIP) over time. The integration of MIP over inspiratory duration (ID) provides the sustained maximal inspiratory pressure (SMIP). Evidence on the reliability and validity of these measurements in COPD is not currently available. Therefore, we assessed the reliability, responsiveness and construct validity of the TIRE measures of inspiratory muscle performance in subjects with COPD. Test-retest reliability, known-groups and convergent validity assessments were implemented simultaneously in 81 male subjects with mild to very severe COPD. TIRE measures were obtained using the portable PrO2 device, following standard guidelines. All TIRE measures were found to be highly reliable, with SMIP demonstrating the strongest test-retest reliability with a nearly perfect intraclass correlation coefficient (ICC) of 0.99, while MIP and ID clustered closely together behind SMIP with ICC values of about 0.97. Our findings also demonstrated known-groups validity of all TIRE measures, with SMIP and ID yielding larger effect sizes when compared to MIP in distinguishing between subjects of different COPD status. Finally, our analyses confirmed convergent validity for both SMIP and ID, but not MIP. The TIRE measures of MIP, SMIP and ID have excellent test-retest reliability and demonstrated known-groups validity in subjects with COPD. SMIP and ID also demonstrated evidence of moderate convergent validity and appear to be more stable measures in this patient population than the traditional MIP.

Psychometric properties of the Perceived Stress Scale (PSS): measurement invariance between athletes and non-athletes and construct validity

PubMed Central

Lin, Ju-Han; Nien, Chiao-Lin; Hsu, Ya-Wen; Liu, Hong-Yu

2016-01-01

Background Although Perceived Stress Scale (PSS, Cohen, Kamarack & Mermelstein, 1983) has been validated and widely used in many domains, there is still no validation in sports by comparing athletes and non-athletes and examining related psychometric indices. Purpose The purpose of this study was to examine the measurement invariance of PSS between athletes and non-athletes, and examine construct validity and reliability in the sports contexts. Methods Study 1 sampled 359 college student-athletes (males = 233; females = 126) and 242 non-athletes (males = 124; females = 118) and examined factorial structure, measurement invariance and internal consistency. Study 2 sampled 196 student-athletes (males = 139, females = 57, Mage = 19.88 yrs, SD = 1.35) and examined discriminant validity and convergent validity of PSS. Study 3 sampled 37 student-athletes to assess test-retest reliability of PSS. Results Results found that 2-factor PSS-10 fitted the model the best and had appropriate reliability. Also, there was a measurement invariance between athletes and non-athletes; and PSS positively correlated with athletic burnout and life stress but negatively correlated with coping efficacy provided evidence of discriminant validity and convergent validity. Further, the test-retest reliability for PSS subscales was significant (r = .66 and r = .50). Discussion It is suggested that 2-factor PSS-10 can be a useful tool in assessing perceived stress either in sports or non-sports settings. We suggest future study may use 2-factor PSS-10 in examining the effects of stress on the athletic injury, burnout, and psychiatry disorders. PMID:27994983

Validity and reliability of instruments aimed at measuring Evidence-Based Practice in Physical Therapy: a systematic review of the literature.

PubMed

Fernández-Domínguez, Juan Carlos; Sesé-Abad, Albert; Morales-Asencio, Jose Miguel; Oliva-Pascual-Vaca, Angel; Salinas-Bueno, Iosune; de Pedro-Gómez, Joan Ernest

2014-12-01

Our goal is to compile and analyse the characteristics - especially validity and reliability - of all the existing international tools that have been used to measure evidence-based clinical practice in physiotherapy. A systematic review conducted with data from exclusively quantitative-type studies synthesized in narrative format. An in-depth search of the literature was conducted in two phases: initial, structured, electronic search of databases and also journals with summarized evidence; followed by a residual-directed search in the bibliographical references of the main articles found in the primary search procedure. The studies included were assigned to members of the research team who acted as peer reviewers. Relevant information was extracted from each of the selected articles using a template that included the general characteristics of the instrument as well as an analysis of the quality of the validation processes carried out, by following the criteria of Terwee. Twenty-four instruments were found to comply with the review screening criteria; however, in all cases, they were found to be limited as regards the 'constructs' included. Besides, they can all be seen to be lacking as regards comprehensiveness associated to the validation process of the psychometric tests used. It seems that what constitutes a rigorously developed assessment instrument for EBP in physical therapy continues to be a challenge. © 2014 John Wiley & Sons, Ltd.

Clinical audit project in undergraduate medical education curriculum: an assessment validation study

PubMed Central

Steketee, Carole; Mak, Donna

2016-01-01

Objectives To evaluate the merit of the Clinical Audit Project (CAP) in an assessment program for undergraduate medical education using a systematic assessment validation framework. Methods A cross-sectional assessment validation study at one medical school in Western Australia, with retrospective qualitative analysis of the design, development, implementation and outcomes of the CAP, and quantitative analysis of assessment data from four cohorts of medical students (2011- 2014). Results The CAP is fit for purpose with clear external and internal alignment to expected medical graduate outcomes.  Substantive validity in students’ and examiners’ response processes is ensured through relevant methodological and cognitive processes. Multiple validity features are built-in to the design, planning and implementation process of the CAP.  There is evidence of high internal consistency reliability of CAP scores (Cronbach’s alpha > 0.8) and inter-examiner consistency reliability (intra-class correlation>0.7). Aggregation of CAP scores is psychometrically sound, with high internal consistency indicating one common underlying construct.  Significant but moderate correlations between CAP scores and scores from other assessment modalities indicate validity of extrapolation and alignment between the CAP and the overall target outcomes of medical graduates.  Standard setting, score equating and fair decision rules justify consequential validity of CAP scores interpretation and use. Conclusions This study provides evidence demonstrating that the CAP is a meaningful and valid component in the assessment program. This systematic framework of validation can be adopted for all levels of assessment in medical education, from individual assessment modality, to the validation of an assessment program as a whole.  PMID:27716612

Clinical audit project in undergraduate medical education curriculum: an assessment validation study.

PubMed

Tor, Elina; Steketee, Carole; Mak, Donna

2016-09-24

To evaluate the merit of the Clinical Audit Project (CAP) in an assessment program for undergraduate medical education using a systematic assessment validation framework. A cross-sectional assessment validation study at one medical school in Western Australia, with retrospective qualitative analysis of the design, development, implementation and outcomes of the CAP, and quantitative analysis of assessment data from four cohorts of medical students (2011- 2014). The CAP is fit for purpose with clear external and internal alignment to expected medical graduate outcomes.  Substantive validity in students' and examiners' response processes is ensured through relevant methodological and cognitive processes. Multiple validity features are built-in to the design, planning and implementation process of the CAP.  There is evidence of high internal consistency reliability of CAP scores (Cronbach's alpha > 0.8) and inter-examiner consistency reliability (intra-class correlation>0.7). Aggregation of CAP scores is psychometrically sound, with high internal consistency indicating one common underlying construct.  Significant but moderate correlations between CAP scores and scores from other assessment modalities indicate validity of extrapolation and alignment between the CAP and the overall target outcomes of medical graduates.  Standard setting, score equating and fair decision rules justify consequential validity of CAP scores interpretation and use. This study provides evidence demonstrating that the CAP is a meaningful and valid component in the assessment program. This systematic framework of validation can be adopted for all levels of assessment in medical education, from individual assessment modality, to the validation of an assessment program as a whole.

Using meta-analytic path analysis to test theoretical predictions in health behavior: An illustration based on meta-analyses of the theory of planned behavior.

PubMed

Hagger, Martin S; Chan, Derwin K C; Protogerou, Cleo; Chatzisarantis, Nikos L D

2016-08-01

Synthesizing research on social cognitive theories applied to health behavior is an important step in the development of an evidence base of psychological factors as targets for effective behavioral interventions. However, few meta-analyses of research on social cognitive theories in health contexts have conducted simultaneous tests of theoretically-stipulated pattern effects using path analysis. We argue that conducting path analyses of meta-analytic effects among constructs from social cognitive theories is important to test nomological validity, account for mediation effects, and evaluate unique effects of theory constructs independent of past behavior. We illustrate our points by conducting new analyses of two meta-analyses of a popular theory applied to health behaviors, the theory of planned behavior. We conducted meta-analytic path analyses of the theory in two behavioral contexts (alcohol and dietary behaviors) using data from the primary studies included in the original meta-analyses augmented to include intercorrelations among constructs and relations with past behavior missing from the original analysis. Findings supported the nomological validity of the theory and its hypotheses for both behaviors, confirmed important model processes through mediation analysis, demonstrated the attenuating effect of past behavior on theory relations, and provided estimates of the unique effects of theory constructs independent of past behavior. Our analysis illustrates the importance of conducting a simultaneous test of theory-stipulated effects in meta-analyses of social cognitive theories applied to health behavior. We recommend researchers adopt this analytic procedure when synthesizing evidence across primary tests of social cognitive theories in health. Copyright © 2016 Elsevier Inc. All rights reserved.

The Development and Validation of the Client Expectations of Massage Scale

PubMed Central

Boulanger, Karen T.; Campo, Shelly; Glanville, Jennifer L.; Lowe, John B; Yang, Jingzhen

2012-01-01

Background: Although there is evidence that client expectations influence client outcomes, a valid and reliable scale for measuring the range of client expectations for both massage therapy and the behaviors of their massage therapists does not exist. Understanding how client expectations influence client outcomes would provide insight into how massage achieves its reported effects. Purpose: To develop and validate the Client Expectations of Massage Scale (CEMS), a measure of clients’ clinical, educational, interpersonal, and outcome expectations. Setting: Offices of licensed massage therapists in Iowa. Research Design: A practice-based research methodology was used to collect data from two samples of massage therapy clients. For Sample 1, 21 volunteer massage therapists collected data from their clients before the massage. Factor analysis was conducted to test construct validity and coefficient alpha was used to assess reliability. Correlational analyses with the CEMS, previous measures of client expectations, and the Life Orientation Test–Revised were examined to test the convergent and discriminant validity of the CEMS. For Sample 2, 24 massage therapists distributed study materials for clients to complete before and after a massage therapy session. Structural equation modeling was used to assess the construct, discriminant, and predictive validity of the CEMS. Participants: Sample 1 involved 320 and Sample 2 involved 321 adult massage clients. Intervention: Standard care provided by licensed massage therapists. Main Outcomes: Numeric Rating Scale for pain and Positive and Negative Affect Schedule–Revised (including the Serenity subscale). Results: The CEMS demonstrated good construct, convergent, discriminant and predictive validity, and adequate reliability. Client expectations were generally positive toward massage and their massage therapists. Positive outcome expectations had a positive effect on clients’ changes in pain and serenity. High interpersonal expectations had a negative effect on clients’ changes in serenity. Conclusions: Client expectations contribute to the nonspecific effects of massage therapy. PMID:23087774

Validity of semi-quantitative scale for brain MRI in unilateral cerebral palsy due to periventricular white matter lesions: Relationship with hand sensorimotor function and structural connectivity.

PubMed

Fiori, Simona; Guzzetta, Andrea; Pannek, Kerstin; Ware, Robert S; Rossi, Giuseppe; Klingels, Katrijn; Feys, Hilde; Coulthard, Alan; Cioni, Giovanni; Rose, Stephen; Boyd, Roslyn N

2015-01-01

To provide first evidence of construct validity of a semi-quantitative scale for brain structural MRI (sqMRI scale) in children with unilateral cerebral palsy (UCP) secondary to periventricular white matter (PWM) lesions, by examining the relationship with hand sensorimotor function and whole brain structural connectivity. Cross-sectional study of 50 children with UCP due to PWM lesions using 3 T (MRI), diffusion MRI and assessment of hand sensorimotor function. We explored the relationship of lobar, hemispheric and global scores on the sqMRI scale, with fractional anisotropy (FA), as a measure of brain white matter microstructure, and with hand sensorimotor measures (Assisting Hand Assessment, AHA; Jebsen-Taylor Test for Hand Function, JTTHF; Melbourne Assessment of Unilateral Upper Limb Function, MUUL; stereognosis; 2-point discrimination). Lobar and hemispheric scores on the sqMRI scale contralateral to the clinical side of hemiplegia correlated with sensorimotor paretic hand function measures and FA of a number of brain structural connections, including connections of brain areas involved in motor control (postcentral, precentral and paracentral gyri in the parietal lobe). More severe lesions correlated with lower sensorimotor performance, with the posterior limb of internal capsule score being the strongest contributor to impaired hand function. The sqMRI scale demonstrates first evidence of construct validity against impaired motor and sensory function measures and brain structural connectivity in a cohort of children with UCP due to PWM lesions. More severe lesions correlated with poorer paretic hand sensorimotor function and impaired structural connectivity in the hemisphere contralateral to the clinical side of hemiplegia. The quantitative structural MRI scoring may be a useful clinical tool for studying brain structure-function relationships but requires further validation in other populations of CP.

Defining mindfulness by how poorly I think I pay attention during everyday awareness and other intractable problems for psychology's (re)invention of mindfulness: comment on Brown et al. (2011).

PubMed

Grossman, Paul

2011-12-01

The Buddhist construct of mindfulness is a central element of mindfulness-based interventions and derives from an age-old systematic phenomenological program to investigate subjective experience. Recent enthusiasm for "mindfulness" in psychology has resulted in proliferation of self-report inventories that purport to measure mindful awareness as a trait. This paper addresses a number of intractable issues regarding these scales, in general, and also specifically highlights vulnerabilities of the adult and adolescent forms of the Mindfulness Attention Awareness Scale. These problems include (a) lack of available external referents for determining the construct validity of these inventories, (b) inadequacy of content validity of measures, (c) lack of evidence that self-reports of mindfulness competencies correspond to actual behavior and evidence that they do not, (d) lack of convergent validity among different mindfulness scales, (e) inequivalence of semantic item interpretation among different groups, (f) response biases related to degree of experience with mindfulness practice, (g) conflation of perceived mindfulness competencies with valuations of importance or meaningfulness, and (h) inappropriateness of samples employed to validate questionnaires. Current self-report attempts to measure mindfulness may serve to denature, distort, and banalize the meaning of mindful awareness in psychological research and may adversely affect further development of mindfulness-based interventions. Opportunities to enrich positivist Western psychological paradigms with a detailed and complex Buddhist phenomenology of the mind are likely to require a depth of understanding of mindfulness that, in turn, depends upon direct and long-term experience with mindfulness practice. Psychologists should consider pursuing this avenue before attempting to characterize and quantify mindfulness.

Development and Validation of Measures of Secondary Science Teachers' PCK for Teaching Photosynthesis

NASA Astrophysics Data System (ADS)

Park, Soonhye; Suh, Jeekyung; Seo, Kyungwoon

2018-06-01

This paper describes procedures by which two types of measures of Pedagogical Content Knowledge (PCK) were developed and validated: (a) PCK Survey and (b) PCK Rubric. Given the topic-specificity of PCK, the measures centered on photosynthesis as taught in high school classrooms. The measures were conceptually grounded in the pentagon model of PCK and designed to measure indispensable PCK that can be applied to any teacher, in any teaching context, for the given topic. Because of the exploratory nature of the study, the measures focus on two key components of PCK: (a) knowledge of students' understanding in science and (b) knowledge of instructional strategies and representations. Both measures have established acceptable levels of reliability as determined by internal consistency and inter-rater agreement. Evidence related to content validity was gathered through expert consultations, while evidence related to construct validity was collected through analysis of think-aloud interviews and factor analyses. Issues and challenges emerging from the course of the measure development, administration, and validation are discussed with strategies for confronting them. Directions for future research are proposed in three areas: (a) relationships between PCK and teaching experiences, (b) differences in PCK between science teachers and scientists, and (c) relationships between PCK and student learning.

Development and Validation of Measures of Secondary Science Teachers' PCK for Teaching Photosynthesis

NASA Astrophysics Data System (ADS)

Park, Soonhye; Suh, Jeekyung; Seo, Kyungwoon

2017-04-01

This paper describes procedures by which two types of measures of Pedagogical Content Knowledge (PCK) were developed and validated: (a) PCK Survey and (b) PCK Rubric. Given the topic-specificity of PCK, the measures centered on photosynthesis as taught in high school classrooms. The measures were conceptually grounded in the pentagon model of PCK and designed to measure indispensable PCK that can be applied to any teacher, in any teaching context, for the given topic. Because of the exploratory nature of the study, the measures focus on two key components of PCK: (a) knowledge of students' understanding in science and (b) knowledge of instructional strategies and representations. Both measures have established acceptable levels of reliability as determined by internal consistency and inter-rater agreement. Evidence related to content validity was gathered through expert consultations, while evidence related to construct validity was collected through analysis of think-aloud interviews and factor analyses. Issues and challenges emerging from the course of the measure development, administration, and validation are discussed with strategies for confronting them. Directions for future research are proposed in three areas: (a) relationships between PCK and teaching experiences, (b) differences in PCK between science teachers and scientists, and (c) relationships between PCK and student learning.

Prolonged grief: where to after Diagnostic and Statistical Manual of Mental Disorders, 5th Edition?

PubMed

Bryant, Richard A

2014-01-01

Although there is much evidence for the construct of prolonged grief, there was much controversy over the proposal to introduce a prolonged grief diagnosis into Diagnostic and Statistical Manual of Mental Disorders, 5th Edition (DSM-5), and it was finally rejected as a diagnosis in DSM-5. This review outlines the evidence for and against the diagnosis, and highlights the implications of the DSM-5 decision. Convergent evidence indicates that prolonged grief characterized by persistently severe yearning for the deceased is a distinct construct from bereavement-related depression and anxiety, is associated with marked functional impairment, is responsive to targeted treatments for prolonged grief, and has been validated across different cultures, age groups, and types of bereavement. Although DSM-5 has rejected the construct as a formal diagnosis, evidence continues to emerge on related mechanisms, including maladaptive appraisals, memory and attentional processes, immunological and arousal responses, and neural circuitry. It is most likely that the International Classification of Diseases (ICD-11) will introduce a diagnosis to recognize prolonged grief, even though DSM-5 has decided against this option. It is probable that the DSM-5 decision may result in more prolonged grief patients being incorrectly diagnosed with depression after bereavement and possibly incorrectly treated. The DSM-5 decision is unlikely to impact on future research agendas.

Systematic review on the effectiveness of augmented reality applications in medical training.

PubMed

Barsom, E Z; Graafland, M; Schijven, M P

2016-10-01

Computer-based applications are increasingly used to support the training of medical professionals. Augmented reality applications (ARAs) render an interactive virtual layer on top of reality. The use of ARAs is of real interest to medical education because they blend digital elements with the physical learning environment. This will result in new educational opportunities. The aim of this systematic review is to investigate to which extent augmented reality applications are currently used to validly support medical professionals training. PubMed, Embase, INSPEC and PsychInfo were searched using predefined inclusion criteria for relevant articles up to August 2015. All study types were considered eligible. Articles concerning AR applications used to train or educate medical professionals were evaluated. Twenty-seven studies were found relevant, describing a total of seven augmented reality applications. Applications were assigned to three different categories. The first category is directed toward laparoscopic surgical training, the second category toward mixed reality training of neurosurgical procedures and the third category toward training echocardiography. Statistical pooling of data could not be performed due to heterogeneity of study designs. Face-, construct- and concurrent validity was proven for two applications directed at laparoscopic training, face- and construct validity for neurosurgical procedures and face-, content- and construct validity in echocardiography training. In the literature, none of the ARAs completed a full validation process for the purpose of use. Augmented reality applications that support blended learning in medical training have gained public and scientific interest. In order to be of value, applications must be able to transfer information to the user. Although promising, the literature to date is lacking to support such evidence.

The Scientific Status of Projective Techniques.

PubMed

Lilienfeld, S O; Wood, J M; Garb, H N

2000-11-01

Although projective techniques continue to be widely used in clinical and forensic settings, their scientific status remains highly controversial. In this monograph, we review the current state of the literature concerning the psychometric properties (norms, reliability, validity, incremental validity, treatment utility) of three major projective instruments: Rorschach Inkblot Test, Thematic Apperception Test (TAT), and human figure drawings. We conclude that there is empirical support for the validity of a small number of indexes derived from the Rorschach and TAT. However, the substantial majority of Rorschach and TAT indexes are not empirically supported. The validity evidence for human figure drawings is even more limited. With a few exceptions, projective indexes have not consistently demonstrated incremental validity above and beyond other psychometric data. In addition, we summarize the results of a new meta-analysis intended to examine the capacity of these three instruments to detect child sexual abuse. Although some projective instruments were better than chance at detecting child sexual abuse, there were virtually no replicated findings across independent investigative teams. This meta-analysis also provides the first clear evidence of substantial file drawer effects in the projectives literature, as the effect sizes from published studies markedly exceeded those from unpublished studies. We conclude with recommendations regarding the (a) construction of projective techniques with adequate validity, (b) forensic and clinical use of projective techniques, and (c) education and training of future psychologists regarding projective techniques. © 2000 Association for Psychological Science.

Cross-cultural evaluation of the French version of the LEIPAD, a health-related quality of life instrument for use in the elderly living at home.

PubMed

Jalenques, I; Auclair, C; Roblin, J; Morand, D; Tourtauchaux, R; May, R; Vaille-Perret, E; Watts, J; Gerbaud, L; De Leo, D

2013-04-01

To cross-culturally adapt a French version of the LEIPAD, a self-administered questionnaire assessing the health-related quality of life (HRQoL) in adults aged 65 years and over living at home, and to evaluate its psychometric properties. After having translated LEIPAD in accordance with guidelines, we studied psychometric properties: reliability and construct validity-factor analysis, relationships between items and scales, internal consistency, concurrent validity with the Medical Outcome Study Short-Form 36 and known-groups validity. The results obtained in a sample of 195 elderly from the general population showed very good acceptability, with response rates superior to 93 %. Exploratory factor analysis extracted eight factors providing a multidimensionality structure with five misclassifications of items in the seven theoretical scales. Good internal consistency (Cronbach's alpha ranging from 0.73 and 0.86) and strong test-retest reliability (ICCs higher than 0.80 for six scales and 0.70 for one) were demonstrated. Concurrent validity with the SF-36 showed small to strong expected correlations. This first evaluation of the French version of LEIPAD's psychometric properties provides evidence in construct validity and reliability. It would allow HRQoL assessment in clinical and common practice, and investigators would be able to take part in national and international research projects.

Reviewing the psychometric properties of contemporary circadian typology measures.

PubMed

Di Milia, Lee; Adan, Ana; Natale, Vincenzo; Randler, Christoph

2013-12-01

The accurate measurement of circadian typology (CT) is critical because the construct has implications for a number of health disorders. In this review, we focus on the evidence to support the reliability and validity of the more commonly used CT scales: the Morningness-Eveningness Questionnaire (MEQ), reduced Morningness-Eveningness Questionnaire (rMEQ), the Composite Scale of Morningness (CSM), and the Preferences Scale (PS). In addition, we also consider the Munich ChronoType Questionnaire (MCTQ). In terms of reliability, the MEQ, CSM, and PS consistently report high levels of reliability (>0.80), whereas the reliability of the rMEQ is satisfactory. The stability of these scales is sound at follow-up periods up to 13 mos. The MCTQ is not a scale; therefore, its reliability cannot be assessed. Although it is possible to determine the stability of the MCTQ, these data are yet to be reported. Validity must be given equal weight in assessing the measurement properties of CT instruments. Most commonly reported is convergent and construct validity. The MEQ, rMEQ, and CSM are highly correlated and this is to be expected, given that these scales share common items. The level of agreement between the MCTQ and the MEQ is satisfactory, but the correlation between these two constructs decreases in line with the number of "corrections" applied to the MCTQ. The interesting question is whether CT is best represented by a psychological preference for behavior or by using a biomarker such as sleep midpoint. Good-quality subjective and objective data suggest adequate construct validity for each of the CT instruments, but a major limitation of this literature is studies that assess the predictive validity of these instruments. We make a number of recommendations with the aim of advancing science. Future studies need to (1) focus on collecting data from representative samples that consider a number of environmental factors; (2) employ longitudinal designs to allow the predictive validity of CT measures to be assessed and preferably make use of objective data; (3) employ contemporary statistical approaches, including structural equation modeling and item-response models; and (4) provide better information concerning sample selection and a rationale for choosing cutoff points.

Reliability and Validity of Assessing User Satisfaction With Web-Based Health Interventions

PubMed Central

Lehr, Dirk; Reis, Dorota; Vis, Christiaan; Riper, Heleen; Berking, Matthias; Ebert, David Daniel

2016-01-01

Background The perspective of users should be taken into account in the evaluation of Web-based health interventions. Assessing the users’ satisfaction with the intervention they receive could enhance the evidence for the intervention effects. Thus, there is a need for valid and reliable measures to assess satisfaction with Web-based health interventions. Objective The objective of this study was to analyze the reliability, factorial structure, and construct validity of the Client Satisfaction Questionnaire adapted to Internet-based interventions (CSQ-I). Methods The psychometric quality of the CSQ-I was analyzed in user samples from 2 separate randomized controlled trials evaluating Web-based health interventions, one from a depression prevention intervention (sample 1, N=174) and the other from a stress management intervention (sample 2, N=111). At first, the underlying measurement model of the CSQ-I was analyzed to determine the internal consistency. The factorial structure of the scale and the measurement invariance across groups were tested by multigroup confirmatory factor analyses. Additionally, the construct validity of the scale was examined by comparing satisfaction scores with the primary clinical outcome. Results Multigroup confirmatory analyses on the scale yielded a one-factorial structure with a good fit (root-mean-square error of approximation =.09, comparative fit index =.96, standardized root-mean-square residual =.05) that showed partial strong invariance across the 2 samples. The scale showed very good reliability, indicated by McDonald omegas of .95 in sample 1 and .93 in sample 2. Significant correlations with change in depressive symptoms (r=−.35, P<.001) and perceived stress (r=−.48, P<.001) demonstrated the construct validity of the scale. Conclusions The proven internal consistency, factorial structure, and construct validity of the CSQ-I indicate a good overall psychometric quality of the measure to assess the user’s general satisfaction with Web-based interventions for depression and stress management. Multigroup analyses indicate its robustness across different samples. Thus, the CSQ-I seems to be a suitable measure to consider the user’s perspective in the overall evaluation of Web-based health interventions. PMID:27582341

Reliability and Validity of Assessing User Satisfaction With Web-Based Health Interventions.

PubMed

Boß, Leif; Lehr, Dirk; Reis, Dorota; Vis, Christiaan; Riper, Heleen; Berking, Matthias; Ebert, David Daniel

2016-08-31

The perspective of users should be taken into account in the evaluation of Web-based health interventions. Assessing the users' satisfaction with the intervention they receive could enhance the evidence for the intervention effects. Thus, there is a need for valid and reliable measures to assess satisfaction with Web-based health interventions. The objective of this study was to analyze the reliability, factorial structure, and construct validity of the Client Satisfaction Questionnaire adapted to Internet-based interventions (CSQ-I). The psychometric quality of the CSQ-I was analyzed in user samples from 2 separate randomized controlled trials evaluating Web-based health interventions, one from a depression prevention intervention (sample 1, N=174) and the other from a stress management intervention (sample 2, N=111). At first, the underlying measurement model of the CSQ-I was analyzed to determine the internal consistency. The factorial structure of the scale and the measurement invariance across groups were tested by multigroup confirmatory factor analyses. Additionally, the construct validity of the scale was examined by comparing satisfaction scores with the primary clinical outcome. Multigroup confirmatory analyses on the scale yielded a one-factorial structure with a good fit (root-mean-square error of approximation =.09, comparative fit index =.96, standardized root-mean-square residual =.05) that showed partial strong invariance across the 2 samples. The scale showed very good reliability, indicated by McDonald omegas of .95 in sample 1 and .93 in sample 2. Significant correlations with change in depressive symptoms (r=-.35, P<.001) and perceived stress (r=-.48, P<.001) demonstrated the construct validity of the scale. The proven internal consistency, factorial structure, and construct validity of the CSQ-I indicate a good overall psychometric quality of the measure to assess the user's general satisfaction with Web-based interventions for depression and stress management. Multigroup analyses indicate its robustness across different samples. Thus, the CSQ-I seems to be a suitable measure to consider the user's perspective in the overall evaluation of Web-based health interventions.

Modeling complex treatment strategies: construction and validation of a discrete event simulation model for glaucoma.

PubMed

van Gestel, Aukje; Severens, Johan L; Webers, Carroll A B; Beckers, Henny J M; Jansonius, Nomdo M; Schouten, Jan S A G

2010-01-01

Discrete event simulation (DES) modeling has several advantages over simpler modeling techniques in health economics, such as increased flexibility and the ability to model complex systems. Nevertheless, these benefits may come at the cost of reduced transparency, which may compromise the model's face validity and credibility. We aimed to produce a transparent report on the construction and validation of a DES model using a recently developed model of ocular hypertension and glaucoma. Current evidence of associations between prognostic factors and disease progression in ocular hypertension and glaucoma was translated into DES model elements. The model was extended to simulate treatment decisions and effects. Utility and costs were linked to disease status and treatment, and clinical and health economic outcomes were defined. The model was validated at several levels. The soundness of design and the plausibility of input estimates were evaluated in interdisciplinary meetings (face validity). Individual patients were traced throughout the simulation under a multitude of model settings to debug the model, and the model was run with a variety of extreme scenarios to compare the outcomes with prior expectations (internal validity). Finally, several intermediate (clinical) outcomes of the model were compared with those observed in experimental or observational studies (external validity) and the feasibility of evaluating hypothetical treatment strategies was tested. The model performed well in all validity tests. Analyses of hypothetical treatment strategies took about 30 minutes per cohort and lead to plausible health-economic outcomes. There is added value of DES models in complex treatment strategies such as glaucoma. Achieving transparency in model structure and outcomes may require some effort in reporting and validating the model, but it is feasible.

Psychometric and cognitive validation of a social capital measurement tool in Peru and Vietnam.

PubMed

De Silva, Mary J; Harpham, Trudy; Tuan, Tran; Bartolini, Rosario; Penny, Mary E; Huttly, Sharon R

2006-02-01

Social capital is a relatively new concept which has attracted significant attention in recent years. No consensus has yet been reached on how to measure social capital, resulting in a large number of different tools available. While psychometric validation methods such as factor analysis have been used by a few studies to assess the internal validity of some tools, these techniques rely on data already collected by the tool and are therefore not capable of eliciting what the questions are actually measuring. The Young Lives (YL) study includes quantitative measures of caregiver's social capital in four countries (Vietnam, Peru, Ethiopia, and India) using a short version of the Adapted Social Capital Assessment Tool (SASCAT). A range of different psychometric methods including factor analysis were used to evaluate the construct validity of SASCAT in Peru and Vietnam. In addition, qualitative cognitive interviews with 20 respondents from Peru and 24 respondents from Vietnam were conducted to explore what each question is actually measuring. We argue that psychometric validation techniques alone are not sufficient to adequately validate multi-faceted social capital tools for use in different cultural settings. Psychometric techniques show SASCAT to be a valid tool reflecting known constructs and displaying postulated links with other variables. However, results from the cognitive interviews present a more mixed picture with some questions being appropriately interpreted by respondents, and others displaying significant differences between what the researchers intended them to measure and what they actually do. Using evidence from a range of methods of assessing validity has enabled the modification of an existing instrument into a valid and low cost tool designed to measure social capital within larger surveys in Peru and Vietnam, with the potential for use in other developing countries following local piloting and cultural adaptation of the tool.

Validity and Reliability of the Clinical Competency Evaluation Instrument for Use among Physiotherapy Students: Pilot study.

PubMed

Muhamad, Zailani; Ramli, Ayiesah; Amat, Salleh

2015-05-01

The aim of this study was to determine the content validity, internal consistency, test-retest reliability and inter-rater reliability of the Clinical Competency Evaluation Instrument (CCEVI) in assessing the clinical performance of physiotherapy students. This study was carried out between June and September 2013 at University Kebangsaan Malaysia (UKM), Kuala Lumpur, Malaysia. A panel of 10 experts were identified to establish content validity by evaluating and rating each of the items used in the CCEVI with regards to their relevance in measuring students' clinical competency. A total of 50 UKM undergraduate physiotherapy students were assessed throughout their clinical placement to determine the construct validity of these items. The instrument's reliability was determined through a cross-sectional study involving a clinical performance assessment of 14 final-year undergraduate physiotherapy students. The content validity index of the entire CCEVI was 0.91, while the proportion of agreement on the content validity indices ranged from 0.83-1.00. The CCEVI construct validity was established with factor loading of ≥0.6, while internal consistency (Cronbach's alpha) overall was 0.97. Test-retest reliability of the CCEVI was confirmed with a Pearson's correlation range of 0.91-0.97 and an intraclass coefficient correlation range of 0.95-0.98. Inter-rater reliability of the CCEVI domains ranged from 0.59 to 0.97 on initial and subsequent assessments. This pilot study confirmed the content validity of the CCEVI. It showed high internal consistency, thereby providing evidence that the CCEVI has moderate to excellent inter-rater reliability. However, additional refinement in the wording of the CCEVI items, particularly in the domains of safety and documentation, is recommended to further improve the validity and reliability of the instrument.

Validation of the Internet Gaming Disorder Scale - Short-Form (IGDS9-SF) in an Italian-speaking sample.

PubMed

Monacis, Lucia; Palo, Valeria de; Griffiths, Mark D; Sinatra, Maria

2016-12-01

Background and aims The inclusion of Internet Gaming Disorder (IGD) in Section III of the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders has increased the interest of researchers in the development of new standardized psychometric tools for the assessment of such a disorder. To date, the nine-item Internet Gaming Disorder Scale - Short-Form (IGDS9-SF) has only been validated in English, Portuguese, and Slovenian languages. Therefore, the aim of this investigation was to examine the psychometric properties of the IGDS9-SF in an Italian-speaking sample. Methods A total of 757 participants were recruited to the present study. Confirmatory factor analysis and multi-group analyses were applied to assess the construct validity. Reliability analyses comprised the average variance extracted, the standard error of measurement, and the factor determinacy coefficient. Convergent and criterion validities were established through the associations with other related constructs. The receiver operating characteristic curve analysis was used to determine an empirical cut-off point. Results Findings confirmed the single-factor structure of the instrument, its measurement invariance at the configural level, and the convergent and criterion validities. Satisfactory levels of reliability and a cut-off point of 21 were obtained. Discussion and conclusions The present study provides validity evidence for the use of the Italian version of the IGDS9-SF and may foster research into gaming addiction in the Italian context.

Validation of a French-Canadian adaptation of the Intuitive Eating Scale-2 for the adult population.

PubMed

Carbonneau, Elise; Carbonneau, Noémie; Lamarche, Benoît; Provencher, Véronique; Bégin, Catherine; Bradette-Laplante, Maude; Laramée, Catherine; Lemieux, Simone

2016-10-01

Intuitive eating is an adaptive eating style based on the reliance on physiological cues to determine when, what, and how much to eat. The Intuitive Eating Scale-2 (IES-2) is a validated four-subscale tool measuring the degree of adherence to intuitive eating principles. The present series of studies aimed at evaluating the psychometric properties of a French-Canadian adaptation of the IES-2 for the adult population. The factor structure, the reliability (internal consistency and test-retest), the construct validity, and the discriminant validity were evaluated in 334 women and 75 men from the Province of Québec, Canada, across two studies. A confirmatory factor analysis upheld that the four-factor structure of the original IES-2 was adequate for the present sample of French-Canadians. The scale demonstrated adequate internal consistency and test-retest reliability. Construct validity evidence was obtained with the significant associations between intuitive eating and psychological and eating-related variables. Intuitive eating was negatively associated with eating disorder symptomatology and with food- and weight-preoccupation, and positively associated with body-esteem and well-being. The French-Canadian IES-2 was also able to discriminate between genders and body mass index categories. The properties of this new version of the IES-2 are demonstrative of a reliable and valid tool to assess intuitive eating in the French-Canadian adult population of the Province of Québec. Copyright © 2016 Elsevier Ltd. All rights reserved.

Preliminary validation and reliability of the Short Form Chronic Respiratory Disease Questionnaire in a lung cancer population.

PubMed

Charalambous, A; Molassiotis, A

2017-01-01

The Short Form Chronic Respiratory Questionnaire (SF-CRQ) is frequently used in patients with obstructive pulmonary disease and it has demonstrated excellent psychometric properties. Since there is no psychometric information for its use with lung cancer patients, this study explored its validity and reliability in this population. Forty-six patients were assessed at two time points (with a 4-week interval) using the SF-CRQ, the modified Borg Scale, five numerical rating scales related to Perceived Severity of Breathlessness, and the Hospital Anxiety and Depression Scale. Internal consistency reliability was investigated by Cronbach's alpha reliability coefficient, test-retest reliability by Spearman-Brown reliability coefficient (P), content validity as well as convergent validity by Pearson's correlation coefficient between the SF-CRQ, and the conceptual similar scales mentioned above were explored. A principal component factor analysis was performed. The internal consistency was high [α = 0.88 (baseline) and 0.91 (after 1 month)]. The SF-CRQ had good stability with test-retest reliability ranging from r = 0.64 to 0.78, P < 0.001. Factor analysis suggests a single construct in this population. The preliminary data analyses supported the convergent, content, and construct validity of the SF-CRQ providing promising evidence that this can be a valid and reliable instrument for the assessment of quality of life related to breathlessness in lung cancer patients. © 2015 John Wiley & Sons Ltd.

Validation of the Internet Gaming Disorder Scale – Short-Form (IGDS9-SF) in an Italian-speaking sample

PubMed Central

Monacis, Lucia; de Palo, Valeria; Griffiths, Mark D.; Sinatra, Maria

2016-01-01

Background and aims The inclusion of Internet Gaming Disorder (IGD) in Section III of the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders has increased the interest of researchers in the development of new standardized psychometric tools for the assessment of such a disorder. To date, the nine-item Internet Gaming Disorder Scale – Short-Form (IGDS9-SF) has only been validated in English, Portuguese, and Slovenian languages. Therefore, the aim of this investigation was to examine the psychometric properties of the IGDS9-SF in an Italian-speaking sample. Methods A total of 757 participants were recruited to the present study. Confirmatory factor analysis and multi-group analyses were applied to assess the construct validity. Reliability analyses comprised the average variance extracted, the standard error of measurement, and the factor determinacy coefficient. Convergent and criterion validities were established through the associations with other related constructs. The receiver operating characteristic curve analysis was used to determine an empirical cut-off point. Results Findings confirmed the single-factor structure of the instrument, its measurement invariance at the configural level, and the convergent and criterion validities. Satisfactory levels of reliability and a cut-off point of 21 were obtained. Discussion and conclusions The present study provides validity evidence for the use of the Italian version of the IGDS9-SF and may foster research into gaming addiction in the Italian context. PMID:27876422

Construct Validity and Scoring Methods of the World Health Organization: Health and Work Performance Questionnaire Among Workers With Arthritis and Rheumatological Conditions.

PubMed

AlHeresh, Rawan; LaValley, Michael P; Coster, Wendy; Keysor, Julie J

2017-06-01

To evaluate construct validity and scoring methods of the world health organization-health and work performance questionnaire (HPQ) for people with arthritis. Construct validity was examined through hypothesis testing using the recommended guidelines of the consensus-based standards for the selection of health measurement instruments (COSMIN). The HPQ using the absolute scoring method showed moderate construct validity as four of the seven hypotheses were met. The HPQ using the relative scoring method had weak construct validity as only one of the seven hypotheses were met. The absolute scoring method for the HPQ is superior in construct validity to the relative scoring method in assessing work performance among people with arthritis and related rheumatic conditions; however, more research is needed to further explore other psychometric properties of the HPQ.

Quantification of construction waste prevented by BIM-based design validation: Case studies in South Korea.

PubMed

Won, Jongsung; Cheng, Jack C P; Lee, Ghang

2016-03-01

Waste generated in construction and demolition processes comprised around 50% of the solid waste in South Korea in 2013. Many cases show that design validation based on building information modeling (BIM) is an effective means to reduce the amount of construction waste since construction waste is mainly generated due to improper design and unexpected changes in the design and construction phases. However, the amount of construction waste that could be avoided by adopting BIM-based design validation has been unknown. This paper aims to estimate the amount of construction waste prevented by a BIM-based design validation process based on the amount of construction waste that might be generated due to design errors. Two project cases in South Korea were studied in this paper, with 381 and 136 design errors detected, respectively during the BIM-based design validation. Each design error was categorized according to its cause and the likelihood of detection before construction. The case studies show that BIM-based design validation could prevent 4.3-15.2% of construction waste that might have been generated without using BIM. Copyright © 2015 Elsevier Ltd. All rights reserved.

The PROactive instruments to measure physical activity in patients with chronic obstructive pulmonary disease

PubMed Central

Gimeno-Santos, Elena; Raste, Yogini; Demeyer, Heleen; Louvaris, Zafeiris; de Jong, Corina; Rabinovich, Roberto A.; Hopkinson, Nicholas S.; Polkey, Michael I.; Vogiatzis, Ioannis; Tabberer, Maggie; Dobbels, Fabienne; Ivanoff, Nathalie; de Boer, Willem I.; van der Molen, Thys; Kulich, Karoly; Serra, Ignasi; Basagaña, Xavier; Troosters, Thierry; Puhan, Milo A.; Karlsson, Niklas

2015-01-01

No current patient-centred instrument captures all dimensions of physical activity in chronic obstructive pulmonary disease (COPD). Our objective was item reduction and initial validation of two instruments to measure physical activity in COPD. Physical activity was assessed in a 6-week, randomised, two-way cross-over, multicentre study using PROactive draft questionnaires (daily and clinical visit versions) and two activity monitors. Item reduction followed an iterative process including classical and Rasch model analyses, and input from patients and clinical experts. 236 COPD patients from five European centres were included. Results indicated the concept of physical activity in COPD had two domains, labelled “amount” and “difficulty”. After item reduction, the daily PROactive instrument comprised nine items and the clinical visit contained 14. Both demonstrated good model fit (person separation index >0.7). Confirmatory factor analysis supported the bidimensional structure. Both instruments had good internal consistency (Cronbach's α>0.8), test–retest reliability (intraclass correlation coefficient ≥0.9) and exhibited moderate-to-high correlations (r>0.6) with related constructs and very low correlations (r<0.3) with unrelated constructs, providing evidence for construct validity. Daily and clinical visit “PROactive physical activity in COPD” instruments are hybrid tools combining a short patient-reported outcome questionnaire and two activity monitor variables which provide simple, valid and reliable measures of physical activity in COPD patients. PMID:26022965

The Work Instability Scale for Rheumatoid Arthritis (RA-WIS): Does it work in osteoarthritis?

PubMed

Tang, Kenneth; Beaton, Dorcas E; Lacaille, Diane; Gignac, Monique A M; Zhang, Wei; Anis, Aslam H; Bombardier, Claire

2010-09-01

To validate the 23-item Work Instability Scale for Rheumatoid Arthritis (RA-WIS) for use in osteoarthritis (OA) using both classical test theory and item response theory approaches. Baseline and 12-month follow-up data were collected from workers with OA recruited from community and clinical settings (n = 130). Fit of RA-WIS data to the Rasch model was evaluated by item- and person-fit statistics (size of residual, chi-sq), assessments of differential item functioning, and tests of unidimensionality and local independence. Internal consistency was assessed by KR-20. Convergent construct validity (Spearman r, known-groups) was evaluated against theoretical constructs that assess impact of health on work. Responsiveness to global indicators of change was assessed by standardized response means (SRM) and area under the receiver operating characteristic curves. Data structure of the RA-WIS showed adequate fit to the Rasch model (chi-sq = 83.2, P = 0.03) after addressing local dependency in three item pairs by creating testlets. High internal consistency (KR-20 = 0.93) and convergent validity with work-oriented constructs (|r| = 0.55-0.77) were evident. The RA-WIS correlated most strongly with the concept of illness intrusiveness (r = 0.77) and was highly responsive to changes (SRM = 1.05 [deterioration]; -0.78 [improvement]). Although developed for RA, the RA-WIS is psychometrically sound for OA and demonstrates interval-level property.

Construct validity for eye-hand coordination skill on a virtual reality laparoscopic surgical simulator.

PubMed

Yamaguchi, Shohei; Konishi, Kozo; Yasunaga, Takefumi; Yoshida, Daisuke; Kinjo, Nao; Kobayashi, Kiichiro; Ieiri, Satoshi; Okazaki, Ken; Nakashima, Hideaki; Tanoue, Kazuo; Maehara, Yoshihiko; Hashizume, Makoto

2007-12-01

This study was carried out to investigate whether eye-hand coordination skill on a virtual reality laparoscopic surgical simulator (the LAP Mentor) was able to differentiate among subjects with different laparoscopic experience and thus confirm its construct validity. A total of 31 surgeons, who were all right-handed, were divided into the following two groups according to their experience as an operator in laparoscopic surgery: experienced surgeons (more than 50 laparoscopic procedures) and novice surgeons (fewer than 10 laparoscopic procedures). The subjects were tested using the eye-hand coordination task of the LAP Mentor, and performance was compared between the two groups. Assessment of the laparoscopic skills was based on parameters measured by the simulator. The experienced surgeons completed the task significantly faster than the novice surgeons. The experienced surgeons also achieved a lower number of movements (NOM), better economy of movement (EOM) and faster average speed of the left instrument than the novice surgeons, whereas there were no significant differences between the two groups for the NOM, EOM and average speed of the right instrument. Eye-hand coordination skill of the nondominant hand, but not the dominant hand, measured using the LAP Mentor was able to differentiate between subjects with different laparoscopic experience. This study also provides evidence of construct validity for eye-hand coordination skill on the LAP Mentor.

The PROactive instruments to measure physical activity in patients with chronic obstructive pulmonary disease.

PubMed

Gimeno-Santos, Elena; Raste, Yogini; Demeyer, Heleen; Louvaris, Zafeiris; de Jong, Corina; Rabinovich, Roberto A; Hopkinson, Nicholas S; Polkey, Michael I; Vogiatzis, Ioannis; Tabberer, Maggie; Dobbels, Fabienne; Ivanoff, Nathalie; de Boer, Willem I; van der Molen, Thys; Kulich, Karoly; Serra, Ignasi; Basagaña, Xavier; Troosters, Thierry; Puhan, Milo A; Karlsson, Niklas; Garcia-Aymerich, Judith

2015-10-01

No current patient-centred instrument captures all dimensions of physical activity in chronic obstructive pulmonary disease (COPD). Our objective was item reduction and initial validation of two instruments to measure physical activity in COPD.Physical activity was assessed in a 6-week, randomised, two-way cross-over, multicentre study using PROactive draft questionnaires (daily and clinical visit versions) and two activity monitors. Item reduction followed an iterative process including classical and Rasch model analyses, and input from patients and clinical experts.236 COPD patients from five European centres were included. Results indicated the concept of physical activity in COPD had two domains, labelled "amount" and "difficulty". After item reduction, the daily PROactive instrument comprised nine items and the clinical visit contained 14. Both demonstrated good model fit (person separation index >0.7). Confirmatory factor analysis supported the bidimensional structure. Both instruments had good internal consistency (Cronbach's α>0.8), test-retest reliability (intraclass correlation coefficient ≥0.9) and exhibited moderate-to-high correlations (r>0.6) with related constructs and very low correlations (r<0.3) with unrelated constructs, providing evidence for construct validity.Daily and clinical visit "PROactive physical activity in COPD" instruments are hybrid tools combining a short patient-reported outcome questionnaire and two activity monitor variables which provide simple, valid and reliable measures of physical activity in COPD patients. Copyright ©ERS 2015.

A French adaptation of the Overt Behaviour Scale (OBS) measuring challenging behaviours following acquired brain injury: The Échelle des comportements observables (ÉCO).

PubMed

Gagnon, Jean; Simpson, Grahame Kenneth; Kelly, Glenn; Godbout, Denis; Ouellette, Michel; Drolet, Jacques

2016-01-01

To develop a French version of the Overt Behaviour Scale (OBS) and examine some of its psychometric properties. The scale was adapted and validated according to standard guidelines for cross-cultural adaptation of questionnaires (Échelle des comportements observables; ÉCO). The reliability and construct validity of the ÉCO were studied among 29 inpatients and outpatients who sustained an acquired brain injury. The instruments were administered by 12 clinicians located at eight rehabilitation centres and the local brain injury association. The ÉCO provided behaviour profile descriptives much like the original scale. It showed excellent reliability and good convergent and divergent validity, as reflected by significant associations with other measures that contained similar behavioural items and by the absence of signification correlations with broader constructs such as physical and cognitive abilities. This study provides evidence that the ÉCO behaves much like the original OBS, has promising initial findings with respect to reliability and validity and is a valuable research and clinical instrument to assess the severity and typology of challenging behaviour after an acquired brain injury and to monitor the evolution of behaviours after intervention in French and bilingual communities.

The Pittsburgh sleep quality index as a screening tool for sleep dysfunction in clinical and non-clinical samples: A systematic review and meta-analysis.

PubMed

Mollayeva, Tatyana; Thurairajah, Pravheen; Burton, Kirsteen; Mollayeva, Shirin; Shapiro, Colin M; Colantonio, Angela

2016-02-01

This review appraises the process of development and the measurement properties of the Pittsburgh sleep quality index (PSQI), gauging its potential as a screening tool for sleep dysfunction in non-clinical and clinical samples; it also compares non-clinical and clinical populations in terms of PSQI scores. MEDLINE, Embase, PsycINFO, and HAPI databases were searched. Critical appraisal of studies of measurement properties was performed using COSMIN. Of 37 reviewed studies, 22 examined construct validity, 19 - known-group validity, 15 - internal consistency, and three - test-retest reliability. Study quality ranged from poor to excellent, with the majority designated fair. Internal consistency, based on Cronbach's alpha, was good. Discrepancies were observed in factor analytic studies. In non-clinical and clinical samples with known differences in sleep quality, the PSQI global scores and all subscale scores, with the exception of sleep disturbance, differed significantly. The best evidence synthesis for the PSQI showed strong reliability and validity, and moderate structural validity in a variety of samples, suggesting the tool fulfills its intended utility. A taxonometric analysis can contribute to better understanding of sleep dysfunction as either a dichotomous or continuous construct. Copyright © 2015 Elsevier Ltd. All rights reserved.

Back to the Consideration of Future Consequences Scale: time to reconsider?

PubMed

Rappange, David R; Brouwer, Werner B F; van Exel, N Job A

2009-10-01

The Consideration of Future Consequences (CFC) Scale is a measure of the extent to which individuals consider and are influenced by the distant outcomes of current behavior. In this study, the authors conducted factor analysis to investigate the factor structure of the 12-item CFC Scale. The authors found evidence for a multiple factor solution including one completely present-oriented factor consisting of all 7 present-oriented items, and one or two future-oriented factors consisting of the remaining future-oriented items. Further evidence indicated that the present-oriented factor and the 12-item CFC Scale perform similarly in terms of internal consistency and convergent validity. The structure and content of the future-oriented factor(s) is unclear. From the findings, the authors raise questions regarding the construct validity of the CFC Scale, the interpretation of its results, and the usefulness of the CFC scale in its current form in applied research.

Development of the Systems Thinking Scale for Adolescent Behavior Change.

PubMed

Moore, Shirley M; Komton, Vilailert; Adegbite-Adeniyi, Clara; Dolansky, Mary A; Hardin, Heather K; Borawski, Elaine A

2018-03-01

This report describes the development and psychometric testing of the Systems Thinking Scale for Adolescent Behavior Change (STS-AB). Following item development, initial assessments of understandability and stability of the STS-AB were conducted in a sample of nine adolescents enrolled in a weight management program. Exploratory factor analysis of the 16-item STS-AB and internal consistency assessments were then done with 359 adolescents enrolled in a weight management program. Test-retest reliability of the STS-AB was .71, p = .03; internal consistency reliability was .87. Factor analysis of the 16-item STS-AB indicated a one-factor solution with good factor loadings, ranging from .40 to .67. Evidence of construct validity was supported by significant correlations with established measures of variables associated with health behavior change. We provide beginning evidence of the reliability and validity of the STS-AB to measure systems thinking for health behavior change in young adolescents.

Development of the Systems Thinking Scale for Adolescent Behavior Change

PubMed Central

Moore, Shirley M.; Komton, Vilailert; Adegbite-Adeniyi, Clara; Dolansky, Mary A.; Hardin, Heather K.; Borawski, Elaine A.

2017-01-01

This report describes the development and psychometric testing of the Systems Thinking Scale for Adolescent Behavior Change (STS-AB). Following item development, initial assessments of understandability and stability of the STS-AB were conducted in a sample of nine adolescents enrolled in a weight management program. Exploratory factor analysis of the 16-item STS-AB and internal consistency assessments were then done with 359 adolescents enrolled in a weight management program. Test–retest reliability of the STS-AB was .71, p = .03; internal consistency reliability was .87. Factor analysis of the 16-item STS-AB indicated a one-factor solution with good factor loadings, ranging from .40 to .67. Evidence of construct validity was supported by significant correlations with established measures of variables associated with health behavior change. We provide beginning evidence of the reliability and validity of the STS-AB to measure systems thinking for health behavior change in young adolescents. PMID:28303755

Factors affecting unsafe behavior in construction projects: development and validation of a new questionnaire.

PubMed

Asilian-Mahabadi, Hassan; Khosravi, Yahya; Hassanzadeh-Rangi, Narmin; Hajizadeh, Ebrahim; Behzadan, Amir H

2018-02-05

Occupational safety in general, and construction safety in particular, is a complex phenomenon. This study was designed to develop a new valid measure to evaluate factors affecting unsafe behavior in the construction industry. A new questionnaire was generated from qualitative research according to the principles of grounded theory. Key measurement properties (face validity, content validity, construct validity, reliability and discriminative validity) were examined using qualitative and quantitative approaches. The receiver operating characteristic curve was used to estimate the discriminating power and the optimal cutoff score. Construct validity revealed an interpretable 12-factor structure which explained 61.87% of variance. Good internal consistency (Cronbach's α = 0.94) and stability (intra-class correlation coefficient = 0.93) were found for the new instrument. The area under the curve, sensitivity and specificity were 0.80, 0.80 and 0.75, respectively. The new instrument also discriminated safety performance among the construction sites with different workers' accident histories (F = 6.40, p < 0.05). The new instrument appears to be a valid, reliable and sensitive instrument that will contribute to investigating the root causes of workers' unsafe behaviors, thus promoting safety performance in the construction industry.

Assessing the Culture of Residency Using the C - Change Resident Survey: Validity Evidence in 34 U.S. Residency Programs.

PubMed

Pololi, Linda H; Evans, Arthur T; Civian, Janet T; Shea, Sandy; Brennan, Robert T

2017-07-01

A practical instrument is needed to reliably measure the clinical learning environment and professionalism for residents. To develop and present evidence of validity of an instrument to assess the culture of residency programs and the clinical learning environment. During 2014-2015, we surveyed residents using the C - Change Resident Survey to assess residents' perceptions of the culture in their programs. Residents in all years of training in 34 programs in internal medicine, pediatrics, and general surgery in 14 geographically diverse public and private academic health systems. The C - Change Resident Survey assessed residents' perceptions of 13 dimensions of the culture: Vitality, Self-Efficacy, Institutional Support, Relationships/Inclusion, Values Alignment, Ethical/Moral Distress, Respect, Mentoring, Work-Life Integration, Gender Equity, Racial/Ethnic Minority Equity, and self-assessed Competencies. We measured the internal reliability of each of the 13 dimensions and evaluated response process, content validity, and construct-related evidence validity by assessing relationships predicted by our conceptual model and prior research. We also assessed whether the measurements were sensitive to differences in specialty and across institutions. A total of 1708 residents completed the survey [internal medicine: n = 956, pediatrics: n = 411, general surgery: n = 311 (51% women; 16% underrepresented in medicine minority)], with a response rate of 70% (range across programs, 51-87%). Internal consistency of each dimension was high (Cronbach α: 0.73-0.90). The instrument was able to detect significant differences in the learning environment across programs and sites. Evidence of validity was supported by a good response process and the demonstration of several relationships predicted by our conceptual model. The C - Change Resident Survey assesses the clinical learning environment for residents, and we encourage further study of validity in different contexts. Results could be used to facilitate and monitor improvements in the clinical learning environment and resident well-being.

Systematic Review of Measurement Property Evidence for 8 Financial Management Instruments in Populations With Acquired Cognitive Impairment.

PubMed

Engel, Lisa; Chui, Adora; Beaton, Dorcas E; Green, Robin E; Dawson, Deirdre R

2018-03-07

To critically appraise the measurement property evidence (ie, psychometric) for 8 observation-based financial management assessment instruments. Seven databases were searched in May 2015. Two reviewers used an independent decision-agreement process to select studies of measurement property evidence relevant to populations with adulthood acquired cognitive impairment, appraise the quality of the evidence, and extract data. Twenty-one articles were selected. This review used the COnsensus-based Standards for the selection of health Measurement Instruments review guidelines and 4-point tool to appraise evidence. After appraising the methodologic quality, the adequacy of results and volume of evidence per instrument were synthesized. Measurement property evidence with high risk of bias was excluded from the synthesis. The volume of measurement property evidence per instrument is low; most instruments had 1 to 3 included studies. Many included studies had poor methodologic quality per measurement property evidence area examined. Six of the 8 instruments reviewed had supporting construct validity/hypothesis-testing evidence of fair methodologic quality. There is a dearth of acceptable quality content validity, reliability, and responsiveness evidence for all 8 instruments. Rehabilitation practitioners assess financial management functions in adults with acquired cognitive impairments. However, there is limited published evidence to support using any of the reviewed instruments. Practitioners should exercise caution when interpreting the results of these instruments. This review highlights the importance of appraising the quality of measurement property evidence before examining the adequacy of the results and synthesizing the evidence. Copyright © 2018 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.

Developing empowering health counseling measurement. Preliminary results.

PubMed

Kettunen, Tarja; Liimatainen, Leena; Villberg, Jari; Perko, Ulla

2006-12-01

This article describes the derivation of an instrument (Empowering Speech Practices Scale) for assessing the empowerment of dyadic counseling, the evaluation of the validity and reliability of the ESPS and the results acquired with the instrument from hospital counseling. ESPS was constructed on the basis of empowerment theory and foregoing conversation analytic research. Nurses and patients assessed the same counseling session by way of parallel statements. Structure and reliability of the scale were evaluated with Cronbach alpha, percentage of agreement, factor analysis and logistic regression analysis. According to these preliminary results, ESPS described the realization of empowerment, directing attention to patient participation. By means of the scale, we assessed 127 counseling sessions and found evidence of the realization of empowering counseling. According to the results, nurses were the most successful in constructing a positive emotional atmosphere and in giving information. We found evidence that nurses need to improve the active mutuality of the counseling relationship by asking for patients' opinions and views, by facilitating the patients' assessment of their personal health and their participation in decision-making and coming up with options for their individual treatment. The developed scale can be utilized, in addition to assessing the quality in hospital care, for improving nursing education programs. Further study is needed to evaluate the usability of the scale and to examine its stability and validity.

Refinement and initial validation of a multidimensional composite scale for use in assessing acute postoperative pain in cats.

PubMed

Brondani, Juliana Tabarelli; Luna, Stelio Pacca Loureiro; Padovani, Carlos Roberto

2011-02-01

To refine and test construct validity and reliability of a composite pain scale for use in assessing acute postoperative pain in cats undergoing ovariohysterectomy. 40 cats that underwent ovariohysterectomy in a previous study. In a previous randomized, double-blind, placebo-controlled study, a composite pain scale was developed to assess postoperative pain in cats that received a placebo or an analgesic (tramadol, vedaprofen, or tramadol-vedaprofen combination). In the present study, the scale was refined via item analysis (distribution frequency and occurrence), a nonparametric ANOVA, and item-to-total score correlation. Construct validity was assessed via factor analysis and known-groups discrimination, and reliability was measured by assessing internal consistency. Respiratory rate and respiratory pattern were rejected after item analysis. Factor analysis resulted in 5 dimensions (F1 [psychomotor change], posture, comfort, activity, mental status, and miscellaneous behaviors; F2 [protection of wound area], reaction to palpation of the surgical wound and palpation of the abdomen and flank; F3 [physiologic variables], systolic arterial blood pressure and appetite; F4 [vocal expression of pain], vocalization; and F5 [heart rate]). Internal consistency was excellent for the overall scale and for F1, F2, and F3; very good for F4; and unacceptable for F5. Except for heart rate, the identified factors and scale total score could be used to detect differences between the analgesic and placebo groups and differences among the analgesic treatments. Results provided initial evidence of construct validity and reliability of a multidimensional composite tool for use in assessing acute postoperative pain in cats undergoing ovariohysterectomy.

Construct, Concurrent and Predictive Validity of the URICA: Data from Two Multi-site Clinical Trials

PubMed Central

Field, Craig A.; Adinoff, Bryon; Harris, T. Robert; Ball, Samuel A.; Carroll, Kathleen M.

2011-01-01

Background A better understanding of how to measure motivation to change and how it relates to behavior change in patients with drug and alcohol dependence would broaden our understanding of the role of motivation in addiction treatment. Methods Two multi-site, randomized clinical trials comparing brief motivational interventions with standard care were conducted in the National Institute on Drug Abuse Clinical Trials Network. Patients with primary drug dependence and alcohol dependence entering outpatient treatment participated in a study of either Motivational Enhancement Therapy (n=431) or Motivational Interviewing (n=423). The construct, concurrent, and predictive validity of two composite measures of motivation to change derived from the University of Rhode Island Change Assessment (URICA): Readiness to Change (RTC) and Committed Action (CA) were evaluated. Results Confirmatory factor analysis confirmed the a priori factor structure of the URICA. RTC was significantly associated with measures of addiction severity at baseline (r=.12-.52, p<.05). Although statistically significant (p<.01), the correlations between treatment outcomes and RTC were low (r=-.15 and -18). Additional analyses did not support a moderating or mediating effect of motivation on treatment retention or substance use. Conclusions The construct validity of the URICA was confirmed separately in a large sample of drug- and alcohol-dependent patients. However, evidence for the predictive validity of composite scores was very limited and there were no moderating or mediating effects of either measure on treatment outcome. Thus, increased motivation to change, as measured by the composite scores of motivation derived from the URICA, does not appear to influence treatment outcome. PMID:19157723

MEASURING SPORT-SPECIFIC PHYSICAL ABILITIES IN MALE GYMNASTS: THE MEN'S GYMNASTICS FUNCTIONAL MEASUREMENT TOOL

PubMed Central

Kenyon, Lisa K.; Elliott, James M; Cheng, M. Samuel

2016-01-01

Purpose/Background Despite the availability of various field-tests for many competitive sports, a reliable and valid test specifically developed for use in men's gymnastics has not yet been developed. The Men's Gymnastics Functional Measurement Tool (MGFMT) was designed to assess sport-specific physical abilities in male competitive gymnasts. The purpose of this study was to develop the MGFMT by establishing a scoring system for individual test items and to initiate the process of establishing test-retest reliability and construct validity. Methods A total of 83 competitive male gymnasts ages 7-18 underwent testing using the MGFMT. Thirty of these subjects underwent re-testing one week later in order to assess test-retest reliability. Construct validity was assessed using a simple regression analysis between total MGFMT scores and the gymnasts’ USA-Gymnastics competitive level to calculate the coefficient of determination (r2). Test-retest reliability was analyzed using Model 1 Intraclass correlation coefficients (ICC). Statistical significance was set at the p<0.05 level. Results The relationship between total MGFMT scores and subjects’ current USA-Gymnastics competitive level was found to be good (r2 = 0.63). Reliability testing of the MGFMT composite test score showed excellent test-retest reliability over a one-week period (ICC = 0.97). Test-retest reliability of the individual component tests ranged from good to excellent (ICC = 0.75-0.97). Conclusions The results of this study provide initial support for the construct validity and test-retest reliability of the MGFMT. Level of Evidence Level 3 PMID:27999723

Validation of Multilevel Constructs: Validation Methods and Empirical Findings for the EDI

ERIC Educational Resources Information Center

Forer, Barry; Zumbo, Bruno D.

2011-01-01

The purposes of this paper are to highlight the foundations of multilevel construct validation, describe two methodological approaches and associated analytic techniques, and then apply these approaches and techniques to the multilevel construct validation of a widely-used school readiness measure called the Early Development Instrument (EDI;…

Does IQ Really Predict Job Performance?

PubMed Central

Richardson, Ken; Norgate, Sarah H.

2015-01-01

IQ has played a prominent part in developmental and adult psychology for decades. In the absence of a clear theoretical model of internal cognitive functions, however, construct validity for IQ tests has always been difficult to establish. Test validity, therefore, has always been indirect, by correlating individual differences in test scores with what are assumed to be other criteria of intelligence. Job performance has, for several reasons, been one such criterion. Correlations of around 0.5 have been regularly cited as evidence of test validity, and as justification for the use of the tests in developmental studies, in educational and occupational selection and in research programs on sources of individual differences. Here, those correlations are examined together with the quality of the original data and the many corrections needed to arrive at them. It is concluded that considerable caution needs to be exercised in citing such correlations for test validation purposes. PMID:26405429

Drive: Theory and Construct Validation

PubMed Central

Petrides, K. V.

2016-01-01

This article explicates the theory of drive and describes the development and validation of two measures. A representative set of drive facets was derived from an extensive corpus of human attributes (Study 1). Operationalised using an International Personality Item Pool version (the Drive:IPIP), a three-factor model was extracted from the facets in two samples and confirmed on a third sample (Study 2). The multi-item IPIP measure showed congruence with a short form, based on single-item ratings of the facets, and both demonstrated cross-informant reliability. Evidence also supported the measures’ convergent, discriminant, concurrent, and incremental validity (Study 3). Based on very promising findings, the authors hope to initiate a stream of research in what is argued to be a rather neglected niche of individual differences and non-cognitive assessment. PMID:27409773

Motivators and barriers of a Healthy Lifestyle Scale: development and psychometric characteristics.

PubMed

Downes, Loureen

2008-01-01

Black individuals suffer disproportionately from diseases that are preventable by lifestyle choices. The purpose of this study was to test the internal consistency and construct validity of the newly devised instrument, Motivators and Barriers of a Healthy Lifestyle Scale (MABS). The MABS was administered to 109 community-dwelling, adult Blacks. Content validity was supported through review of the literature and the judgment of three content experts. Exploratory factor analysis supported the two dimensions, that is, motivators and barriers. The Cronbach's alphas for the motivators and barriers dimensions were .88 and .90, respectively. Results provide initial evidence that the MABS is a valid, internally consistent measure of factors that motivate or inhibit healthy lifestyle behaviors. Screening with the MABS could encourage more focused health promotion discussions between patients and practitioners.

Development and validation of a multi-dimensional measure of intellectual humility

PubMed Central

Alfano, Mark; Iurino, Kathryn; Stey, Paul; Robinson, Brian; Christen, Markus; Yu, Feng; Lapsley, Daniel

2017-01-01

This paper presents five studies on the development and validation of a scale of intellectual humility. This scale captures cognitive, affective, behavioral, and motivational components of the construct that have been identified by various philosophers in their conceptual analyses of intellectual humility. We find that intellectual humility has four core dimensions: Open-mindedness (versus Arrogance), Intellectual Modesty (versus Vanity), Corrigibility (versus Fragility), and Engagement (versus Boredom). These dimensions display adequate self-informant agreement, and adequate convergent, divergent, and discriminant validity. In particular, Open-mindedness adds predictive power beyond the Big Six for an objective behavioral measure of intellectual humility, and Intellectual Modesty is uniquely related to Narcissism. We find that a similar factor structure emerges in Germanophone participants, giving initial evidence for the model’s cross-cultural generalizability. PMID:28813478

Exploring the validity of the Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT) with established emotions measures.

PubMed

Roberts, Richard D; Schulze, Ralf; O'Brien, Kristin; MacCann, Carolyn; Reid, John; Maul, Andy

2006-11-01

Emotions measures represent an important means of obtaining construct validity evidence for emotional intelligence (EI) tests because they have the same theoretical underpinnings. Additionally, the extent to which both emotions and EI measures relate to intelligence is poorly understood. The current study was designed to address these issues. Participants (N = 138) completed the Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT), two emotions measures, as well as four intelligence tests. Results provide mixed support for the model hypothesized to underlie the MSCEIT, with emotions research and EI measures failing to load on the same factor. The emotions measures loaded on the same factor as intelligence measures. The validity of certain EI components (in particular, Emotion Perception), as currently assessed, appears equivocal. Copyright 2006 APA, all rights reserved.

Developing an index to measure the voluntariness of consent to research.

PubMed

Dugosh, Karen L; Festinger, David S; Marlowe, Douglas B; Clements, Nicolle T

2014-10-01

The goals of the current study were to expand the content domain and further validate the Coercion Assessment Scale (CAS), a measure of perceived coercion for criminally involved substance abusers being recruited into research. Unlike the few existing measures of this construct, the CAS identifies specific external sources of pressure that may influence one's decision to participate. In Phase 1, we conducted focus groups with criminal justice clients and stakeholders to expand the instrument by identifying additional sources of pressure. In Phase 2, we evaluated the expanded measure (i.e., endorsement rates, reliability, validity) in an ongoing research trial. Results identified new sources of pressure and provided evidence supporting the CAS's utility and reliability over time as well as convergent and discriminative validity. © The Author(s) 2014.

The Development and Validation of the Rational and Intuitive Decision Styles Scale.

PubMed

Hamilton, Katherine; Shih, Shin-I; Mohammed, Susan

2016-01-01

Decision styles reflect the typical manner by which individuals make decisions. The purpose of this research was to develop and validate a decision style scale that addresses conceptual and psychometric problems with current measures. The resulting 10-item scale captures a broad range of the rational and intuitive styles construct domain. Results from 5 independent samples provide initial support for the dimensionality and reliability of the new scale, as demonstrated by a clear factor structure and high internal consistency. In addition, our results show evidence of convergent and discriminant validity through expected patterns of correlations across decision-making individual differences and the International Personality Item Pool (IPIP) Big Five traits. Research domains that would benefit from incorporating the concept of decision styles are discussed.

Development and Validation of a Cross-Cultural Knowledge, Attitudes, and Practices Survey Instrument for Chronic Kidney Disease in a Swahili-Speaking Population

PubMed Central

Stanifer, John W.; Karia, Francis; Voils, Corrine I.; Turner, Elizabeth L.; Maro, Venance; Shimbi, Dionis; Kilawe, Humphrey; Lazaro, Matayo; Patel, Uptal D.

2015-01-01

Introduction Non-communicable diseases are a growing global burden, and structured surveys can identify critical gaps to address this epidemic. In sub-Saharan Africa, there are very few well-tested survey instruments measuring population attributes related to non-communicable diseases. To meet this need, we have developed and validated the first instrument evaluating knowledge, attitudes and practices pertaining to chronic kidney disease in a Swahili-speaking population. Methods and Results Between December 2013 and June 2014, we conducted a four-stage, mixed-methods study among adults from the general population of northern Tanzania. In stage 1, the survey instrument was constructed in English by a group of cross-cultural experts from multiple disciplines and through content analysis of focus group discussions to ensure local significance. Following translation, in stage 2, we piloted the survey through cognitive and structured interviews, and in stage 3, in order to obtain initial evidence of reliability and construct validity, we recruited and then administered the instrument to a random sample of 606 adults. In stage 4, we conducted analyses to establish test-retest reliability and known-groups validity which was informed by thematic analysis of the qualitative data in stages 1 and 2. The final version consisted of 25 items divided into three conceptual domains: knowledge, attitudes and practices. Each item demonstrated excellent test-retest reliability with established content and construct validity. Conclusions We have developed a reliable and valid cross-cultural survey instrument designed to measure knowledge, attitudes and practices of chronic kidney disease in a Swahili-speaking population of Northern Tanzania. This instrument may be valuable for addressing gaps in non-communicable diseases care by understanding preferences regarding healthcare, formulating educational initiatives, and directing development of chronic disease management programs that incorporate chronic kidney disease across sub-Saharan Africa. PMID:25811781

Is the Berg Balance Scale an effective tool for the measurement of early postural control impairments in patients with Parkinson's disease? Evidence from Rasch analysis.

PubMed

La Porta, F; Giordano, A; Caselli, S; Foti, C; Franchignoni, F

2015-12-01

It is unclear whether the BBS is an effective tool for the measurement of early postural control impairments in patients with Parkinson's disease (PD). The aim of this paper was to evaluate BBS' content validity, internal construct validity, reliability and targeting in patients with PD within the Rasch analysis framework. Observational, cross-sectional study. Outpatient Rehabilitation Unit. A sample of 285 outpatients with PD. The content validity of the BBS was assessed using standard linking techniques. The BBS was administered by trained physiotherapists. The data collected then underwent Rasch analysis. Content validity analysis showed a lack of items assessing postural responses to tripping and slips and stability during walking. On Rasch analysis, the BBS failed the requirements of monotonicity, local independence, unidimensionality and invariance. After rescoring 7 items, grouping of locally dependent items into testlets, and deletion of the static sitting balance item because mistargeted and underdiscriminating, the Rasch-modified BBS for PD (BBS-PD) showed adequate internal construct validity (χ(2)24=39.693; P=0.023), including absence of differential item functioning (DIF) across gender and age, and was, as a whole, sufficiently precise for individual person measurement (PSI=0.894). However, the scale was not well targeted to the sample in view of the prevalence of higher scores. This study demonstrated the internal construct validity and reliability of the BBS-PD as a measurement tool for patients with PD within the Rasch analysis framework. However, the lack of items critical to the assessment of postural control impairments typical of PD, affected negatively the targeting, so that a significant percentage of patients was located in the higher ability range of the measurement continuum, where precision of measurement is reduced. These findings suggest that the BBS, even if modified, may not be an effective tool for the measurement of early postural control in patients with PD.

Investigation of Item-Pair Presentation and Construct Validity of the Navy Computer Adaptive Personality Scales (NCAPS)

DTIC Science & Technology

2006-10-01

Investigation of Item-Pair Presentation and Construct Validity of the Navy Computer Adaptive Personality Scales ( NCAPS ) Christina M. Underhill, Ph.D...Construct Validity of the Navy Computer Adaptive Personality Scales ( NCAPS ) Christina M. Underhill, Ph.D. Reviewed and Approved by Jacqueline A. Mottern...and Construct Validity of the Navy Computer Adaptive Personality Scales ( NCAPS ) 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 0602236N and 0603236N 6

Reliability and validity of the test of incremental respiratory endurance measures of inspiratory muscle performance in COPD

PubMed Central

Formiga, Magno F; Roach, Kathryn E; Vital, Isabel; Urdaneta, Gisel; Balestrini, Kira; Calderon-Candelario, Rafael A

2018-01-01

Purpose The Test of Incremental Respiratory Endurance (TIRE) provides a comprehensive assessment of inspiratory muscle performance by measuring maximal inspiratory pressure (MIP) over time. The integration of MIP over inspiratory duration (ID) provides the sustained maximal inspiratory pressure (SMIP). Evidence on the reliability and validity of these measurements in COPD is not currently available. Therefore, we assessed the reliability, responsiveness and construct validity of the TIRE measures of inspiratory muscle performance in subjects with COPD. Patients and methods Test–retest reliability, known-groups and convergent validity assessments were implemented simultaneously in 81 male subjects with mild to very severe COPD. TIRE measures were obtained using the portable PrO2 device, following standard guidelines. Results All TIRE measures were found to be highly reliable, with SMIP demonstrating the strongest test–retest reliability with a nearly perfect intraclass correlation coefficient (ICC) of 0.99, while MIP and ID clustered closely together behind SMIP with ICC values of about 0.97. Our findings also demonstrated known-groups validity of all TIRE measures, with SMIP and ID yielding larger effect sizes when compared to MIP in distinguishing between subjects of different COPD status. Finally, our analyses confirmed convergent validity for both SMIP and ID, but not MIP. Conclusion The TIRE measures of MIP, SMIP and ID have excellent test–retest reliability and demonstrated known-groups validity in subjects with COPD. SMIP and ID also demonstrated evidence of moderate convergent validity and appear to be more stable measures in this patient population than the traditional MIP. PMID:29805255

Reliability, validity, and responsiveness of the Persian version of Shoulder Activity Scale in a group of patients with shoulder disorders.

PubMed

Negahban, Hossein; Mohtasebi, Elham; Goharpey, Shahin

2015-01-01

The aim of this methodological study was to cross-culturally translate the Shoulder Activity Scale (SAS) into the Persian and determine its clinimetric properties including reliability, validity, and responsiveness in patients with shoulder disorders. Persian version of the SAS was obtained after standard forward-backward translation. Three questionnaires were completed by the respondents: SAS, shoulder pain and disability index (SPADI), and Short-Form 36 Health Survey (SF-36). The patients completed the SAS, 1 week after the first visit to evaluate the test-retest reliability. Construct validity was evaluated by examining the associations between the scores on the SAS and the scores obtained from the SPADI, SF-36, and age of the patients. To assess responsiveness, data were collected in the first visit and then again after 4 weeks physiotherapy intervention. Test-retest reliability and internal consistency were assessed using Intra-class Correlation Coefficient (ICC) and Cronbach's alpha, respectively. To evaluate construct validity, Spearman's rank correlation was used. The ability of the SAS to detect changes was evaluated by the receiver-operating characteristics method. No problem or language difficulties were reported during translation process. Test-retest reliability of the SAS was excellent with an ICC of 0.98. Also, the marginal Cronbach's alpha level of 0.64 was obtained. The correlation between the SAS and the SPADI was low, proving divergent validity, whereas the correlations between the SAS and the SF-36/age were moderate proving convergent validity. A marginally acceptable responsiveness was achieved for the Persian SAS. The study provides some evidences to support the test-retest reliability, internal consistency, construct validity, and responsiveness of the Persian version of the SAS in patients with shoulder disorders. Therefore, it seems that this instrument is a useful measure of shoulder activity level in research setting and clinical practice. The shoulder activity scale (SAS) is a reliable, valid, and responsive measure of shoulder activity level in Persian-speaking patients with different shoulder disorders. The results on clinimetric properties of the Persian SAS are comparable with its original, English version. Persian version of the SAS can be used in "clinical" and "research" settings of patients with shoulder disorders.

Evidence-based algorithm for heparin dosing before cardiopulmonary bypass. Part 1: Development of the algorithm.

PubMed

McKinney, Mark C; Riley, Jeffrey B

2007-12-01

The incidence of heparin resistance during adult cardiac surgery with cardiopulmonary bypass has been reported at 15%-20%. The consistent use of a clinical decision-making algorithm may increase the consistency of patient care and likely reduce the total required heparin dose and other problems associated with heparin dosing. After a directed survey of practicing perfusionists regarding treatment of heparin resistance and a literature search for high-level evidence regarding the diagnosis and treatment of heparin resistance, an evidence-based decision-making algorithm was constructed. The face validity of the algorithm decisive steps and logic was confirmed by a second survey of practicing perfusionists. The algorithm begins with review of the patient history to identify predictors for heparin resistance. The definition for heparin resistance contained in the algorithm is an activated clotting time < 450 seconds with > 450 IU/kg heparin loading dose. Based on the literature, the treatment for heparin resistance used in the algorithm is anti-thrombin III supplement. The algorithm seems to be valid and is supported by high-level evidence and clinician opinion. The next step is a human randomized clinical trial to test the clinical procedure guideline algorithm vs. current standard clinical practice.

A constructive Indian country response to the evidence-based program mandate.

PubMed

Walker, R Dale; Bigelow, Douglas A

2011-01-01

Over the last 20 years governmental mandates for preferentially funding evidence-based "model" practices and programs has become doctrine in some legislative bodies, federal agencies, and state agencies. It was assumed that what works in small sample, controlled settings would work in all community settings, substantially improving safety, effectiveness, and value-for-money. The evidence-based "model" programs mandate has imposed immutable "core components," fidelity testing, alien programming and program developers, loss of familiar programs, and resource capacity requirements upon tribes, while infringing upon their tribal sovereignty and consultation rights. Tribal response in one state (Oregon) went through three phases: shock and rejection; proposing an alternative approach using criteria of cultural appropriateness, aspiring to evaluability; and adopting logic modeling. The state heard and accepted the argument that the tribal way of knowing is different and valid. Currently, a state-authorized tribal logic model and a review panel process are used to approve tribal best practices for state funding. This constructive response to the evidence-based program mandate elevates tribal practices in the funding and regulatory world, facilitates continuing quality improvement and evaluation, while ensuring that practices and programs remain based on local community context and culture. This article provides details of a model that could well serve tribes facing evidence-based model program mandates throughout the country.

Validating MEDIQUAL Constructs

NASA Astrophysics Data System (ADS)

Lee, Sang-Gun; Min, Jae H.

In this paper, we validate MEDIQUAL constructs through the different media users in help desk service. In previous research, only two end-users' constructs were used: assurance and responsiveness. In this paper, we extend MEDIQUAL constructs to include reliability, empathy, assurance, tangibles, and responsiveness, which are based on the SERVQUAL theory. The results suggest that: 1) five MEDIQUAL constructs are validated through the factor analysis. That is, importance of the constructs have relatively high correlations between measures of the same construct using different methods and low correlations between measures of the constructs that are expected to differ; and 2) five MEDIQUAL constructs are statistically significant on media users' satisfaction in help desk service by regression analysis.

Variance composition, measurement invariance by gender, and construct validity of the Femininity Ideology Scale-Short Form.

PubMed

Levant, Ronald F; Alto, Kathleen M; McKelvey, Daniel K; Richmond, Katherine A; McDermott, Ryon C

2017-11-01

The current study extended prior work on the Femininity Ideology Scale (FIS), a multidimensional measure of traditional femininity ideology (TFI), in several ways. First, we conducted exploratory factor and bifactor analyses, which revealed a general TFI factor and 3 specific factors: dependence/deference, purity, and emotionality/traditional roles. Second, based on these results we developed the 12-item FIS-Short Form (FIS-SF). Third, we assessed the FIS-SF using confirmatory factor analysis on a separate sample, finding that the items loaded on the general factor and 3 specific factors as hypothesized, and that the bifactor model fit better than common factors and unidimensional models. Fourth, model-based reliability estimates tentatively support the use of raw scores to represent the general TFI factor and the emotionality/traditional roles specific factor, but the other 2 specific factors are best measured using SEM or by ipsatizing their scores. Fifth, we assessed measurement invariance across 2 gender groups, finding evidence for configural invariance for all factors, and for partial metric invariance for the specific factors. Sixth, we found evidence for the convergent construct validity of the FIS-SF general factor and the emotionality/traditional roles specific factors by examining relationships with the latent variables of several constructs in the nomological network. The results are discussed in relationship to prior literature, future research directions, applications to counseling practice, and limitations. Data (N = 1,472, 907 women, 565 men, 530 people of color) were from community and college participants who responded to an online survey. (PsycINFO Database Record (c) 2017 APA, all rights reserved).

An integrated assessment instrument: Developing and validating instrument for facilitating critical thinking abilities and science process skills on electrolyte and nonelectrolyte solution matter

NASA Astrophysics Data System (ADS)

Astuti, Sri Rejeki Dwi; Suyanta, LFX, Endang Widjajanti; Rohaeti, Eli

2017-05-01

The demanding of assessment in learning process was impact by policy changes. Nowadays, assessment is not only emphasizing knowledge, but also skills and attitudes. However, in reality there are many obstacles in measuring them. This paper aimed to describe how to develop integrated assessment instrument and to verify instruments' validity such as content validity and construct validity. This instrument development used test development model by McIntire. Development process data was acquired based on development test step. Initial product was observed by three peer reviewer and six expert judgments (two subject matter experts, two evaluation experts and two chemistry teachers) to acquire content validity. This research involved 376 first grade students of two Senior High Schools in Bantul Regency to acquire construct validity. Content validity was analyzed used Aiken's formula. The verifying of construct validity was analyzed by exploratory factor analysis using SPSS ver 16.0. The result show that all constructs in integrated assessment instrument are asserted valid according to content validity and construct validity. Therefore, the integrated assessment instrument is suitable for measuring critical thinking abilities and science process skills of senior high school students on electrolyte solution matter.

Observed Parenting Behavior with Teens: Measurement Invariance and Predictive Validity Across Race

PubMed Central

Skinner, Martie L.; MacKenzie, Elizabeth P.; Haggerty, Kevin P.; Hill, Karl G.; Roberson, Kendra C.

2011-01-01

Previous reports supporting measurement equality between European American and African American families have often focused on self-reported risk factors or observed parent behavior with young children. This study examines equality of measurement of observer ratings of parenting behavior with adolescents during structured tasks; mean levels of observed parenting; and predictive validity of teen self-reports of antisocial behaviors and beliefs using a sample of 163 African American and 168 European American families. Multiple-group confirmatory factor analyses supported measurement invariance across ethnic groups for 4 measures of observed parenting behavior: prosocial rewards, psychological costs, antisocial rewards, and problem solving. Some mean-level differences were found: African American parents exhibited lower levels of prosocial rewards, higher levels of psychological costs, and lower problem solving when compared to European Americans. No significant mean difference was found in rewards for antisocial behavior. Multigroup structural equation models suggested comparable relationships across race (predictive validity) between parenting constructs and youth antisocial constructs (i.e., drug initiation, positive drug attitudes, antisocial attitudes, problem behaviors) in all but one of the tested relationships. This study adds to existing evidence that family-based interventions targeting parenting behaviors can be generalized to African American families. PMID:21787057

The Perceived Leadership Communication Questionnaire (PLCQ): Development and Validation.

PubMed

Schneider, Frank M; Maier, Michaela; Lovrekovic, Sara; Retzbach, Andrea

2015-01-01

The Perceived Leadership Communication Questionnaire (PLCQ) is a short, reliable, and valid instrument for measuring leadership communication from both perspectives of the leader and the follower. Drawing on a communication-based approach to leadership and following a theoretical framework of interpersonal communication processes in organizations, this article describes the development and validation of a one-dimensional 6-item scale in four studies (total N = 604). Results from Study 1 and 2 provide evidence for the internal consistency and factorial validity of the PLCQ's self-rating version (PLCQ-SR)-a version for measuring how leaders perceive their own communication with their followers. Results from Study 3 and 4 show internal consistency, construct validity, and criterion validity of the PLCQ's other-rating version (PLCQ-OR)-a version for measuring how followers perceive the communication of their leaders. Cronbach's α had an average of.80 over the four studies. All confirmatory factor analyses yielded good to excellent model fit indices. Convergent validity was established by average positive correlations of.69 with subdimensions of transformational leadership and leader-member exchange scales. Furthermore, nonsignificant correlations with socially desirable responding indicated discriminant validity. Last, criterion validity was supported by a moderately positive correlation with job satisfaction (r =.31).

Mapping the MMPI-2-RF Specific Problems Scales Onto Extant Psychopathology Structures.

PubMed

Sellbom, Martin

2017-01-01

A main objective in developing the Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF; Ben-Porath & Tellegen, 2008 ) was to link the hierarchical structure of the instrument's scales to contemporary psychopathology and personality models for greater enhancement of construct validity. Initial evidence published with the Restructured Clinical scales has indicated promising results in that the higher order structure of these measures maps onto those reported in the extant psychopathology literature. This study focused on evaluating the internal structure of the Specific Problems and Interest scales, which have not yet been examined in this manner. Two large, mixed-gender outpatient and correctional samples were used. Exploratory factor analyses revealed consistent evidence for a 4-factor structure representing somatization, negative affect, externalizing, and social detachment. Convergent and discriminant validity analyses in the outpatient sample yielded a pattern of results consistent with expectations. These findings add further evidence to indicate that the MMPI-2-RF hierarchy of scales map onto extant psychopathology literature, and also add support to the notion that somatization and detachment should be considered important higher order domains in the psychopathology literature.

Validity evidence for the situational judgment test paradigm in emotional intelligence measurement.

PubMed

Libbrecht, Nele; Lievens, Filip

2012-01-01

To date, various measurement approaches have been proposed to assess emotional intelligence (EI). Recently, two new EI tests have been developed based on the situational judgment test (SJT) paradigm: the Situational Test of Emotional Understanding (STEU) and the Situational Test of Emotion Management (STEM). Initial attempts have been made to examine the construct-related validity of these new tests; we extend these findings by placing the tests in a broad nomological network. To this end, 850 undergraduate students completed a personality inventory, a cognitive ability test, a self-report EI test, a performance-based EI measure, the STEU, and the STEM. The SJT-based EI tests were not strongly correlated with personality and fluid cognitive ability. Regarding their relation with existing EI measures, the tests did not capture the same construct as self-report EI measures, but corresponded rather to performance-based EI measures. Overall, these results lend support for the SJT paradigm for measuring EI as an ability.

Developmentally and Culturally Appropriate Screening in Primary Care: Development of the Behavioral Health Checklist

PubMed Central

Koshy, Anson J.; Watkins, Marley W.; Cassano, Michael C.; Wahlberg, Andrea C.; Mautone, Jennifer A.; Blum, Nathan J.

2013-01-01

Objective To evaluate the construct validity of the Behavioral Health Checklist (BHCL) for children aged from 4 to 12 years from diverse backgrounds. Method The parents of 4–12-year-old children completed the BHCL in urban and suburban primary care practices affiliated with a tertiary-care children’s hospital. Across practices, 1,702 were eligible and 1,406 (82.6%) provided consent. Children of participating parents were primarily non-Hispanic black/African American and white/Caucasian from low- to middle-income groups. Confirmatory factor analyses examined model fit for the total sample and subsamples defined by demographic characteristics. Results The findings supported the hypothesized 3-factor structure: Internalizing Problems, Externalizing Problems, and Inattention/Hyperactivity. The model demonstrated adequate to good fit across age-groups, gender, races, income groups, and suburban versus urban practices. Conclusion The findings provide strong evidence of the construct validity, developmental appropriateness, and cultural sensitivity of the BHCL when used for screening in primary care. PMID:23978505

The generalised anxiety stigma scale (GASS): psychometric properties in a community sample

PubMed Central

2011-01-01

Background Although there is substantial concern about negative attitudes to mental illness, little is known about the stigma associated with Generalised Anxiety Disorder (GAD) or its measurement. The aim of this study was to develop a multi-item measure of Generalised Anxiety Disorder stigma (the GASS). Methods Stigma items were developed from a thematic analysis of web-based text about the stigma associated with GAD. Six hundred and seventeen members of the public completed a survey comprising the resulting 20 stigma items and measures designed to evaluate construct validity. Follow-up data were collected for a subset of the participants (n = 212). Results The factor structure comprised two components: Personal Stigma (views about Generalised Anxiety Disorder); and Perceived Stigma (views about the beliefs of most others in the community). There was evidence of good construct validity and reliability for each of the Generalised Anxiety Stigma Scale (GASS) subscales. Conclusions The GASS is a promising brief measure of the stigma associated with Generalised Anxiety Disorder. PMID:22108099

Gendered sexuality: a new model and measure of attraction and intimacy.

PubMed

Starks, Tyrel J; Gilbert, Brenda O; Fischer, Ann R; Weston, Rebecca; DiLalla, David L

2009-01-01

Currently, the literature related to sexual orientation is ambiguous with regard to the relationship of sexual orientation, sexual identity, attraction, and intimacy. In order to explore the relationships of self-identified categorical sexual identity (which is the most popular method of sexual orientation assessment) with attraction and intimacy, it is imperative that researchers have access to a reliable and valid measure of the latter. The present study proposes a model for conceptualizing attraction and intimacy, termed gendered sexuality, and examines the factor structure of a measure designed to assess the construct. Results suggest that four factors adequately accounted for the variance in gendered sexuality in a large sample of young adults. These factors assess attraction to females, attraction to males, intimacy with females, and intimacy with males. Exploratory analyses provided preliminary evidence of potential construct validity and suggested that discrepancies between desired and available behavior predict dissatisfaction in interpersonal role as measured by the Outcome Questionnaire 45.2.

Measuring Hope Among Children Affected by Armed Conflict: Cross-Cultural Construct Validity of the Children's Hope Scale.

PubMed

Haroz, Emily E; Jordans, Mark; de Jong, Joop; Gross, Alden; Bass, Judith; Tol, Wietse

2017-06-01

We investigated the cross-cultural construct validity of hope, a factor associated with mental health protection and promotion, using the Children's Hope Scale (CHS). The sample ( n = 1,057; 48% girls) included baseline data from three cluster-randomized controlled trials with children affected by armed conflict ( n = 329 Burundi; n = 403 Indonesia; n = 325 Nepal). The confirmatory factor analysis in each country indicated good fit for the hypothesized two-factor model. Analysis by gender indicated that configural invariance was supported and that scalar invariance was demonstrated in Indonesia. However, metric and scalar invariance were not supported in Burundi and Nepal. In country comparisons, configural and metric invariance were met, but scalar invariance was not supported. Evidence from this study supports the use of the CHS within various sociocultural settings and across genders, but direct comparisons of CHS scores across groups should be done with caution. Rigorous evaluations of the measurement properties of mental health protective and promotive factors are necessary to inform both research and practice.

Profiling Perceptual Learning Styles of Chinese as a Second Language Learners in University Settings.

PubMed

Sun, Peijian Paul; Teng, Lin Sophie

2017-12-01

This study revisited Reid's (1987) perceptual learning style preference questionnaire (PLSPQ) in an attempt to answer whether the PLSPQ fits in the Chinese-as-a-second-language (CSL) context. If not, what are CSL learners' learning styles drawing on the PLSPQ? The PLSPQ was first re-examined through reliability analysis and confirmatory factor analysis (CFA) with 224 CSL learners. The results showed that Reid's six-factor PLSPQ could not satisfactorily explain the CSL learners' learning styles. Exploratory factor analyses were, therefore, performed to explore the dimensionality of the PLSPQ in the CSL context. A four-factor PLSPQ was successfully constructed including auditory/visual, kinaesthetic/tactile, group, and individual styles. Such a measurement model was cross-validated through CFAs with 118 CSL learners. The study not only lends evidence to the literature that Reid's PLSPQ lacks construct validity, but also provides CSL teachers and learners with insightful and practical guidance concerning learning styles. Implications and limitations of the present study are discussed.

A Philosophical Perspective on Construct Validation: Application of Inductive Logic to the Analysis of Experimental Episode Construct Validity.

ERIC Educational Resources Information Center

Rossi, Robert Joseph

Methods drawn from four logical theories associated with studies of inductive processes are applied to the assessment and evaluation of experimental episode construct validity. It is shown that this application provides for estimates of episode informativeness with respect to the person examined in terms of the construct and to the construct…

Determining the Scoring Validity of a Co-Constructed CEFR-Based Rating Scale

ERIC Educational Resources Information Center

Deygers, Bart; Van Gorp, Koen

2015-01-01

Considering scoring validity as encompassing both reliable rating scale use and valid descriptor interpretation, this study reports on the validation of a CEFR-based scale that was co-constructed and used by novice raters. The research questions this paper wishes to answer are (a) whether it is possible to construct a CEFR-based rating scale with…

Development of the Assessment of Belief Conflict in Relationship-14 (ABCR-14).

PubMed

Kyougoku, Makoto; Teraoka, Mutsumi; Masuda, Noriko; Ooura, Mariko; Abe, Yasushi

2015-01-01

Nurses and other healthcare workers frequently experience belief conflict, one of the most important, new stress-related problems in both academic and clinical fields. In this study, using a sample of 1,683 nursing practitioners, we developed The Assessment of Belief Conflict in Relationship-14 (ABCR-14), a new scale that assesses belief conflict in the healthcare field. Standard psychometric procedures were used to develop and test the scale, including a qualitative framework concept and item-pool development, item reduction, and scale development. We analyzed the psychometric properties of ABCR-14 according to entropy, polyserial correlation coefficient, exploratory factor analysis, confirmatory factor analysis, average variance extracted, Cronbach's alpha, Pearson product-moment correlation coefficient, and multidimensional item response theory (MIRT). The results of the analysis supported a three-factor model consisting of 14 items. The validity and reliability of ABCR-14 was suggested by evidence from high construct validity, structural validity, hypothesis testing, internal consistency reliability, and concurrent validity. The result of the MIRT offered strong support for good item response of item slope parameters and difficulty parameters. However, the ABCR-14 Likert scale might need to be explored from the MIRT point of view. Yet, as mentioned above, there is sufficient evidence to support that ABCR-14 has high validity and reliability. The ABCR-14 demonstrates good psychometric properties for nursing belief conflict. Further studies are recommended to confirm its application in clinical practice.

Measuring Psychobiosocial States in Sport: Initial Validation of a Trait Measure

PubMed Central

Bertollo, Maurizio; Ruiz, Montse C.; Bortoli, Laura

2016-01-01

We examined the item characteristics, the factor structure, and the concurrent validity of a trait measure of psychobiosocial states. In Study 1, Italian athletes (N = 342, 228 men, 114 women, Mage = 23.93, SD = 6.64) rated the intensity, the frequency, and the perceived impact dimensions of a psychobiosocial states scale, trait version (PBS-ST), which is composed of 20 items (10 functional and 10 dysfunctional) referring to how they usually felt before an important competition. In Study 2, the scale was cross validated in an independent sample (N = 251, 181 men, 70 women, Mage = 24.35, SD = 7.25). The concurrent validity of the PBS-ST scale scores were also examined in comparison with two sport-specific emotion-related measures and a general measure of affect. Exploratory structural equation modeling and confirmatory factor analysis of the data of Study 1 showed that a 2-factor, 15-item solution of the PBS-ST scale (8 functional items and 7 dysfunctional items) reached satisfactory fit indices for the three dimensions (i.e., intensity, frequency, and perceived impact). Results of Study 2 provided evidence of substantial measurement and structural invariance of all dimensions across samples. The low association of the PBS-ST scale with other measures suggests that the scale taps unique constructs. Findings of the two studies offer initial validity evidence for a sport-specific tool to measure psychobiosocial states. PMID:27907111

Construct Validity of Neuropsychological Tests in Schizophrenia.

ERIC Educational Resources Information Center

Allen, Daniel N.; Aldarondo, Felito; Goldstein, Gerald; Huegel, Stephen G.; Gilbertson, Mark; van Kammen, Daniel P.

1998-01-01

The construct validity of neuropsychological tests in patients with schizophrenia was studied with 39 patients who were evaluated with a battery of six tests assessing attention, memory, and abstract reasoning abilities. Results support the construct validity of the neuropsychological tests in patients with schizophrenia. (SLD)

Exploring depressive personality traits in youth: origins, correlates, and developmental consequences.

PubMed

Rudolph, Karen D; Klein, Daniel N

2009-01-01

Research suggests that depressive personality (DP) disorder may represent a persistent, trait-based form of depression that lies along an affective spectrum ranging from personality traits to diagnosable clinical disorders. A significant gap in this area of research concerns the development of DP and its applicability to youth. The present research explored the construct of DP traits in youth. Specifically, this study examined the reliability, stability, and validity of the construct, potential origins of DP traits, and the developmental consequences of DP traits. A sample of 143 youth (mean age = 12.37 years, SD = 1.26) and their caregivers completed semistructured interviews and questionnaires on two occasions, separated by a 12-month interval. The measure of DP traits was reliable and moderately stable over time. Providing evidence of construct validity, DP traits were associated with a network of constructs, including a negative self-focus, high-negative and low-positive emotionality, and heightened stress reactivity. Moreover, several potential origins of DP traits were identified, including a history of family adversity, maternal DP traits, and maternal depression. Consistent with hypotheses regarding their developmental significance, DP traits predicted the generation of stress and the emergence of depression (but not nondepressive psychopathology) during the pubertal transition. Finally, depression predicted subsequent DP traits, suggesting a reciprocal process whereby DP traits heighten risk for depression, which then exacerbates these traits. These findings support the construct of DP traits in youth, and suggest that these traits may be a useful addition to developmental models of risk for youth depression.

Assessing the capacity of ministries of health to use research in decision-making: conceptual framework and tool.

PubMed

Rodríguez, Daniela C; Hoe, Connie; Dale, Elina M; Rahman, M Hafizur; Akhter, Sadika; Hafeez, Assad; Irava, Wayne; Rajbangshi, Preety; Roman, Tamlyn; Ţîrdea, Marcela; Yamout, Rouham; Peters, David H

2017-08-01

The capacity to demand and use research is critical for governments if they are to develop policies that are informed by evidence. Existing tools designed to assess how government officials use evidence in decision-making have significant limitations for low- and middle-income countries (LMICs); they are rarely tested in LMICs and focus only on individual capacity. This paper introduces an instrument that was developed to assess Ministry of Health (MoH) capacity to demand and use research evidence for decision-making, which was tested for reliability and validity in eight LMICs (Bangladesh, Fiji, India, Lebanon, Moldova, Pakistan, South Africa, Zambia). Instrument development was based on a new conceptual framework that addresses individual, organisational and systems capacities, and items were drawn from existing instruments and a literature review. After initial item development and pre-testing to address face validity and item phrasing, the instrument was reduced to 54 items for further validation and item reduction. In-country study teams interviewed a systematic sample of 203 MoH officials. Exploratory factor analysis was used in addition to standard reliability and validity measures to further assess the items. Thirty items divided between two factors representing organisational and individual capacity constructs were identified. South Africa and Zambia demonstrated the highest level of organisational capacity to use research, whereas Pakistan and Bangladesh were the lowest two. In contrast, individual capacity was highest in Pakistan, followed by South Africa, whereas Bangladesh and Lebanon were the lowest. The framework and related instrument represent a new opportunity for MoHs to identify ways to understand and improve capacities to incorporate research evidence in decision-making, as well as to provide a basis for tracking change.

Student mathematical imagination instruments: construction, cultural adaptation and validity

NASA Astrophysics Data System (ADS)

Dwijayanti, I.; Budayasa, I. K.; Siswono, T. Y. E.

2018-03-01

Imagination has an important role as the center of sensorimotor activity of the students. The purpose of this research is to construct the instrument of students’ mathematical imagination in understanding concept of algebraic expression. The researcher performs validity using questionnaire and test technique and data analysis using descriptive method. Stages performed include: 1) the construction of the embodiment of the imagination; 2) determine the learning style questionnaire; 3) construct instruments; 4) translate to Indonesian as well as adaptation of learning style questionnaire content to student culture; 5) perform content validation. The results stated that the constructed instrument is valid by content validation and empirical validation so that it can be used with revisions. Content validation involves Indonesian linguists, english linguists and mathematics material experts. Empirical validation is done through a legibility test (10 students) and shows that in general the language used can be understood. In addition, a questionnaire test (86 students) was analyzed using a biserial point correlation technique resulting in 16 valid items with a reliability test using KR 20 with medium reability criteria. While the test instrument test (32 students) to find all items are valid and reliability test using KR 21 with reability is 0,62.

Development of the Muscle Dysmorphia Inventory (MDI).

PubMed

Rhea, D J; Lantz, C D; Cornelius, A E

2004-12-01

The development of the 6-factor, 27-item Muscle Dysmorphia Inventory (MDI) was based on Lantz et al. proposed model of characteristics associated with Muscle Dysmorphia. quantitative procedures including item-to-total correlations, exploratory and confirmatory factor analyses, and structure equation modeling confirmed the construct validity of the scale. Convergent validity was also tested. bodybuilding and powerlifting competition venues, weight training facilities, and university athletic venues. the 1(st) study consisted of 77 experienced male free weight lifters. The 2(nd) study consisted of 156 male non-competitive bodybuilders and weight lifters and 168 elite level powerlifters and bodybuilders. The 3(rd) study consisted of 151 male and female bodybuilders and weight lifters. each participant completed demographic information, the MDI, Drive for Thinness subscale of the Eating Disorder Inventory, and the Training Dependency subscale of the Bodybuilding Dependence Scale. Reliability estimates (Cronbach's a) ranged from 0.72 to 0.94. Factor loadings in all 3 studies supported the 6-factor structure (size/symmetry, supplement use, exercise dependence, pharmacological use, dietary behavior, and physique protection). Much of the scale validation was focused on construct validity, however, correlations with the MDI's subscales and the Training Dependency subscale of the Bodybuilding Dependence Scale and the Drive for Thinness subscale of the Eating Disorder Inventory provided evidence of convergent validity also. From these preliminary results, the MDI appears to contribute to the identification of a newly formed disorder by offering a multi-dimensional measure of factors related to Muscle Dysmorphia.

Developing and validating a nutrition knowledge questionnaire: key methods and considerations.

PubMed

Trakman, Gina Louise; Forsyth, Adrienne; Hoye, Russell; Belski, Regina

2017-10-01

To outline key statistical considerations and detailed methodologies for the development and evaluation of a valid and reliable nutrition knowledge questionnaire. Literature on questionnaire development in a range of fields was reviewed and a set of evidence-based guidelines specific to the creation of a nutrition knowledge questionnaire have been developed. The recommendations describe key qualitative methods and statistical considerations, and include relevant examples from previous papers and existing nutrition knowledge questionnaires. Where details have been omitted for the sake of brevity, the reader has been directed to suitable references. We recommend an eight-step methodology for nutrition knowledge questionnaire development as follows: (i) definition of the construct and development of a test plan; (ii) generation of the item pool; (iii) choice of the scoring system and response format; (iv) assessment of content validity; (v) assessment of face validity; (vi) purification of the scale using item analysis, including item characteristics, difficulty and discrimination; (vii) evaluation of the scale including its factor structure and internal reliability, or Rasch analysis, including assessment of dimensionality and internal reliability; and (viii) gathering of data to re-examine the questionnaire's properties, assess temporal stability and confirm construct validity. Several of these methods have previously been overlooked. The measurement of nutrition knowledge is an important consideration for individuals working in the nutrition field. Improved methods in the development of nutrition knowledge questionnaires, such as the use of factor analysis or Rasch analysis, will enable more confidence in reported measures of nutrition knowledge.

Measuring Work Functioning: Validity of a Weighted Composite Work Functioning Approach.

PubMed

Boezeman, Edwin J; Sluiter, Judith K; Nieuwenhuijsen, Karen

2015-09-01

To examine the construct validity of a weighted composite work functioning measurement approach. Workers (health-impaired/healthy) (n = 117) completed a composite measure survey that recorded four central work functioning aspects with existing scales: capacity to work, quality of work performance, quantity of work, and recovery from work. Previous derived weights reflecting the relative importance of these aspects of work functioning were used to calculate the composite weighted work functioning score of the workers. Work role functioning, productivity, and quality of life were used for validation. Correlations were calculated and norms applied to examine convergent and divergent construct validity. A t test was conducted and a norm applied to examine discriminative construct validity. Overall the weighted composite work functioning measure demonstrated construct validity. As predicted, the weighted composite score correlated (p < .001) strongly (r > .60) with work role functioning and productivity (convergent construct validity), and moderately (.30 < r < .60) with physical quality of life and less strongly than work role functioning and productivity with mental quality of life (divergent validity). Further, the weighted composite measure detected that health-impaired workers show with a large effect size (Cohen's d > .80) significantly worse work functioning than healthy workers (discriminative validity). The weighted composite work functioning measurement approach takes into account the relative importance of the different work functioning aspects and demonstrated good convergent, fair divergent, and good discriminative construct validity.

Subluxation: dogma or science?

PubMed Central

Keating, Joseph C; Charlton, Keith H; Grod, Jaroslaw P; Perle, Stephen M; Sikorski, David; Winterstein, James F

2005-01-01

Subluxation syndrome is a legitimate, potentially testable, theoretical construct for which there is little experimental evidence. Acceptable as hypothesis, the widespread assertion of the clinical meaningfulness of this notion brings ridicule from the scientific and health care communities and confusion within the chiropractic profession. We believe that an evidence-orientation among chiropractors requires that we distinguish between subluxation dogma vs. subluxation as the potential focus of clinical research. We lament efforts to generate unity within the profession through consensus statements concerning subluxation dogma, and believe that cultural authority will continue to elude us so long as we assert dogma as though it were validated clinical theory. PMID:16092955

Assessing Student Understanding of the "New Biology": Development and Evaluation of a Criterion-Referenced Genomics and Bioinformatics Assessment

NASA Astrophysics Data System (ADS)

Campbell, Chad Edward

Over the past decade, hundreds of studies have introduced genomics and bioinformatics (GB) curricula and laboratory activities at the undergraduate level. While these publications have facilitated the teaching and learning of cutting-edge content, there has yet to be an evaluation of these assessment tools to determine if they are meeting the quality control benchmarks set forth by the educational research community. An analysis of these assessment tools indicated that <10% referenced any quality control criteria and that none of the assessments met more than one of the quality control benchmarks. In the absence of evidence that these benchmarks had been met, it is unclear whether these assessment tools are capable of generating valid and reliable inferences about student learning. To remedy this situation the development of a robust GB assessment aligned with the quality control benchmarks was undertaken in order to ensure evidence-based evaluation of student learning outcomes. Content validity is a central piece of construct validity, and it must be used to guide instrument and item development. This study reports on: (1) the correspondence of content validity evidence gathered from independent sources; (2) the process of item development using this evidence; (3) the results from a pilot administration of the assessment; (4) the subsequent modification of the assessment based on the pilot administration results and; (5) the results from the second administration of the assessment. Twenty-nine different subtopics within GB (Appendix B: Genomics and Bioinformatics Expert Survey) were developed based on preliminary GB textbook analyses. These subtopics were analyzed using two methods designed to gather content validity evidence: (1) a survey of GB experts (n=61) and (2) a detailed content analyses of GB textbooks (n=6). By including only the subtopics that were shown to have robust support across these sources, 22 GB subtopics were established for inclusion in the assessment. An expert panel subsequently developed, evaluated, and revised two multiple-choice items to align with each of the 22 subtopics, producing a final item pool of 44 items. These items were piloted with student samples of varying content exposure levels. Both Classical Test Theory (CTT) and Item Response Theory (IRT) methodologies were used to evaluate the assessment's validity, reliability and ability inferences, and its ability to differentiate students with different magnitudes of content exposure. A total of 18 items were subsequently modified and reevaluated by an expert panel. The 26 original and 18 modified items were once again piloted with student samples of varying content exposure levels. Both CTT and IRT methodologies were once again used to evaluate student responses in order to evaluate the assessment's validity and reliability inferences as well as its ability to differentiate students with different magnitudes of content exposure. Interviews with students from different content exposure levels were also performed in order to gather convergent validity evidence (external validity evidence) as well as substantive validity evidence. Also included are the limitations of the assessment and a set of guidelines on how the assessment can best be used.

A new instrument to measure quality of life of heart failure family caregivers.

PubMed

Nauser, Julie A; Bakas, Tamilyn; Welch, Janet L

2011-01-01

Family caregivers of heart failure (HF) patients experience poor physical and mental health leading to poor quality of life. Although several quality-of-life measures exist, they are often too generic to capture the unique experience of this population. The purpose of this study was to evaluate the psychometric properties of the Family Caregiver Quality of Life (FAMQOL) Scale that was designed to assess the physical, psychological, social, and spiritual dimensions of quality of life among caregivers of HF patients. Psychometric testing of the FAMQOL with 100 HF family caregivers was conducted using item analysis, Cronbach α, intraclass correlation, factor analysis, and hierarchical multiple regression guided by a conceptual model. Caregivers were predominately female (89%), white, (73%), and spouses (62%). Evidence of internal consistency reliability (α=.89) was provided for the FAMQOL, with item-total correlations of 0.39 to 0.74. Two-week test-retest reliability was supported by an intraclass correlation coefficient of 0.91. Using a 1-factor solution and principal axis factoring, loadings ranged from 0.31 to 0.78, with 41% of the variance explained by the first factor (eigenvalue=6.5). With hierarchical multiple regression, 56% of the FAMQOL variance was explained by model constructs (F8,91=16.56, P<.001). Criterion-related validity was supported by correlations with SF-36 General (r=0.45, P<.001) and Mental (r=0.59, P<.001) Health subscales and Bakas Caregiving Outcomes Scale (r=0.73, P<.001). Evidence of internal and test-retest reliability and construct and criterion validity was provided for physical, psychological, and social well-being subscales. The 16-item FAMQOL is a brief, easy-to-administer instrument that has evidence of reliability and validity in HF family caregivers. Physical, psychological, and social well-being can be measured with 4-item subscales. The FAMQOL scale could serve as a valuable measure in research, as well as an assessment tool to identify caregivers in need of intervention.

Evaluation of Pharmacy and Therapeutic (P&T) Committee member knowledge, attitudes and ability regarding the use of comparative effectiveness research (CER) in health care decision-making.

PubMed

Tang, D H; Warholak, T L; Hines, L E; Hurwitz, J; Brown, M; Taylor, A M; Brixner, D; Malone, D C

2014-01-01

Comparative effectiveness research (CER) is a constellation of research methods designed to improve health care decision making. Educational programs that improve health care decision makers' CER knowledge and awareness may ultimately lead to more cost-effective use of health care resources. This study was conducted to evaluate changes in CER knowledge, attitudes, and ability among Pharmacy and Therapeutics (P&T) Committee members and support staff after attending a tailored educational program. Physicians and pharmacists from two professional societies and the Indian Health Service who participated in the P&T process were invited via email to participate in this study. Participants completed a questionnaire, designed specifically for this study, prior to and following the 4-hour live, educational program on CER to determine the impact on their related knowledge, attitudes, and ability to use CER in decision-making. Rasch analysis was used to assess validity and reliability of subsections of the questionnaire and regression analysis was used to assess programmatic impact on CER knowledge, attitude, and ability. One hundred and forty of the 199 participants completed both the pre- and post-CER session questionnaires (response rate = 70.4%). Most participants (>75%) correctly answered eight of the ten knowledge items after attending the educational session. More than 60% of the respondents had a positive attitude toward CER both before and after the program. Compared to baseline (pretest), participants reported significant improvements in their perceived ability to use CER after attending the session in these areas: using CER reviews, knowledge of CER methods, identifying problems with randomized controlled trials, identifying threats to validity, understanding of evidence synthesis approaches, and evaluating the quality of CER (all P values < 0.001). The questionnaire demonstrated acceptable reliability and validity evidence (limited evidence of construct under-representation and construct irrelevant variance). The CER educational program was effective in increasing participants' CER knowledge and self-perceived ability to evaluate relevant evidence. Improving knowledge and awareness of CER and its applicability is a critical first step in improving its use in health care decision making. Copyright © 2014 Elsevier Inc. All rights reserved.

Construct validity of the individual work performance questionnaire.

PubMed

Koopmans, Linda; Bernaards, Claire M; Hildebrandt, Vincent H; de Vet, Henrica C W; van der Beek, Allard J

2014-03-01

To examine the construct validity of the Individual Work Performance Questionnaire (IWPQ). A total of 1424 Dutch workers from three occupational sectors (blue, pink, and white collar) participated in the study. First, IWPQ scores were correlated with related constructs (convergent validity). Second, differences between known groups were tested (discriminative validity). First, IWPQ scores correlated weakly to moderately with absolute and relative presenteeism, and work engagement. Second, significant differences in IWPQ scores were observed for workers differing in job satisfaction, and workers differing in health. Overall, the results indicate acceptable construct validity of the IWPQ. Researchers are provided with a reliable and valid instrument to measure individual work performance comprehensively and generically, among workers from different occupational sectors, with and without health problems.

Validity of three clinical performance assessments of internal medicine clerks.

PubMed

Hull, A L; Hodder, S; Berger, B; Ginsberg, D; Lindheim, N; Quan, J; Kleinhenz, M E

1995-06-01

To analyze the construct validity of three methods to assess the clinical performances of internal medicine clerks. A multitrait-multimethod (MTMM) study was conducted at the Case Western Reserve University School of Medicine to determine the convergent and divergent validity of a clinical evaluation form (CEF) completed by faculty and residents, an objective structured clinical examination (OSCE), and the medicine subject test of the National Board of Medical Examiners. Three traits were involved in the analysis: clinical skills, knowledge, and personal characteristics. A correlation matrix was computed for 410 third-year students who completed the clerkship between August 1988 and July 1991. There was a significant (p < .01) convergence of the four correlations that assessed the same traits by using different methods. However, the four convergent correlations were of moderate magnitude (ranging from .29 to .47). Divergent validity was assessed by comparing the magnitudes of the convergence correlations with the magnitudes of correlations among unrelated assessments (i.e., different traits by different methods). Seven of nine possible coefficients were smaller than the convergent coefficients, suggesting evidence of divergent validity. A significant CEF method effect was identified. There was convergent validity and some evidence of divergent validity with a significant method effect. The findings were similar for correlations corrected for attenuation. Four conclusions were reached: (1) the reliability of the OSCE must be improved, (2) the CEF ratings must be redesigned to further discriminate among the specific traits assessed, (3) additional methods to assess personal characteristics must be instituted, and (4) several assessment methods should be used to evaluate individual student performances.

Types of Anxiety and Depression: Theoretical Assumptions and Development of the Anxiety and Depression Questionnaire

PubMed Central

Fajkowska, Małgorzata; Domaradzka, Ewa; Wytykowska, Agata

2018-01-01

The present paper is addressed to (1) the validation of a recently proposed typology of anxiety and depression, and (2) the presentation of a new tool—the Anxiety and Depression Questionnaire (ADQ)—based on this typology. Empirical data collected across two stages—construction and validation—allowed us to offer the final form of the ADQ, designed to measure arousal anxiety, apprehension anxiety, valence depression, anhedonic depression, and mixed types of anxiety and depression. The results support the proposed typology of anxiety and depression and provide evidence that the ADQ is a reliable and valid self-rating measure of affective types, and accordingly its use in scientific research is recommended. PMID:29410638

Development of the Perioperative Nursing Data Set.

PubMed

Kleinbeck, S V

1999-07-01

Nursing practice is a major component of health care. Yet, it remains undervalued and essentially invisible because little data exist to substantiate the influence of nurses on patient outcomes. The research-based Perioperative Nursing Data Set (PNDS), with an easily automated nomenclature capable of describing the specialty practice of perioperative nursing, was designed to fill this gap. Four domains (i.e., safety, physiologic response to surgery, patient and family behavioral response to surgery, health system) form the foundation of the PNDS. Each domain, with accompanying desired outcomes, nursing interventions, and nursing diagnoses, has reliability, content validity, and evidence of construct validity. The purpose of this article is to introduce the conceptual framework, taxonomy, and potential clinical applications of the PNDS.

Measuring teacher self-report on classroom practices: Construct validity and reliability of the Classroom Strategies Scale-Teacher Form.

PubMed

Reddy, Linda A; Dudek, Christopher M; Fabiano, Gregory A; Peters, Stephanie

2015-12-01

This article presents information about the construct validity and reliability of a new teacher self-report measure of classroom instructional and behavioral practices (the Classroom Strategies Scales-Teacher Form; CSS-T). The theoretical underpinnings and empirical basis for the instructional and behavioral management scales are presented. Information is provided about the construct validity, internal consistency, test-retest reliability, and freedom from item-bias of the scales. Given previous investigations with the CSS Observer Form, it was hypothesized that internal consistency would be adequate and that confirmatory factor analyses (CFA) of CSS-T data from 293 classrooms would offer empirical support for the CSS-T's Total, Composite and subscales, and yield a similar factor structure to that of the CSS Observer Form. Goodness-of-fit indices of χ2/df, Root Mean Square Error of Approximation, Goodness of Fit Index, and Adjusted Goodness of Fit Index suggested satisfactory fit of proposed CFA models whereas the Comparative Fit Index did not. Internal consistency estimates of .93 and .94 were obtained for the Instructional Strategies and Behavioral Strategies Total scales respectively. Adequate test-retest reliability was found for instructional and behavioral total scales (r = .79, r = .84, percent agreement 93% and 93%). The CSS-T evidences freedom from item bias on important teacher demographics (age, educational degree, and years of teaching experience). Implications of results are discussed. (c) 2015 APA, all rights reserved).

Development and Construct Validation of an Academic Emotions Scale

ERIC Educational Resources Information Center

Govaerts, Sophie; Gregoire, Jacques

2008-01-01

This article describes the development and two studies on the construct validity of the Academic Emotions Scale (AES). The AES is a French self-report questionnaire assessing six emotions in the context of school learning: enjoyment, hope, pride, anxiety, shame and frustration. Its construct validity was studied through exploratory and…

Construction and Validation of a Professional Suitability Scale for Social Work Practice

ERIC Educational Resources Information Center

Tam, Dora M. Y.; Coleman, Heather

2009-01-01

This article reports on the construction and validation of a professional suitability scale, designed for assessing students' suitability for social work practice. Data were collected from 188 field supervisors who provided usable questionnaires, representing a response rate of 74%. Construct validation by exploratory factor analysis identified a…