Hopes and Cautions for Instrument-Based Evaluation of Consent Capacity: Results of a Construct Validity Study of Three Instruments

PubMed Central

Moye, Jennifer; Azar, Annin R.; Karel, Michele J.; Gurrera, Ronald J.

2016-01-01

Does instrument based evaluation of consent capacity increase the precision and validity of competency assessment or does ostensible precision provide a false sense of confidence without in fact improving validity? In this paper we critically examine the evidence for construct validity of three instruments for measuring four functional abilities important in consent capacity: understanding, appreciation, reasoning, and expressing a choice. Instrument based assessment of these abilities is compared through investigation of a multi-trait multi-method matrix in 88 older adults with mild to moderate dementia. Results find variable support for validity. There appears to be strong evidence for good hetero-method validity for the measurement of understanding, mixed evidence for validity in the measurement of reasoning, and strong evidence for poor hetero-method validity for the concepts of appreciation and expressing a choice, although the latter is likely due to extreme range restrictions. The development of empirically based tools for use in capacity evaluation should ultimately enhance the reliability and validity of assessment, yet clearly more research is needed to define and measure the constructs of decisional capacity. We would also emphasize that instrument based assessment of capacity is only one part of a comprehensive evaluation of competency which includes consideration of diagnosis, psychiatric and/or cognitive symptomatology, risk involved in the situation, and individual and cultural differences. PMID:27330455

Design and validation of the INICIARE instrument, for the assessment of dependency level in acutely ill hospitalised patients.

PubMed

Morales-Asencio, José Miguel; Porcel-Gálvez, Ana María; Oliveros-Valenzuela, Rosa; Rodríguez-Gómez, Susana; Sánchez-Extremera, Lucrecia; Serrano-López, Francisco Andrés; Aranda-Gallardo, Marta; Canca-Sánchez, José Carlos; Barrientos-Trigo, Sergio

2015-03-01

The aim of this study was to establish the validity and reliability of an instrument (Inventario del NIvel de Cuidados mediante IndicAdores de clasificación de Resultados de Enfermería) used to assess the dependency level in acutely hospitalised patients. This instrument is novel, and it is based on the Nursing Outcomes Classification. Multiple existing instruments for needs assessment have been poorly validated and based predominately on interventions. Standardised Nursing Languages offer an ideal framework to develop nursing sensitive instruments. A cross-sectional validation study in two acute care hospitals in Spain. This study was implemented in two phases. First, the research team developed the instrument to be validated. In the second phase, the validation process was performed by experts, and the data analysis was conducted to establish the psychometric properties of the instrument. Seven hundred and sixty-one patient ratings performed by nurses were collected during the course of the research study. Data analysis yielded a Cronbach's alpha of 0·91. An exploratory factorial analysis identified three factors (Physiological, Instrumental and Cognitive-behavioural), which explained 74% of the variance. Inventario del NIvel de Cuidados mediante IndicAdores de clasificación de Resultados de Enfermería was demonstrated to be a valid and reliable instrument based on its use in acutely hospitalised patients to assess the level of dependency. Inventario del NIvel de Cuidados mediante IndicAdores de clasificación de Resultados de Enfermería can be used as an assessment tool in hospitalised patients during the nursing process throughout the entire hospitalisation period. It contributes information to support decisions on nursing diagnoses, interventions and outcomes. It also enables data codification in large databases. © 2014 John Wiley & Sons Ltd.

Assessing the overuse of antibiotics in children in Saudi Arabia: validation of the parental perception on antibiotics scale (PAPA scale)

PubMed Central

2013-01-01

Background Antibiotics overuse is a global public health issue influenced by several factors, of which some are parent-related psychosocial factors that can only be measured using valid and reliable psychosocial measurement instruments. The PAPA scale was developed to measure these factors and the content validity of this instrument was assessed. Aim This study further validated the recently developed instrument in terms of (1) face validity and (2) construct validity including: deciding the number and nature of factors, and item selection. Methods Questionnaires were self-administered to parents of children between the ages of 0 and 12 years old. Parents were conveniently recruited from schools’ parental meetings in the Eastern Province, Saudi Arabia. Face validity was assessed with regards to questionnaire clarity and unambiguity. Construct validity and item selection processes were conducted using Exploratory factor analysis. Results Parallel analysis and Exploratory factor analysis using principal axis factoring produced six factors in the developed instrument: knowledge and beliefs, behaviours, sources of information, adherence, awareness about antibiotics resistance, and parents’ perception regarding doctors’ prescribing behaviours. Reliability was assessed (Cronbach’s alpha = 0.78) which demonstrates the instrument as being reliable. Conclusion The ‘factors’ produced in this study coincide with the constructs contextually identified in the development phase of other instruments used to study antibiotic use. However, no other study considering perceptions of antibiotic use had gone beyond content validation of such instruments. This study is the first to constructively validate the factors underlying perceptions regarding antibiotic use in any population and in parents in particular. PMID:23497151

Validity and reliability of the Turkish version of the pressure ulcer prevention knowledge assessment instrument.

PubMed

Tulek, Zeliha; Polat, Cansu; Ozkan, Ilknur; Theofanidis, Dimitris; Togrol, Rifat Erdem

2016-11-01

Sound knowledge of pressure ulcers is important to enable good prevention. There are limited instruments assessing pressure ulcer knowledge. The Pressure Ulcer Prevention Knowledge Assessment Instrument is among the scales of which psychometric properties have been studied rigorously and reflects the latest evidence. This study aimed to evaluate the validity and reliability of the Turkish version of the Pressure Ulcer Prevention Knowledge Assessment Instrument (PUPKAI-T), an instrument that assesses knowledge of pressure ulcer prevention by using multiple-choice questions. Linguistic validity was verified through front-to-back translation. Psychometric properties of the instrument were studied on a sample of 150 nurses working in a tertiary hospital in Istanbul, Turkey. The content validity index of the translated instrument was 0.94, intra-class correlation coefficients were between 0.37 and 0.80, item difficulty indices were between 0.21 and 0.88, discrimination indices were 0.20-0.78, and the Kuder Richardson for the internal consistency was 0.803. The PUPKAI-T was found to be a valid and reliable tool to evaluate nurses' knowledge on pressure ulcer prevention. The PUPKAI-T may be a useful tool for determining educational needs of nurses on pressure ulcer prevention. Copyright © 2016 Tissue Viability Society. Published by Elsevier Ltd. All rights reserved.

The Development and Validation of a Concise Instrument for Formative Assessment of Team Leader Performance During Simulated Pediatric Resuscitations.

PubMed

Nadkarni, Lindsay D; Roskind, Cindy G; Auerbach, Marc A; Calhoun, Aaron W; Adler, Mark D; Kessler, David O

2018-04-01

The aim of this study was to assess the validity of a formative feedback instrument for leaders of simulated resuscitations. This is a prospective validation study with a fully crossed (person × scenario × rater) study design. The Concise Assessment of Leader Management (CALM) instrument was designed by pediatric emergency medicine and graduate medical education experts to be used off the shelf to evaluate and provide formative feedback to resuscitation leaders. Four experts reviewed 16 videos of in situ simulated pediatric resuscitations and scored resuscitation leader performance using the CALM instrument. The videos consisted of 4 pediatric emergency department resuscitation teams each performing in 4 pediatric resuscitation scenarios (cardiac arrest, respiratory arrest, seizure, and sepsis). We report on content and internal structure (reliability) validity of the CALM instrument. Content validity was supported by the instrument development process that involved professional experience, expert consensus, focused literature review, and pilot testing. Internal structure validity (reliability) was supported by the generalizability analysis. The main component that contributed to score variability was the person (33%), meaning that individual leaders performed differently. The rater component had almost zero (0%) contribution to variance, which implies that raters were in agreement and argues for high interrater reliability. These results provide initial evidence to support the validity of the CALM instrument as a reliable assessment instrument that can facilitate formative feedback to leaders of pediatric simulated resuscitations.

The Faculty Self-Reported Assessment Survey (FRAS): Differentiating Faculty Knowledge and Experience in Assessment

PubMed Central

Hanauer, David I.; Bauerle, Cynthia

2015-01-01

Science, technology, engineering, and mathematics education reform efforts have called for widespread adoption of evidence-based teaching in which faculty members attend to student outcomes through assessment practice. Awareness about the importance of assessment has illuminated the need to understand what faculty members know and how they engage with assessment knowledge and practice. The Faculty Self-Reported Assessment Survey (FRAS) is a new instrument for evaluating science faculty assessment knowledge and experience. Instrument validation was composed of two distinct studies: an empirical evaluation of the psychometric properties of the FRAS and a comparative known-groups validation to explore the ability of the FRAS to differentiate levels of faculty assessment experience. The FRAS was found to be highly reliable (α = 0.96). The dimensionality of the instrument enabled distinction of assessment knowledge into categories of program design, instrumentation, and validation. In the known-groups validation, the FRAS distinguished between faculty groups with differing levels of assessment experience. Faculty members with formal assessment experience self-reported higher levels of familiarity with assessment terms, higher frequencies of assessment activity, increased confidence in conducting assessment, and more positive attitudes toward assessment than faculty members who were novices in assessment. These results suggest that the FRAS can reliably and validly differentiate levels of expertise in faculty knowledge of assessment. PMID:25976653

Validation of the process criteria for assessment of a hospital nursing service.

PubMed

Feldman, Liliane Bauer; Cunha, Isabel Cristina Kowal Olm; D'Innocenzo, Maria

2013-01-01

to validate an instrument containing process criteria for assessment of a hospital nursing service based on the National Accreditation Organization program. a descriptive, quantitative methodological study performed in stages. An instrument constructed with 69 process criteria was assessed by 49 nurses from accredited hospitals in 2009, according to a Likert scale, and validated by 16 judges through Delphi rounds in 2010. the original instrument assessed by nurses with 69 process criteria was judged by the degree of importance, and changed to 39 criteria. In the first Delphi round, the 39 criteria reached consensus among the 19 judges, with a medium reliability by Cronbach's alpha. In the second round, 40 converging criteria were validated by 16 judges, with high reliability. The criteria addressed management, costs, teaching, education, indicators, protocols, human resources, communication, among others. the 40 process criteria formed a validated instrument to assess the hospital nursing service which, when measured, can better direct interventions by nurses in reaching and strengthening outcomes.

Development of an instrument to measure medical students' perceptions of the assessment environment: initial validation.

PubMed

Sim, Joong Hiong; Tong, Wen Ting; Hong, Wei-Han; Vadivelu, Jamuna; Hassan, Hamimah

2015-01-01

Assessment environment, synonymous with climate or atmosphere, is multifaceted. Although there are valid and reliable instruments for measuring the educational environment, there is no validated instrument for measuring the assessment environment in medical programs. This study aimed to develop an instrument for measuring students' perceptions of the assessment environment in an undergraduate medical program and to examine the psychometric properties of the new instrument. The Assessment Environment Questionnaire (AEQ), a 40-item, four-point (1=Strongly Disagree to 4=Strongly Agree) Likert scale instrument designed by the authors, was administered to medical undergraduates from the authors' institution. The response rate was 626/794 (78.84%). To establish construct validity, exploratory factor analysis (EFA) with principal component analysis and varimax rotation was conducted. To examine the internal consistency reliability of the instrument, Cronbach's α was computed. Mean scores for the entire AEQ and for each factor/subscale were calculated. Mean AEQ scores of students from different academic years and sex were examined. Six hundred and eleven completed questionnaires were analysed. EFA extracted four factors: feedback mechanism (seven items), learning and performance (five items), information on assessment (five items), and assessment system/procedure (three items), which together explained 56.72% of the variance. Based on the four extracted factors/subscales, the AEQ was reduced to 20 items. Cronbach's α for the 20-item AEQ was 0.89, whereas Cronbach's α for the four factors/subscales ranged from 0.71 to 0.87. Mean score for the AEQ was 2.68/4.00. The factor/subscale of 'feedback mechanism' recorded the lowest mean (2.39/4.00), whereas the factor/subscale of 'assessment system/procedure' scored the highest mean (2.92/4.00). Significant differences were found among the AEQ scores of students from different academic years. The AEQ is a valid and reliable instrument. Initial validation supports its use to measure students' perceptions of the assessment environment in an undergraduate medical program.

Register of Validated Short Dietary Assessment Instruments

Cancer.gov

The register contains descriptive information about the instruments identified (over 135) along with any associated validation studies and publications, and copies of the instruments themselves when available.

Psychometric Validation of the Academic Motivation Scale in a Dental Student Sample.

PubMed

Orsini, Cesar; Binnie, Vivian; Evans, Phillip; Ledezma, Priscilla; Fuentes, Fernando; Villegas, Maria J

2015-08-01

The Academic Motivation Scale is one of the most frequently used instruments to assess academic motivation. It relies on the self-determination theory of human motivation. However, motivation has been understudied in dental education. Therefore, to address the lack of valid instruments to assess academic motivation in dental education and contribute to future research in the field, the aim of this study was to analyze the psychometric properties of this instrument in a sample of dental students. Participants were 989 Chilean undergraduate dental students (86% response rate) who completed a survey containing a Chilean face-valid version of the Spanish Academic Motivation Scale and three other motivation-related instruments to assess the survey's construct and criterion validity. Later, 76 of the students (out of 100 invited) took the survey again to assess its test-retest stability. The instrument's construct validity was supported by the superior goodness of fit of the seven-subscale Academic Motivation Scale over competing models through confirmatory factor analysis and by the expected correlations among its subscales. The concurrent criterion validity was supported by the confirmation of correlations between its subscales and external criteria. Adequate internal consistency and test-retest correlations were also found. The evidence from this study suggests that the Academic Motivation Scale is a preliminarily valid and reliable instrument to assess motivation in the predoctoral dental context. Future research in this area is needed to confirm or refute these results.

School Age Populations Research Needs - NCS Dietary Assessment Literature Review

Cancer.gov

Drawing conclusions about the validity of available dietary assessment instruments in school age children is hampered by the differences in instruments, research design, reference methods, and populations in the validation literature.

Measuring metacognitive ability based on science literacy in dynamic electricity topic

NASA Astrophysics Data System (ADS)

Warni; Sunyono; Rosidin

2018-01-01

This study aims to produce an instrument of metacognition ability assessment based on science literacy on theoretically and empirically feasible dynamic electrical material. The feasibility of the assessment instrument includes theoretical validity on material, construction, and language aspects, as well as empirical validity, reliability, difficulty, distinguishing, and distractor indices. The development of assessment instruments refers to the Dick and Carey development model which includes the preliminary study stage, initial product development, validation and revision, and piloting. The instrument was tested to 32 students of class IX in SMP Negeri 20 Bandar Lampung, using the design of One Group Pretest-Postest Design. The result shows that the metacognition ability assessment instrument based on science literacy is feasible theoretically with theoretical validity percentage of 95.44% and empirical validity of 43.75% for the high category, 43.75% for the medium category, and 12.50 % for low category questions; Reliability of assessment instruments of 0.83 high categories; Difficulty level of difficult item is about 31.25% and medium category is equal to 68.75%. Item that has very good distinguishing power is 12.50%, 62.50% for good stage, and medium category is 25.00%; As well as the duplexing function on a matter of multiple choice is 80.00% including good category and 20.00% for medium category.

Development of self and peer performance assessment on iodometric titration experiment

NASA Astrophysics Data System (ADS)

Nahadi; Siswaningsih, W.; Kusumaningtyas, H.

2018-05-01

This study aims to describe the process in developing of reliable and valid assessment to measure students’ performance on iodometric titration and the effect of the self and peer assessment on students’ performance. The self and peer-instrument provides valuable feedback for the student performance improvement. The developed assessment contains rubric and task for facilitating self and peer assessment. The participants are 24 students at the second-grade student in certain vocational high school in Bandung. The participants divided into two groups. The first 12 students involved in the validity test of the developed assessment, while the remain 12 students participated for the reliability test. The content validity was evaluated based on the judgment experts. Test result of content validity based on judgment expert show that the developed performance assessment instrument categorized as valid on each task with the realibity classified as very good. Analysis of the impact of the self and peer assessment implementation showed that the peer instrument supported the self assessment.

Group power through the lens of the 21st century and beyond: further validation of the Sieloff-King Assessment of Group Power within Organizations.

PubMed

Sieloff, Christina L; Bularzik, Anne M

2011-11-01

The purpose was to determine the content validity of a semantic revision of items on a reliable and valid instrument, the Sieloff-King Assessment of Group Power within Organizations (SKAGPO). Research participants expressed negative perceptions regarding the use of the concept of 'power' in SKAGPO items. The SKAGPO is the only instrument measuring a nursing group's power or outcome attainment. Using a survey method, the instrument and grading scale were sent to 12 expert judges. Six participants completed the grading scale. The Content Validity Index (CVI) for seven questions was at or above 83% agreement. Overall, the CVI for the eight revised questions was 93.75%. Subsequently, the instrument was renamed the Sieloff-King Assessment of Group Outcome Attainment within Organizations (SKAGOAO). The semantic revision demonstrated content validity for the revised SKAGOAO. When used by nursing groups to assess their level of outcome attainment, the instrument should continue to be psychometrically evaluated. A nursing group of any size can use the SKAGOAO to both assess the group's level of outcome attainment or empowerment and direct plans to further improve that level. © 2011 Blackwell Publishing Ltd.

Trainees' Perceptions of Feedback: Validity Evidence for Two FEEDME (Feedback in Medical Education) Instruments.

PubMed

Bing-You, Robert; Ramesh, Saradha; Hayes, Victoria; Varaklis, Kalli; Ward, Denham; Blanco, Maria

2018-01-01

Construct: Medical educators consider feedback a core component of the educational process. Effective feedback allows learners to acquire new skills, knowledge, and attitudes. Learners' perceptions of feedback are an important aspect to assess with valid methods in order to improve the feedback skills of educators and the feedback culture. Although guidelines for delivering effective feedback have existed for several decades, medical students and residents often indicate that they receive little feedback. A recent scoping review on feedback in medical education did not reveal any validity evidence on instruments to assess learner's perceptions of feedback. The purpose of our study was to gather validity evidence on two novel FEEDME (Feedback in Medical Education) instruments to assess medical students' and residents' perceptions of the feedback that they receive. After the authors developed an initial instrument with 54 items, cognitive interviews with medical students and residents suggested that 2 separate instruments were needed, one focused on the feedback culture (FEEDME-Culture) and the other on the provider of feedback (FEEDME-Provider). A Delphi study with 17 medical education experts and faculty members assessed content validity. The response process was explored involving 31 medical students and residents at 2 academic institutions. Exploratory factor analysis and reliability analyses were performed on completed instruments. Two Delphi consultation rounds refined the wording of items and eliminated several items. Learners found both instruments easy and quick to answer; it took them less than 5 minutes to complete. Learners preferred an electronic format of the instruments over paper. Factor analysis revealed a two- and three-factor solution for the FEEDME-Culture and FEEDME-Provider instruments, respectively. Cronbach's alpha was greater than 0.80 for all factors. Items on both instruments were moderately to highly correlated (range, r = .3-.7). Our results provide preliminary validity evidence of 2 novel feedback instruments. After further validation of both FEEDME instruments, sharing the results of the FEEDME-Culture instrument with educational leaders and faculty may improve the culture of feedback on specific educational rotations and at the institutional level. The FEEDME-Provider instrument could be useful for faculty development targeting feedback skills. Additional research studies could assess whether both instruments may be used to help learners receive feedback and prompt reflective learning.

Developing and testing instruments for improving cooperation and patient's participation in mental health care.

PubMed

Latvala, E; Saranto, K; Pekkala, E

2004-10-01

The main purpose of the project was to develop computerized instruments that could be used by nurses and patients to assess their cooperation and mutual contributions to care. This paper presents a part of the project: the reliability and validity testing phase of a process of instrument development. To test the validity and reliability of the instruments, data were collected with questionnaires from nurses (n = 146) and patients (n = 286). The validity evaluated as construct validity and the reliability evaluated as internal consistency of the instruments were quite good. Construct validity was tested by factor analysis, and internal consistency was tested by Cronbach's alpha coefficient, which varied from 0.69 to 0.79. The instruments, which consisted of a software application that can be operated in a www environment, were meant to be used as tools in the psychiatric nursing context for assessing the cooperation between the nurses and patients and the patient's participation in his/her care. Furthermore, the computer programme can be used as a tool for developing and assessing the patient orientation in nursing.

Opportunity integrated assessment facilitating critical thinking and science process skills measurement on acid base matter

NASA Astrophysics Data System (ADS)

Sari, Anggi Ristiyana Puspita; Suyanta, LFX, Endang Widjajanti; Rohaeti, Eli

2017-05-01

Recognizing the importance of the development of critical thinking and science process skills, the instrument should give attention to the characteristics of chemistry. Therefore, constructing an accurate instrument for measuring those skills is important. However, the integrated instrument assessment is limited in number. The purpose of this study is to validate an integrated assessment instrument for measuring students' critical thinking and science process skills on acid base matter. The development model of the test instrument adapted McIntire model. The sample consisted of 392 second grade high school students in the academic year of 2015/2016 in Yogyakarta. Exploratory Factor Analysis (EFA) was conducted to explore construct validity, whereas content validity was substantiated by Aiken's formula. The result shows that the KMO test is 0.714 which indicates sufficient items for each factor and the Bartlett test is significant (a significance value of less than 0.05). Furthermore, content validity coefficient which is based on 8 experts is obtained at 0.85. The findings support the integrated assessment instrument to measure critical thinking and science process skills on acid base matter.

Reliability and validity of generalizable skills instruments for students who are deaf, blind, or visually impaired.

PubMed

Loeding, B L; Greenan, J P

1998-12-01

The study examined the validity and reliability of four assessments, with three instruments per domain. Domains included generalizable mathematics, communication, interpersonal relations, and reasoning skills. Participants were deaf, legally blind, or visually impaired students enrolled in vocational classes at residential secondary schools. The researchers estimated the internal consistency reliability, test-retest reliability, and construct validity correlations of three subinstruments: student self-ratings, teacher ratings, and performance assessments. The data suggest that these instruments are highly internally consistent measures of generalizable vocational skills. Four performance assessments have high-to-moderate test-retest reliability estimates, and were generally considered to possess acceptable validity and reliability.

'Mechanical restraint-confounders, risk, alliance score': testing the clinical validity of a new risk assessment instrument.

PubMed

Deichmann Nielsen, Lea; Bech, Per; Hounsgaard, Lise; Alkier Gildberg, Frederik

2017-08-01

Unstructured risk assessment, as well as confounders (underlying reasons for the patient's risk behaviour and alliance), risk behaviour, and parameters of alliance, have been identified as factors that prolong the duration of mechanical restraint among forensic mental health inpatients. To clinically validate a new, structured short-term risk assessment instrument called the Mechanical Restraint-Confounders, Risk, Alliance Score (MR-CRAS), with the intended purpose of supporting the clinicians' observation and assessment of the patient's readiness to be released from mechanical restraint. The content and layout of MR-CRAS and its user manual were evaluated using face validation by forensic mental health clinicians, content validation by an expert panel, and pilot testing within two, closed forensic mental health inpatient units. The three sub-scales (Confounders, Risk, and a parameter of Alliance) showed excellent content validity. The clinical validations also showed that MR-CRAS was perceived and experienced as a comprehensible, relevant, comprehensive, and useable risk assessment instrument. MR-CRAS contains 18 clinically valid items, and the instrument can be used to support the clinical decision-making regarding the possibility of releasing the patient from mechanical restraint. The present three studies have clinically validated a short MR-CRAS scale that is currently being psychometrically tested in a larger study.

Development of student performance assessment based on scientific approach for a basic physics practicum in simple harmonic motion materials

NASA Astrophysics Data System (ADS)

Serevina, V.; Muliyati, D.

2018-05-01

This research aims to develop students’ performance assessment instrument based on scientific approach is valid and reliable in assessing the performance of students on basic physics lab of Simple Harmonic Motion (SHM). This study uses the ADDIE consisting of stages: Analyze, Design, Development, Implementation, and Evaluation. The student performance assessment developed can be used to measure students’ skills in observing, asking, conducting experiments, associating and communicate experimental results that are the ‘5M’ stages in a scientific approach. Each grain of assessment in the instrument is validated by the instrument expert and the evaluation with the result of all points of assessment shall be eligible to be used with a 100% eligibility percentage. The instrument is then tested for the quality of construction, material, and language by panel (lecturer) with the result: 85% or very good instrument construction aspect, material aspect 87.5% or very good, and language aspect 83% or very good. For small group trial obtained instrument reliability level of 0.878 or is in the high category, where r-table is 0.707. For large group trial obtained instrument reliability level of 0.889 or is in the high category, where r-table is 0.320. Instruments declared valid and reliable for 5% significance level. Based on the result of this research, it can be concluded that the student performance appraisal instrument based on the developed scientific approach is declared valid and reliable to be used in assessing student skill in SHM experimental activity.

Assessing the Validity and Reliability of the Peristomal Skin Lesion Assessment Instrument Adapted for Use in Turkey.

PubMed

Ay, Ali; Bulut, Hulya

2015-08-01

Many ostomy patients experience peristomal skin lesions. A descriptive study was conducted to assess the validity, usability, and reliability of the Peristomal Skin Lesions Assessment instrument (SACS instrument) adapted to Turkish from English. The SACS Instrument consists of 2 main assessments: lesion type (utilizing definitions and photographs) and lesion area by location around the ostomy. The study was performed in 2 stages: 1) the SACS language was changed and its content validity established; and 2) the instrument\\'92s content validity and inter-observer agreement (consistency) were determined among pairs of nurses who used the tool to assess peristomal skin lesions. Patients (included if they were >18 years old and receiving treatment/observation at 1 of the 4 participating stomatherapy units) and 8 stomatherapy nurses also completed appropriate sociodemographic questionnaires. Of the 393 patients screened during the 7-month study, 100 (average age 56.74 \\'b1 14.03 years, 55 men) participated; most (79) had a planned operation. A little more than half (59) of the patients had colorectal cancer and 28 had their stoma site marked preoperatively by a stomatherapy nurse. The most common peristomal skin lesion risk factors were having an ileostomy and unplanned surgery. The content validity index of the entire Turkish SACS instrument was 1, and the inter-observer agreement Kappa statistic was very good (K = 0.90, 95% CI 0.80- 0.99). Individual SACS item K values ranged from K = 0.84 (95% CI 0.63\\'961) to K = 1 (95% CI 1). Most (62.5%) nurses found the terms and pictures used in the SACS classification adequate and suitable, and 50% believed the Turkish version of the SACS instrument was a valid and suitable assessment tool for use by Turkish stomatherapy nurses. Validity and reliability studies involving larger and more diverse patient and nurse samples are warranted.

Instruments to Identify Prescription Medication Misuse, Abuse, and Related Events in Clinical Trials: An ACTTION Systematic Review.

PubMed

Smith, Shannon M; Paillard, Florence; McKeown, Andrew; Burke, Laurie B; Edwards, Robert R; Katz, Nathaniel P; Papadopoulos, Elektra J; Rappaport, Bob A; Slagle, Ashley; Strain, Eric C; Wasan, Ajay D; Turk, Dennis C; Dworkin, Robert H

2015-05-01

Measurement of inappropriate medication use events (eg, abuse or misuse) in clinical trials is important in characterizing a medication's abuse potential. However, no gold standard assessment of inappropriate use events in clinical trials has been identified. In this systematic review, we examine the measurement properties (ie, content validity, cross-sectional reliability and construct validity, longitudinal construct validity, ability to detect change, and responder definitions) of instruments assessing inappropriate use of opioid and nonopioid prescription medications to identify any that meet U.S. and European regulatory agencies' rigorous standards for outcome measures in clinical trials. Sixteen published instruments were identified, most of which were not designed for the selected concept of interest and context of use. For this reason, many instruments were found to lack adequate content validity (or documentation of content validity) to evaluate current inappropriate medication use events; for example, evaluating inappropriate use across the life span rather than current use, including items that did not directly assess inappropriate use (eg, questions about anger), or failing to capture information pertinent to inappropriate use events (eg, intention and route of administration). In addition, the psychometric data across all instruments were generally limited in scope. A further limitation is the heterogeneous, nonstandardized use of inappropriate medication use terminology. These observations suggest that available instruments are not well suited for assessing current inappropriate medication use within the specific context of clinical trials. Further effort is needed to develop reliable and valid instruments to measure current inappropriate medication use events in clinical trials. This systematic review evaluates the measurement properties of inappropriate medication use (eg, abuse or misuse) instruments to determine whether any meet regulatory standards for clinical trial outcome measures to assess abuse potential. Copyright © 2015 American Pain Society. All rights reserved.

Development and validation of self-reported line drawings of the modified Beighton score for the assessment of generalised joint hypermobility.

PubMed

Cooper, Dale J; Scammell, Brigitte E; Batt, Mark E; Palmer, Debbie

2018-01-17

The impracticalities and comparative expense of carrying out a clinical assessment is an obstacle in many large epidemiological studies. The purpose of this study was to develop and validate a series of electronic self-reported line drawing instruments based on the modified Beighton scoring system for the assessment of self-reported generalised joint hypermobility. Five sets of line drawings were created to depict the 9-point Beighton score criteria. Each instrument consisted of an explanatory question whereby participants were asked to select the line drawing which best represented their joints. Fifty participants completed the self-report online instrument on two occasions, before attending a clinical assessment. A blinded expert clinical observer then assessed participants' on two occasions, using a standardised goniometry measurement protocol. Validity of the instrument was assessed by participant-observer agreement and reliability by participant repeatability and observer repeatability using unweighted Cohen's kappa (k). Validity and reliability were assessed for each item in the self-reported instrument separately, and for the sum of the total scores. An aggregate score for generalised joint hypermobility was determined based on a Beighton score of 4 or more out of 9. Observer-repeatability between the two clinical assessments demonstrated perfect agreement (k 1.00; 95% CI 1.00, 1.00). Self-reported participant-repeatability was lower but it was still excellent (k 0.91; 95% CI 0.74, 1.00). The participant-observer agreement was excellent (k 0.96; 95% CI 0.87, 1.00). Validity was excellent for the self-report instrument, with a good sensitivity of 0.87 (95% CI 0.81, 0.91) and excellent specificity of 0.99 (95% CI 0.98, 1.00). The self-reported instrument provides a valid and reliable assessment of the presence of generalised joint hypermobility and may have practical use in epidemiological studies.

Older adult mistreatment risk screening: contribution to the validation of a screening tool in a domestic setting.

PubMed

Lindenbach, Jeannette M; Larocque, Sylvie; Lavoie, Anne-Marise; Garceau, Marie-Luce

2012-06-01

ABSTRACTThe hidden nature of older adult mistreatment renders its detection in the domestic setting particularly challenging. A validated screening instrument that can provide a systematic assessment of risk factors can facilitate this detection. One such instrument, the "expanded Indicators of Abuse" tool, has been previously validated in the Hebrew language in a hospital setting. The present study has contributed to the validation of the "e-IOA" in an English-speaking community setting in Ontario, Canada. It consisted of two phases: (a) a content validity review and adaptation of the instrument by experts throughout Ontario, and (b) an inter-rater reliability assessment by home visiting nurses. The adaptation, the "Mistreatment of Older Adult Risk Factors" tool, offers a comprehensive tool for screening in the home setting. This instrument is significant to professional practice as practitioners working with older adults will be better equipped to assess for risk of mistreatment.

Development and Validation of a Consumer Quality Assessment Instrument for Dentistry.

ERIC Educational Resources Information Center

Johnson, Jeffrey D.; And Others

1990-01-01

This paper reviews the literature on consumer involvement in dental quality assessment, argues for inclusion of this information in quality assessment measures, outlines a conceptual model for measuring dental consumer quality assessment, and presents data relating to the development and validation of an instrument based on the conceptual model.…

The quality of instruments to assess the process of shared decision making: A systematic review.

PubMed

Gärtner, Fania R; Bomhof-Roordink, Hanna; Smith, Ian P; Scholl, Isabelle; Stiggelbout, Anne M; Pieterse, Arwen H

2018-01-01

To inventory instruments assessing the process of shared decision making and appraise their measurement quality, taking into account the methodological quality of their validation studies. In a systematic review we searched seven databases (PubMed, Embase, Emcare, Cochrane, PsycINFO, Web of Science, Academic Search Premier) for studies investigating instruments measuring the process of shared decision making. Per identified instrument, we assessed the level of evidence separately for 10 measurement properties following a three-step procedure: 1) appraisal of the methodological quality using the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist, 2) appraisal of the psychometric quality of the measurement property using three possible quality scores, 3) best-evidence synthesis based on the number of studies, their methodological and psychometrical quality, and the direction and consistency of the results. The study protocol was registered at PROSPERO: CRD42015023397. We included 51 articles describing the development and/or evaluation of 40 shared decision-making process instruments: 16 patient questionnaires, 4 provider questionnaires, 18 coding schemes and 2 instruments measuring multiple perspectives. There is an overall lack of evidence for their measurement quality, either because validation is missing or methods are poor. The best-evidence synthesis indicated positive results for a major part of instruments for content validity (50%) and structural validity (53%) if these were evaluated, but negative results for a major part of instruments when inter-rater reliability (47%) and hypotheses testing (59%) were evaluated. Due to the lack of evidence on measurement quality, the choice for the most appropriate instrument can best be based on the instrument's content and characteristics such as the perspective that they assess. We recommend refinement and validation of existing instruments, and the use of COSMIN-guidelines to help guarantee high-quality evaluations.

A systematic review of instruments to assess organizational readiness for knowledge translation in health care.

PubMed

Gagnon, Marie-Pierre; Attieh, Randa; Ghandour, El Kebir; Légaré, France; Ouimet, Mathieu; Estabrooks, Carole A; Grimshaw, Jeremy

2014-01-01

The translation of research into practices has been incomplete. Organizational readiness for change (ORC) is a potential facilitator of effective knowledge translation (KT). However we know little about the best way to assess ORC. Therefore, we sought to systematically review ORC measurement instruments. We searched for published studies in bibliographic databases (Pubmed, Embase, CINAHL, PsychINFO, Web of Science, etc.) up to November 1st, 2012. We included publications that developed ORC measures and/or empirically assessed ORC using an instrument at the organizational level in the health care context. We excluded articles if they did not refer specifically to ORC, did not concern the health care domain or were limited to individual-level change readiness. We focused on identifying the psychometric properties of instruments that were developed to assess readiness in an organization prior to implementing KT interventions in health care. We used the Standards for Educational and Psychological Testing to assess the psychometric properties of identified ORC measurement instruments. We found 26 eligible instruments described in 39 publications. According to the Standards for Educational and Psychological Testing, 18 (69%) of a total of 26 measurement instruments presented both validity and reliability criteria. The Texas Christian University -ORC (TCU-ORC) scale reported the highest instrument validity with a score of 4 out of 4. Only one instrument, namely the Modified Texas Christian University - Director version (TCU-ORC-D), reported a reliability score of 2 out of 3. No information was provided regarding the reliability and validity of five (19%) instruments. Our findings indicate that there are few valid and reliable ORC measurement instruments that could be applied to KT in the health care sector. The TCU-ORC instrument presents the best evidence in terms of validity testing. Future studies using this instrument could provide more knowledge on its relevance to diverse clinical contexts.

A Systematic Review of Instruments to Assess Organizational Readiness for Knowledge Translation in Health Care

PubMed Central

Gagnon, Marie-Pierre; Attieh, Randa; Ghandour, El Kebir; Légaré, France; Ouimet, Mathieu; Estabrooks, Carole A.; Grimshaw, Jeremy

2014-01-01

Background The translation of research into practices has been incomplete. Organizational readiness for change (ORC) is a potential facilitator of effective knowledge translation (KT). However we know little about the best way to assess ORC. Therefore, we sought to systematically review ORC measurement instruments. Methods We searched for published studies in bibliographic databases (Pubmed, Embase, CINAHL, PsychINFO, Web of Science, etc.) up to November 1st, 2012. We included publications that developed ORC measures and/or empirically assessed ORC using an instrument at the organizational level in the health care context. We excluded articles if they did not refer specifically to ORC, did not concern the health care domain or were limited to individual-level change readiness. We focused on identifying the psychometric properties of instruments that were developed to assess readiness in an organization prior to implementing KT interventions in health care. We used the Standards for Educational and Psychological Testing to assess the psychometric properties of identified ORC measurement instruments. Findings We found 26 eligible instruments described in 39 publications. According to the Standards for Educational and Psychological Testing, 18 (69%) of a total of 26 measurement instruments presented both validity and reliability criteria. The Texas Christian University –ORC (TCU-ORC) scale reported the highest instrument validity with a score of 4 out of 4. Only one instrument, namely the Modified Texas Christian University – Director version (TCU-ORC-D), reported a reliability score of 2 out of 3. No information was provided regarding the reliability and validity of five (19%) instruments. Conclusion Our findings indicate that there are few valid and reliable ORC measurement instruments that could be applied to KT in the health care sector. The TCU-ORC instrument presents the best evidence in terms of validity testing. Future studies using this instrument could provide more knowledge on its relevance to diverse clinical contexts. PMID:25474622

Reliability and Validity of Three Instruments (DSM-IV, CPGI, and PPGM) in the Assessment of Problem Gambling in South Korea.

PubMed

Back, Ki-Joon; Williams, Robert J; Lee, Choong-Ki

2015-09-01

Most research on the assessment, epidemiology, and treatment of problem gambling has occurred in Western jurisdictions. This potentially limits the cross-cultural validity of problem gambling assessment instruments as well as etiological models of problem gambling. The primary objective of the present research was to investigate the reliability and validity of three problem gambling assessment instruments within a South Korean context. A total of 4,330 South Korean adults participated in a comprehensive assessment of their gambling behavior that included the administration of the DSM-IV criteria for pathological gambling (NODS), the Canadian Problem Gambling Index (CPGI), and the Problem and Pathological Gambling Measure (PPGM). Cronbach alpha showed that all three instruments had good internal consistency. Concurrent validity was established by the significant associations observed between scores on the instruments and measures of gambling involvement (number of gambling formats engaged in; frequency of gambling; and gambling expenditure). Most importantly, kappa statistics showed that all instruments have satisfactory classification accuracy against clinical assessment of problem gambling conducted by South Korean clinicians (NODS κ = .66; PPGM κ = .62; CPGI κ = .51). These results confirm that Western-derived operationalizations of problem gambling have applicability in a South Korean setting.

Adolescent Populations Research Needs - NCS Dietary Assessment Literature Review

Cancer.gov

As with school age children, it is difficult to make conclusions about the validity of available dietary assessment instruments for adolescents because of the differences in instruments, research designs, reference methods, and populations in the validation literature.

A Comparative Study of Adolescent Risk Assessment Instruments: Predictive and Incremental Validity

ERIC Educational Resources Information Center

Welsh, Jennifer L.; Schmidt, Fred; McKinnon, Lauren; Chattha, H. K.; Meyers, Joanna R.

2008-01-01

Promising new adolescent risk assessment tools are being incorporated into clinical practice but currently possess limited evidence of predictive validity regarding their individual and/or combined use in risk assessments. The current study compares three structured adolescent risk instruments, Youth Level of Service/Case Management Inventory…

Screening emergency department patients for opioid drug use: A qualitative systematic review.

PubMed

Sahota, Preet Kaur; Shastry, Siri; Mukamel, Dana B; Murphy, Linda; Yang, Narisu; Lotfipour, Shahram; Chakravarthy, Bharath

2018-05-24

The opioid drug epidemic is a major public health concern and an economic burden in the United States. The purpose of this systematic review is to assess the reliability and validity of screening instruments used in emergency medicine settings to detect opioid use in patients and to assess psychometric data for each screening instrument. PubMed/MEDLINE, PsycINFO, Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, Web of Science, Cumulative Index to Nursing and Allied Health Literature and ClinicalTrials.gov were searched for articles published up to May 2018. The extracted articles were independently screened for eligibility by two reviewers. We extracted 1555 articles for initial screening and 95 articles were assessed for full-text eligibility. Six articles were extracted from the full-text assessment. Six instruments were identified from the final article list: Screener and Opioid Assessment for Patients with Pain - Revised; Drug Abuse Screening Test; Opioid Risk Tool; Current Opioid Misuse Measure; an Emergency Medicine Providers Clinician Assessment Questionnaire; and an Emergency Provider Impression Data Collection Form. Screening instrument characteristics, and reliability and validity data were extracted from the six studies. A meta-analysis was not conducted due to heterogeneity between the studies. There is a lack of validity and reliability evidence in all six articles; and sensitivity, specificity and predictive values varied between the different instruments. These instruments cannot be validated for use in emergency medicine settings. There is no clear evidence to state which screening instruments are appropriate for use in detecting opioid use disorders in emergency medicine patients. There is a need for brief, reliable, valid and feasible opioid use screening instruments in the emergency medicine setting. Copyright © 2018 Elsevier Ltd. All rights reserved.

Assessing students' communication skills: validation of a global rating.

PubMed

Scheffer, Simone; Muehlinghaus, Isabel; Froehmel, Annette; Ortwein, Heiderose

2008-12-01

Communication skills training is an accepted part of undergraduate medical programs nowadays. In addition to learning experiences its importance should be emphasised by performance-based assessment. As detailed checklists have been shown to be not well suited for the assessment of communication skills for different reasons, this study aimed to validate a global rating scale. A Canadian instrument was translated to German and adapted to assess students' communication skills during an end-of-semester-OSCE. Subjects were second and third year medical students at the reformed track of the Charité-Universitaetsmedizin Berlin. Different groups of raters were trained to assess students' communication skills using the global rating scale. Validity testing included concurrent validity and construct validity: Judgements of different groups of raters were compared to expert ratings as a defined gold standard. Furthermore, the amount of agreement between scores obtained with this global rating scale and a different instrument for assessing communication skills was determined. Results show that communication skills can be validly assessed by trained non-expert raters as well as standardised patients using this instrument.

Patient-Reported Outcome Instruments for Surgical and Traumatic Scars: A Systematic Review of their Development, Content, and Psychometric Validation.

PubMed

Mundy, Lily R; Miller, H Catherine; Klassen, Anne F; Cano, Stefan J; Pusic, Andrea L

2016-10-01

Patient-reported outcomes (PROs) are of growing importance in research and clinical care and may be used as primary outcomes or as compliments to traditional surgical outcomes. In assessing the impact of surgical and traumatic scars, PROs are often the most meaningful. To assess outcomes from the patient perspective, rigorously developed and validated PRO instruments are essential. The authors conducted a systematic literature review to identify PRO instruments developed and/or validated for patients with surgical and/or non-burn traumatic scars. Identified instruments were assessed for content, development process, and validation under recommended guidelines for PRO instrument development. The systematic review identified 6534 articles. After review, we identified four PRO instruments meeting inclusion criteria: patient and observer scar assessment scale (POSAS), bock quality of life questionnaire for patients with keloid and hypertrophic scarring (Bock), patient scar assessment questionnaire (PSAQ), and patient-reported impact of scars measure (PRISM). Common concepts measured were symptoms and psychosocial well-being. Only PSAQ had a dedicated appearance domain. Qualitative data were used to inform content for the PSAQ and PRISM, and a modern psychometric approach (Rasch Measurement Theory) was used to develop PRISM and to test POSAS. Overall, PRISM demonstrated the most rigorous design and validation process, however, was limited by the lack of a dedicated appearance domain. PRO instruments to evaluate outcomes in scars exist but vary in terms of concepts measured and psychometric soundness. This review discusses the strengths and weaknesses of existing instruments, highlighting the need for future scar-focused PRO instrument development. This journal requires that authors assign a level of evidence to each article. For a full description of these Evidence-Based Medicine ratings, please refer to Table of Contents or the online Instructions to Authors www.springer.com/00266 .

Reliability and validity evidence of the Assessment of Language Use in Social Contexts for Adults (ALUSCA).

PubMed

Valente, Ana Rita S; Hall, Andreia; Alvelos, Helena; Leahy, Margaret; Jesus, Luis M T

2018-04-12

The appropriate use of language in context depends on the speaker's pragmatic language competencies. A coding system was used to develop a specific and adult-focused self-administered questionnaire to adults who stutter and adults who do not stutter, The Assessment of Language Use in Social Contexts for Adults, with three categories: precursors, basic exchanges, and extended literal/non-literal discourse. This paper presents the content validity, item analysis, reliability coefficients and evidences of construct validity of the instrument. Content validity analysis was based on a two-stage process: first, 11 pragmatic questionnaires were assessed to identify items that probe each pragmatic competency and to create the first version of the instrument; second, items were assessed qualitatively by an expert panel composed by adults who stutter and controls, and quantitatively and qualitatively by an expert panel composed by clinicians. A pilot study was conducted with five adults who stutter and five controls to analyse items and calculate reliability. Construct validity evidences were obtained using the hypothesized relationships method and factor analysis with 28 adults who stutter and 28 controls. Concerning content validity, the questionnaires assessed up to 13 pragmatic competencies. Qualitative and quantitative analysis revealed ambiguities in items construction. Disagreement between experts was solved through item modification. The pilot study showed that the instrument presented internal consistency and temporal stability. Significant differences between adults who stutter and controls and different response profiles revealed the instrument's underlying construct. The instrument is reliable and presented evidences of construct validity.

Development of the evaluation instrument use CIPP on the implementation of project assessment topic optik

NASA Astrophysics Data System (ADS)

Asfaroh, Jati Aurum; Rosana, Dadan; Supahar

2017-08-01

This research aims to develop an evaluation instrument models CIPP valid and reliable as well as determine the feasibility and practicality of an evaluation instrument models CIPP. An evaluation instrument models CIPP to evaluate the implementation of the project assessment topic optik to measure problem-solving skills of junior high school class VIII in the Yogyakarta region. This research is a model of development that uses 4-D. Subject of product trials are students in class VIII SMP N 1 Galur and SMP N 1 Sleman. Data collection techniques in this research using non-test techniques include interviews, questionnaires and observations. Validity in this research was analyzed using V'Aikens. Reliability analyzed using ICC. This research uses 7 raters are derived from two lecturers expert (expert judgment), two practitioners (science teacher) and three colleagues. The results of this research is the evaluation's instrument model of CIPP is used to evaluate the implementation of the implementation of the project assessment instruments. The validity result of evaluation instrument have V'Aikens values between 0.86 to 1, which means a valid and 0.836 reliability values into categories so well that it has been worth used as an evaluation instrument.

Validation of a scenario-based assessment of critical thinking using an externally validated tool.

PubMed

Buur, Jennifer L; Schmidt, Peggy; Smylie, Dean; Irizarry, Kris; Crocker, Carlos; Tyler, John; Barr, Margaret

2012-01-01

With medical education transitioning from knowledge-based curricula to competency-based curricula, critical thinking skills have emerged as a major competency. While there are validated external instruments for assessing critical thinking, many educators have created their own custom assessments of critical thinking. However, the face validity of these assessments has not been challenged. The purpose of this study was to compare results from a custom assessment of critical thinking with the results from a validated external instrument of critical thinking. Students from the College of Veterinary Medicine at Western University of Health Sciences were administered a custom assessment of critical thinking (ACT) examination and the externally validated instrument, California Critical Thinking Skills Test (CCTST), in the spring of 2011. Total scores and sub-scores from each exam were analyzed for significant correlations using Pearson correlation coefficients. Significant correlations between ACT Blooms 2 and deductive reasoning and total ACT score and deductive reasoning were demonstrated with correlation coefficients of 0.24 and 0.22, respectively. No other statistically significant correlations were found. The lack of significant correlation between the two examinations illustrates the need in medical education to externally validate internal custom assessments. Ultimately, the development and validation of custom assessments of non-knowledge-based competencies will produce higher quality medical professionals.

Development and Validation of an Assessment Instrument for Course Experience in a General Education Integrated Science Course

ERIC Educational Resources Information Center

Liu, Juhong Christie; St. John, Kristen; Courtier, Anna M. Bishop

2017-01-01

Identifying instruments and surveys to address geoscience education research (GER) questions is among the high-ranked needs in a 2016 survey of the GER community (St. John et al., 2016). The purpose of this study was to develop and validate a student-centered assessment instrument to measure course experience in a general education integrated…

Development and Validation of an Instrument to Assess Youth Motivation to Participate in Career Development Events

ERIC Educational Resources Information Center

Knobloch, Neil A.; Brady, Colleen M.; Orvis, Kathryn S.; Carroll, Natalie J.

2016-01-01

Career development events develop career and life skills in youth, but limited work has been done to assess the motivation of students who participate in these events. The purpose of this study was to validate an instrument developed to measure youth motivation to participate in career development events. An instrument grounded in expectancy-value…

Development and Validation of an Instrument to Assess the Implementation of Family-Centred Care in Traditional Open Bay Neonatal Intensive Care Units

ERIC Educational Resources Information Center

Al-Motlaq, Mohammad A.; Abuidhail, Jamila; Salameh, Taghreed; Awwad, Wesam

2017-01-01

Objective: To develop an instrument to study family-centred care (FCC) in traditional open bay Neonatal Intensive Care Units (NICUs). Methods: The development process involved constructing instrument's items, establishing content validity by an expert panel and testing the instrument for validity and reliability with a convenience sample of 25…

Adaptation and validation of the Inventory of Family Protective Factors for the Portuguese culture

PubMed Central

Augusto, Cláudia Cristina Vieira Carvalho de Oliveira Ferreira; Araújo, Beatriz Rodrigues; Rodrigues, Vítor Manuel Costa Pereira; de Figueiredo, Maria do Céu Aguiar Barbieri

2014-01-01

OBJECTIVES: to adapt and validate the Inventory of Family Protective Factors (IFPF) for the Portuguese culture. This instrument assesses protective factors that contribute to family resilience. Studies addressing resilience are embedded within the salutogenic paradigm, i.e. it addresses protective factors of individuals or groups without underestimating risk factors or vulnerability. METHOD: in order to assess the IFPF's linguistic and conceptual equivalence, the instrument was translated, retro-translated and the think-aloud protocol was used. We then verified the instrument's sensitiveness, reliability and validity of results to assess its psychometric characteristics. A factor analysis was performed of the principal components with varimax rotation of the scale's items and Cronbach's alpha coefficient was calculated for each dimension. A total of 85 families with disabled children, selected through simple random sampling, self-administered the instrument. RESULTS: the IFPF presents psychometric characteristics that are appropriate for the Portuguese population (Cronbach's alpha = .90). CONCLUSION: the IFPF was adapted and validated for the Portuguese culture and is an instrument to be used in studies intended to assess protective factors of family resilience. PMID:25591096

Adaptation and validation of the Inventory of Family Protective Factors for the Portuguese culture.

PubMed

Augusto, Cláudia Cristina Vieira Carvalho de Oliveira Ferreira; Araújo, Beatriz Rodrigues; Rodrigues, Vítor Manuel Costa Pereira; de Figueiredo, Maria do Céu Aguiar Barbieri

2014-01-01

to adapt and validate the Inventory of Family Protective Factors (IFPF) for the Portuguese culture. This instrument assesses protective factors that contribute to family resilience. Studies addressing resilience are embedded within the salutogenic paradigm, i.e. it addresses protective factors of individuals or groups without underestimating risk factors or vulnerability. in order to assess the IFPF's linguistic and conceptual equivalence, the instrument was translated, retro-translated and the think-aloud protocol was used. We then verified the instrument's sensitiveness, reliability and validity of results to assess its psychometric characteristics. A factor analysis was performed of the principal components with varimax rotation of the scale's items and Cronbach's alpha coefficient was calculated for each dimension. A total of 85 families with disabled children, selected through simple random sampling, self-administered the instrument. the IFPF presents psychometric characteristics that are appropriate for the Portuguese population (Cronbach's alpha = .90). the IFPF was adapted and validated for the Portuguese culture and is an instrument to be used in studies intended to assess protective factors of family resilience.

Design and validation of a three-instrument toolkit for the assessment of competence in electrocardiogram rhythm recognition.

PubMed

Hernández-Padilla, José M; Granero-Molina, José; Márquez-Hernández, Verónica V; Suthers, Fiona; López-Entrambasaguas, Olga M; Fernández-Sola, Cayetano

2017-06-01

Rapid and accurate interpretation of cardiac arrhythmias by nurses has been linked with safe practice and positive patient outcomes. Although training in electrocardiogram rhythm recognition is part of most undergraduate nursing programmes, research continues to suggest that nurses and nursing students lack competence in recognising cardiac rhythms. In order to promote patient safety, nursing educators must develop valid and reliable assessment tools that allow the rigorous assessment of this competence before nursing students are allowed to practise without supervision. The aim of this study was to develop and psychometrically evaluate a toolkit to holistically assess competence in electrocardiogram rhythm recognition. Following a convenience sampling technique, 293 nursing students from a nursing faculty in a Spanish university were recruited for the study. The following three instruments were developed and psychometrically tested: an electrocardiogram knowledge assessment tool (ECG-KAT), an electrocardiogram skills assessment tool (ECG-SAT) and an electrocardiogram self-efficacy assessment tool (ECG-SES). Reliability and validity (content, criterion and construct) of these tools were meticulously examined. A high Cronbach's alpha coefficient demonstrated the excellent reliability of the instruments (ECG-KAT=0.89; ECG-SAT=0.93; ECG-SES=0.98). An excellent context validity index (scales' average content validity index>0.94) and very good criterion validity were evidenced for all the tools. Regarding construct validity, principal component analysis revealed that all items comprising the instruments contributed to measure knowledge, skills or self-efficacy in electrocardiogram rhythm recognition. Moreover, known-groups analysis showed the tools' ability to detect expected differences in competence between groups with different training experiences. The three-instrument toolkit developed showed excellent psychometric properties for measuring competence in electrocardiogram rhythm recognition.

Development and Validation of an Instrument to Evaluate Perceived Wellbeing Associated with the Ingestion of Water: The Water Ingestion-Related Wellbeing Instrument (WIRWI)

PubMed Central

Espinosa-Montero, Juan; Monterrubio-Flores, Eric A.; Sanchez-Estrada, Marcela; Buendia-Jimenez, Inmaculada; Lieberman, Harris R.; Allaert, François-Andre; Barquera, Simon

2016-01-01

Background Ingestion of water has been associated with general wellbeing. When water intake is insufficient, symptoms such as thirst, fatigue and impaired memory result. Currently there are no instruments to assess water consumption associated with wellbeing. The objective of our study was to develop and validate such an instrument in urban, low socioeconomic, adult Mexican population. Methods To construct the Water Ingestion-Related Wellbeing Instrument (WIRWI), a qualitative study in which wellbeing related to everyday practices and experiences in water consumption were investigated. To validate the WIRWI a formal, five-process procedure was used. Face and content validation were addressed, consistency was assessed by exploratory and confirmatory psychometric factor analyses, repeatability, reproducibility and concurrent validity were assessed by conducting correlation tests with other measures of wellbeing such as a quality of life instrument, the SF-36, and objective parameters such as urine osmolality, 24-hour urine total volume and others. Results The final WIRWI is composed of 17 items assessing physical and mental dimensions. Items were selected based on their content and face validity. Exploratory and confirmatory factor analyses yielded Cronbach's alpha of 0.87 and 0.86, respectively. The final confirmatory factor analysis demonstrated that the model estimates were satisfactory for the constructs. Statistically significant correlations with the SF-36, total liquid consumption and simple water consumption were observed. Conclusion The resulting WIRWI is a reliable tool for assessing wellbeing associated with consumption of plain water in Mexican adults and could be useful for similar groups. PMID:27388902

Development and Validation of an Instrument to Evaluate Perceived Wellbeing Associated with the Ingestion of Water: The Water Ingestion-Related Wellbeing Instrument (WIRWI).

PubMed

Espinosa-Montero, Juan; Monterrubio-Flores, Eric A; Sanchez-Estrada, Marcela; Buendia-Jimenez, Inmaculada; Lieberman, Harris R; Allaert, François-Andre; Barquera, Simon

2016-01-01

Ingestion of water has been associated with general wellbeing. When water intake is insufficient, symptoms such as thirst, fatigue and impaired memory result. Currently there are no instruments to assess water consumption associated with wellbeing. The objective of our study was to develop and validate such an instrument in urban, low socioeconomic, adult Mexican population. To construct the Water Ingestion-Related Wellbeing Instrument (WIRWI), a qualitative study in which wellbeing related to everyday practices and experiences in water consumption were investigated. To validate the WIRWI a formal, five-process procedure was used. Face and content validation were addressed, consistency was assessed by exploratory and confirmatory psychometric factor analyses, repeatability, reproducibility and concurrent validity were assessed by conducting correlation tests with other measures of wellbeing such as a quality of life instrument, the SF-36, and objective parameters such as urine osmolality, 24-hour urine total volume and others. The final WIRWI is composed of 17 items assessing physical and mental dimensions. Items were selected based on their content and face validity. Exploratory and confirmatory factor analyses yielded Cronbach's alpha of 0.87 and 0.86, respectively. The final confirmatory factor analysis demonstrated that the model estimates were satisfactory for the constructs. Statistically significant correlations with the SF-36, total liquid consumption and simple water consumption were observed. The resulting WIRWI is a reliable tool for assessing wellbeing associated with consumption of plain water in Mexican adults and could be useful for similar groups.

Development and validation of a survey instrument for assessing prescribers' perception of computerized drug-drug interaction alerts.

PubMed

Zheng, Kai; Fear, Kathleen; Chaffee, Bruce W; Zimmerman, Christopher R; Karls, Edward M; Gatwood, Justin D; Stevenson, James G; Pearlman, Mark D

2011-12-01

To develop a theoretically informed and empirically validated survey instrument for assessing prescribers' perception of computerized drug-drug interaction (DDI) alerts. The survey is grounded in the unified theory of acceptance and use of technology and an adapted accident causation model. Development of the instrument was also informed by a review of the extant literature on prescribers' attitude toward computerized medication safety alerts and common prescriber-provided reasons for overriding. To refine and validate the survey, we conducted a two-stage empirical validation study consisting of a pretest with a panel of domain experts followed by a field test among all eligible prescribers at our institution. The resulting survey instrument contains 28 questionnaire items assessing six theoretical dimensions: performance expectancy, effort expectancy, social influence, facilitating conditions, perceived fatigue, and perceived use behavior. Satisfactory results were obtained from the field validation; however, a few potential issues were also identified. We analyzed these issues accordingly and the results led to the final survey instrument as well as usage recommendations. High override rates of computerized medication safety alerts have been a prevalent problem. They are usually caused by, or manifested in, issues of poor end user acceptance. However, standardized research tools for assessing and understanding end users' perception are currently lacking, which inhibits knowledge accumulation and consequently forgoes improvement opportunities. The survey instrument presented in this paper may help fill this methodological gap. We developed and empirically validated a survey instrument that may be useful for future research on DDI alerts and other types of computerized medication safety alerts more generally.

Instruments to identify prescription medication misuse, abuse, and related events in clinical trials: an ACTTION systematic review

PubMed Central

Smith, Shannon M.; Paillard, Florence; McKeown, Andrew; Burke, Laurie B.; Edwards, Robert R.; Katz, Nathaniel P.; Papadopoulos, Elektra J.; Rappaport, Bob A.; Slagle, Ashley; Strain, Eric C.; Wasan, Ajay D.; Turk, Dennis C.; Dworkin, Robert H.

2017-01-01

Measurement of inappropriate medication use events (e.g., abuse, misuse) in clinical trials is important in characterizing a medication’s abuse potential. However, no “gold standard” assessment of inappropriate use events in clinical trials has been identified. In this systematic review, we examine the measurement properties (i.e., content validity, cross-sectional reliability and construct validity, longitudinal construct validity, ability to detect change, and responder definitions) of instruments assessing inappropriate use of opioid and non-opioid prescription medications to identify any that meet U.S. and European regulatory agencies’ rigorous standards for outcome measures in clinical trials. Sixteen published instruments were identified, most of which were not designed for the selected concept of interest and context of use. For this reason, many instruments were found to lack adequate content validity (or documentation of content validity) to evaluate current inappropriate medication use events; for example, evaluating inappropriate use across the lifespan rather than current use, including items that did not directly assess inappropriate use (e.g., questions about anger), or failing to capture information pertinent to inappropriate use events (e.g., intention, route of administration). In addition, the psychometric data across all instruments were generally limited in scope. A further limitation is the heterogeneous, non-standardized use of inappropriate medication use terminology. These observations suggest that available instruments are not well suited for assessing current inappropriate medication use within the specific context of clinical trials. Further effort is needed to develop reliable and valid instruments to measure current inappropriate medication use events in clinical trials. PMID:25660826

Performance assessment instrument to assess the senior high students' psychomotor for the salt hydrolysis material

NASA Astrophysics Data System (ADS)

Nahadi, Firman, Harry; Yulina, Erlis

2016-02-01

The purposes of this study were to develop a performance assessment instrument for assessing the competence of psychomotor high school students on salt hydrolysis concepts. The design used in this study was the Research & Development which consists of three phases: development, testing and application of instruments. Subjects in this study were high school students in class XI science, which amounts to 93 students. In the development phase, seven validators validated 17 tasks instrument. In the test phase, we divided 19 students into three-part different times to conduct performance test in salt hydrolysis lab work and observed by six raters. The first, the second, and the third groups recpectively consist of five, six, and eight students. In the application phase, two raters observed the performance of 74 students in the salt hydrolysis lab work in several times. The results showed that 16 of 17 tasks of performance assessment instrument developed can be stated to be valid with CVR value of 1,00 and 0,714. While, the rest was not valid with CVR value was 0.429, below the critical value (0.622). In the test phase, reliability value of instrument obtained were 0,951 for the five-student group, 0,806 for the six-student group and 0,743 for the eight-student group. From the interviews, teachers strongly agree with the performance instrument developed. They stated that the instrument was feasible to use for maximum number of students were six in a single observation.

Developing Instruments to Assess and Compare the Quality of Engineering Education: The Case of China and Russia

ERIC Educational Resources Information Center

Kardanova, Elena; Loyalka, Prashant; Chirikov, Igor; Liu, Lydia; Li, Guirong; Wang, Huan; Enchikova, Ekaterina; Shi, Henry; Johnson, Natalie

2016-01-01

Relatively little is known about differences in the quality of engineering education within and across countries because of the lack of valid instruments that allow for the assessment and comparison of engineering students' skill gains. The purpose of our study is to develop and validate instruments that can be used to compare student skill gains…

Pain assessment in children: theoretical and empirical validity.

PubMed

Villarruel, A M; Denyes, M J

1991-12-01

Valid assessment of pain in children is foundational for both the nursing practice and research domains, yet few validated methods of pain measurement are currently available for young children. This article describes an innovative research approach used in the development of photographic instruments to measure pain intensity in young African-American and Hispanic children. The instruments were designed to enable children to participate actively in their own care and to do so in ways that are congruent with their developmental and cultural heritage. Conceptualization of the instruments, methodological development, and validation processes grounded in Orem's Self-Care Deficit Theory of Nursing are described. The authors discuss the ways in which the gaps between nursing theory, research, and practice are narrowed when development of instruments to measure clinical nursing phenomena are grounded in nursing theory, validated through research and utilized in practice settings.

[Development and validation of an instrument for initial nursing assessment].

PubMed

Fernández-Sola, Cayetano; Granero-Molina, José; Mollinedo-Mallea, Judith; de Gonzales, María Hilda Peredo; Aguilera-Manrique, Gabriel; Ponce, Mara Luna

2012-12-01

The objective of this study, conducted in Bolivia from April to July of 2008, is the design and validation of an initial nursing assessment instrument to be used in clinical and educational environments in Santa Cruz (Bolivia). Twelve Bolivian nurses participated; both document analysis as well as consensus techniques were used to determine the categories and criteria to be assessed. Categories included in the nursing assessment instrument are a physical assessment and the eleven Gordon's Functional Health Patterns. The nursing assessment instrument stands out as being concise, easy to complete and utilizing a nursing approach. It does not include items for advanced nursing assessment. However, it incorporates items regarding lifestyle and the patient's autonomy. The nursing assessment instrument contributes to improving the quality of clinical records, supports the nursing diagnosis and implementation of the nursing process, promotes the nurse's role and helps to standardize practice.

The Need, Development, and Validation of the Innovation Test Instrument

ERIC Educational Resources Information Center

Wheadon, Jacob; Wright, Geoff A.; West, Richard E.; Skaggs, Paul

2017-01-01

This study discusses the need, development, and validation of the Innovation Test Instrument (ITI). This article outlines how the researchers identified the content domain of the assessment and created test items. Then, it describes initial validation testing of the instrument. The findings suggest that the ITI is a good first step in creating an…

Development of functional oral health literacy assessment instruments: application of literacy and cognitive theories.

PubMed

Bridges, Susan M; Parthasarathy, Divya S; Au, Terry K F; Wong, Hai Ming; Yiu, Cynthia K Y; McGrath, Colman P

2014-01-01

This paper describes the development of a new literacy assessment instrument, the Hong Kong Oral Health Literacy Assessment Task for Paediatric Dentistry (HKOHLAT-P). Its relationship to literacy theory is analyzed to establish content and face validity. Implications for construct validity are examined by analyzing cognitive demand to determine how "comprehension" is measured. Key influences from literacy assessment were identified to analyze item development. Cognitive demand was analyzed using an established taxonomy. The HKOHLAT-P focuses on the functional domain of health literacy assessment. Items had strong content and face validity reflecting established principles from modern literacy theory. Inclusion of new text types signified relevant developments in the area of new literacies. Analysis of cognitive demand indicated that this instrument assesses the "comprehension" domain, specifically the areas of factual and procedural knowledge, with some assessment of conceptual knowledge. Metacognitive knowledge was not assessed. Comprehension tasks assessing patient health literacy predominantly examine functional health literacy at the lower levels of comprehension. Item development is influenced by the fields of situated and authentic literacy. Inclusion of content regarding multiliteracies is suggested for further research. Development of functional health literacy assessment instruments requires careful consideration of the clinical context in determining construct validity. © 2013 American Association of Public Health Dentistry.

The Assessment, Development, Assurance Pharmacist's Tool (ADAPT) for ensuring quality implementation of health promotion programs.

PubMed

Truong, Hoai-An; Taylor, Catherine R; DiPietro, Natalie A

2012-02-10

To develop and validate the Assessment, Development, Assurance Pharmacist's Tool (ADAPT), an instrument for pharmacists and student pharmacists to use in developing and implementing health promotion programs. The 36-item ADAPT instrument was developed using the framework of public health's 3 core functions (assessment, policy development, and assurance) and 10 essential services. The tool's content and usage was assessed and conducted through peer-review and initial validity testing processes. Over 20 faculty members, preceptors, and student pharmacists at 5 institutions involved in planning and implementing health promotion initiatives reviewed the instrument and conducted validity testing. The instrument took approximately 15 minutes to complete and the findings resulted in changes and improvements to elements of the programs evaluated. The ADAPT instrument fills a need to more effectively plan, develop, implement, and evaluate pharmacist-directed public health programs that are evidence-based, high-quality, and compliant with laws and regulations and facilitates documentation of pharmacists' contributions to public health.

Attributional Bias Instrument (ABI): Validation of a Measure to Assess Ability and Effort Explanations for Math Performance

ERIC Educational Resources Information Center

Espinoza, Penelope P.; Quezada, Stephanie A.; Rincones, Rodolfo; Strobach, E. Natalia; Gutierrez, Maria Armida Estrada

2012-01-01

The present work investigates the validation of a newly developed instrument, the attributional bias instrument, based on achievement attribution theories that distinguish between effort and ability explanations of behavior. The instrument further incorporates the distinction between explanations for success versus failure in academic performance.…

Psychometric properties concerning four instruments measuring job satisfaction, strain, and stress of conscience in a residential care context.

PubMed

Orrung Wallin, Anneli; Edberg, Anna-Karin; Beck, Ingela; Jakobsson, Ulf

2013-01-01

There are many instruments assessing the wellbeing of staff, but far from all have been psychometrically investigated. When evaluating supportive interventions directed toward nurse assistants in residential care, valid and reliable instruments are needed in order to detect possible changes. The aim of the study was to investigate validity in terms of data quality, construct validity, convergent and divergent validity and reliability in terms of the internal consistency and stability of the Job Satisfaction Questionnaire, the Psychosocial Aspects of Job Satisfaction, the Strain in Dementia Care Scale (SDCS), and the Stress of Conscience Questionnaire (SCQ) in a residential care context. The psychometric properties of the instruments were investigated in terms of data quality, construct validity, convergent and divergent validity and reliability, including test-retest reliability, in a residential care context with a sample consisting of nurse assistants (n=114). The four instruments responded with different psychometric-related problems such as internal missing data, floor and ceiling effects, problems with construct validity and low test-retest reliability, especially when assessed on the item level. These problems were however reduced or disappeared completely when assessed for total and factor scores. From a psychometric perspective, the SDCS seemed to stand out as the best instrument. However, it should be modified in order to reduce floor effects on item level and thereby gain sensitivity. The Job Satisfaction Questionnaire seemed to have problems both with the construct validity and test-retest reliability. The final choice of instrument must, however, be made dependent on what one intends to measure. Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.

Development and Validation of an Instrument for Assessing Mathematics Classroom Environment in Tertiary Institutions

ERIC Educational Resources Information Center

Yin, Hongbiao; Lu, Genshu

2014-01-01

This report describes the development and validation of an instrument, the University Mathematics Classroom Environment Questionnaire (UMCEQ), for assessing the mathematics classroom environment in tertiary institutions in China. Through the use of multiple methods, including exploratory and confirmatory factor analyses, on two independent samples…

Non-technical skills assessment for prelicensure nursing students: An integrative review.

PubMed

Pires, Sara; Monteiro, Sara; Pereira, Anabela; Chaló, Daniela; Melo, Elsa; Rodrigues, Alexandre

2017-11-01

In nursing, non-technical skills are recognized as playing an important role to increase patient safety and successful clinical outcomes (Pearson and McLafferty, 2011). Non-technical skills are cognitive and social resource skills that complement technical skills and contribute to safe and efficient task performance (Flin et al., 2008). In order to effectively provide non-technical skills training, it is essential to have an instrument to measure these skills. An online search was conducted. Articles were selected if they referred to and/or described instruments assessing non-technical skills for nurses and/or prelicensure nursing students in educational, clinical and/or simulated settings with validation evidence (inclusion criteria). Of the 53 articles located, 26 met the inclusion criteria. Those referred to and/or described 16 instruments with validation evidence developed to assess non-technical skills in multidisciplinary teams including nurses. Although articles have shown 16 valid and reliable instruments, to our knowledge, no instrument has been published or developed and validated for the assessment of non-technical skills of only nurses in general, relevant for use in high-fidelity simulation-based training for prelicensure nursing students. Therefore, there is a need for the development of such an instrument. Copyright © 2017 Elsevier Ltd. All rights reserved.

Assessing Sleep Disturbance in Low Back Pain: The Validity of Portable Instruments

PubMed Central

Alsaadi, Saad M.; McAuley, James H.; Hush, Julia M.; Bartlett, Delwyn J.; McKeough, Zoe M.; Grunstein, Ronald R.; Dungan, George C.; Maher, Chris G.

2014-01-01

Although portable instruments have been used in the assessment of sleep disturbance for patients with low back pain (LBP), the accuracy of the instruments in detecting sleep/wake episodes for this population is unknown. This study investigated the criterion validity of two portable instruments (Armband and Actiwatch) for assessing sleep disturbance in patients with LBP. 50 patients with LBP performed simultaneous overnight sleep recordings in a university sleep laboratory. All 50 participants were assessed by Polysomnography (PSG) and the Armband and a subgroup of 33 participants wore an Actiwatch. Criterion validity was determined by calculating epoch-by-epoch agreement, sensitivity, specificity and prevalence and bias- adjusted kappa (PABAK) for sleep versus wake between each instrument and PSG. The relationship between PSG and the two instruments was assessed using intraclass correlation coefficients (ICC 2, 1). The study participants showed symptoms of sub-threshold insomnia (mean ISI = 13.2, 95% CI = 6.36) and poor sleep quality (mean PSQI = 9.20, 95% CI = 4.27). Observed agreement with PSG was 85% and 88% for the Armband and Actiwatch. Sensitivity was 0.90 for both instruments and specificity was 0.54 and 0.67 and PABAK of 0.69 and 0.77 for the Armband and Actiwatch respectively. The ICC (95%CI) was 0.76 (0.61 to 0.86) and 0.80 (0.46 to 0.92) for total sleep time, 0.52 (0.29 to 0.70) and 0.55 (0.14 to 0.77) for sleep efficiency, 0.64 (0.45 to 0.78) and 0.52 (0.23 to 0.73) for wake after sleep onset and 0.13 (−0.15 to 0.39) and 0.33 (−0.05 to 0.63) for sleep onset latency, for the Armband and Actiwatch, respectively. The findings showed that both instruments have varied criterion validity across the sleep parameters from excellent validity for measures of total sleep time, good validity for measures of sleep efficiency and wake after onset to poor validity for sleep onset latency. PMID:24763506

Assessing sleep disturbance in low back pain: the validity of portable instruments.

PubMed

Alsaadi, Saad M; McAuley, James H; Hush, Julia M; Bartlett, Delwyn J; McKeough, Zoe M; Grunstein, Ronald R; Dungan, George C; Maher, Chris G

2014-01-01

Although portable instruments have been used in the assessment of sleep disturbance for patients with low back pain (LBP), the accuracy of the instruments in detecting sleep/wake episodes for this population is unknown. This study investigated the criterion validity of two portable instruments (Armband and Actiwatch) for assessing sleep disturbance in patients with LBP. 50 patients with LBP performed simultaneous overnight sleep recordings in a university sleep laboratory. All 50 participants were assessed by Polysomnography (PSG) and the Armband and a subgroup of 33 participants wore an Actiwatch. Criterion validity was determined by calculating epoch-by-epoch agreement, sensitivity, specificity and prevalence and bias- adjusted kappa (PABAK) for sleep versus wake between each instrument and PSG. The relationship between PSG and the two instruments was assessed using intraclass correlation coefficients (ICC 2, 1). The study participants showed symptoms of sub-threshold insomnia (mean ISI = 13.2, 95% CI = 6.36) and poor sleep quality (mean PSQI = 9.20, 95% CI = 4.27). Observed agreement with PSG was 85% and 88% for the Armband and Actiwatch. Sensitivity was 0.90 for both instruments and specificity was 0.54 and 0.67 and PABAK of 0.69 and 0.77 for the Armband and Actiwatch respectively. The ICC (95%CI) was 0.76 (0.61 to 0.86) and 0.80 (0.46 to 0.92) for total sleep time, 0.52 (0.29 to 0.70) and 0.55 (0.14 to 0.77) for sleep efficiency, 0.64 (0.45 to 0.78) and 0.52 (0.23 to 0.73) for wake after sleep onset and 0.13 (-0.15 to 0.39) and 0.33 (-0.05 to 0.63) for sleep onset latency, for the Armband and Actiwatch, respectively. The findings showed that both instruments have varied criterion validity across the sleep parameters from excellent validity for measures of total sleep time, good validity for measures of sleep efficiency and wake after onset to poor validity for sleep onset latency.

Assessing child and adolescent pragmatic language competencies: toward evidence-based assessments.

PubMed

Russell, Robert L; Grizzle, Kenneth L

2008-06-01

Using language appropriately and effectively in social contexts requires pragmatic language competencies (PLCs). Increasingly, deficits in PLCs are linked to child and adolescent disorders, including autism spectrum, externalizing, and internalizing disorders. As the role of PLCs expands in diagnosis and treatment of developmental psychopathology, psychologists and educators will need to appraise and select clinical and research PLC instruments for use in assessments and/or studies. To assist in this appraisal, 24 PLC instruments, containing 1,082 items, are assessed by addressing four questions: (1) Can PLC domains targeted by assessment items be reliably identified?, (2) What are the core PLC domains that emerge across the 24 instruments?, (3) Do PLC questionnaires and tests assess similar PLC domains?, and (4) Do the instruments achieve content, structural, diagnostic, and ecological validity? Results indicate that test and questionnaire items can be reliably categorized into PLC domains, that PLC domains featured in questionnaires and tests significantly differ, and that PLC instruments need empirical confirmation of their dimensional structure, content validity across all developmental age bands, and ecological validity. Progress in building a better evidence base for PLC assessments should be a priority in future research.

The knowledge, efficacy, and practices instrument for oral health providers: a validity study with dental students.

PubMed

Behar-Horenstein, Linda S; Garvan, Cyndi W; Moore, Thomas E; Catalanotto, Frank A

2013-08-01

Valid and reliable instruments to measure and assess cultural competence for oral health care providers are scarce in the literature, and most published scales have been contested due to a lack of item analysis and internal estimates of reliability. The purposes of this study were, first, to develop a standardized instrument to measure dental students' knowledge of diversity, skills in culturally competent patient-centered communication, and use of culture-centered practices in patient care and, second, to provide preliminary validity support for this instrument. The initial instrument used in this study was a thirty-six-item Likert-scale survey entitled the Knowledge, Efficacy, and Practices Instrument for Oral Health Providers (KEPI-OHP). This instrument is an adaption of an initially thirty-three-item version of the Multicultural Awareness, Knowledge, and Skills Scale-Counselor Edition (MAKSS-CE), a scale that assesses factors related to social justice, cultural differences among clients, and cross-cultural client management. After the authors conducted cognitive and expert interviews, focus groups, pilot testing, and item analysis, their initial instrument was reduced to twenty-eight items. The KEPI-OHP was then distributed to 916 dental students (response rate=48.6 percent) across the United States to measure its reliability and assess its validity. Both exploratory and confirmatory factor analyses were conducted to test the scale's validity. The modification of the survey into a sensible instrument with a relatively clear factor structure using factor analysis resulted in twenty items. A scree test suggested three expressive factors, which were retained for rotation. Bentler's comparative fit and Bentler and Bonnett's non-normed indices were 0.95 and 0.92, respectively. A three-factor solution, including efficacy of assessment, knowledge of diversity, and culture-centered practice subscales, comprised of twenty-items was identified. The KEPI-OHP was found to have reasonable internal consistency reliability to warrant its use for baseline and repeated measures in assessing changes in dental students' growth in cultural competence across four-year dental curricula.

The quality of instruments to assess the process of shared decision making: A systematic review

PubMed Central

Bomhof-Roordink, Hanna; Smith, Ian P.; Scholl, Isabelle; Stiggelbout, Anne M.; Pieterse, Arwen H.

2018-01-01

Objective To inventory instruments assessing the process of shared decision making and appraise their measurement quality, taking into account the methodological quality of their validation studies. Methods In a systematic review we searched seven databases (PubMed, Embase, Emcare, Cochrane, PsycINFO, Web of Science, Academic Search Premier) for studies investigating instruments measuring the process of shared decision making. Per identified instrument, we assessed the level of evidence separately for 10 measurement properties following a three-step procedure: 1) appraisal of the methodological quality using the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist, 2) appraisal of the psychometric quality of the measurement property using three possible quality scores, 3) best-evidence synthesis based on the number of studies, their methodological and psychometrical quality, and the direction and consistency of the results. The study protocol was registered at PROSPERO: CRD42015023397. Results We included 51 articles describing the development and/or evaluation of 40 shared decision-making process instruments: 16 patient questionnaires, 4 provider questionnaires, 18 coding schemes and 2 instruments measuring multiple perspectives. There is an overall lack of evidence for their measurement quality, either because validation is missing or methods are poor. The best-evidence synthesis indicated positive results for a major part of instruments for content validity (50%) and structural validity (53%) if these were evaluated, but negative results for a major part of instruments when inter-rater reliability (47%) and hypotheses testing (59%) were evaluated. Conclusions Due to the lack of evidence on measurement quality, the choice for the most appropriate instrument can best be based on the instrument’s content and characteristics such as the perspective that they assess. We recommend refinement and validation of existing instruments, and the use of COSMIN-guidelines to help guarantee high-quality evaluations. PMID:29447193

Diagnostic Criteria for Temporomandibular Disorders (DC/TMD) for Clinical and Research Applications: Recommendations of the International RDC/TMD Consortium Network* and Orofacial Pain Special Interest Group†

PubMed Central

Schiffman, Eric; Ohrbach, Richard; Truelove, Edmond; Look, John; Anderson, Gary; Goulet, Jean-Paul; List, Thomas; Svensson, Peter; Gonzalez, Yoly; Lobbezoo, Frank; Michelotti, Ambra; Brooks, Sharon L.; Ceusters, Werner; Drangsholt, Mark; Ettlin, Dominik; Gaul, Charly; Goldberg, Louis J.; Haythornthwaite, Jennifer A.; Hollender, Lars; Jensen, Rigmor; John, Mike T.; De Laat, Antoon; de Leeuw, Reny; Maixner, William; van der Meulen, Marylee; Murray, Greg M.; Nixdorf, Donald R.; Palla, Sandro; Petersson, Arne; Pionchon, Paul; Smith, Barry; Visscher, Corine M.; Zakrzewska, Joanna; Dworkin, Samuel F.

2015-01-01

Aims The original Research Diagnostic Criteria for Temporomandibular Disorders (RDC/TMD) Axis I diagnostic algorithms have been demonstrated to be reliable. However, the Validation Project determined that the RDC/TMD Axis I validity was below the target sensitivity of ≥ 0.70 and specificity of ≥ 0.95. Consequently, these empirical results supported the development of revised RDC/TMD Axis I diagnostic algorithms that were subsequently demonstrated to be valid for the most common pain-related TMD and for one temporomandibular joint (TMJ) intra-articular disorder. The original RDC/TMD Axis II instruments were shown to be both reliable and valid. Working from these findings and revisions, two international consensus workshops were convened, from which recommendations were obtained for the finalization of new Axis I diagnostic algorithms and new Axis II instruments. Methods Through a series of workshops and symposia, a panel of clinical and basic science pain experts modified the revised RDC/TMD Axis I algorithms by using comprehensive searches of published TMD diagnostic literature followed by review and consensus via a formal structured process. The panel's recommendations for further revision of the Axis I diagnostic algorithms were assessed for validity by using the Validation Project's data set, and for reliability by using newly collected data from the ongoing TMJ Impact Project—the follow-up study to the Validation Project. New Axis II instruments were identified through a comprehensive search of the literature providing valid instruments that, relative to the RDC/TMD, are shorter in length, are available in the public domain, and currently are being used in medical settings. Results The newly recommended Diagnostic Criteria for TMD (DC/TMD) Axis I protocol includes both a valid screener for detecting any pain-related TMD as well as valid diagnostic criteria for differentiating the most common pain-related TMD (sensitivity ≥ 0.86, specificity ≥ 0.98) and for one intra-articular disorder (sensitivity of 0.80 and specificity of 0.97). Diagnostic criteria for other common intra-articular disorders lack adequate validity for clinical diagnoses but can be used for screening purposes. Inter-examiner reliability for the clinical assessment associated with the validated DC/TMD criteria for pain-related TMD is excellent (kappa ≥ 0.85). Finally, a comprehensive classification system that includes both the common and less common TMD is also presented. The Axis II protocol retains selected original RDC/TMD screening instruments augmented with new instruments to assess jaw function as well as behavioral and additional psychosocial factors. The Axis II protocol is divided into screening and comprehensive self-report instrument sets. The screening instruments’ 41 questions assess pain intensity, pain-related disability, psychological distress, jaw functional limitations, and parafunctional behaviors, and a pain drawing is used to assess locations of pain. The comprehensive instruments, composed of 81 questions, assess in further detail jaw functional limitations and psychological distress as well as additional constructs of anxiety and presence of comorbid pain conditions. Conclusion The recommended evidence-based new DC/TMD protocol is appropriate for use in both clinical and research settings. More comprehensive instruments augment short and simple screening instruments for Axis I and Axis II. These validated instruments allow for identification of patients with a range of simple to complex TMD presentations. PMID:24482784

Measuring teamwork in health care settings: a review of survey instruments.

PubMed

Valentine, Melissa A; Nembhard, Ingrid M; Edmondson, Amy C

2015-04-01

Teamwork in health care settings is widely recognized as an important factor in providing high-quality patient care. However, the behaviors that comprise effective teamwork, the organizational factors that support teamwork, and the relationship between teamwork and patient outcomes remain empirical questions in need of rigorous study. To identify and review survey instruments used to assess dimensions of teamwork so as to facilitate high-quality research on this topic. We conducted a systematic review of articles published before September 2012 to identify survey instruments used to measure teamwork and to assess their conceptual content, psychometric validity, and relationships to outcomes of interest. We searched the ISI Web of Knowledge database, and identified relevant articles using the search terms team, teamwork, or collaboration in combination with survey, scale, measure, or questionnaire. We found 39 surveys that measured teamwork. Surveys assessed different dimensions of teamwork. The most commonly assessed dimensions were communication, coordination, and respect. Of the 39 surveys, 10 met all of the criteria for psychometric validity, and 14 showed significant relationships to nonself-report outcomes. Evidence of psychometric validity is lacking for many teamwork survey instruments. However, several psychometrically valid instruments are available. Researchers aiming to advance research on teamwork in health care should consider using or adapting one of these instruments before creating a new one. Because instruments vary considerably in the behavioral processes and emergent states of teamwork that they capture, researchers must carefully evaluate the conceptual consistency between instrument, research question, and context.

Development and validation of an instrument to assess job satisfaction in eye-care personnel.

PubMed

Paudel, Prakash; Cronjé, Sonja; O'Connor, Patricia M; Khadka, Jyoti; Rao, Gullapalli N; Holden, Brien A

2017-11-01

The aim was to develop and validate an instrument to measure job satisfaction in eye-care personnel and assess the job satisfaction of one-year trained vision technicians in India. A pilot instrument for assessing job satisfaction was developed, based on a literature review and input from a public health expert panel. Rasch analysis was used to assess psychometric properties and to undertake an iterative item reduction. The instrument was then administered to vision technicians in vision centres of Andhra Pradesh in India. Associations between vision technicians' job satisfaction and factors such as age, gender and experience were analysed using t-test and one-way analysis of variance. Rasch analysis confirmed that the 15-item job satisfaction in eye-care personnel (JSEP) was a unidimensional instrument with good fit statistics, measurement precisions and absence of differential item functioning. Overall, vision technicians reported high rates of job satisfaction (0.46 logits). Age, gender and experience were not associated with high job satisfaction score. Item score analysis showed non-financial incentives, salary and workload were the most important determinants of job satisfaction. The 15-item JSEP instrument is a valid instrument for assessing job satisfaction among eye-care personnel. Overall, vision technicians in India demonstrated high rates of job satisfaction. © 2016 Optometry Australia.

Construction of a survey to assess workload and fatigue among AMT operators in Mexico.

PubMed

Hernández Arellano, Juan Luis; Ibarra Mejía, Gabriel; Serratos Pérez, J Nieves; García Alcaraz, Jorge Luis; Brunette, María Julia

2012-01-01

Operators of machinery classified as Advanced Manufacturing Technology (AMT) are exposed to high levels of workload and fatigue. However, only few studies have been conducted on this topic in Hispanic-American countries workers. Several instruments be used to assess workload and fatigue; however, only few of them have been adapted to Spanish language. This paper reports on the development and validity testing of a survey instrument in Spanish, aiming to subjectively assess workload and fatigue among AMT operators in Mexico. After an exhaustive literature review in search of already available measurement instruments, they were adapted for content and later translated into Spanish; a pilot test was conducted to evaluate validity and reliability; afterwards appropriate modifications were made to the testing instruments. Final version of the instrument was applied to a group of 121 operators of CNC lathes. Reliability was analyzed using KMO and Cronbach alpha indices. For the assessment of workload, both NASA-TLX and ISTAS 21 methods were incorporated to the survey instrument. As for fatigue assessment tools, these were SOFI-S, FAS and OFER questionnaires. RESULTS show KMO value and Cronbach alpha above 0.6.Conclusions. The survey instrument as designed, allows the collection of reliable and valid data regarding workload and fatigue among AMT operators in Mexico.

An instrument to assess subjective task value beliefs regarding the decision to pursue postgraduate training.

PubMed

Hagemeier, Nicholas E; Murawski, Matthew M

2014-02-12

To develop and validate an instrument to assess subjective ratings of the perceived value of various postgraduate training paths followed using expectancy-value as a theoretical framework; and to explore differences in value beliefs across type of postgraduate training pursued and type of pharmacy training completed prior to postgraduate training. A survey instrument was developed to sample 4 theoretical domains of subjective task value: intrinsic value, attainment value, utility value, and perceived cost. Retrospective self-report methodology was employed to examine respondents' (N=1,148) subjective task value beliefs specific to their highest level of postgraduate training completed. Exploratory and confirmatory factor analytic techniques were used to evaluate and validate value belief constructs. Intrinsic, attainment, utility, cost, and financial value constructs resulted from exploratory factor analysis. Cross-validation resulted in a 26-item instrument that demonstrated good model fit. Differences in value beliefs were noted across type of postgraduate training pursued and pharmacy training characteristics. The Postgraduate Training Value Instrument demonstrated evidence of reliability and construct validity. The survey instrument can be used to assess value beliefs regarding multiple postgraduate training options in pharmacy and potentially inform targeted recruiting of individuals to those paths best matching their own value beliefs.

Nursing Intensive-Care Satisfaction Scale [NICSS]: Development and validation of a patient-centred instrument.

PubMed

Romero-García, Marta; de la Cueva-Ariza, Laura; Benito-Aracil, Llucia; Lluch-Canut, Teresa; Trujols-Albet, Joan; Martínez-Momblan, Maria Antonia; Juvé-Udina, Maria-Eulàlia; Delgado-Hito, Pilar

2018-06-01

The aim of this study was to develop and validate the Nursing Intensive-Care Satisfaction Scale to measures satisfaction with nursing care from the critical care patient's perspective. Instruments that measure satisfaction with nursing cares have been designed and validated without taking the patient's perspective into consideration. Despite the benefits and advances in measuring satisfaction with nursing care, none instrument is specifically designed to assess satisfaction in intensive care units. Instrument development. The population were all discharged patients (January 2013 - January 2015) from three Intensive Care Units of a third level hospital (N = 200). All assessment instruments were given to discharged patients and 48 hours later, to analyse the temporal stability, only the questionnaire was given again. The validation process of the scale included the analysis of internal consistency, temporal stability; validity of construct through a confirmatory factor analysis; and criterion validity. Reliability was 0.95. The intraclass correlation coefficient for the total scale was 0.83 indicating a good temporal stability. Construct validity showed an acceptable fit and factorial structure with four factors, in accordance with the theoretical model, being Consequences factor the best correlated with other factors. Criterion validity, presented a correlation between low and high (range: 0.42-0.68). The scale has been designed and validated incorporating the perspective of critical care patients. Thanks to its reliability and validity, this questionnaire can be used both in research and in clinical practice. The scale offers a possibility to assess and develop interventions to improve patient satisfaction with nursing care. © 2018 John Wiley & Sons Ltd.

Nurses' Attitudes Regarding the Safe Handling of Patients Who Are Morbidly Obese: Instrument Development and Psychometric Analysis.

PubMed

Bejciy-Spring, Susan; Vermillion, Brenda; Morgan, Sally; Newton, Cheryl; Chucta, Sheila; Gatens, Cindy; Zadvinskis, Inga; Holloman, Christopher; Chipps, Esther

2016-12-01

Nurses' attitudes play an important role in the consistent practice of safe patient handling behaviors. The purposes of this study were to develop and assess the psychometric properties of a newly developed instrument measuring attitudes of nurses related to the care and safe handling of patients who are obese. Phases of instrument development included (a) item generation, (b) content validity assessment, (c) reliability assessment, (d) cognitive interviewing, and (e) construct validity assessment through factor analysis. The final data from the exploratory factor analysis produced a 26-item multidimensional instrument that contains 9 subscales. Based on the factor analysis, a 26-item instrument can be used to examine nurses' attitudes regarding patients who are morbidly obese and related safe handling practices.

The Spiritual Needs Assessment for Patients (SNAP): development and validation of a comprehensive instrument to assess unmet spiritual needs.

PubMed

Sharma, Rashmi K; Astrow, Alan B; Texeira, Kenneth; Sulmasy, Daniel P

2012-07-01

Unmet spiritual needs have been associated with decreased patient ratings of quality of care, satisfaction, and quality of life. There is a need for a well-validated, psychometrically sound instrument to describe and measure spiritual needs. To develop a valid and reliable instrument to assess patients' spiritual needs. Instrument development was based on a literature review, clinical and pastoral evaluation, and cognitive pretesting (n=15 ambulatory cancer patients). Forty-seven ambulatory cancer patients completed cross-sectional and longitudinal surveys to test instrument validity and reliability. Internal reliability was assessed by Cronbach's α, test-retest reliability by Spearman's correlation coefficients, and construct validity by comparing instrument scores to a previously used single-item spiritual needs question. The Spiritual Needs Assessment for Patients (SNAP) comprises a total of 23 items in three domains: psychosocial (n=5), spiritual (n=13), and religious (n=5). Sixty percent of participants were white, 21% black, 13% Hispanic, and 6% Asian or other. Fifty-eight percent were Catholic, 13% Jewish, 11% Protestant, 2% Buddhist, 2% Muslim, and 2% Hindu. Sixty-eight percent described themselves as spiritual but not religious; 15% reported unmet spiritual needs; 19% wanted help meeting their spiritual needs. Cronbach's α for the total SNAP was 0.95, and for the subscales was psychosocial=0.74, spiritual=0.93, and religious needs=0.86. Test-retest correlation coefficients were total SNAP=0.69, psychosocial needs=0.51, spiritual needs=0.70, and religious needs=0.65. Participants reporting unmet spiritual needs had significantly higher mean scores on the total SNAP (66.3 vs. 49.4, P=0.03) and on the spiritual needs subscale (39.0 vs. 28.3, P=0.02). The results provide preliminary evidence that the SNAP is a valid and reliable instrument for measuring spiritual needs in a diverse patient population. Copyright © 2012 U.S. Cancer Pain Relief Committee. Published by Elsevier Inc. All rights reserved.

Self-management in chronic conditions: partners in health scale instrument validation.

PubMed

Peñarrieta-de Córdova, Isabel; Barrios, Flores Florabel; Gutierrez-Gomes, Tranquilina; Piñonez-Martinez, Ma del Socorro; Quintero-Valle, Luz Maria; Castañeda-Hidalgo, Hortensia

2014-03-01

This article describes a study that aimed to validate the Self-care in Chronic Conditions Partners in Health Scale instrument in the Mexican population. The instrument has been validated in Australia for use as a screening tool by primary healthcare professionals to assess the self-care skills and abilities of people with a chronic illness. Validation was conducted using baseline data for 552 people with diabetes, hypertension and cancer aged 18 or older who were users of healthcare centres in Tampico, Tamaulipas, Mexico. Results show high reliability and validity of the instrument and three themes were identified: knowledge, adherence, and dealing with and managing side effects. The findings suggest the scale is useful as a generic self-rated clinical tool for assessing self-management in a range of chronic conditions, and provides an outcome measure for comparing populations and change in patient self-management knowledge and behaviour. The authors recommend validating the scale in other Latin-American settings with more research into the effect of gender on self- management.

Validation of an instrument for assessing teacher knowledge of basic language constructs of literacy.

PubMed

Binks-Cantrell, Emily; Joshi, R Malatesha; Washburn, Erin K

2012-10-01

Recent national reports have stressed the importance of teacher knowledge in teaching reading. However, in the past, teachers' knowledge of language and literacy constructs has typically been assessed with instruments that are not fully tested for validity. In the present study, an instrument was developed; and its reliability, item difficulty, and item discrimination were computed and examined to identify model fit by applying exploratory factor analysis. Such analyses showed that the instrument demonstrated adequate estimates of reliability in assessing teachers' knowledge of language constructs. The implications for professional development of in-service teachers as well as preservice teacher education are also discussed.

Validation of the organizational culture assessment instrument.

PubMed

Heritage, Brody; Pollock, Clare; Roberts, Lynne

2014-01-01

Organizational culture is a commonly studied area in industrial/organizational psychology due to its important role in workplace behaviour, cognitions, and outcomes. Jung et al.'s [1] review of the psychometric properties of organizational culture measurement instruments noted many instruments have limited validation data despite frequent use in both theoretical and applied situations. The Organizational Culture Assessment Instrument (OCAI) has had conflicting data regarding its psychometric properties, particularly regarding its factor structure. Our study examined the factor structure and criterion validity of the OCAI using robust analysis methods on data gathered from 328 (females = 226, males = 102) Australian employees. Confirmatory factor analysis supported a four factor structure of the OCAI for both ideal and current organizational culture perspectives. Current organizational culture data demonstrated expected reciprocally-opposed relationships between three of the four OCAI factors and the outcome variable of job satisfaction but ideal culture data did not, thus indicating possible weak criterion validity when the OCAI is used to assess ideal culture. Based on the mixed evidence regarding the measure's properties, further examination of the factor structure and broad validity of the measure is encouraged.

Validation of the Organizational Culture Assessment Instrument

PubMed Central

Heritage, Brody; Pollock, Clare; Roberts, Lynne

2014-01-01

Organizational culture is a commonly studied area in industrial/organizational psychology due to its important role in workplace behaviour, cognitions, and outcomes. Jung et al.'s [1] review of the psychometric properties of organizational culture measurement instruments noted many instruments have limited validation data despite frequent use in both theoretical and applied situations. The Organizational Culture Assessment Instrument (OCAI) has had conflicting data regarding its psychometric properties, particularly regarding its factor structure. Our study examined the factor structure and criterion validity of the OCAI using robust analysis methods on data gathered from 328 (females = 226, males = 102) Australian employees. Confirmatory factor analysis supported a four factor structure of the OCAI for both ideal and current organizational culture perspectives. Current organizational culture data demonstrated expected reciprocally-opposed relationships between three of the four OCAI factors and the outcome variable of job satisfaction but ideal culture data did not, thus indicating possible weak criterion validity when the OCAI is used to assess ideal culture. Based on the mixed evidence regarding the measure's properties, further examination of the factor structure and broad validity of the measure is encouraged. PMID:24667839

The Faculty Self-Reported Assessment Survey (FRAS): differentiating faculty knowledge and experience in assessment.

PubMed

Hanauer, David I; Bauerle, Cynthia

2015-01-01

Science, technology, engineering, and mathematics education reform efforts have called for widespread adoption of evidence-based teaching in which faculty members attend to student outcomes through assessment practice. Awareness about the importance of assessment has illuminated the need to understand what faculty members know and how they engage with assessment knowledge and practice. The Faculty Self-Reported Assessment Survey (FRAS) is a new instrument for evaluating science faculty assessment knowledge and experience. Instrument validation was composed of two distinct studies: an empirical evaluation of the psychometric properties of the FRAS and a comparative known-groups validation to explore the ability of the FRAS to differentiate levels of faculty assessment experience. The FRAS was found to be highly reliable (α = 0.96). The dimensionality of the instrument enabled distinction of assessment knowledge into categories of program design, instrumentation, and validation. In the known-groups validation, the FRAS distinguished between faculty groups with differing levels of assessment experience. Faculty members with formal assessment experience self-reported higher levels of familiarity with assessment terms, higher frequencies of assessment activity, increased confidence in conducting assessment, and more positive attitudes toward assessment than faculty members who were novices in assessment. These results suggest that the FRAS can reliably and validly differentiate levels of expertise in faculty knowledge of assessment. © 2015 D. I. Hanauer and C. Bauerle. CBE—Life Sciences Education © 2015 The American Society for Cell Biology. This article is distributed by The American Society for Cell Biology under license from the author(s). It is available to the public under an Attribution–Noncommercial–Share Alike 3.0 Unported Creative Commons License (http://creativecommons.org/licenses/by-nc-sa/3.0).

Movie for the Assessment of Social Cognition (MASC): Spanish Validation

ERIC Educational Resources Information Center

Lahera, G.; Boada, L.; Pousa, E.; Mirapeix, I.; Morón-Nozaleda, G.; Marinas, L.; Gisbert, L.; Pamiàs, M.; Parellada, M.

2014-01-01

We present the Spanish validation of the "Movie for the Assessment of Social Cognition" instrument (MASC-SP). We recruited 22 adolescents and young adults with Asperger syndrome and 26 participants with typical development. The MASC-SP and three other social cognition instruments (Ekman Pictures of Facial Affect test, Reading the Mind in…

Development and Cross-National Validation of a Laboratory Classroom Environment Instrument for Senior High School Science.

ERIC Educational Resources Information Center

Fraser, Barry J.; And Others

1993-01-01

Describes the development of the Science Laboratory Environment Inventory (SLEI) instrument for assessing perceptions of the psychosocial environment in science laboratory classrooms, and reports validation information for samples of senior high school students from six different countries. The SLEI assesses five dimensions of the actual and…

The Construction and Initial Validation of the "Student Teachers' Efficacy Scale for Teaching Students with Disabilities"

ERIC Educational Resources Information Center

Zhang, Dake; Wang, Qiu; Stegall, Joanna; Losinki, Mickey; Katsiyannis, Antonis

2018-01-01

Although there have been numerous instruments for assessing teacher efficacy in teaching general education students, there is a need to develop and validate an instrument that specifically assesses teachers' efficacy in teaching students with disabilities. This study constructed and evaluated the psychometric properties of the "Student…

Validation of the translation of an instrument to measure reliability of written information on treatment choices: a study on attention deficit/hyperactivity disorder (ADHD).

PubMed

Montoya, A; Llopis, N; Gilaberte, I

2011-12-01

DISCERN is an instrument designed to help patients assess the reliability of written information on treatment choices. Originally created in English, there is no validated Spanish version of this instrument. This study seeks to validate the Spanish translation of the DISCERN instrument used as a primary measure on a multicenter study aimed to assess the reliability of web-based information on treatment choices for attention deficit/hyperactivity disorder (ADHD). We used a modified version of a method for validating translated instruments in which the original source-language version is formally compared with the back-translated source-language version. Each item was ranked in terms of comparability of language, similarity of interpretability, and degree of understandability. Responses used Likert scales ranging from 1 to 7, where 1 indicates the best interpretability, language and understandability, and 7 indicates the worst. Assessments were performed by 20 raters fluent in the source language. The Spanish translation of DISCERN, based on ratings of comparability, interpretability and degree of understandability (mean score (SD): 1.8 (1.1), 1.4 (0.9) and 1.6 (1.1), respectively), was considered extremely comparable. All items received a score of less than three, therefore no further revision of the translation was needed. The validation process showed that the quality of DISCERN translation was high, validating the comparable language of the tool translated on assessing written information on treatment choices for ADHD.

Assessing cross-cultural validity of scales: a methodological review and illustrative example.

PubMed

Beckstead, Jason W; Yang, Chiu-Yueh; Lengacher, Cecile A

2008-01-01

In this article, we assessed the cross-cultural validity of the Women's Role Strain Inventory (WRSI), a multi-item instrument that assesses the degree of strain experienced by women who juggle the roles of working professional, student, wife and mother. Cross-cultural validity is evinced by demonstrating the measurement invariance of the WRSI. Measurement invariance is the extent to which items of multi-item scales function in the same way across different samples of respondents. We assessed measurement invariance by comparing a sample of working women in Taiwan with a similar sample from the United States. Structural equation models (SEMs) were employed to determine the invariance of the WRSI and to estimate the unique validity variance of its items. This article also provides nurse-researchers with the necessary underlying measurement theory and illustrates how SEMs may be applied to assess cross-cultural validity of instruments used in nursing research. Overall performance of the WRSI was acceptable but our analysis showed that some items did not display invariance properties across samples. Item analysis is presented and recommendations for improving the instrument are discussed.

Two-Tiered Violence Risk Estimates: a validation study of an integrated-actuarial risk assessment instrument.

PubMed

Mills, Jeremy F; Gray, Andrew L

2013-11-01

This study is an initial validation study of the Two-Tiered Violence Risk Estimates instrument (TTV), a violence risk appraisal instrument designed to support an integrated-actuarial approach to violence risk assessment. The TTV was scored retrospectively from file information on a sample of violent offenders. Construct validity was examined by comparing the TTV with instruments that have shown utility to predict violence that were prospectively scored: The Historical-Clinical-Risk Management-20 (HCR-20) and Lifestyle Criminality Screening Form (LCSF). Predictive validity was examined through a long-term follow-up of 12.4 years with a sample of 78 incarcerated offenders. Results show the TTV to be highly correlated with the HCR-20 and LCSF. The base rate for violence over the follow-up period was 47.4%, and the TTV was equally predictive of violent recidivism relative to the HCR-20 and LCSF. Discussion centers on the advantages of an integrated-actuarial approach to the assessment of violence risk.

Assessing patient capacity to consent to treatment: an integrative review of instruments and tools.

PubMed

Lamont, Scott; Jeon, Yun-Hee; Chiarella, Mary

2013-09-01

To provide a narrative synthesis of research findings on instruments or tools designed to aid assessment of patient capacity to consent to treatment. Capacity assessment is of significant priority within health care as a finding of incapacity is a vehicle for the removal of many of an individual's fundamental rights. Despite there being many instruments and tools available to aid health professionals in the assessment of patient capacity, there are no standardised guidelines from professional bodies that inform the assessment of mental capacity. Integrative review. Primary studies of instruments or tools concerning assessment of patient capacity to consent to treatment, published in English in peer-reviewed journals between January 2005-December 2010, were included in the review. Review papers of capacity assessment instruments were included for years including and prior to 2006. Nineteen instruments were found which assess patient capacity to consent. Key themes were identified in terms of capacity domains assessed, psychometric properties, instrument implementation, patient populations studied and instrument versus clinician judgement. Despite a plethora of capacity assessment instruments and tools available, only a small number of instruments were found to have demonstrated both reliability and validity. Further research is required to improve the validity of existing capacity assessment instruments. Increased attention to patient rights and autonomy arguably places a considerable burden on healthcare professionals to facilitate capacity assessments across a continuum of health care. Despite a plethora of capacity assessment instruments and tools being available to healthcare professionals, a comprehensive assessment requires time and is often difficult in the acute care setting. A strictly formulaic approach to the assessment of capacity is unlikely to capture specific individual nuances; therefore, capacity assessment instruments should support, but not replace, experienced clinical judgement. © 2013 John Wiley & Sons Ltd.

The Construct and Predictive Validity of a Dynamic Assessment of Young Children Learning to Read: Implications for RTI Frameworks

ERIC Educational Resources Information Center

Fuchs, Douglas; Compton, Donald L.; Fuchs, Lynn S.; Bouton, Bobette; Caffrey, Erin

2011-01-01

The purpose of this study was to examine the construct and predictive validity of a dynamic assessment (DA) of decoding learning. Students (N = 318) were assessed in the fall of first grade on an array of instruments that were given in hopes of forecasting responsiveness to reading instruction. These instruments included DA as well as…

Validation of Modified Soft Skills Assessment Instrument (MOSSAI) for Use in Nigeria

ERIC Educational Resources Information Center

Aworanti, O. A.; Taiwo, M. B.; Iluobe, O. I.

2015-01-01

Currently, it has become an accepted norm nearly all over the globe to teach and assess soft skills. However, in Nigeria, it is an emerging area of interest that needs to be addressed squarely. In the light of the fore-going, this study validated a modified version of Measuring and Assessment Soft Skills (MASS) (an instrument developed and used by…

Assessing medical professionalism: A systematic review of instruments and their measurement properties.

PubMed

Li, Honghe; Ding, Ning; Zhang, Yuanyuan; Liu, Yang; Wen, Deliang

2017-01-01

Over the last three decades, various instruments were developed and employed to assess medical professionalism, but their measurement properties have yet to be fully evaluated. This study aimed to systematically evaluate these instruments' measurement properties and the methodological quality of their related studies within a universally acceptable standardized framework and then provide corresponding recommendations. A systematic search of the electronic databases PubMed, Web of Science, and PsycINFO was conducted to collect studies published from 1990-2015. After screening titles, abstracts, and full texts for eligibility, the articles included in this study were classified according to their respective instrument's usage. A two-phase assessment was conducted: 1) methodological quality was assessed by following the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist; and 2) the quality of measurement properties was assessed according to Terwee's criteria. Results were integrated using best-evidence synthesis to look for recommendable instruments. After screening 2,959 records, 74 instruments from 80 existing studies were included. The overall methodological quality of these studies was unsatisfactory, with reasons including but not limited to unknown missing data, inadequate sample sizes, and vague hypotheses. Content validity, cross-cultural validity, and criterion validity were either unreported or negative ratings in most studies. Based on best-evidence synthesis, three instruments were recommended: Hisar's instrument for nursing students, Nurse Practitioners' Roles and Competencies Scale, and Perceived Faculty Competency Inventory. Although instruments measuring medical professionalism are diverse, only a limited number of studies were methodologically sound. Future studies should give priority to systematically improving the performance of existing instruments and to longitudinal studies.

The development and testing of a qualitative instrument designed to assess critical thinking

NASA Astrophysics Data System (ADS)

Clauson, Cynthia Louisa

This study examined a qualitative approach to assess critical thinking. An instrument was developed that incorporates an assessment process based on Dewey's (1933) concepts of self-reflection and critical thinking as problem solving. The study was designed to pilot test the critical thinking assessment process with writing samples collected from a heterogeneous group of students. The pilot test included two phases. Phase 1 was designed to determine the validity and inter-rater reliability of the instrument using two experts in critical thinking, problem solving, and literacy development. Validity of the instrument was addressed by requesting both experts to respond to ten questions in an interview. The inter-rater reliability was assessed by analyzing the consistency of the two experts' scorings of the 20 writing samples to each other, as well as to my scoring of the same 20 writing samples. Statistical analyses included the Spearman Rho and the Kuder-Richardson (Formula 20). Phase 2 was designed to determine the validity and reliability of the critical thinking assessment process with seven science teachers. Validity was addressed by requesting the teachers to respond to ten questions in a survey and interview. Inter-rater reliability was addressed by comparing the seven teachers' scoring of five writing samples with my scoring of the same five writing samples. Again, the Spearman Rho and the Kuder-Richardson (Formula 20) were used to determine the inter-rater reliability. The validity results suggest that the instrument is helpful as a guide for instruction and provides a systematic method to teach and assess critical thinking while problem solving with students in the classroom. The reliability results show the critical thinking assessment instrument to possess fairly high reliability when used by the experts, but weak reliability when used by classroom teachers. A major conclusion was drawn that teachers, as well as students, would need to receive instruction in critical thinking and in how to use the assessment process in order to gain more consistent interpretations of the six problem-solving steps. Specific changes needing to be made in the instrument to improve the quality are included.

Instruments assessing attitudes toward or capability regarding self-management of osteoarthritis: a systematic review of measurement properties.

PubMed

Eyles, J P; Hunter, D J; Meneses, S R F; Collins, N J; Dobson, F; Lucas, B R; Mills, K

2017-08-01

To make a recommendation on the "best" instrument to assess attitudes toward and/or capabilities regarding self-management of osteoarthritis (OA) based on available measurement property evidence. Electronic searches were performed in MEDLINE, EMBASE, CINAHL and PsychINFO (inception to 27 December 2016). Two reviewers independently rated measurement properties using the Consensus-based Standards for the selection of Health Measurement Instruments (COSMIN) 4-point scale. Best evidence synthesis was determined by considering COSMIN ratings for measurement property results and the level of evidence available for each measurement property of each instrument. Eight studies out of 5653 publications met the inclusion criteria, with eight instruments identified for evaluation: Multidimensional Health Locus of Control (MHLC), Perceived Behavioural Control (PBC), Patient Activation Measure (PAM), Educational Needs Assessment (ENAT), Stages of Change Questionnaire in Osteoarthritis (SCQOA), Effective Consumer Scale (EC-17) and Perceived Efficacy in Patient-Physician Interactions five item (PEPPI-5) and ten item scales. Measurement properties assessed for these instruments included internal consistency (k = 8), structural validity (k = 8), test-retest reliability (k = 2), measurement error (k = 1), hypothesis testing (k = 3) and cross-cultural validity (k = 3). No information was available for content validity, responsiveness or minimal important change (MIC)/minimal important difference (MID). The Dutch PEPPI-5 demonstrated the best measurement property evidence; strong evidence for internal consistency and structural validity but limited evidence for reliability and construct validity. Although PEPPI-5 was identified as having the best measurement properties, overall there is a poor level of evidence currently available concerning measurement properties of instruments to assess attitudes toward and/or capabilities regarding osteoarthritis self-management. Further well-designed studies investigating measurement properties of existing instruments are required. Copyright © 2017 Osteoarthritis Research Society International. All rights reserved.

Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 1--eliciting concepts for a new PRO instrument.

PubMed

Patrick, Donald L; Burke, Laurie B; Gwaltney, Chad J; Leidy, Nancy Kline; Martin, Mona L; Molsen, Elizabeth; Ring, Lena

2011-12-01

The importance of content validity in developing patient reported outcomes (PRO) instruments is stressed by both the US Food and Drug Administration and the European Medicines Agency. Content validity is the extent to which an instrument measures the important aspects of concepts that developers or users purport it to assess. A PRO instrument measures the concepts most significant and relevant to a patient's condition and its treatment. For PRO instruments, items and domains as reflected in the scores of an instrument should be important to the target population and comprehensive with respect to patient concerns. Documentation of target population input in item generation, as well as evaluation of patient understanding through cognitive interviewing, can provide the evidence for content validity. Developing content for, and assessing respondent understanding of, newly developed PRO instruments for medical product evaluation will be discussed in this two-part ISPOR PRO Good Research Practices Task Force Report. Topics include the methods for generating items, documenting item development, coding of qualitative data from item generation, cognitive interviewing, and tracking item development through the various stages of research and preparing this tracking for submission to regulatory agencies. Part 1 covers elicitation of key concepts using qualitative focus groups and/or interviews to inform content and structure of a new PRO instrument. Part 2 covers the instrument development process, the assessment of patient understanding of the draft instrument using cognitive interviews and steps for instrument revision. The two parts are meant to be read together. They are intended to offer suggestions for good practices in planning, executing, and documenting qualitative studies that are used to support the content validity of PRO instruments to be used in medical product evaluation. Copyright © 2011 International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. All rights reserved.

Fitness to plead: Development and validation of a standardised assessment instrument

PubMed Central

Stahl, Daniel; Appiah-Kusi, Elizabeth; Brewer, Rebecca; Watts, Michael; Peay, Jill; Blackwood, Nigel

2018-01-01

The ability of an individual to participate in courtroom proceedings is assessed by clinicians using legal ‘fitness to plead’ criteria. Findings of ‘unfitness’ are so rare that there is considerable professional unease concerning the utility of the current subjective assessment process. As a result, mentally disordered defendants may be subjected unfairly to criminal trials. The Law Commission in England and Wales has proposed legal reform, as well as the utilisation of a defined psychiatric instrument to assist in fitness to plead assessments. Similar legal reforms are occurring in other jurisdictions. Our objective was to produce and validate a standardised assessment instrument of fitness to plead employing a filmed vignette of criminal proceedings. The instrument was developed in consultation with legal and clinical professionals, and was refined using standard item reduction methods in two initial rounds of testing (n = 212). The factorial structure, test-retest reliability and convergent validity of the resultant instrument were assessed in a further round (n = 160). As a result of this iterative process a 25-item scale was produced, with an underlying two-factor structure representing the foundational and decision-making abilities underpinning fitness to plead. The sub-scales demonstrate good internal consistency (factor 1: 0·76; factor 2: 0·65) and test-retest stability (0·7) as well as excellent convergent validity with scores of intelligence, executive function and mentalising abilities (p≤0·01 in all domains). Overall the standardised Fitness to Plead Assessment instrument has good psychometric properties. It has the potential to ensure that the significant numbers of mentally ill and cognitively impaired individuals who face trial are objectively assessed, and the courtroom process critically informed. PMID:29698396

Validation of a method for assessing resident physicians' quality improvement proposals.

PubMed

Leenstra, James L; Beckman, Thomas J; Reed, Darcy A; Mundell, William C; Thomas, Kris G; Krajicek, Bryan J; Cha, Stephen S; Kolars, Joseph C; McDonald, Furman S

2007-09-01

Residency programs involve trainees in quality improvement (QI) projects to evaluate competency in systems-based practice and practice-based learning and improvement. Valid approaches to assess QI proposals are lacking. We developed an instrument for assessing resident QI proposals--the Quality Improvement Proposal Assessment Tool (QIPAT-7)-and determined its validity and reliability. QIPAT-7 content was initially obtained from a national panel of QI experts. Through an iterative process, the instrument was refined, pilot-tested, and revised. Seven raters used the instrument to assess 45 resident QI proposals. Principal factor analysis was used to explore the dimensionality of instrument scores. Cronbach's alpha and intraclass correlations were calculated to determine internal consistency and interrater reliability, respectively. QIPAT-7 items comprised a single factor (eigenvalue = 3.4) suggesting a single assessment dimension. Interrater reliability for each item (range 0.79 to 0.93) and internal consistency reliability among the items (Cronbach's alpha = 0.87) were high. This method for assessing resident physician QI proposals is supported by content and internal structure validity evidence. QIPAT-7 is a useful tool for assessing resident QI proposals. Future research should determine the reliability of QIPAT-7 scores in other residency and fellowship training programs. Correlations should also be made between assessment scores and criteria for QI proposal success such as implementation of QI proposals, resident scholarly productivity, and improved patient outcomes.

Development and Psychometric Evaluation of an Instrument to Assess Cross-Cultural Competence of Healthcare Professionals (CCCHP)

PubMed Central

Bernhard, Gerda; Knibbe, Ronald A.; von Wolff, Alessa; Dingoyan, Demet; Schulz, Holger; Mösko, Mike

2015-01-01

Background Cultural competence of healthcare professionals (HCPs) is recognized as a strategy to reduce cultural disparities in healthcare. However, standardised, valid and reliable instruments to assess HCPs’ cultural competence are notably lacking. The present study aims to 1) identify the core components of cultural competence from a healthcare perspective, 2) to develop a self-report instrument to assess cultural competence of HCPs and 3) to evaluate the psychometric properties of the new instrument. Methods The conceptual model and initial item pool, which were applied to the cross-cultural competence instrument for the healthcare profession (CCCHP), were derived from an expert survey (n = 23), interviews with HCPs (n = 12), and a broad narrative review on assessment instruments and conceptual models of cultural competence. The item pool was reduced systematically, which resulted in a 59-item instrument. A sample of 336 psychologists, in advanced psychotherapeutic training, and 409 medical students participated, in order to evaluate the construct validity and reliability of the CCCHP. Results Construct validity was supported by principal component analysis, which led to a 32-item six-component solution with 50% of the total variance explained. The different dimensions of HCPs’ cultural competence are: Cross-Cultural Motivation/Curiosity, Cross-Cultural Attitudes, Cross-Cultural Skills, Cross-Cultural Knowledge/Awareness and Cross-Cultural Emotions/Empathy. For the total instrument, the internal consistency reliability was .87 and the dimension’s Cronbach’s α ranged from .54 to .84. The discriminating power of the CCCHP was indicated by statistically significant mean differences in CCCHP subscale scores between predefined groups. Conclusions The 32-item CCCHP exhibits acceptable psychometric properties, particularly content and construct validity to examine HCPs’ cultural competence. The CCCHP with its five dimensions offers a comprehensive assessment of HCPs’ cultural competence, and has the ability to distinguish between groups that are expected to differ in cultural competence. This instrument can foster professional development through systematic self-assessment and thus contributes to improve the quality of patient care. PMID:26641876

Development and Psychometric Evaluation of an Instrument to Assess Cross-Cultural Competence of Healthcare Professionals (CCCHP).

PubMed

Bernhard, Gerda; Knibbe, Ronald A; von Wolff, Alessa; Dingoyan, Demet; Schulz, Holger; Mösko, Mike

2015-01-01

Cultural competence of healthcare professionals (HCPs) is recognized as a strategy to reduce cultural disparities in healthcare. However, standardised, valid and reliable instruments to assess HCPs' cultural competence are notably lacking. The present study aims to 1) identify the core components of cultural competence from a healthcare perspective, 2) to develop a self-report instrument to assess cultural competence of HCPs and 3) to evaluate the psychometric properties of the new instrument. The conceptual model and initial item pool, which were applied to the cross-cultural competence instrument for the healthcare profession (CCCHP), were derived from an expert survey (n = 23), interviews with HCPs (n = 12), and a broad narrative review on assessment instruments and conceptual models of cultural competence. The item pool was reduced systematically, which resulted in a 59-item instrument. A sample of 336 psychologists, in advanced psychotherapeutic training, and 409 medical students participated, in order to evaluate the construct validity and reliability of the CCCHP. Construct validity was supported by principal component analysis, which led to a 32-item six-component solution with 50% of the total variance explained. The different dimensions of HCPs' cultural competence are: Cross-Cultural Motivation/Curiosity, Cross-Cultural Attitudes, Cross-Cultural Skills, Cross-Cultural Knowledge/Awareness and Cross-Cultural Emotions/Empathy. For the total instrument, the internal consistency reliability was .87 and the dimension's Cronbach's α ranged from .54 to .84. The discriminating power of the CCCHP was indicated by statistically significant mean differences in CCCHP subscale scores between predefined groups. The 32-item CCCHP exhibits acceptable psychometric properties, particularly content and construct validity to examine HCPs' cultural competence. The CCCHP with its five dimensions offers a comprehensive assessment of HCPs' cultural competence, and has the ability to distinguish between groups that are expected to differ in cultural competence. This instrument can foster professional development through systematic self-assessment and thus contributes to improve the quality of patient care.

Structural Validation of the Holistic Wellness Assessment

ERIC Educational Resources Information Center

Brown, Charlene; Applegate, E. Brooks; Yildiz, Mustafa

2015-01-01

The Holistic Wellness Assessment (HWA) is a relatively new assessment instrument based on an emergent transdisciplinary model of wellness. This study validated the factor structure identified via exploratory factor analysis (EFA), assessed test-retest reliability, and investigated concurrent validity of the HWA in three separate samples. The…

Guidelines, Criteria, and Rules of Thumb for Evaluating Normed and Standardized Assessment Instruments in Psychology.

ERIC Educational Resources Information Center

Cicchetti, Domenic V.

1994-01-01

In the context of developing assessment instruments in psychology, issues of standardization, norming procedures, and test reliability and validity are discussed. Criteria, guidelines, and rules of thumb are provided to help the clinician with instrument selection for a given psychological assessment. (SLD)

A systematic review of instruments for assessing parent satisfaction with family-centred care in neonatal intensive care units.

PubMed

Dall'Oglio, Immacolata; Mascolo, Rachele; Gawronski, Orsola; Tiozzo, Emanuela; Portanova, Anna; Ragni, Angela; Alvaro, Rosaria; Rocco, Gennaro; Latour, Jos M

2018-03-01

This systematic review synthesised and described instruments measuring parent satisfaction with the increasing standard practice of family-centred care (FCC) in neonatal intensive care units. We evaluated 11 studies published from January 2006 to March 2016: two studies validated a parent satisfaction questionnaire, and nine developed or modified previous questionnaires to use as outcome measures in their local settings. Most instruments were not tested on reliability and validity. Only two validated instruments included all six of the FCC principles and could assess parent satisfaction with FCC in neonatal intensive care units and be considered as outcome indicators for further research. ©2017 Foundation Acta Paediatrica. Published by John Wiley & Sons Ltd.

Format Effects of Empirically Derived Multiple-Choice versus Free-Response Instruments When Assessing Graphing Abilities

ERIC Educational Resources Information Center

Berg, Craig; Boote, Stacy

2017-01-01

Prior graphing research has demonstrated that clinical interviews and free-response instruments produce very different results than multiple-choice instruments, indicating potential validity problems when using multiple-choice instruments to assess graphing skills (Berg & Smith in "Science Education," 78(6), 527-554, 1994). Extending…

Assessing the Generalizable Skills of Post-Secondary Vocational Students. A Validation Study.

ERIC Educational Resources Information Center

Greenan, James P.; Smith, Brandon B.

A study examined the feasibility, reliability, and validity of two instruments designed to assess the degree to which postsecondary vocational students possessed those generalizable skills that are believed to be functionally relevant to success in a vocational program. The instruments, a student self-rating and a teacher rating form, contained 81…

The Development and Validation of the Inventory of Family Protective Factors: A Brief Assessment for Family Counseling

ERIC Educational Resources Information Center

Gardner, Deborah L.; Huber, Charles H.; Steiner, Robert; Vazquez, Luis A.; Savage, Todd A.

2008-01-01

This article describes the three-stage protocol employed in development and validation of the Inventory of Family Protective Factors (IFPF), a brief-form formal instrument intended to assess the primary protective factors that contribute to family resilience. Following construction of the instrument, data collections and analyses of a total sample…

Analysis of the Reliability and Validity of a Mentor's Assessment for Principal Internships

ERIC Educational Resources Information Center

Koonce, Glenn L.; Kelly, Michael D.

2014-01-01

In this study, researchers analyzed the reliability and validity of the mentor's assessment for principal internships at a university in the Southeast region of the United States. The results of the study yielded how trustworthy and dependable the instrument is and the effectiveness of the instrument in the current principal preparation program.…

Assessing personal talent determinants in young racquet sport players: a systematic review.

PubMed

Faber, Irene R; Bustin, Paul M J; Oosterveld, Frits G J; Elferink-Gemser, Marije T; Nijhuis-Van der Sanden, Maria W G

2016-01-01

Since junior performances have little predictive value for future success, other solutions are sought to assess a young player's potential. The objectives of this systematic review are (1) to provide an overview of instruments measuring personal talent determinants of young players in racquet sports, and (2) to evaluate these instruments regarding their validity for talent development. Electronic searches were conducted in PubMed, PsychINFO, Web of Knowledge, ScienceDirect and SPORTDiscus (1990 to 31 March 2014). Search terms represented tennis, table tennis, badminton and squash, the concept of talent, methods of testing and children. Thirty articles with information regarding over 100 instruments were included. Validity evaluation showed that instruments focusing on intellectual and perceptual abilities, and coordinative skills discriminate elite from non-elite players and/or are related to current performance, but their predictive validity is not confirmed. There is moderate evidence that the assessments of mental and goal management skills predict future performance. Data on instruments measuring physical characteristics prohibit a conclusion due to conflicting findings. This systematic review yielded an ambiguous end point. The lack of longitudinal studies precludes verification of the instrument's capacity to forecast future performance. Future research should focus on instruments assessing multidimensional talent determinants and their predictive value in longitudinal designs.

Validity and reliability of a Malay version of the Lawton instrumental activities of daily living scale among the Malay speaking elderly in Malaysia.

PubMed

Kadar, Masne; Ibrahim, Suhaili; Razaob, Nor Afifi; Chai, Siaw Chui; Harun, Dzalani

2018-02-01

The Lawton Instrumental Activities of Daily Living Scale is a tool often used to assess independence among elderly at home. Its suitability to be used with the elderly population in Malaysia has not been validated. This current study aimed to assess the validity and reliability of the Lawton Instrumental Activities of Daily Living Scale - Malay Version to Malay speaking elderly in Malaysia. This study was divided into three phases: (1) translation and linguistic validity involving both forward and backward translations; (2) establishment of face validity and content validity; and (3) establishment of reliability involving inter-rater, test-retest and internal consistency analyses. Data used for these analyses were obtained by interviewing 65 elderly respondents. Percentages of Content Validity Index for 4 criteria were from 88.89 to 100.0. The Cronbach α coefficient for internal consistency was 0.838. Intra-class Correlation Coefficient of inter-rater reliability and test-retest reliability was 0.957 and 0.950 respectively. The result shows that the Lawton Instrumental Activities of Daily Living Scale - Malay Version has excellent reliability and validity for use with the Malay speaking elderly people in Malaysia. This scale could be used by professionals to assess functional ability of elderly who live independently in community. © 2018 Occupational Therapy Australia.

Quality appraisal of generic self-reported instruments measuring health-related productivity changes: a systematic review

PubMed Central

2014-01-01

Background Health impairments can result in disability and changed work productivity imposing considerable costs for the employee, employer and society as a whole. A large number of instruments exist to measure health-related productivity changes; however their methodological quality remains unclear. This systematic review critically appraised the measurement properties in generic self-reported instruments that measure health-related productivity changes to recommend appropriate instruments for use in occupational and economic health practice. Methods PubMed, PsycINFO, Econlit and Embase were systematically searched for studies whereof: (i) instruments measured health-related productivity changes; (ii) the aim was to evaluate instrument measurement properties; (iii) instruments were generic; (iv) ratings were self-reported; (v) full-texts were available. Next, methodological quality appraisal was based on COSMIN elements: (i) internal consistency; (ii) reliability; (iii) measurement error; (iv) content validity; (v) structural validity; (vi) hypotheses testing; (vii) cross-cultural validity; (viii) criterion validity; and (ix) responsiveness. Recommendations are based on evidence syntheses. Results This review included 25 articles assessing the reliability, validity and responsiveness of 15 different generic self-reported instruments measuring health-related productivity changes. Most studies evaluated criterion validity, none evaluated cross-cultural validity and information on measurement error is lacking. The Work Limitation Questionnaire (WLQ) was most frequently evaluated with moderate respectively strong positive evidence for content and structural validity and negative evidence for reliability, hypothesis testing and responsiveness. Less frequently evaluated, the Stanford Presenteeism Scale (SPS) showed strong positive evidence for internal consistency and structural validity, and moderate positive evidence for hypotheses testing and criterion validity. The Productivity and Disease Questionnaire (PRODISQ) yielded strong positive evidence for content validity, evidence for other properties is lacking. The other instruments resulted in mostly fair-to-poor quality ratings with limited evidence. Conclusions Decisions based on the content of the instrument, usage purpose, target country and population, and available evidence are recommended. Until high-quality studies are in place to accurately assess the measurement properties of the currently available instruments, the WLQ and, in a Dutch context, the PRODISQ are cautiously preferred based on its strong positive evidence for content validity. Based on its strong positive evidence for internal consistency and structural validity, the SPS is cautiously recommended. PMID:24495301

The measurement of collaboration within healthcare settings: a systematic review of measurement properties of instruments.

PubMed

Walters, Stephen John; Stern, Cindy; Robertson-Malt, Suzanne

2016-04-01

There is a growing call by consumers and governments for healthcare to adopt systems and approaches to care to improve patient safety. Collaboration within healthcare settings is an important factor for improving systems of care. By using validated measurement instruments a standardized approach to assessing collaboration is possible, otherwise it is only an assumption that collaboration is occurring in any healthcare setting. The objective of this review was to evaluate and compare measurement properties of instruments that measure collaboration within healthcare settings, specifically those which have been psychometrically tested and validated. Participants could be healthcare professionals, the patient or any non-professional who contributes to a patient's care, for example, family members, chaplains or orderlies. The term participant type means the designation of any one participant; for example 'nurse', 'social worker' or 'administrator'. More than two participant types was mandatory. The focus of this review was the validity of tools used to measure collaboration within healthcare settings. The types of studies considered for inclusion were validation studies, but quantitative study designs such as randomized controlled trials, controlled trials and case studies were also eligible for inclusion. Studies that focused on Interprofessional Education, were published as an abstract only, contained patient self-reporting only or were not about care delivery were excluded. The outcome of interest was validation and interpretability of the instrument being assessed and included content validity, construct validity and reliability. Interpretability is characterized by statistics such as mean and standard deviation which can be translated to a qualitative meaning. The search strategy aimed to find both published and unpublished studies. A three-step search strategy was utilized in this review. The databases searched included PubMed, CINAHL, Embase, Cochrane Central Register of Controlled Trials, Emerald Fulltext, MD Consult Australia, PsycARTICLES, Psychology and Behavioural Sciences Collection, PsycINFO, Informit Health Databases, Scopus, UpToDate and Web of Science. The search for unpublished studies included EThOS (Electronic Thesis Online Service), Index to Theses and ProQuest- Dissertations and Theses. The assessment of methodological quality of the included studies was undertaken using the COSMIN checklist which is a validated tool that assesses the process of design and validation of healthcare measurement instruments. An Excel spreadsheet version of COSMIN was developed for data collection which included a worksheet for extracting participant characteristics and interpretability data. Statistical pooling of data was not possible for this review. Therefore, the findings are presented in a narrative form including tables and figures to aid in data presentation. To make a synthesis of the assessments of methodological quality of the different studies, each instrument was rated by accounting for the number of studies performed with an instrument, the appraisal of methodological quality and the consistency of results between studies. Twenty-one studies of 12 instruments were included in the review. The studies were diverse in their theoretical underpinnings, target population/setting and measurement objectives. Measurement objectives included: investigating beliefs, behaviors, attitudes, perceptions and relationships associated with collaboration; measuring collaboration between different levels of care or within a multi-rater/target group; assessing collaboration across teams; or assessing internal participation of both teams and patients.Studies produced validity or interpretability data but none of the studies assessed all validity and reliability properties. However, most of the included studies produced a factor structure or referred to prior factor analysis. A narrative synthesis of the individual study factor structures was generated consisting of nine headings: organizational settings, support structures, purpose and goals; communication; reflection on process; cooperation; coordination; role interdependence and partnership; relationships; newly created professional activities; and professional flexibility. Among the many instruments that measure collaboration within healthcare settings, the quality of each instrument varies; instruments are designed for specific populations and purposes, and are validated in various settings. Selecting an instrument requires careful consideration of the qualities of each. Therefore, referring to systematic reviews of measurement properties of instruments may be helpful to clinicians or researchers in instrument selection. Systematic reviews of measurement properties of instruments are valuable in aiding in instrument selection. This systematic review may be useful in instrument selection for the measurement of collaboration within healthcare settings with a complex mix of participant types. Evaluating collaboration provides important information on the strengths and limitations of different healthcare settings and the opportunities for continuous improvement via any remedial actions initiated. Development of a tool that can be used to measure collaboration within teams of healthcare professionals and non-professionals is important for practice. The use of different statistical modelling techniques, such as Item Response Theory modelling and the translation of models into Computer Adaptive Tests, may prove useful. Measurement equivalence is an important consideration for future instrument development and validation. Further development of the COSMIN tool should include appraisal for measurement equivalence. Researchers developing and validating measurement tools should consider multi-method research designs.

[Development of an Instrument to Assess the Quality of Childbirth Care from the Mother's Perspective].

PubMed

Jeong, Geum Hee; Kim, Hyun Kyoung; Kim, Young Hee; Kim, Sun Hee; Lee, Sun Hee; Kim, Kyung Won

2018-02-01

This study aimed to develop an instrument to assess the quality of childbirth care from the perspective of a mother after delivery. The instrument was developed from a literature review, interviews, and item validation. Thirty-eight items were compiled for the instrument. The data for validity and reliability testing were collected using a questionnaire survey conducted on 270 women who had undergone normal vaginal delivery in Korea and analyzed with descriptive statistics, exploratory factor analysis, and reliability coefficients. The exploratory factor analysis reduced the number of items in the instrument to 28 items that were factored into four subscales: family-centered care, personal care, emotional empowerment, and information provision. With respect to convergence validation, there was positive correlation between this instrument and birth satisfaction scale (r=.34, p<.001). The internal consistency reliability was acceptable (Cronbach's alpha =.96). This instrument could be used as a measure of the quality of nursing care for women who have a normal vaginal delivery. © 2018 Korean Society of Nursing Science.

Development and Validation of a Novel Generic Health-related Quality of Life Instrument With 20 Items (HINT-20).

PubMed

Jo, Min-Woo; Lee, Hyeon-Jeong; Kim, Soo Young; Kim, Seon-Ha; Chang, Hyejung; Ahn, Jeonghoon; Ock, Minsu

2017-01-01

Few attempts have been made to develop a generic health-related quality of life (HRQoL) instrument and to examine its validity and reliability in Korea. We aimed to do this in our present study. After a literature review of existing generic HRQoL instruments, a focus group discussion, in-depth interviews, and expert consultations, we selected 30 tentative items for a new HRQoL measure. These items were evaluated by assessing their ceiling effects, difficulty, and redundancy in the first survey. To validate the HRQoL instrument that was developed, known-groups validity and convergent/discriminant validity were evaluated and its test-retest reliability was examined in the second survey. Of the 30 items originally assessed for the HRQoL instrument, four were excluded due to high ceiling effects and six were removed due to redundancy. We ultimately developed a HRQoL instrument with a reduced number of 20 items, known as the Health-related Quality of Life Instrument with 20 items (HINT-20), incorporating physical, mental, social, and positive health dimensions. The results of the HINT-20 for known-groups validity were poorer in women, the elderly, and those with a low income. For convergent/discriminant validity, the correlation coefficients of items (except vitality) in the physical health dimension with the physical component summary of the Short Form 36 version 2 (SF-36v2) were generally higher than the correlations of those items with the mental component summary of the SF-36v2, and vice versa. Regarding test-retest reliability, the intraclass correlation coefficient of the total HINT-20 score was 0.813 (p<0.001). A novel generic HRQoL instrument, the HINT-20, was developed for the Korean general population and showed acceptable validity and reliability.

The construction and validation of an instrument for the assessment of graduates of undergraduate nursing courses 1

PubMed Central

Vieira, Maria Aparecida; Ohara, Conceição Vieira da Silva; de Domenico, Edvane Birelo Lopes

2016-01-01

Abstract Objective: to construct an instrument for the assessment of graduates of undergraduate nursing courses and to validate this instrument through the consensus of specialists. Method: methodological study. In order to elaborate the instrument, documental analysis and a literature review were undertaken. Validation took place through use of the Delphi Conference, between September 2012 and September 2013, in which 36 specialists from Brazilian Nursing participated. In order to analyze reliability, the Cronbach alpha coefficient, the item/total correlation, and the Pearson correlation coefficient were calculated. Results: the instrument was constructed with the participation of specialist nurses representing all regions of Brazil, with experience in lecturing and research. The first Delphi round led to changes in the first instrument, which was restructured and submitted to another round, with a response rate of 94.44%. In the second round, the instrument was validated with a Cronbach alpha of 0.75. Conclusion: the final instrument possessed three dimensions related to the characterization of the graduate, insertion in the job market, and evaluation of the professional training process. This instrument may be used across the territory of Brazil as it is based on the curricular guidelines and contributes to the process of regulation of the quality of the undergraduate courses in Nursing. PMID:27305184

Measurement properties of instruments evaluating self-care and related concepts in people with chronic obstructive pulmonary disease: A systematic review.

PubMed

Clari, Marco; Matarese, Maria; Alvaro, Rosaria; Piredda, Michela; De Marinis, Maria Grazia

2016-01-01

The use of valid and reliable instruments for assessing self-care is crucial for the evaluation of chronic obstructive pulmonary disease (COPD) management programs. The aim of this review is to evaluate the measurement properties and theoretical foundations of instruments for assessing self-care and related concepts in people with COPD. A systematic review was conducted of articles describing the development and validation of self-care instruments. The methodological quality of the measurement properties was assessed using the COSMIN checklist. Ten studies were included evaluating five instruments: three for assessing self-care and self-management and two for assessing self-efficacy. The COPD Self-Efficacy Scale was the most studied instrument, but due to poor study methodological quality, evidence about its measurement properties is inconclusive. Evidence from the COPD Self-Management Scale is more promising, but only one study tested its properties. Due to inconclusive evidence of their measurement properties, no instrument can be recommended for clinical use. Copyright © 2016 Elsevier Inc. All rights reserved.

Updating the OMERACT Filter: Implications for imaging and soluble biomarkers

PubMed Central

D’Agostino, Maria-Antonietta; Boers, Maarten; Kirwan, John; van der Heijde, Desirée; Østergaard, Mikkel; Schett, Georg; Landewé, Robert B.M.; Maksymowych, Walter P.; Naredo, Esperanza; Dougados, Maxime; Iagnocco, Annamaria; Bingham, Clifton O.; Brooks, Peter; Beaton, Dorcas; Gandjbakhch, Frederique; Gossec, Laure; Guillemin, Francis; Hewlett, Sarah; Kloppenburg, Margreet; March, Lyn; Mease, Philip J; Moller, Ingrid; Simon, Lee S; Singh, Jasvinder A; Strand, Vibeke; Wakefield, Richard J; Wells, George; Tugwell, Peter; Conaghan, Philip G

2014-01-01

Objective The OMERACT Filter provides a framework for the validation of outcome measures for use in rheumatology clinical research. However, imaging and biochemical measures may face additional validation challenges due to their technical nature. The Imaging and Soluble Biomarker Session at OMERACT 11 aimed to provide a guide for the iterative development of an imaging or biochemical measurement instrument so it can be used in therapeutic assessment. Methods A hierarchical structure was proposed, reflecting 3 dimensions needed for validating an imaging or biochemical measurement instrument: outcome domain(s), study setting and performance of the instrument. Movement along the axes in any dimension reflects increasing validation. For a given test instrument, the 3-axis structure assesses the extent to which the instrument is a validated measure for the chosen domain, whether it assesses a patient or disease centred-variable, and whether its technical performance is adequate in the context of its application. Some currently used imaging and soluble biomarkers for rheumatoid arthritis, spondyloarthritis and knee osteoarthritis were then evaluated using the original OMERACT filter and the newly proposed structure. Break-out groups critically reviewed the extent to which the candidate biomarkers complied with the proposed step-wise approach, as a way of examining the utility of the proposed 3 dimensional structure. Results Although there was a broad acceptance of the value of the proposed structure in general, some areas for improvement were suggested including clarification of criteria for achieving a certain level of validation and how to deal with extension of the structure to areas beyond clinical trials. Conclusion General support was obtained for a proposed tri-axis structure to assess validation of imaging and soluble biomarkers; nevertheless, additional work is required to better evaluate its place within the OMERACT Filter 2.0. PMID:24584916

Updating the OMERACT filter: implications for imaging and soluble biomarkers.

PubMed

D'Agostino, Maria-Antonietta; Boers, Maarten; Kirwan, John; van der Heijde, Désirée; Østergaard, Mikkel; Schett, Georg; Landewé, Robert B; Maksymowych, Walter P; Naredo, Esperanza; Dougados, Maxime; Iagnocco, Annamaria; Bingham, Clifton O; Brooks, Peter M; Beaton, Dorcas E; Gandjbakhch, Frederique; Gossec, Laure; Guillemin, Francis; Hewlett, Sarah E; Kloppenburg, Margreet; March, Lyn; Mease, Philip J; Moller, Ingrid; Simon, Lee S; Singh, Jasvinder A; Strand, Vibeke; Wakefield, Richard J; Wells, George A; Tugwell, Peter; Conaghan, Philip G

2014-05-01

The Outcome Measures in Rheumatology (OMERACT) Filter provides a framework for the validation of outcome measures for use in rheumatology clinical research. However, imaging and biochemical measures may face additional validation challenges because of their technical nature. The Imaging and Soluble Biomarker Session at OMERACT 11 aimed to provide a guide for the iterative development of an imaging or biochemical measurement instrument so it can be used in therapeutic assessment. A hierarchical structure was proposed, reflecting 3 dimensions needed for validating an imaging or biochemical measurement instrument: outcome domain(s), study setting, and performance of the instrument. Movement along the axes in any dimension reflects increasing validation. For a given test instrument, the 3-axis structure assesses the extent to which the instrument is a validated measure for the chosen domain, whether it assesses a patient-centered or disease-centered variable, and whether its technical performance is adequate in the context of its application. Some currently used imaging and soluble biomarkers for rheumatoid arthritis, spondyloarthritis, and knee osteoarthritis were then evaluated using the original OMERACT Filter and the newly proposed structure. Breakout groups critically reviewed the extent to which the candidate biomarkers complied with the proposed stepwise approach, as a way of examining the utility of the proposed 3-dimensional structure. Although there was a broad acceptance of the value of the proposed structure in general, some areas for improvement were suggested including clarification of criteria for achieving a certain level of validation and how to deal with extension of the structure to areas beyond clinical trials. General support was obtained for a proposed tri-axis structure to assess validation of imaging and soluble biomarkers; nevertheless, additional work is required to better evaluate its place within the OMERACT Filter 2.0.

Adaptation and Validation of the Arabic Version of the Infant Breastfeeding Knowledge Questionnaire among Lebanese Women.

PubMed

Tamim, Hani; Ghandour, Lilian A; Shamsedine, Lama; Charafeddine, Lama; Nasser, Fatima; Khalil, Yvette; Nabulsi, Mona

2016-11-01

Valid instruments that can reliably assess maternal breastfeeding knowledge in Arabic-speaking populations are nonexistent. The availability of such an instrument is essential for investigators working in this field. This study aimed to describe the adaptation and validation of the Arabic Breastfeeding Knowledge Questionnaire (BFK-A) from the original 20-item English version. A translated version of the 20-item BFK was validated among 417 Lebanese women after pilot testing for clarity, comprehension, length, and cultural appropriateness. Exploratory factor analysis was run to examine dimensionality of the instrument and Kuder-Richardson-20 (KR-20) was used to assess its internal consistency. The BFK-A is a unidimensional scale with acceptable internal consistency reliability (KR-20 = 0.652) after the exclusion of 4 items. Higher breastfeeding knowledge levels were strongly and statistically significantly associated with higher mean scores for the validated Arabic Iowa Infant Feeding Attitude Scale ( P < .001), thus confirming its construct validity. The Arabic 16-item BFK-A has an acceptable reliability, similar to the original instrument. Further studies are encouraged to confirm the validity of the 16-item BFK-A among other Arab populations. There is also a need to develop more reliable instruments to use in lactation research in this context.

Assessing the Role of Spirituality in Coping Among African Americans Diagnosed with Cancer

PubMed Central

Schulz, Emily; Caplan, Lee; Blake, Victor; Southward, Vivian L.; Buckner, Ayanna V.

2013-01-01

Spirituality plays an important role in cancer coping among African Americans. The purpose of this study was to report on the initial psychometric properties of instruments specific to the cancer context, assessing the role of spirituality in coping. Items were developed based on a theoretical model of spirituality and qualitative patient interviews. The instruments reflected connections to self, others, God, and the world. One hundred African American cancer survivors completed the instruments by telephone. The instruments showed adequate internal reliability, mixed convergent validity, discriminant validity, and interpretable factor structures. PMID:21246282

Reliability and concurrent validity of the adapted Chinese version of Scoliosis Research Society-22 (SRS-22) questionnaire.

PubMed

Cheung, Kenneth M C; Senkoylu, Alpaslan; Alanay, Ahmet; Genc, Yasemin; Lau, Sarah; Luk, Keith D

2007-05-01

Validation study to define validity and reliability of an adapted and translated questionnaire. Assessment of the concurrent validity and reliability of a Chinese version of SRS-22 outcome instrument. No valid health-related quality of life (HRQL) outcome instrument exists for patients with spinal deformity in Chinese. The modified SRS-22 questionnaire was proven to be an appropriate outcome instrument in English, and has already been translated and validated in several other languages. The English version of the SRS-22 questionnaire was adapted to Chinese according to the International Quality of Life Assessment Project guidelines. To assess reliability, 48 subjects with adolescent idiopathic scoliosis (mean age, 16.5 years) filled the questionnaire on 2 separate occasions (Group 1). To assess concurrent validity, 50 subjects (mean age, 21 years) filled in the same questionnaire and a previously validated Chinese version of the Short Form-36 (SF36) questionnaire (Group 2). Internal consistency, reproducibility and concurrent validity were determined with Cronbach's alpha coefficient, interclass correlation coefficient and Pearson correlation coefficient, respectively. Cronbach's alpha coefficient for the 4 major domains (function/activity, pain, self-image/appearance and mental health) were high. Intraclass correlation was also excellent for all domains. For concurrent validity, excellent correlation was found in 1 domain, good in 12 domains, moderate in 3 domains, and poor in 1 domain of the 17 relevant domains. Both cultural adaptation and linguistic translation are essential in any attempt to use a HRQL questionnaire across cultures. The Chinese version of the SRS-22 outcome instrument has satisfactory internal consistency and excellent reproducibility. It is ready for use in clinical studies on idiopathic scoliosis in Chinese-speaking societies.

Validation of the Communication Skills Questionnaire (CSQ) in people with schizophrenia.

PubMed

Prat, Gemma; Casas-Anguera, Emma; Garcia-Franco, Mar; Escandell, Maria José; Martin, José Ramón; Vilamala, Sonia; Villalta-Gil, Victoria; Gimenez-Salinas, Jordi; Hernández-Rambla, Carla; Ochoa, Susana

2014-12-15

This present study describes the validation of the Communication Skills Questionnaire (CSQ) in people with schizophrenia. A total of 125 clinically stable people in rehabilitation treatment who were diagnosed with schizophrenia were included. For convergent and discriminant validity the following tests were administered; the Gambrill and Richie (GR) Assertiveness Inventory, the Social Functioning Scale (SFS), Life Skills Profile (LSP), Clinical Global Impression scale for schizophrenia (CGI-S) and the Global Assessment of Functioning (GAF) scale. Internal consistency of the CSQ had a Cronbach׳s alpha of 0.96. Test-retest reliability showed coefficients between 0.60 and 0.70. Convergent validity showed significant relations at p<0.0001 for all instruments assessed. None of the subscales used for assessing discriminant validity showed a significant correlation with the CSQ except for the CGI-S depression subscale. The instrument shows good psychometric properties and demonstrates that it is a useful instrument for evaluating communication skills in people with schizophrenia. Copyright © 2014 Elsevier Ireland Ltd. All rights reserved.

[Development and validity of workplace bullying in nursing-type inventory (WPBN-TI)].

PubMed

Lee, Younju; Lee, Mihyoung

2014-04-01

The purpose of this study was to develop an instrument to assess bullying of nurses, and test the validity and reliability of the instrument. The initial thirty items of WPBN-TI were identified through a review of the literature on types bullying related to nursing and in-depth interviews with 14 nurses who experienced bullying at work. Sixteen items were developed through 2 content validity tests by 9 experts and 10 nurses. The final WPBN-TI instrument was evaluated by 458 nurses from five general hospitals in the Incheon metropolitan area. SPSS 18.0 program was used to assess the instrument based on internal consistency reliability, construct validity, and criterion validity. WPBN-TI consisted of 16 items with three distinct factors (verbal and nonverbal bullying, work-related bullying, and external threats), which explained 60.3% of the total variance. The convergent validity and determinant validity for WPBN-TI were 100.0%, 89.7%, respectively. Known-groups validity of WPBN-TI was proven through the mean difference between subjective perception of bullying. The satisfied criterion validity for WPBN-TI was more than .70. The reliability of WPBN-TI was Cronbach's α of .91. WPBN-TI with high validity and reliability is suitable to determine types of bullying in nursing workplace.

Measuring organizational readiness for knowledge translation in chronic care.

PubMed

Gagnon, Marie-Pierre; Labarthe, Jenni; Légaré, France; Ouimet, Mathieu; Estabrooks, Carole A; Roch, Geneviève; Ghandour, El Kebir; Grimshaw, Jeremy

2011-07-13

Knowledge translation (KT) is an imperative in order to implement research-based and contextualized practices that can answer the numerous challenges of complex health problems. The Chronic Care Model (CCM) provides a conceptual framework to guide the implementation process in chronic care. Yet, organizations aiming to improve chronic care require an adequate level of organizational readiness (OR) for KT. Available instruments on organizational readiness for change (ORC) have shown limited validity, and are not tailored or adapted to specific phases of the knowledge-to-action (KTA) process. We aim to develop an evidence-based, comprehensive, and valid instrument to measure OR for KT in healthcare. The OR for KT instrument will be based on core concepts retrieved from existing literature and validated by a Delphi study. We will specifically test the instrument in chronic care that is of an increasing importance for the health system. Phase one: We will conduct a systematic review of the theories and instruments assessing ORC in healthcare. The retained theoretical information will be synthesized in a conceptual map. A bibliography and database of ORC instruments will be prepared after appraisal of their psychometric properties according to the standards for educational and psychological testing. An online Delphi study will be carried out among decision makers and knowledge users across Canada to assess the importance of these concepts and measures at different steps in the KTA process in chronic care.Phase two: A final OR for KT instrument will be developed and validated both in French and in English and tested in chronic disease management to measure OR for KT regarding the adoption of comprehensive, patient-centered, and system-based CCMs. This study provides a comprehensive synthesis of current knowledge on explanatory models and instruments assessing OR for KT. Moreover, this project aims to create more consensus on the theoretical underpinnings and the instrumentation of OR for KT in chronic care. The final product--a comprehensive and valid OR for KT instrument--will provide the chronic care settings with an instrument to assess their readiness to implement evidence-based chronic care.

Measuring organizational readiness for knowledge translation in chronic care

PubMed Central

2011-01-01

Background Knowledge translation (KT) is an imperative in order to implement research-based and contextualized practices that can answer the numerous challenges of complex health problems. The Chronic Care Model (CCM) provides a conceptual framework to guide the implementation process in chronic care. Yet, organizations aiming to improve chronic care require an adequate level of organizational readiness (OR) for KT. Available instruments on organizational readiness for change (ORC) have shown limited validity, and are not tailored or adapted to specific phases of the knowledge-to-action (KTA) process. We aim to develop an evidence-based, comprehensive, and valid instrument to measure OR for KT in healthcare. The OR for KT instrument will be based on core concepts retrieved from existing literature and validated by a Delphi study. We will specifically test the instrument in chronic care that is of an increasing importance for the health system. Methods Phase one: We will conduct a systematic review of the theories and instruments assessing ORC in healthcare. The retained theoretical information will be synthesized in a conceptual map. A bibliography and database of ORC instruments will be prepared after appraisal of their psychometric properties according to the standards for educational and psychological testing. An online Delphi study will be carried out among decision makers and knowledge users across Canada to assess the importance of these concepts and measures at different steps in the KTA process in chronic care. Phase two: A final OR for KT instrument will be developed and validated both in French and in English and tested in chronic disease management to measure OR for KT regarding the adoption of comprehensive, patient-centered, and system-based CCMs. Discussion This study provides a comprehensive synthesis of current knowledge on explanatory models and instruments assessing OR for KT. Moreover, this project aims to create more consensus on the theoretical underpinnings and the instrumentation of OR for KT in chronic care. The final product--a comprehensive and valid OR for KT instrument--will provide the chronic care settings with an instrument to assess their readiness to implement evidence-based chronic care. PMID:21752264

Satisfaction and adherence in patients with iron overload receiving iron chelation therapy as assessed by a newly developed patient instrument.

PubMed

Rofail, Diana; Abetz, Linda; Viala, Muriel; Gait, Claire; Baladi, Jean-Francois; Payne, Krista

2009-01-01

This study assesses satisfaction with iron chelation therapy (ICT) based on a reliable and valid instrument, and explores the relationship between satisfaction and adherence to ICT. Patients in the USA and UK completed a new "Satisfaction with ICT" (SICT) instrument consisting of 28 items, three pertaining to adherence. Simple and multivariate regression analyses assessed the relationship between satisfaction with different aspects of ICT and adherence. First assessments of the SICT instrument indicate its validity and reliability. Recommended thresholds for internal consistency, convergent validity, discriminant validity, and floor and ceiling effects were met. A number of variables were identified in the simple linear regression analyses as significant predictors of "never thinking about stopping ICT," a proxy for adherence. These significant variables were entered into the multivariate model to assess the combined factor effects, explaining 42% of the total variance of "never thinking about stopping ICT." A significant and positive relationship was demonstrated between "never thinking about stopping ICT" and age (P = 0.04), Perceived Effectiveness of ICT (P = 0.003), low Burden of ICT (P = 0.002), and low Side Effects of ICT (P = 0.01). The SICT is a reliable and valid instrument which will be useful in ICT clinical trials. Furthermore, the administration of ICT by slow subcutaneous infusion negatively impacts on satisfaction with ICT which was shown to be a determinant of adherence. This points to the need for new more convenient and less burdensome oral iron chelators to increase adherence, and ultimately to improve patient outcomes.

Validity and reliability assessment of a peer evaluation method in team-based learning classes.

PubMed

Yoon, Hyun Bae; Park, Wan Beom; Myung, Sun-Jung; Moon, Sang Hui; Park, Jun-Bean

2018-03-01

Team-based learning (TBL) is increasingly employed in medical education because of its potential to promote active group learning. In TBL, learners are usually asked to assess the contributions of peers within their group to ensure accountability. The purpose of this study is to assess the validity and reliability of a peer evaluation instrument that was used in TBL classes in a single medical school. A total of 141 students were divided into 18 groups in 11 TBL classes. The students were asked to evaluate their peers in the group based on evaluation criteria that were provided to them. We analyzed the comments that were written for the highest and lowest achievers to assess the validity of the peer evaluation instrument. The reliability of the instrument was assessed by examining the agreement among peer ratings within each group of students via intraclass correlation coefficient (ICC) analysis. Most of the students provided reasonable and understandable comments for the high and low achievers within their group, and most of those comments were compatible with the evaluation criteria. The average ICC of each group ranged from 0.390 to 0.863, and the overall average was 0.659. There was no significant difference in inter-rater reliability according to the number of members in the group or the timing of the evaluation within the course. The peer evaluation instrument that was used in the TBL classes was valid and reliable. Providing evaluation criteria and rules seemed to improve the validity and reliability of the instrument.

An Adaptation of the Original Fresno Test to Measure Evidence-Based Practice Competence in Pediatric Bedside Nurses.

PubMed

Laibhen-Parkes, Natasha; Kimble, Laura P; Melnyk, Bernadette Mazurek; Sudia, Tanya; Codone, Susan

2018-06-01

Instruments used to assess evidence-based practice (EBP) competence in nurses have been subjective, unreliable, or invalid. The Fresno test was identified as the only instrument to measure all the steps of EBP with supportive reliability and validity data. However, the items and psychometric properties of the original Fresno test are only relevant to measure EBP with medical residents. Therefore, the purpose of this paper is to describe the development of the adapted Fresno test for pediatric nurses, and provide preliminary validity and reliability data for its use with Bachelor of Science in Nursing-prepared pediatric bedside nurses. General adaptations were made to the original instrument's case studies, item content, wording, and format to meet the needs of a pediatric nursing sample. The scoring rubric was also modified to complement changes made to the instrument. Content and face validity, and intrarater reliability of the adapted Fresno test were assessed during a mixed-methods pilot study conducted from October to December 2013 with 29 Bachelor of Science in Nursing-prepared pediatric nurses. Validity data provided evidence for good content and face validity. Intrarater reliability estimates were high. The adapted Fresno test presented here appears to be a valid and reliable assessment of EBP competence in Bachelor of Science in Nursing-prepared pediatric nurses. However, further testing of this instrument is warranted using a larger sample of pediatric nurses in diverse settings. This instrument can be a starting point for evaluating the impact of EBP competence on patient outcomes. © 2018 Sigma Theta Tau International.

An Ethical Issue Scale for Community Pharmacy Setting (EISP): Development and Validation.

PubMed

Crnjanski, Tatjana; Krajnovic, Dusanka; Tadic, Ivana; Stojkov, Svetlana; Savic, Mirko

2016-04-01

Many problems that arise when providing pharmacy services may contain some ethical components and the aims of this study were to develop and validate a scale that could assess difficulties of ethical issues, as well as the frequency of those occurrences in everyday practice of community pharmacists. Development and validation of the scale was conducted in three phases: (1) generating items for the initial survey instrument after qualitative analysis; (2) defining the design and format of the instrument; (3) validation of the instrument. The constructed Ethical Issue scale for community pharmacy setting has two parts containing the same 16 items for assessing the difficulty and frequency thereof. The results of the 171 completely filled out scales were analyzed (response rate 74.89%). The Cronbach's α value of the part of the instrument that examines difficulties of the ethical situations was 0.83 and for the part of the instrument that examined frequency of the ethical situations was 0.84. Test-retest reliability for both parts of the instrument was satisfactory with all Interclass correlation coefficient (ICC) values above 0.6, (for the part that examines severity ICC = 0.809, for the part that examines frequency ICC = 0.929). The 16-item scale, as a self assessment tool, demonstrated a high degree of content, criterion, and construct validity and test-retest reliability. The results support its use as a research tool to asses difficulty and frequency of ethical issues in community pharmacy setting. The validated scale needs to be further employed on a larger sample of pharmacists.

Development and Psychometric Validation of HIPER-Q to Assess Knowledge of Hypertensive Patients in Cardiac Rehabilitation.

PubMed

Santos, Rafaella Zulianello Dos; Bonin, Christiani Decker Batista; Martins, Eliara Ten Caten; Pereira Junior, Moacir; Ghisi, Gabriela Lima de Melo; Macedo, Kassia Rosangela Paz de; Benetti, Magnus

2018-01-01

The absence of instruments capable of measuring the level of knowledge of hypertensive patients in cardiac rehabilitation programs about their disease reflects the lack of specific recommendations for these patients. To develop and validate a questionnaire to evaluate the knowledge of hypertensive patients in cardiac rehabilitation programs about their disease. A total of 184 hypertensive patients (mean age 60.5 ± 10 years, 66.8% men) were evaluated. Reproducibility was assessed by calculation of the intraclass correlation coefficient using the test-retest method. Internal consistency was assessed by the Cronbach's alpha and the construct validity by the exploratory factorial analysis. The final version of the instrument had 17 questions organized in areas considered important for patient education. The instrument proposed showed a clarity index of 8.7 (0.25). The intraclass correlation coefficient was 0.804 and the Cronbach's correlation coefficient was 0.648. Factor analysis revealed five factors associated with knowledge areas. Regarding the criterion validity, patients with higher education level and higher family income showed greater knowledge about hypertension. The instrument has a satisfactory clarity index and adequate validity, and can be used to evaluate the knowledge of hypertensive participants in cardiac rehabilitation programs.

What do Demand-Control and Effort-Reward work stress questionnaires really measure? A discriminant content validity study of relevance and representativeness of measures.

PubMed

Bell, Cheryl; Johnston, Derek; Allan, Julia; Pollard, Beth; Johnston, Marie

2017-05-01

The Demand-Control (DC) and Effort-Reward Imbalance (ERI) models predict health in a work context. Self-report measures of the four key constructs (demand, control, effort, and reward) have been developed and it is important that these measures have good content validity uncontaminated by content from other constructs. We assessed relevance (whether items reflect the constructs) and representativeness (whether all aspects of the construct are assessed, and all items contribute to that assessment) across the instruments and items. Two studies examined fourteen demand/control items from the Job Content Questionnaire and seventeen effort/reward items from the Effort-Reward Imbalance measure using discriminant content validation and a third study developed new methods to assess instrument representativeness. Both methods use judges' ratings and construct definitions to get transparent quantitative estimates of construct validity. Study 1 used dictionary definitions while studies 2 and 3 used published phrases to define constructs. Overall, 3/5 demand items, 4/9 control items, 1/6 effort items, and 7/11 reward items were uniquely classified to the appropriate theoretical construct and were therefore 'pure' items with discriminant content validity (DCV). All pure items measured a defining phrase. However, both the DC and ERI assessment instruments failed to assess all defining aspects. Finding good discriminant content validity for demand and reward measures means these measures are usable and our quantitative results can guide item selection. By contrast, effort and control measures had limitations (in relevance and representativeness) presenting a challenge to the implementation of the theories. Statement of contribution What is already known on this subject? While the reliability and construct validity of Demand-Control and Effort-Reward-Imbalance (DC and ERI) work stress measures are routinely reported, there has not been adequate investigation of their content validity. This paper investigates their content validity in terms of both relevance and representativeness and provides a model for the investigation of content validity of measures in health psychology more generally. What does this study add? A new application of an existing method, discriminant content validity, and a new method of assessing instrument representativeness. 'Pure' DC and ERI items are identified, as are constructs that are not fully represented by their assessment instruments. The findings are important for studies attempting to distinguish between the main DC and ERI work stress constructs. The quantitative results can be used to guide item selection for future studies. © 2017 The British Psychological Society.

Development of Assessment Instrument of Critical Thinking in Physics at Senior High School

NASA Astrophysics Data System (ADS)

Sugiarti, T.; Kaniawati, I.; Aviyanti, L.

2017-02-01

The result of preliminary study shows that the assessment of physics in school did not train students’ critical thinking skill. The assessment instrument just measured low cognitive aspects. Supposedly, critical thinking skill is trained in the assessment activity. The study aims to determine the characteristics and the quality of critical thinking skill instrument. It employs descriptive-qualitative method with research and development as the research design. The research participants are 35 students involved in the limited trial and 188 students in the wider trial from three public senior high school in Ciamis which in high level school. The data was collected through expert validation, tests and interviews. The results indicate that the characteristics of the assessment instrument of critical thinking skill is open-ended. The instrument fulfills some indicators namely analyzing argument, deduction, induction, and display information in the form of scenario, text, graphic and table. In addition, the data processing through V4 Anates program shows that the instrument reliability achieves 0.67 with high interpretation of 0.67 and the validity is 0.47 with enough interpretation. Thus, the assessment instrument of critical thinking skill in the form of open-ended essay meets the criteria of quality test, so it can use as instrument of assessment critical thinking skill.

An Instrument to Assess Adults' Orientations toward Control versus Autonomy with Children: Reflections on Intrinsic Motivation and Perceived Competence.

ERIC Educational Resources Information Center

Deci, Edward L.; And Others

1981-01-01

This article describes the development and validation of an instrument to assess adults' orientations toward control versus autonomy in their interactions with children. The responses from 68 teachers had a good range and were internally consistent and temporally stable. Further, the measure was shown to be externally valid. (Author/BW)

The Development and Validation of the Measure of Acceptance of the Theory of Evolution Instrument.

ERIC Educational Resources Information Center

Rutledge, Michael L.; Warden, Melissa A.

1999-01-01

Describes the development and validation of the Measure of Acceptance of the Theory of Evolution (MATE), a 20-item, Likert-scaled instrument that assesses teachers' overall acceptance of evolutionary theory. (Author/CCM)

Development and preliminary evidence for the validity of an instrument assessing implementation of human-factors principles in medication-related decision-support systems—I-MeDeSA

PubMed Central

Zachariah, Marianne; Seidling, Hanna M; Neri, Pamela M; Cresswell, Kathrin M; Duke, Jon; Bloomrosen, Meryl; Volk, Lynn A; Bates, David W

2011-01-01

Background Medication-related decision support can reduce the frequency of preventable adverse drug events. However, the design of current medication alerts often results in alert fatigue and high over-ride rates, thus reducing any potential benefits. Methods The authors previously reviewed human-factors principles for relevance to medication-related decision support alerts. In this study, instrument items were developed for assessing the appropriate implementation of these human-factors principles in drug–drug interaction (DDI) alerts. User feedback regarding nine electronic medical records was considered during the development process. Content validity, construct validity through correlation analysis, and inter-rater reliability were assessed. Results The final version of the instrument included 26 items associated with nine human-factors principles. Content validation on three systems resulted in the addition of one principle (Corrective Actions) to the instrument and the elimination of eight items. Additionally, the wording of eight items was altered. Correlation analysis suggests a direct relationship between system age and performance of DDI alerts (p=0.0016). Inter-rater reliability indicated substantial agreement between raters (κ=0.764). Conclusion The authors developed and gathered preliminary evidence for the validity of an instrument that measures the appropriate use of human-factors principles in the design and display of DDI alerts. Designers of DDI alerts may use the instrument to improve usability and increase user acceptance of medication alerts, and organizations selecting an electronic medical record may find the instrument helpful in meeting their clinicians' usability needs. PMID:21946241

Content validity of the Geriatric Health Assessment Instrument

PubMed Central

Pedreira, Rhaine Borges Santos; Rocha, Saulo Vasconcelos; dos Santos, Clarice Alves; Vasconcelos, Lélia Renata Carneiro; Reis, Martha Cerqueira

2016-01-01

ABSTRACT Objective Assess the content validity of the Elderly Health Assessment Tool with low education. Methods The data collection instrument/questionnaire was prepared and submitted to an expert panel comprising four healthcare professionals experienced in research on epidemiology of aging. The experts were allowed to suggest item inclusion/exclusion and were asked to rate the ability of individual items in questionnaire blocks to encompass target dimensions as “not valid”, “somewhat valid” or “valid”, using an interval scale. Percent agreement and the Content Validity Index were used as measurements of inter-rater agreement; the minimum acceptable inter-rater agreement was set at 80%. Results The mean instrument percent agreement rate was 86%, ranging from 63 to 99%, and from 50 to 100% between and within blocks respectively. The Mean Content Validity Index score was 93.47%, ranging from 50 to 100% between individual items. Conclusion The instrument showed acceptable psychometric properties for application in geriatric populations with low levels of education. It enabled identifying diseases and assisted in choice of strategies related to health of the elderly. PMID:27462889

Development and Validation of Teaching Practice Evaluation Instrument for Assessing Chemistry Students' Teaching Skills

ERIC Educational Resources Information Center

Ezeudu, F. O.; Chiaha, G. T. U.; Eze, J. U.

2013-01-01

The study was designed to develop and factorially validate an instrument for measuring teaching practice skills of chemistry student-teachers in University of Nigeria, Nsukka. Two research questions guided the study. The design of the study was instrumentation. All the chemistry student-teachers in the Department of Science Education, University…

Validity and Reliability of the Clinical Competency Evaluation Instrument for Use among Physiotherapy Students: Pilot study.

PubMed

Muhamad, Zailani; Ramli, Ayiesah; Amat, Salleh

2015-05-01

The aim of this study was to determine the content validity, internal consistency, test-retest reliability and inter-rater reliability of the Clinical Competency Evaluation Instrument (CCEVI) in assessing the clinical performance of physiotherapy students. This study was carried out between June and September 2013 at University Kebangsaan Malaysia (UKM), Kuala Lumpur, Malaysia. A panel of 10 experts were identified to establish content validity by evaluating and rating each of the items used in the CCEVI with regards to their relevance in measuring students' clinical competency. A total of 50 UKM undergraduate physiotherapy students were assessed throughout their clinical placement to determine the construct validity of these items. The instrument's reliability was determined through a cross-sectional study involving a clinical performance assessment of 14 final-year undergraduate physiotherapy students. The content validity index of the entire CCEVI was 0.91, while the proportion of agreement on the content validity indices ranged from 0.83-1.00. The CCEVI construct validity was established with factor loading of ≥0.6, while internal consistency (Cronbach's alpha) overall was 0.97. Test-retest reliability of the CCEVI was confirmed with a Pearson's correlation range of 0.91-0.97 and an intraclass coefficient correlation range of 0.95-0.98. Inter-rater reliability of the CCEVI domains ranged from 0.59 to 0.97 on initial and subsequent assessments. This pilot study confirmed the content validity of the CCEVI. It showed high internal consistency, thereby providing evidence that the CCEVI has moderate to excellent inter-rater reliability. However, additional refinement in the wording of the CCEVI items, particularly in the domains of safety and documentation, is recommended to further improve the validity and reliability of the instrument.

A model of scientific attitudes assessment by observation in physics learning based scientific approach: case study of dynamic fluid topic in high school

NASA Astrophysics Data System (ADS)

Yusliana Ekawati, Elvin

2017-01-01

This study aimed to produce a model of scientific attitude assessment in terms of the observations for physics learning based scientific approach (case study of dynamic fluid topic in high school). Development of instruments in this study adaptation of the Plomp model, the procedure includes the initial investigation, design, construction, testing, evaluation and revision. The test is done in Surakarta, so that the data obtained are analyzed using Aiken formula to determine the validity of the content of the instrument, Cronbach’s alpha to determine the reliability of the instrument, and construct validity using confirmatory factor analysis with LISREL 8.50 program. The results of this research were conceptual models, instruments and guidelines on scientific attitudes assessment by observation. The construct assessment instruments include components of curiosity, objectivity, suspended judgment, open-mindedness, honesty and perseverance. The construct validity of instruments has been qualified (rated load factor > 0.3). The reliability of the model is quite good with the Alpha value 0.899 (> 0.7). The test showed that the model fits the theoretical models are supported by empirical data, namely p-value 0.315 (≥ 0.05), RMSEA 0.027 (≤ 0.08)

Psychometric Properties of the Chinese Family Assessment Instrument in Chinese Adolescents in Hong Kong

ERIC Educational Resources Information Center

Siu, Andrew M. H.; Shek, Daniel T. L.

2005-01-01

This paper reports evidence on the factor structure, reliability, and validity of the Chinese Family Assessment Instrument (C-FAI), an instrument developed to assess family functioning in Chinese populations. A convenience sample of 1,462 adolescents from junior secondary schools completed the C-FAI and measures of parent-adolescent conflict.…

AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews.

PubMed

Shea, Beverley J; Hamel, Candyce; Wells, George A; Bouter, Lex M; Kristjansson, Elizabeth; Grimshaw, Jeremy; Henry, David A; Boers, Maarten

2009-10-01

Our purpose was to measure the agreement, reliability, construct validity, and feasibility of a measurement tool to assess systematic reviews (AMSTAR). We randomly selected 30 systematic reviews from a database. Each was assessed by two reviewers using: (1) the enhanced quality assessment questionnaire (Overview of Quality Assessment Questionnaire [OQAQ]); (2) Sacks' instrument; and (3) our newly developed measurement tool (AMSTAR). We report on reliability (interobserver kappas of the 11 AMSTAR items), intraclass correlation coefficients (ICCs) of the sum scores, construct validity (ICCs of the sum scores of AMSTAR compared with those of other instruments), and completion times. The interrater agreement of the individual items of AMSTAR was substantial with a mean kappa of 0.70 (95% confidence interval [CI]: 0.57, 0.83) (range: 0.38-1.0). Kappas recorded for the other instruments were 0.63 (95% CI: 0.38, 0.78) for enhanced OQAQ and 0.40 (95% CI: 0.29, 0.50) for the Sacks' instrument. The ICC of the total score for AMSTAR was 0.84 (95% CI: 0.65, 0.92) compared with 0.91 (95% CI: 0.82, 0.96) for OQAQ and 0.86 (95% CI: 0.71, 0.94) for the Sacks' instrument. AMSTAR proved easy to apply, each review taking about 15 minutes to complete. AMSTAR has good agreement, reliability, construct validity, and feasibility. These findings need confirmation by a broader range of assessors and a more diverse range of reviews.

Development and validation of self-reported line drawings for assessment of knee malalignment and foot rotation: a cross-sectional comparative study

PubMed Central

2010-01-01

Background For large scale epidemiological studies clinical assessments and radiographs can be impractical and expensive to apply to more than just a sample of the population examined. The study objectives were to develop and validate two novel instruments for self-reported knee malalignment and foot rotation suitable for use in questionnaire studies of knee pain and osteoarthritis. Methods Two sets of line drawings were developed using similar methodology. Each instrument consisted of an explanatory question followed by a set of drawings showing straight alignment, then two each at 7.5° angulation and 15° angulation in the varus/valgus (knee) and inward/outward (foot) directions. Forty one participants undertaking a community study completed the instruments on two occasions. Participants were assessed once by a blinded expert clinical observer with demonstrated excellent reproducibility. Validity was assessed by sensitivity, specificity and likelihood ratio (LR) using the observer as the reference standard. Reliability was assessed using weighted kappa (κ). Knee malalignment was measured on 400 knee radiographs. General linear model was used to assess for the presence of a linear increase in knee alignment angle (measured medially) from self-reported severe varus to mild varus, straight, mild valgus and severe valgus deformity. Results Observer reproducibility (κ) was 0.89 and 0.81 for the knee malalignment and foot rotation instruments respectively. Self-reported participant reproducibility was also good for the knee (κ 0.73) and foot (κ 0.87) instruments. Validity was excellent for the knee malalignment instrument, with a sensitivity of 0.74 (95%CI 0.54, 0.93) and specificity of 0.97 (95%CI 0.94, 1.00). Similarly the foot rotation instrument was also found to have high sensitivity (0.92, 95%CI 0.83, 1.01) and specificity (0.96, 95%CI 0.93, 1.00). The knee alignment angle increased progressively from self reported severe varus to mild varus, straight, mild valgus and severe valgus knee malalignment (ptrend <0.001). Conclusions The two novel instruments appear to provide a valid and reliable assessment of self-reported knee malalignment and foot rotation, and may have a practical use in epidemiological studies. PMID:20565825

Is Validation Always Valid? Cross-Cultural Complexities of Standard-Based Instruments Migrating out of Their Context

ERIC Educational Resources Information Center

Pastori, Giulia; Pagani, Valentina

2017-01-01

The international application of standard-based measures to assess ECEC quality raises crucial questions concerning the cultural complexities and the problematic validity of instruments migrating out of their cultural cradle; nevertheless the topic has received only marginal attention in the literature. This paper, which aims to address this gap,…

The psychometric properties and practicability of self-report instruments to identify medication nonadherence in adult transplant patients: a systematic review.

PubMed

Dobbels, Fabienne; Berben, Lut; De Geest, Sabina; Drent, Gerda; Lennerling, Annette; Whittaker, Clare; Kugler, Christiane

2010-07-27

Nonadherence to immunosuppressive therapy is recognized as a key prognostic indicator for poor posttransplantation long-term outcomes. Several methods aiming to measure medication nonadherence have been suggested in the literature. Although combining measurement methods is regarded as the gold standard for measuring nonadherence, self-report is generally considered a central component of adherence assessment. However, no systematic review currently exists to determine which instrument(s) are most appropriate for use in transplant populations. The transplant360 Task Force first performed a survey of the self-report adherence instruments currently used in European centers. Next, a systematic literature review of self-report instruments assessing medication adherence in chronically ill patients was conducted. Self-report instruments were evaluated to assess those which were: (a) short and easy to score; (b) assessed both the taking and timing of medication intake; and (c) had established reliability and validity. Fourteen instruments were identified from our survey of European centers, of which the Basel Assessment of Adherence Scale for Immunosuppressives met the aforementioned criteria. The systematic review found 20 self-report instruments, of which only two qualified for use in transplantation, that is, the Brief Antiretroviral Adherence Index and the Medication Adherence Self-Report Inventory. The three selected self-report scales may assist transplant professionals in detecting nonadherence. However, these scales were only validated in patients with HIV. Although HIV shares similar characteristics with transplantation, including the importance of taking and timing of medication, further validation in transplant populations is required.

Initial evaluation of psychometric properties of a structured work task application for the Assessment of Work Performance in a constructed environment.

PubMed

Karlsson, Elin A; Liedberg, Gunilla M; Sandqvist, Jan L

2017-06-22

The Swedish Social Insurance Administration has developed a new assessment tool for sickness insurance. This study is a part of the initial evaluation of the application, called the Assessment of Work Performance, Structured Activities, and focuses on evaluation of the psychometric properties of social validity, content validity, and utility. This was a qualitative study using semi-structured telephone interviews with occupational therapists. A convenience sample was used and participants who fulfilled inclusion criteria (n = 15) were interviewed. Data were analyzed using content analysis with a directed approach. The results indicate that the application provides valuable information and that it is socially valid. Assessors found work tasks suitable for a diverse group of clients and reported that clients accepted the assessments. Improvements were suggested, for example, expanding the application with more work tasks. The instrument has benefits; however, further development is desired. The use of a constructed environment in assessments may be a necessary option to supplement a real environment. But depending on organizational factors such as time and other resources, the participants had different opportunities to do so. Further evaluations regarding ecological validity are essential to ensure that assessments are fair and realistic when using constructed environments. Implications for rehabilitation This study indicates that assessment in a constructed environment can provide a secure and protected context for clients being assessed. Psychometric evaluations are a never-ending process and this assessment instrument needs further development. However, this initial evaluation provides guidance in development of the instrument but also what studies to give priority to. It is important to evaluate social validity in order to ensure that clients and assessors perceive assessment methods fair and meaningful. In this study, participants found the work tasks appropriate and usable when assessing their clients but client's perspective must also be included in following studies. This assessment instrument is the only activity-based assessment instrument within the Swedish Social Security Insurance. Psychometric evaluations are important since it affects so many individuals in Sweden.

Estimating functional cognition in older adults using observational assessments of task performance in complex everyday activities: A systematic review and evaluation of measurement properties.

PubMed

Wesson, Jacqueline; Clemson, Lindy; Brodaty, Henry; Reppermund, Simone

2016-09-01

Functional cognition is a relatively new concept in assessment of older adults with mild cognitive impairment or dementia. Instruments need to be reliable and valid, hence we conducted a systematic review of observational assessments of task performance used to estimate functional cognition in this population. Two separate database searches were conducted: firstly to identify instruments; and secondly to identify studies reporting on the psychometric properties of the instruments. Studies were analysed using a published checklist and their quality reviewed according to specific published criteria. Clinical utility was reviewed and the information formulated into a best evidence synthesis. We found 21 instruments and included 58 studies reporting on measurement properties. The majority of studies were rated as being of fair methodological quality and the range of properties investigated was restricted. Most instruments had studies reporting on construct validity (hypothesis testing), none on content validity and there were few studies reporting on reliability. Overall the evidence on psychometric properties is lacking and there is an urgent need for further evaluation of instruments. Copyright © 2016 Elsevier Ltd. All rights reserved.

Validation of an instrument for mathematics enhancement teaching efficacy of Pacific Northwest agricultural educators

NASA Astrophysics Data System (ADS)

Jansen, Daniel J.

Teacher efficacy continues to be an important area of study in educational research. This study tested an instrument designed to assess the perceived efficacy of agricultural education teachers when engaged in lessons involving mathematics instruction. The study population of Oregon and Washington agricultural educators utilized in the validation of the instrument revealed important demographic findings and specific results related to teacher efficacy for the study population. An instrument was developed from the assimilation of three scales previously used and validated in efficacy research. Participants' mathematics teaching efficacy was assessed using a portion of the Mathematics Teaching Efficacy Beliefs Instrument (MTEBI), and personal mathematics efficacy was evaluated by the mathematics self-belief instrument which was derived from the Betz and Hackett's Mathematics Self-Efficacy Scale. The final scale, the Teachers' Sense of Efficacy Scale (TSES) created by Tschannen-Moran and Woolfolk Hoy, examined perceived personal teaching efficacy. Structural equation modeling was used as the statistical analyses tool to validate the instrument and examine correlations between efficacy constructs used to determine potential professional development needs of the survey population. As part of the data required for validation of the Mathematics Enhancement Teaching Efficacy instrument, demographic information defining the population of Oregon and Washington agricultural educators was obtained and reported. A hypothetical model derived from teacher efficacy literature was found to be an acceptable model to verify construct validity and determine strength of correlations between the scales that defined the instrument. The instrument produced an alpha coefficient of .905 for reliability. Both exploratory and confirmatory factor analyses were used to verify construct and discriminate validity. Specifics results related to the survey population of agricultural educators concluded that personal mathematics efficacy has a stronger correlation with mathematics teaching efficacy than personal teaching efficacy of teachers for this population. The implications of such findings suggest that professional development and pre-service preparation should be more focused on mathematics content knowledge rather than pedagogical knowledge when the objective is to enhance mathematics in interdisciplinary lessons.

Face validity and reliability of a pictorial instrument for assessing fundamental movement skill perceived competence in young children.

PubMed

Barnett, Lisa M; Ridgers, Nicola D; Zask, Avigdor; Salmon, Jo

2015-01-01

To determine reliability and face validity of an instrument to assess young children's perceived fundamental movement skill competence. Validation and reliability study. A pictorial instrument based on the Test Gross Motor Development-2 assessed perceived locomotor (six skills) and object control (six skills) competence using the format and item structure from the physical competence subscale of the Pictorial Scale of Perceived Competence and Acceptance for Young Children. Sample 1 completed object control items in May (n=32) and locomotor items in October 2012 (n=23) at two time points seven days apart. Children were asked at the end of the test-retest their understanding of what was happening in each picture to determine face validity. Sample 2 (n=58) completed 12 items in November 2012 on a single occasion to test internal reliability only. Sample 1 children were aged 5-7 years (M=6.0, SD=0.8) at object control assessment and 5-8 years at locomotor assessment (M=6.5, SD=0.9). Sample 2 children were aged 6-8 years (M=7.2, SD=0.73). Intra-class correlations assessed in Sample 1 children were excellent for object control (intra-class correlation=0.78), locomotor (intra-class correlation=0.82) and all 12 skills (intra-class correlations=0.83). Face validity was acceptable. Internal consistency was adequate in both samples for each subscale and all 12 skills (alpha range 0.60-0.81). This study has provided preliminary evidence for instrument reliability and face validity. This enables future alignment between the measurement of perceived and actual fundamental movement skill competence in young children. Crown Copyright © 2014. Published by Elsevier Ltd. All rights reserved.

Development and validation of the Vellore Occupational Therapy Evaluation Scale to assess functioning in people with mental illness.

PubMed

Samuel, Reema; Russell, Paul Ss; Paraseth, Tapan Kumar; Ernest, Sharmila; Jacob, K S

2016-08-26

Available occupational therapy assessment scales focus on specific areas of functioning. There is a need for comprehensive evaluation of diverse aspects of functioning in people with mental illness. To develop a comprehensive assessment scale to evaluate diverse aspects of functioning among people with mental illness and to assess its validity and reliability. Available instruments, which evaluate diverse aspects of functioning in people with mental illness, were retrieved. Relevant items, which evaluate specific functions, were selected by a committee of mental health experts and combined to form a comprehensive instrument. Face and content validity and feasibility were assessed and the new instrument was piloted among 60 patients with mental illness. The final version of the instrument was employed in 151 consecutive clients, between 18 and 60 years of age, who were also assessed using Global Assessment of Functioning (GAF), Occupational Therapy Task Observation Scale (OTTOS), Social Functioning Questionnaire (SFQ), Rosenberg Self Esteem Scale (RSES) and Pai and Kapur Family Burden Interview Schedule (FBIS) by two therapists. The inter-rater reliability and test-retest reliability of the new instrument (Vellore Occupational Therapy Evaluation Scale (VOTES)) were also evaluated. The new scale had good internal consistency (Cronbach's alpha = .817), inter-rater reliability .928 (.877-.958) and test-retest reliability .928 (.868-.961). The correlation between the general behaviour domain (Pearson's Correlation Coefficient [PCC] = -.763, p = .000), task behaviour (PCC = -.829, p = .000), social skills (PCC = -.351, p = .000), intrapersonal skills (PCC = -.208, p = .010), instrumental activities of daily living (IADL) (PCC = -.329, p = .038) and leisure activities (PCC = -.433, p = .005) scores of VOTES with the corresponding domains in the scales used for comparison was statistically significant. The correlation between the total score of VOTES and the total scores of OTTOS, SFQ and RSES was also statistically significant suggesting convergent validity. The correlation between the total score of VOTES with the total score of FBI is not statistically significant, implying good divergent validity. VOTES seems to be a promising tool to assess overall functioning of people with mental illness. © The Author(s) 2016.

Development and Validation of an Instrument to Assess Student Attitudes toward Science across Grades 5 through 10

ERIC Educational Resources Information Center

Summers, Ryan; Abd-El-Khalick, Fouad

2018-01-01

The aim of the present study is to enable future studies into students' attitudes toward science, and related constructs, by developing and validating an instrument suitable for cross-sectional designs. Following a thorough review of the literature it was determined that many extant instruments included design aspects that appeared to be limited…

The Reliability, Validity and Practical Utility of Measuring Supports Using the I-CAN Instrument: Part II

ERIC Educational Resources Information Center

Riches, Vivienne C.; Parmenter, Trevor R.; Llewellyn, Gwynnyth; Hindmarsh, Gabrielle; Chan, Jeff

2009-01-01

Background: There is an urgent need for developing reliable, valid and practical instruments that assess and classify the support needed by persons with disability to function in their chosen living, working and social environments. I-CAN is an instrument that addresses the frequency and level of support needed (not individual skills or deficits)…

Content validity of governing in Building Information Modelling (BIM) implementation assessment instrument

NASA Astrophysics Data System (ADS)

Hadzaman, N. A. H.; Takim, R.; Nawawi, A. H.; Mohamad Yusuwan, N.

2018-04-01

BIM governance assessment instrument is a process of analysing the importance in developing BIM governance solution to tackle the existing problems during team collaboration in BIM-based projects. Despite the deployment of integrative technologies in construction industry particularly BIM, it is still insufficient compare to other sectors. Several studies have been established the requirements of BIM implementation concerning all technical and non-technical BIM adoption issues. However, the data are regarded as inadequate to develop a BIM governance framework. Hence, the objective of the paper is to evaluate the content validity of the BIM governance instrument prior to the main data collection. Two methods were employed in the form of literature review and questionnaire survey. Based on the literature review, 273 items with six main constructs are suggested to be incorporated in the BIM governance instrument. The Content Validity Ratio (CVR) scores revealed that 202 out of 273 items are considered as the utmost critical by the content experts. The findings for Item Level Content Validity Index (I-CVI) and Modified Kappa Coefficient however revealed that 257 items in BIM governance instrument are appropriate and excellent. The instrument is highly reliable for future strategies and the development of BIM projects in Malaysia.

Development of an interprofessional lean facilitator assessment scale.

PubMed

Bravo-Sanchez, Cindy; Dorazio, Vincent; Denmark, Robert; Heuer, Albert J; Parrott, J Scott

2018-05-01

High reliability is important for optimising quality and safety in healthcare organisations. Reliability efforts include interprofessional collaborative practice (IPCP) and Lean quality/process improvement strategies, which require skilful facilitation. Currently, no validated Lean facilitator assessment tool for interprofessional collaboration exists. This article describes the development and pilot evaluation of such a tool; the Interprofessional Lean Facilitator Assessment Scale (ILFAS), which measures both technical and 'soft' skills, which have not been measured in other instruments. The ILFAS was developed using methodologies and principles from Lean/Shingo, IPCP, metacognition research and Bloom's Taxonomy of Learning Domains. A panel of experts confirmed the initial face validity of the instrument. Researchers independently assessed five facilitators, during six Lean sessions. Analysis included quantitative evaluation of rater agreement. Overall inter-rater agreement of the assessment of facilitator performance was high (92%), and discrepancies in the agreement statistics were analysed. Face and content validity were further established, and usability was evaluated, through primary stakeholder post-pilot feedback, uncovering minor concerns, leading to tool revision. The ILFAS appears comprehensive in the assessment of facilitator knowledge, skills, abilities, and may be useful in the discrimination between facilitators of different skill levels. Further study is needed to explore instrument performance and validity.

Validity and test-retest reliability of an at-work production loss instrument.

PubMed

Aboagye, E; Jensen, I; Bergström, G; Hagberg, J; Axén, I; Lohela-Karlsson, M

2016-07-01

Besides causing ill health, a poor work environment may contribute to production loss. Production loss assessment instruments emphasize health-related consequences but there is no instrument to measure reduced work performance related to the work environment. To examine convergent validity and test-retest reliability of health-related production loss (HRPL) and work environment-related production loss (WRPL) against a valid comparable instrument, the Health and Work Performance Questionnaire (HPQ). Cross-sectional study of employees, not on sick leave, who were asked to self-rate their work performance and production losses. Using the Pearson correlation and Bland and Altman's Test of Agreement, convergent validity was examined. Subgroup analyses were performed for employees recording problem-specific reduced work performance. Consistency of pairs of HRPL and WRPL for samples responding to both assessments was expressed using Intraclass Correlation Coefficient (ICC) and tests of repeatability. A total of 88 employees participated and 44 responded to both assessments. Test of agreement between measurements estimates a mean difference of 0.34 for HRPL and -0.03 for WRPL compared with work performance. This indicates that the production loss questions are valid and moderately associated with work performance for the total sample and subgroups. ICC for paired HRPL assessments was 0.90 and 0.91 for WRPL, i.e. the test-retest reliability was good and suggests stability in the instrument. HRPL and WRPL can be used to measure production loss due to health-related and work environment-related problems. These results may have implications for advancing methods of assessing production loss, which represents an important cost to employers. © The Author 2016. Published by Oxford University Press on behalf of the Society of Occupational Medicine. All rights reserved. For Permissions, please email: journals.permissions@oup.com.

Organizational member involvement in physical activity coalitions across the United States: development and testing of a novel survey instrument for assessing coalition functioning.

PubMed

Bornstein, Daniel B; Pate, Russell R; Beets, Michael W; Saunders, Ruth P; Blair, Steven N

2015-06-01

Coalitions are often composed of member organizations. Member involvement is thought to be associated with coalition success. No instrument currently exists for evaluating organizational member involvement in physical activity coalitions. This study aimed to develop a survey instrument for evaluating organizational member involvement in physical activity coalitions. The study was carried out in three phases: (a) developing a draft survey, (b) assessing the content validity of the draft survey, and (c) assessing the underlying factor structure, reliability, and validity of the survey. A cross-sectional design was employed. In Phase 1, a team of experts in survey development produced a draft survey. In Phase 2, the content validity of the draft survey was evaluated by a panel of individuals with expertise in physical activity coalitions. In Phase 3, the survey was administered to 120 individuals on local-, state-, and national-level physical activity coalitions. Responses were subjected to an exploratory factor analysis in order to determine the survey's underlying factor structure, reliability, and validity. Phases 1 and 2yielded a survey instrument with demonstrated content validity. Phase 3 yielded a three-factor model with three subscales: Strategic Alignment, Organizational Alignment, and Providing Input. Each subscale demonstrated high internal consistency reliability and construct validity. The survey instrument developed here demonstrated sound psychometric properties and provides new insight into organizational member involvement in physical activity coalitions. This instrument may be an important tool in developing a more complete picture of coalition functioning in physical activity coalitions specifically and health-based coalitions overall. © 2014 Society for Public Health Education.

Assessing medical professionalism: A systematic review of instruments and their measurement properties

PubMed Central

Li, Honghe; Liu, Yang; Wen, Deliang

2017-01-01

Background Over the last three decades, various instruments were developed and employed to assess medical professionalism, but their measurement properties have yet to be fully evaluated. This study aimed to systematically evaluate these instruments’ measurement properties and the methodological quality of their related studies within a universally acceptable standardized framework and then provide corresponding recommendations. Methods A systematic search of the electronic databases PubMed, Web of Science, and PsycINFO was conducted to collect studies published from 1990–2015. After screening titles, abstracts, and full texts for eligibility, the articles included in this study were classified according to their respective instrument’s usage. A two-phase assessment was conducted: 1) methodological quality was assessed by following the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist; and 2) the quality of measurement properties was assessed according to Terwee’s criteria. Results were integrated using best-evidence synthesis to look for recommendable instruments. Results After screening 2,959 records, 74 instruments from 80 existing studies were included. The overall methodological quality of these studies was unsatisfactory, with reasons including but not limited to unknown missing data, inadequate sample sizes, and vague hypotheses. Content validity, cross-cultural validity, and criterion validity were either unreported or negative ratings in most studies. Based on best-evidence synthesis, three instruments were recommended: Hisar’s instrument for nursing students, Nurse Practitioners’ Roles and Competencies Scale, and Perceived Faculty Competency Inventory. Conclusion Although instruments measuring medical professionalism are diverse, only a limited number of studies were methodologically sound. Future studies should give priority to systematically improving the performance of existing instruments and to longitudinal studies. PMID:28498838

Measuring Nutrition Literacy in Spanish-Speaking Latinos: An Exploratory Validation Study.

PubMed

Gibbs, Heather D; Camargo, Juliana M T B; Owens, Sarah; Gajewski, Byron; Cupertino, Ana Paula

2017-11-21

Nutrition is important for preventing and treating chronic diseases highly prevalent among Latinos, yet no tool exists for measuring nutrition literacy among Spanish speakers. This study aimed to adapt the validated Nutrition Literacy Assessment Instrument for Spanish-speaking Latinos. This study was developed in two phases: adaptation and validity testing. Adaptation included translation, expert item content review, and interviews with Spanish speakers. For validity testing, 51 participants completed the Short Assessment of Health Literacy-Spanish (SAHL-S), the Nutrition Literacy Assessment Instrument in Spanish (NLit-S), and socio-demographic questionnaire. Validity and reliability statistics were analyzed. Content validity was confirmed with a Scale Content Validity Index of 0.96. Validity testing demonstrated NLit-S scores were strongly correlated with SAHL-S scores (r = 0.52, p < 0.001). Entire reliability was substantial at 0.994 (CI 0.992-0.996) and internal consistency was excellent (Cronbach's α = 0.92). The NLit-S demonstrates validity and reliability for measuring nutrition literacy among Spanish-speakers.

Methods for assessing the preventability of adverse drug events: a systematic review.

PubMed

Hakkarainen, Katja Marja; Andersson Sundell, Karolina; Petzold, Max; Hägg, Staffan

2012-02-01

Preventable adverse drug events (ADEs) are common in both outpatient and inpatient settings. However, the proportion of preventable ADEs varies considerably in different studies, even when conducted in the same setting, and methods for assessing the preventability of ADEs are diverse. The aim of this article is to identify and systematically evaluate methods for assessing the preventability of ADEs. Seven databases (Cochrane, CINAHL, EMBASE, IPA, MEDLINE, PsycINFO and Web of Science) were searched in September 2010 utilizing the databases' index terms and other common terminology on preventable ADEs. No limits for the years of publication were set. Reference lists of included original articles and relevant review articles were also screened. After applying predetermined inclusion and exclusion criteria on 4161 unique citations, 142 (3.4%) original research articles were included in the review. One additional article was included from reference lists. Outcome measures of included studies had to include the frequency of ADEs and the assessment of their preventability. Studies were excluded if they focused on individuals with one specific type of treatment, medical condition, medical procedure or ADE. Measurement instruments for determining the preventability of ADEs in each article were extracted and unique instruments were compared. The process of assessing the preventability of ADEs was described based on reported actions taken to standardize and conduct the assessment, and on information about the reliability and validity of the assessment. Eighteen unique instruments for determining the preventability of ADEs were identified. They fell under the following four groups: (i) instruments using a definition of preventability only (n = 3); (ii) instruments with a definition of preventability and an assessment scale for determining preventability (n = 5); (iii) instruments with specific criteria for each preventability category (n = 3); and (iv) instruments with an algorithm for determining preventability (n = 7). Of actions to standardize the assessment process, performing a pilot study was reported in 21 (15%), and use of a standardized protocol was reported in 18 (13%), of the included 143 articles. Preventability was assessed by physicians in 86 (60%) articles and by pharmacists in 41 (29%) articles. In 29 (20%) articles, persons conducting the assessment were described as trained for or experienced in preventability assessment. In 94 (66%) articles, more than one person assessed the preventability of each case. Among these 94 articles, assessment was done independently in 73 (51%) articles. Procedures for managing conflicting assessments were diverse. The reliability of the preventability assessment was tested in 39 (27%) articles, and 16 (11%) articles referred to a previous reliability assessment. Reliability ranged from poor to excellent (kappa 0.19-0.98; overall agreement 26-97%). Four (3%) articles mentioned assessing validity, but no sensitivity or specificity analyses or negative or positive predictive values were presented. Instruments for assessing the preventability of ADEs vary from implicit instruments to explicit algorithms. There is limited evidence for the validity of the identified instruments, and instrument reliability varied significantly. The process of assessing the preventability of ADEs is also commonly imprecisely described, which hinders the interpretation and comparison of studies. For measuring the preventability of ADEs more accurately and precisely in future, we believe that existing instruments should be further studied and developed, or that one or more new instruments should be developed, and the validity and reliability of the existing and new instruments be established.

Reliability and validity of a nutrition and physical activity environmental self-assessment for child care

PubMed Central

Benjamin, Sara E; Neelon, Brian; Ball, Sarah C; Bangdiwala, Shrikant I; Ammerman, Alice S; Ward, Dianne S

2007-01-01

Background Few assessment instruments have examined the nutrition and physical activity environments in child care, and none are self-administered. Given the emerging focus on child care settings as a target for intervention, a valid and reliable measure of the nutrition and physical activity environment is needed. Methods To measure inter-rater reliability, 59 child care center directors and 109 staff completed the self-assessment concurrently, but independently. Three weeks later, a repeat self-assessment was completed by a sub-sample of 38 directors to assess test-retest reliability. To assess criterion validity, a researcher-administered environmental assessment was conducted at 69 centers and was compared to a self-assessment completed by the director. A weighted kappa test statistic and percent agreement were calculated to assess agreement for each question on the self-assessment. Results For inter-rater reliability, kappa statistics ranged from 0.20 to 1.00 across all questions. Test-retest reliability of the self-assessment yielded kappa statistics that ranged from 0.07 to 1.00. The inter-quartile kappa statistic ranges for inter-rater and test-retest reliability were 0.45 to 0.63 and 0.27 to 0.45, respectively. When percent agreement was calculated, questions ranged from 52.6% to 100% for inter-rater reliability and 34.3% to 100% for test-retest reliability. Kappa statistics for validity ranged from -0.01 to 0.79, with an inter-quartile range of 0.08 to 0.34. Percent agreement for validity ranged from 12.9% to 93.7%. Conclusion This study provides estimates of criterion validity, inter-rater reliability and test-retest reliability for an environmental nutrition and physical activity self-assessment instrument for child care. Results indicate that the self-assessment is a stable and reasonably accurate instrument for use with child care interventions. We therefore recommend the Nutrition and Physical Activity Self-Assessment for Child Care (NAP SACC) instrument to researchers and practitioners interested in conducting healthy weight intervention in child care. However, a more robust, less subjective measure would be more appropriate for researchers seeking an outcome measure to assess intervention impact. PMID:17615078

Instruments for measuring meaningful learning in healthcare students: a systematic psychometric review.

PubMed

Cadorin, Lucia; Bagnasco, Annamaria; Tolotti, Angela; Pagnucci, Nicola; Sasso, Loredana

2016-09-01

To identify, evaluate and describe the psychometric properties of instruments that measure learning outcomes in healthcare students. Meaningful learning is an active process that enables a wider and deeper understanding of concepts. It is the result of an interaction between new and prior knowledge and produces a long-standing change in knowledge and skills. In the field of education, validated and reliable instruments for assessing meaningful learning are needed. A psychometric systematic review. MEDLINE CINAHL, SCOPUS, ERIC, Cochrane Library, Psychology & Behavioural Sciences Collection Database from 1990-December 2013. Using pre-determined inclusion criteria, three reviewers independently identified studies for full-text review. Then they extracted data for quality appraisal and graded instrument validity using the Consensus-based Standards for the selection of the health status Measurement INstruments checklist and the Psychometric Grading Framework. Of the 57 studies identified for full-text review, 16 met the inclusion criteria and 13 different instruments were assessed. Following quality assessment, only one instrument was considered of good quality but it measured meaningful learning only in part; the others were either fair or poor. The Psychometric Grading Framework indicated that one instrument was weak, while the others were very weak. No instrument displayed adequate validity. The systematic review produced a synthesis of the psychometric properties of tools that measure learning outcomes in students of healthcare disciplines. Measuring learning outcomes is very important when educating health professionals. The identified tools may constitute a starting point for the development of other assessment tools. © 2016 John Wiley & Sons Ltd.

Development and Validation of the Musical Ear Training Assessment (META)

ERIC Educational Resources Information Center

Wolf, Anna; Kopiez, Reinhard

2018-01-01

In the following study, we have developed an assessment instrument for the practice-dependent skill of analytical hearing following a strict test theoretical validation, resulting in the Musical Ear Training Assessment (META). By means of three pilot studies, a developmental study, and a validation study, we verified a one-dimensional test model…

A Psychometric Analysis of Quality of Life Tools in Lung Cancer Patients Who Smoke

PubMed Central

Browning, Kristine K.; Ferketich, Amy K.; Otterson, Gregory A.; Reynolds, Nancy R.; Wewers, Mary Ellen

2009-01-01

Lung cancer is the leading cause of cancer death for both men and women in the United States. Patient quality of life (QOL) prior to cancer treatment is known to be a strong predictor of survival and toleration of treatment toxicities. A lung cancer patient’s self-assessment of QOL is highly valued among clinicians as it guides treatment-related decisions and impacts clinical outcomes. Smokers are known to report a lower QOL. Limited research has been conducted on QOL outcomes in lung cancer patients who continue to smoke. To assess QOL, a reliable and valid QOL measure specific to lung cancer is required. The Functional Assessment of Cancer Therapy-Lung Cancer (FACT-L) and Lung Cancer Symptom Scale (LCSS) are instruments that specifically examine QOL among lung cancer patients. The LCSS is a focused QOL instrument that includes physical and functional domains of QOL and disease symptomatology. The FACT-L is a broader QOL instrument that includes physical, functional, social and emotional domains and disease symptomatology. Both are psychometrically valid and are widely used in the literature, but have not been exclusively evaluated in smokers. Furthermore, there is no ‘gold standard’ instrument since there has never been a correlation study to compare estimates of reliability and validity between these instruments. The purpose of this study is to report the internal consistency and convergence validity of the FACT-L and the LCSS among newly diagnosed lung cancer patients who smoke. This data were collected and analyzed from a larger study examining smoking behavior among newly diagnosed lung cancer patients (n=51). Descriptive statistics were calculated on the FACT-L and LCSS scores, internal consistency was assessed by estimating Cronbach’s alpha coefficients, and Pearson correlation coefficients were estimated between the two scales. Internal consistency coefficients demonstrated good reliability for both scales, and the two instruments demonstrated a strong correlation, suggesting good convergence validity. Either of these instruments are appropriate measures for QOL in lung cancer patients who smoke. Given the conceptual difference between the two instruments, it is important to carefully consider the research aims when selecting the appropriate QOL measurement instrument. PMID:19181418

Brief comprehensive quality of life assessment after stroke: the assessment of quality of life instrument in the north East melbourne stroke incidence study (NEMESIS).

PubMed

Sturm, Jonathan W; Osborne, Richard H; Dewey, Helen M; Donnan, Geoffrey A; Macdonell, Richard A L; Thrift, Amanda G

2002-12-01

Generic utility health-related quality of life instruments are useful in assessing stroke outcome because they facilitate a broader description of the disease and outcomes, allow comparisons between diseases, and can be used in cost-benefit analysis. The aim of this study was to validate the Assessment of Quality of Life (AQoL) instrument in a stroke population. Ninety-three patients recruited from the community-based North East Melbourne Stroke Incidence Study between July 13, 1996, and April 30, 1997, were interviewed 3 months after stroke. Validity of the AQoL was assessed by examining associations between the AQoL and comparator instruments: the Medical Outcomes Short-Form Health Survey (SF-36); London Handicap Scale; Barthel Index; National Institutes of Health Stroke Scale; and Irritability, Depression, Anxiety scale. Sensitivity of the AQoL was assessed by comparing AQoL scores from groups of patients categorized by severity of impairment and disability and with total anterior circulation syndrome (TACS) versus non-TACS. Predictive validity was assessed by examining the association between 3-month AQoL scores and outcomes of death or institutionalization 12 months after stroke. Overall AQoL utility scores and individual dimension scores were most highly correlated with relevant scales on the comparator instruments. AQoL scores clearly differentiated between patients in categories of severity of impairment and disability and between patients with TACS and non-TACS. AQoL scores at 3 months after stroke predicted death and institutionalization at 12 months. The AQoL demonstrated strong psychometric properties and appears to be a valid and sensitive measure of health-related QoL after stroke.

Development of the Human Factors Skills for Healthcare Instrument: a valid and reliable tool for assessing interprofessional learning across healthcare practice settings.

PubMed

Reedy, Gabriel B; Lavelle, Mary; Simpson, Thomas; Anderson, Janet E

2017-10-01

A central feature of clinical simulation training is human factors skills, providing staff with the social and cognitive skills to cope with demanding clinical situations. Although these skills are critical to safe patient care, assessing their learning is challenging. This study aimed to develop, pilot and evaluate a valid and reliable structured instrument to assess human factors skills, which can be used pre- and post-simulation training, and is relevant across a range of healthcare professions. Through consultation with a multi-professional expert group, we developed and piloted a 39-item survey with 272 healthcare professionals attending training courses across two large simulation centres in London, one specialising in acute care and one in mental health, both serving healthcare professionals working across acute and community settings. Following psychometric evaluation, the final 12-item instrument was evaluated with a second sample of 711 trainees. Exploratory factor analysis revealed a 12-item, one-factor solution with good internal consistency (α=0.92). The instrument had discriminant validity, with newly qualified trainees scoring significantly lower than experienced trainees ( t (98)=4.88, p<0.001) and was sensitive to change following training in acute and mental health settings, across professional groups (p<0.001). Confirmatory factor analysis revealed an adequate model fit (RMSEA=0.066). The Human Factors Skills for Healthcare Instrument provides a reliable and valid method of assessing trainees' human factors skills self-efficacy across acute and mental health settings. This instrument has the potential to improve the assessment and evaluation of human factors skills learning in both uniprofessional and interprofessional clinical simulation training.

Development and Validation of an Instrument to Measure University Students' Biotechnology Attitude

NASA Astrophysics Data System (ADS)

Erdogan, Mehmet; Özel, Murat; Uşak, Muhammet; Prokop, Pavol

2009-06-01

The impact of biotechnologies on peoples' everyday lives continuously increases. Measuring young peoples' attitudes toward biotechnologies is therefore very important and its results are useful not only for science curriculum developers and policy makers, but also for producers and distributors of genetically modified products. Despite of substantial number of instruments which focused on measuring student attitudes toward biotechnology, a majority of them were not rigorously validated. This study deals with the development and validation of an attitude questionnaire toward biotechnology. Detailed information on development and validation process of the instrument is provided. Data gathered from 326 university students provided evidence for the validity and reliability of the new instrument which consists of 28 attitude items on a five point likert type scale. It is believed that the instrument will serve as a valuable tool for both instructors and researchers in science education to assess students' biotechnology attitudes.

Measurement properties of adult quality-of-life measurement instruments for eczema: protocol for a systematic review.

PubMed

Apfelbacher, Christian J; Heinl, Daniel; Prinsen, Cecilia A C; Deckert, Stefanie; Chalmers, Joanne; Ofenloch, Robert; Humphreys, Rosemary; Sach, Tracey; Chamlin, Sarah; Schmitt, Jochen

2015-04-16

Eczema is a common chronic or chronically relapsing skin disease that has a substantial impact on quality of life (QoL). By means of a consensus-based process, the Harmonising Outcome Measures in Eczema (HOME) initiative has identified QoL as one of the four core outcome domains to be assessed in all eczema trials (Allergy 67(9):1111-7, 2012). Various measurement instruments exist to measure QoL in adults with eczema, but there is a great variability in both content and quality (for example, reliability and validity) of the instruments used, and it is not always clear if the best instrument is being used. Therefore, the aim of the proposed research is a comprehensive systematic assessment of the measurement properties of the existing measurement instruments that were developed and/or validated for the measurement of patient-reported QoL in adults with eczema. This study is a systematic review of the measurement properties of patient-reported measures of QoL developed and/or validated for adults with eczema. Medline via PubMed and EMBASE will be searched using a selection of relevant search terms. Eligible studies will be primary empirical studies evaluating, describing, or comparing measurement properties of QoL instruments for adult patients with eczema. Eligibility assessment and data abstraction will be performed independently by two reviewers. Evidence tables will be generated for study characteristics, instrument characteristics, measurement properties, and interpretability. The quality of the measurement properties will be assessed using predefined criteria. Methodological quality of studies will be assessed using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. A best evidence synthesis will be undertaken if more than one study has investigated a particular measurement property. The proposed systematic review will produce a comprehensive assessment of measurement properties of existing QoL instruments in adult patients with eczema. We aim to identify one best currently available instrument to measure QoL in eczema patients. PROSPERO CRD42015017138.

Gender Differences of Children's Developmental Trajectory from 6 to 60 Months in the Taiwan Birth Cohort Pilot Study

ERIC Educational Resources Information Center

Lung, For-Wey; Chiang, Tung-Liang; Lin, Shio-Jean; Feng, Jui-Ying; Chen, Po-Fei; Shu, Bih-Ching

2011-01-01

The parental report instrument is the most efficient developmental detection method and has shown high validity with professional assessment instruments. The reliability and validity of the Taiwan Birth Cohort Study (TBCS) 6-, 18- and 36-month scales have already been established. In this study, the reliability and validity of the 60-month scale…

The Intra-Rater Reliability of Nine Content-Validated Technical Skill Assessment Instruments (TSAI) for Athletic Taping Skills

ERIC Educational Resources Information Center

Lagumen, Niko G.; Butterwick, Dale J.; Paskevich, David M.; Fung, Tak S.; Donnon, Tyrone L.

2008-01-01

Objective: To establish the intra-rater reliability of nine content-validated Technical Skill Assessment Instruments (TSAI) for the skills of athletic taping. Setting: University of Calgary. Subjects: Canadian Certified Athletic Therapists, CAT(C), with a mean ± SD of 9.6 ± 10.8 years as a CAT(C), 7.8 ± 10.9 years as a Supervisory Athletic…

Response Distortion in Normal Personality Assessment: Investigating Proposed Validity Scales for the NEO-PI-R in a College Student Sample

ERIC Educational Resources Information Center

Fineran, Kerrie R. J.

2009-01-01

The NEO-PI-R (Costa & McCrae, 1992b) is an assessment of normal personality composition that is used in clinical counseling contexts as well as for personnel selection. There has been some debate regarding the necessity and usefulness of validity scales to detect response distortion on this instrument. Because the authors of the instrument,…

Pre-donation cognitions of potential living organ donors: the development of the Donation Cognition Instrument in potential kidney donors.

PubMed

Wirken, Lieke; van Middendorp, Henriët; Hooghof, Christina W; Sanders, Jan-Stephan F; Dam, Ruth E; van der Pant, Karlijn A M I; Berendsen, Elsbeth C M; Wellink, Hiske; Dackus, Henricus J A; Hoitsma, Andries J; Hilbrands, Luuk B; Evers, Andrea W M

2017-03-01

Cognitions surrounding living organ donation, including the motivation to donate, expectations of donation and worries about donation, are relevant themes during living donor evaluation. However, there is no reliable psychometric instrument assessing all these different cognitions. This study developed and validated a questionnaire to assess pre-donation motivations, expectations and worries regarding donation, entitled the Donation Cognition Instrument (DCI). Psychometric properties of the DCI were examined using exploratory factor analysis for scale structure and associations with validated questionnaires for construct validity assessment. From seven Dutch transplantation centres, 719 potential living kidney donors were included. The DCI distinguishes cognitions about donor benefits, recipient benefits, idealistic incentives, gratitude and worries about donation (Cronbach's alpha 0.76-0.81). Scores on pre-donation cognitions differed with regard to gender, age, marital status, religion and donation type. With regard to construct validity, the DCI was moderately correlated with expectations regarding donor's personal well-being and slightly to moderately to health-related quality of life. The DCI is found to be a reliable instrument assessing cognitions surrounding living organ donation, which might add to pre-donation quality of life measures in facilitating psychosocial donor evaluation by healthcare professionals. © The Author 2017. Published by Oxford University Press on behalf of ERA-EDTA. All rights reserved.

Toward Automated Computer-Based Visualization and Assessment of Team-Based Performance

ERIC Educational Resources Information Center

Ifenthaler, Dirk

2014-01-01

A considerable amount of research has been undertaken to provide insights into the valid assessment of team performance. However, in many settings, manual and therefore labor-intensive assessment instruments for team performance have limitations. Therefore, automated assessment instruments enable more flexible and detailed insights into the…

Predictive Validity of the HKT-R Risk Assessment Tool: Two and 5-Year Violent Recidivism in a Nationwide Sample of Dutch Forensic Psychiatric Patients.

PubMed

Bogaerts, Stefan; Spreen, Marinus; Ter Horst, Paul; Gerlsma, Coby

2018-06-01

This study has examined the predictive validity of the Historical Clinical Future [ Historisch Klinisch Toekomst] Revised risk assessment scheme in a cohort of 347 forensic psychiatric patients, which were discharged between 2004 and 2008 from any of 12 highly secure forensic centers in the Netherlands. Predictive validity was measured 2 and 5 years after release. Official reconviction data obtained from the Dutch Ministry of Security and Justice were used as outcome measures. Violent reoffending within 2 and 5 years after discharge was assessed. With regard to violent reoffending, results indicated that the predictive validity of the Historical domain was modest for 2 (area under the curve [AUC] = .75) and 5 (AUC = .74) years. The predictive validity of the Clinical domain was marginal for 2 (admission: AUC = .62; discharge: AUC = .63) and 5 (admission: AUC = .69; discharge: AUC = .62) years after release. The predictive validity of the Future domain was modest (AUC = .71) for 2 years and low for 5 (AUC = .58) years. The total score of the instrument was modest for 2 years (AUC = .78) and marginal for 5 (AUC = .68) years. Finally, the Final Risk Judgment was modest for 2 years (AUC = .78) and marginal for 5 (AUC = .63) years time at risk. It is concluded that this risk assessment instrument appears to be a satisfactory instrument for risk assessment.

Development and Validation of a Novel Generic Health-related Quality of Life Instrument With 20 Items (HINT-20)

PubMed Central

2017-01-01

Objectives Few attempts have been made to develop a generic health-related quality of life (HRQoL) instrument and to examine its validity and reliability in Korea. We aimed to do this in our present study. Methods After a literature review of existing generic HRQoL instruments, a focus group discussion, in-depth interviews, and expert consultations, we selected 30 tentative items for a new HRQoL measure. These items were evaluated by assessing their ceiling effects, difficulty, and redundancy in the first survey. To validate the HRQoL instrument that was developed, known-groups validity and convergent/discriminant validity were evaluated and its test-retest reliability was examined in the second survey. Results Of the 30 items originally assessed for the HRQoL instrument, four were excluded due to high ceiling effects and six were removed due to redundancy. We ultimately developed a HRQoL instrument with a reduced number of 20 items, known as the Health-related Quality of Life Instrument with 20 items (HINT-20), incorporating physical, mental, social, and positive health dimensions. The results of the HINT-20 for known-groups validity were poorer in women, the elderly, and those with a low income. For convergent/discriminant validity, the correlation coefficients of items (except vitality) in the physical health dimension with the physical component summary of the Short Form 36 version 2 (SF-36v2) were generally higher than the correlations of those items with the mental component summary of the SF-36v2, and vice versa. Regarding test-retest reliability, the intraclass correlation coefficient of the total HINT-20 score was 0.813 (p<0.001). Conclusions A novel generic HRQoL instrument, the HINT-20, was developed for the Korean general population and showed acceptable validity and reliability. PMID:28173686

Development and evaluation of an instrument for assessing brief behavioral change interventions.

PubMed

Strayer, Scott M; Martindale, James R; Pelletier, Sandra L; Rais, Salehin; Powell, Jon; Schorling, John B

2011-04-01

To develop an observational coding instrument for evaluating the fidelity and quality of brief behavioral change interventions based on the behavioral theories of the 5 A's, Stages of Change and Motivational Interviewing. Content and face validity were assessed prior to an intervention where psychometric properties were evaluated with a prospective cohort of 116 medical students. Properties assessed included the inter-rater reliability of the instrument, internal consistency of the full scale and sub-scales and descriptive statistics of the instrument. Construct validity was assessed based on student's scores. Inter-rater reliability for the instrument was 0.82 (intraclass correlation). Internal consistency for the full scale was 0.70 (KR20). Internal consistencies for the sub-scales were as follows: MI intervention component (KR20=.7); stage-appropriate MI-based intervention (KR20=.55); MI spirit (KR20=.5); appropriate assessment (KR20=.45) and appropriate assisting (KR20=.56). The instrument demonstrated good inter-rater reliability and moderate overall internal consistency when used to assess performing brief behavioral change interventions by medical students. This practical instrument can be used with minimal training and demonstrates promising psychometric properties when evaluated with medical students counseling standardized patients. Further testing is required to evaluate its usefulness in clinical settings. Copyright © 2010 Elsevier Ireland Ltd. All rights reserved.

Multidimensional measures validated for home health needs of older persons: A systematic review.

PubMed

de Rossi Figueiredo, Daniela; Paes, Lucilene Gama; Warmling, Alessandra Martins; Erdmann, Alacoque Lorenzini; de Mello, Ana Lúcia Schaefer Ferreira

2018-01-01

To conduct a systematic review of the literature on valid and reliable multidimensional instruments to assess home health needs of older persons. Systematic review. Electronic databases, PubMed/Medline, Web of Science, Scopus, Cumulative Index to Nursing and Allied Health Literature, Scientific Electronic Library Online and the Latin American and Caribbean Health Sciences Information. All English, Portuguese and Spanish literature which included studies of reliability and validity of instruments that assessed at least two dimensions: physical, psychological, social support and functional independence, self-rated health behaviors and contextual environment and if such instruments proposed interventions after evaluation and/or monitoring changes over a period of time. Older persons aged 60 years or older. Of the 2397 studies identified, 32 were considered eligible. Two-thirds of the instruments proposed the physical, psychological, social support and functional independence dimensions. Inter-observer and intra-observer reliability and internal consistency values were 0.7 or above. More than two-thirds of the studies included validity (n=26) and more than one validity was tested in 15% (n=4) of these. Only 7% (n=2) proposed interventions after evaluation and/or monitoring changes over a period of time. Although the multidimensional assessment was performed, and the reliability values of the reviewed studies were satisfactory, different validity tests were not present in several studies. A gap at the instrument conception was observed related to interventions after evaluation and/or monitoring changes over a period of time. Further studies with this purpose are necessary for home health needs of the older persons. Copyright © 2017 Elsevier Ltd. All rights reserved.

‘‘Time after time’’: A Quali-T method for assessing music's impact on well-being

PubMed Central

2013-01-01

This article considers the question of how to produce ecologically valid assessments of music's role as a health technology. To address this question, I consider critically some of the standard quantitative instruments used to assess well-being and quality of life. I suggest that these instruments do not lend themselves well to the production of ecologically valid assessments and understandings for two reasons: (1) the process of data elicitation is removed from everyday meanings and practices and therefore risks producing data that is an artifact of the situation in which it is elicited (2) standard, quantitative instruments are not neutral but are rather discursive texts that are inevitably imbued with a politics of expertise and an image of the health care client. For these reasons, I suggest that we consider the question of how to develop ecologically valid, client-centered assessment measures. To that end, I introduce a third critique of the standard quantitative instruments, namely that they are associated with, and promote, an ontology of wellness/illness that downplays the temporally variable and situationally emergent nature of both wellness/illness and musical interventions themselves. As an alternative mode of assessment, I suggest that we reconsider the value of singular case studies and I describe a set of principles that can assist researchers to produce ecologically valid assessments. To this end I introduce the concept of the musical event as a more ecologically valid means for illuminating the specific mechanisms by which music aids well-being. I suggest that the case study approach is temporally sensitive, that it lends itself to an emergent ontology of wellness/illness, and that it is client-centered (and can also be user-led). PMID:23930990

Performance-based comparison of neonatal intubation training outcomes: simulator and live animal.

PubMed

Andreatta, Pamela B; Klotz, Jessica J; Dooley-Hash, Suzanne L; Hauptman, Joe G; Biddinger, Bea; House, Joseph B

2015-02-01

The purpose of this article was to establish psychometric validity evidence for competency assessment instruments and to evaluate the impact of 2 forms of training on the abilities of clinicians to perform neonatal intubation. To inform the development of assessment instruments, we conducted comprehensive task analyses including each performance domain associated with neonatal intubation. Expert review confirmed content validity. Construct validity was established using the instruments to differentiate between the intubation performance abilities of practitioners (N = 294) with variable experience (novice through expert). Training outcomes were evaluated using a quasi-experimental design to evaluate performance differences between 294 subjects randomly assigned to 1 of 2 training groups. The training intervention followed American Heart Association Pediatric Advanced Life Support and Neonatal Resuscitation Program protocols with hands-on practice using either (1) live feline or (2) simulated feline models. Performance assessment data were captured before and directly following the training. All data were analyzed using analysis of variance with repeated measures and statistical significance set at P < .05. Content validity, reliability, and consistency evidence were established for each assessment instrument. Construct validity for each assessment instrument was supported by significantly higher scores for subjects with greater levels of experience, as compared with those with less experience (P = .000). Overall, subjects performed significantly better in each assessment domain, following the training intervention (P = .000). After controlling for experience level, there were no significant differences among the cognitive, performance, and self-efficacy outcomes between clinicians trained with live animal model or simulator model. Analysis of retention scores showed that simulator trained subjects had significantly higher performance scores after 18 weeks (P = .01) and 52 weeks (P = .001) and cognitive scores after 52 weeks (P = .001). The results of this study demonstrate the feasibility of using valid, reliable assessment instruments to assess clinician competency and self-efficacy in the performance of neonatal intubation. We demonstrated the relative equivalency of live animal and simulation-based models as tools to support acquisition of neonatal intubation skills. Retention of performance abilities was greater for subjects trained using the simulator, likely because it afforded greater opportunity for repeated practice. Outcomes in each assessment area were influenced by the previous intubation experience of participants. This suggests that neonatal intubation training programs could be tailored to the level of provider experience to make efficient use of time and educational resources. Future research focusing on the uses of assessment in the applied clinical environment, as well as identification of optimal training cycles for performance retention, is merited.

Critical review of the validity of patient satisfaction questionnaires pertaining to oral health care.

PubMed

Nair, Rahul; Ishaque, Sana; Spencer, Andrew John; Luzzi, Liana; Do, Loc Giang

2018-03-30

Review the validation process reported for oral healthcare satisfaction scales that intended to measure general oral health care that is not restricted to specific subspecialties or interventions. After preliminary searches, PUBMED and EMBASE were searched using a broad search strategy, followed by a snowball strategy using the references of the publications included from database searches. Title and abstract were screened for assessing inclusion, followed by a full-text screening of these publications. English language publications on multi-item questionnaires that report on a scale measuring patient satisfaction for oral health care were included. Publications were excluded when they did not report on any psychometric validation, or the scales were addressing specific treatments or subspecialities in oral health care. Fourteen instruments were identified from as many publications that report on their initial validation, while five more publications reported on further testing of the validity of these instruments. Number of items (range: 8-42) and dimension reported (range: 2-13) were often dissimilar between the assessed measurement instruments. There was also a lack of methodologies to incorporate patient's subjective perspective. Along with a limited reporting of psychometric properties of instruments, cross-cultural adaptations were limited to translation processes. The extent of validity and reliability of the included instruments was largely unassessed, and appropriate instruments for populations outside of those belonging to general adult populations were not present. © 2018 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

Translation and validation of the Self-care of Heart Failure Index into Persian.

PubMed

Siabani, Soraya; Leeder, Stephen R; Davidson, Patricia M; Najafi, Farid; Hamzeh, Behrooz; Solimani, Akram; Siahbani, Sara; Driscoll, Tim

2014-01-01

Chronic heart failure (CHF) is a common burdensome health problem worldwide. Self-care improves outcomes in patients with CHF. The Self-care of Heart Failure Index (SCHFI) is a well-known scale for assessing self-care. A reliable, valid, and culturally acceptable instrument is needed to develop and test self-care interventions in Iran. We sought to translate and validate the Persian version of SCHFI v 6.2 (pSCHFI). We translated the SCHFI into Persian (pSCHFI) using standardized methods. The reliability was evaluated by assessing Cronbach's α coefficient. Expert opinion, discussion with patients, and confirmatory factor analysis were used to assess face validity, content validity, and construct validity, respectively. The analysis, using 184 participants, showed acceptable internal consistency and construct validity for the 3 subscales of pSCHFI-self-care maintenance, self-care management, and self-care self-confidence. The pSCHFI is a valid instrument with an acceptable reliability for evaluating self-care in Persian patients with heart failure.

Development of performance assessment instrument based contextual learning for measuring students laboratory skills

NASA Astrophysics Data System (ADS)

Susilaningsih, E.; Khotimah, K.; Nurhayati, S.

2018-04-01

The assessment of laboratory skill in general hasn’t specific guideline in assessment, while the individual assessment of students during a performance and skill in performing laboratory is still not been observed and measured properly. Alternative assessment that can be used to measure student laboratory skill is use performance assessment. The purpose of this study was to determine whether the performance assessment instrument that the result of research can be used to assess basic skills student laboratory. This research was conducted by the Research and Development. The result of the data analysis performance assessment instruments developed feasible to implement and validation result 62.5 with very good categories for observation sheets laboratory skills and all of the components with the very good category. The procedure is the preliminary stages of research and development stages. Preliminary stages are divided in two, namely the field studies and literature studies. The development stages are divided into several parts, namely 1) development of the type instrument, 2) validation by an expert, 3) a limited scale trial, 4) large-scale trials and 5) implementation of the product. The instrument included in the category of effective because 26 from 29 students have very high laboratory skill and high laboratory skill. The research of performance assessment instrument is standard and can be used to assess basic skill student laboratory.

Assessing reliability and validity measures in managed care studies.

PubMed

Montoya, Isaac D

2003-01-01

To review the reliability and validity literature and develop an understanding of these concepts as applied to managed care studies. Reliability is a test of how well an instrument measures the same input at varying times and under varying conditions. Validity is a test of how accurately an instrument measures what one believes is being measured. A review of reliability and validity instructional material was conducted. Studies of managed care practices and programs abound. However, many of these studies utilize measurement instruments that were developed for other purposes or for a population other than the one being sampled. In other cases, instruments have been developed without any testing of the instrument's performance. The lack of reliability and validity information may limit the value of these studies. This is particularly true when data are collected for one purpose and used for another. The usefulness of certain studies without reliability and validity measures is questionable, especially in cases where the literature contradicts itself

Development of the Informing Relatives Inventory (IRI): Assessing Index Patients' Knowledge, Motivation and Self-Efficacy Regarding the Disclosure of Hereditary Cancer Risk Information to Relatives.

PubMed

de Geus, Eveline; Aalfs, Cora M; Menko, Fred H; Sijmons, Rolf H; Verdam, Mathilde G E; de Haes, Hanneke C J M; Smets, Ellen M A

2015-08-01

Despite the use of genetic services, counselees do not always share hereditary cancer information with at-risk relatives. Reasons for not informing relatives may be categorized as a lack of: knowledge, motivation, and/or self-efficacy. This study aims to develop and test the psychometric properties of the Informing Relatives Inventory, a battery of instruments that intend to measure counselees' knowledge, motivation, and self-efficacy regarding the disclosure of hereditary cancer risk information to at-risk relatives. Guided by the proposed conceptual framework, existing instruments were selected and new instruments were developed. We tested the instruments' acceptability, dimensionality, reliability, and criterion-related validity in consecutive index patients visiting the Clinical Genetics department with questions regarding hereditary breast and/or ovarian cancer or colon cancer. Data of 211 index patients were included (response rate = 62%). The Informing Relatives Inventory (IRI) assesses three barriers in disclosure representing seven domains. Instruments assessing index patients' (positive) motivation and self-efficacy were acceptable and reliable and suggested good criterion-related validity. Psychometric properties of instruments assessing index patients knowledge were disputable. These items were moderately accepted by index patients and the criterion-related validity was weaker. This study presents a first conceptual framework and associated inventory (IRI) that improves insight into index patients' barriers regarding the disclosure of genetic cancer information to at-risk relatives. Instruments assessing (positive) motivation and self-efficacy proved to be reliable measurements. Measuring index patients knowledge appeared to be more challenging. Further research is necessary to ensure IRI's dimensionality and sensitivity to change.

Validity and reliability of the multidimensional assessment of fatigue scale in Iranian patients with relapsing-remitting subtype of multiple sclerosis.

PubMed

Behrangrad, Shabnam; Kordi Yoosefinejad, Amin

2018-03-01

The purpose of this study is to investigate the validity and reliability of the Persian version of the Multidimensional Assessment of Fatigue Scale (MAFS) in an Iranian population with multiple sclerosis. A self-reported survey on fatigue including the MAFS, Fatigue Impact Scale and demographic measures was completed by 130 patients with multiple sclerosis and 60 healthy persons sampled with a convenience method. Test-retest reliability and validity were evaluated 3 days apart. Construct validity of the MAFS was assessed with the Fatigue Impact Scale. The MAFS had high internal consistency (Cronbach's alpha >0.9) and 3-d test-retest reliability (intraclass correlation coefficient = 0.99). Correlation between the Fatigue Impact Scale and MAFS was high (r = 0.99). Correlation between MAFS scores and the Expanded Disability Status Scale was also strong (r = 0.85). Questionnaire items showed acceptable item-scale correlation (0.968-0.993). The Persian version of the MAFS appears to be a valid and reliable questionnaire. It is an appropriate short multidimensional instrument to assess fatigue in patients with multiple sclerosis in clinical practice and research. Implications for Rehabilitation The Persian version of Multidimensional Assessment of Fatigue is a valid and reliable instrument for the assessment and monitoring the fatigue in Persian-language patients with multiple sclerosis. It is very easy to administer and a time efficient scale in comparison to other instruments evaluating fatigue in patients with multiple sclerosis.

Translation and validation of the Malay version of the Stroke Knowledge Test.

PubMed

Sowtali, Siti Noorkhairina; Yusoff, Dariah Mohd; Harith, Sakinah; Mohamed, Monniaty

2016-04-01

To date, there is a lack of published studies on assessment tools to evaluate the effectiveness of stroke education programs. This study developed and validated the Malay language version of the Stroke Knowledge Test research instrument. This study involved translation, validity, and reliability phases. The instrument underwent backward and forward translation of the English version into the Malay language. Nine experts reviewed the content for consistency, clarity, difficulty, and suitability for inclusion. Perceived usefulness and utilization were obtained from experts' opinions. Later, face validity assessment was conducted with 10 stroke patients to determine appropriateness of sentences and grammar used. A pilot study was conducted with 41 stroke patients to determine the item analysis and reliability of the translated instrument using the Kuder Richardson 20 or Cronbach's alpha. The final Malay version Stroke Knowledge Test included 20 items with good content coverage, acceptable item properties, and positive expert review ratings. Psychometric investigations suggest that Malay version Stroke Knowledge Test had moderate reliability with Kuder Richardson 20 or Cronbach's alpha of 0.58. Improvement is required for Stroke Knowledge Test items with unacceptable difficulty indices. Overall, the average rating of perceived usefulness and perceived utility of the instruments were both 72.7%, suggesting that reviewers were likely to use the instruments in their facilities. Malay version Stroke Knowledge Test was a valid and reliable tool to assess educational needs and to evaluate stroke knowledge among participants of group-based stroke education programs in Malaysia.

Multiple Subtypes among Vocationally Undecided College Students: A Model and Assessment Instrument.

ERIC Educational Resources Information Center

Jones, Lawrence K.; Chenery, Mary Faeth

1980-01-01

A model of vocational decision status was developed, and an instrument was constructed and used to assess its three dimensions. Results demonstrated the utility of the model, supported the reliability and validity of the instrument, and illustrated the value of viewing vocationally undecided students as multiple subtypes. (Author)

Development of performance test instrument in the experiment of law of conservation mass using self and peer assessment’s technique

NASA Astrophysics Data System (ADS)

Siswaningsih, W.; Nahadi; Firmansyah, D. R.

2018-05-01

The purpose of this research is to develop the instrument of performance assessment of law of mass conservation using self and peer assessment technique that meet valid and reliable criteria. The instrument components consist of task and rubric. The method used is development and validation.Value of the instrument reliability obtained from twice observations that are at four and six students every group with three same observers. Cronbach alpha value for four and six students every group consecutively are 0.94 and 0.76, indicating that value shows that the instrument is reliable. Optimum amount of the students that can be observed are four students. The implementation of the instrument to limited group of students showed that All of the students give positive responses to the instrument used with the interpretation of questionnaire scores >90% that categorized as good.

Development of a quality instrument for assessing the spontaneous reports of ADR/ADE using Delphi method in China.

PubMed

Chen, Lixun; Jiang, Ling; Shen, Aizong; Wei, Wei

2016-09-01

The frequently low quality of submitted spontaneous reports is of an increasing concern; to our knowledge, no validated instrument exists for assessing case reports' quality comprehensively enough. This work was conducted to develop such a quality instrument for assessing the spontaneous reports of adverse drug reaction (ADR)/adverse drug event (ADE) in China. Initial evaluation indicators were generated using systematic and literature data analysis. Final indicators and their weights were identified using Delphi method. The final quality instrument was developed by adopting the synthetic scoring method. A consensus was reached after four rounds of Delphi survey. The developed quality instrument consisted of 6 first-rank indicators, 18 second-rank indicators, and 115 third-rank indicators, and each rank indicator has been weighted. It evaluates the quality of spontaneous reports of ADR/ADE comprehensively and quantitatively on six parameters: authenticity, duplication, regulatory, completeness, vigilance level, and reporting time frame. The developed instrument was tested with good reliability and validity, which can be used to comprehensively and quantitatively assess the submitted spontaneous reports of ADR/ADE in China.

[Validity of an instrument for assessing food consumption, food habits and cooking skills in 8-11 years old students].

PubMed

Lera, Lydia; Fretes, Gabriela; González, Carmen Gloria; Salinas, Judith; Vio del Rio, Fernando

2015-05-01

An instrument to measure food knowledge, food consumption, cooking skills, food habits and food expenses at school is necessary to assess changes in food practices. To validate an instrument to measure changes in food knowledge, food consumption, cooking skills, food habits and food expenses in Chilean school children 8 - 11 years from third to fifth grade. A validation of a questionnaire with 42 questions was conducted in two stages: the first to assess temporal stability, concordance and internal consistency in 45 children. The second one to apply the survey, modified with the results of the first stage, in 90 children assessing internal consistency. The first survey with 42 questions showed a reasonable temporal stability, concordance and internal consistency for cooking skills, habits and food expenditure at school. Internal consistency was good for food consumption, but not so good for food knowledge. In the final validation with 90 children, there was good consistency for food consumption but bad for food knowledge. Besides, children with cooking skills ate more healthy food and those who expended more money at school, consumed less healthy food. Food knowledge questions were eliminated from the instrument, which was elaborated with 28 questions about food consumption, cooking skills, food habits and food expenses at school. This instrument is useful to assess changes in food and nutrition education interventions in 8 -11 years children, in particular to measure cooking skills and food expenses at school. Copyright AULA MEDICA EDICIONES 2014. Published by AULA MEDICA. All rights reserved.

Validation of Yoon's Critical Thinking Disposition Instrument.

PubMed

Shin, Hyunsook; Park, Chang Gi; Kim, Hyojin

2015-12-01

The lack of reliable and valid evaluation tools targeting Korean nursing students' critical thinking (CT) abilities has been reported as one of the barriers to instructing and evaluating students in undergraduate programs. Yoon's Critical Thinking Disposition (YCTD) instrument was developed for Korean nursing students, but few studies have assessed its validity. This study aimed to validate the YCTD. Specifically, the YCTD was assessed to identify its cross-sectional and longitudinal measurement invariance. This was a validation study in which a cross-sectional and longitudinal (prenursing and postnursing practicum) survey was used to validate the YCTD using 345 nursing students at three universities in Seoul, Korea. The participants' CT abilities were assessed using the YCTD before and after completing an established pediatric nursing practicum. The validity of the YCTD was estimated and then group invariance test using multigroup confirmatory factor analysis was performed to confirm the measurement compatibility of multigroups. A test of the seven-factor model showed that the YCTD demonstrated good construct validity. Multigroup confirmatory factor analysis findings for the measurement invariance suggested that this model structure demonstrated strong invariance between groups (i.e., configural, factor loading, and intercept combined) but weak invariance within a group (i.e., configural and factor loading combined). In general, traditional methods for assessing instrument validity have been less than thorough. In this study, multigroup confirmatory factor analysis using cross-sectional and longitudinal measurement data allowed validation of the YCTD. This study concluded that the YCTD can be used for evaluating Korean nursing students' CT abilities. Copyright © 2015. Published by Elsevier B.V.

The modified patient enablement instrument: a Portuguese cross-cultural adaptation, validity and reliability study.

PubMed

Remelhe, Mafalda; Teixeira, Pedro M; Lopes, Irene; Silva, Luís; Correia de Sousa, Jaime

2017-01-12

Enabling patients with asthma to obtain the knowledge, confidence and skills they need in order to assume a major role in the management of their disease is cost effective. It should be an integral part of any plan for long-term control of asthma. The modified Patient Enablement Instrument (mPEI) is an easily administered questionnaire that was adapted in the United Kingdom to measure patient enablement in asthma, but its applicability in Portugal is not known. Validity and reliability of questionnaires should be tested before use in settings different from those of the original version. The purpose of this study was to test the applicability of the mPEI to Portuguese asthma patients after translation and cross-cultural adaptation, and to verify the structural validity, internal consistency and reproducibility of the instrument. The mPEI was translated to Portuguese and back translated to English. Its content validity was assessed by a debriefing interview with 10 asthma patients. The translated instrument was then administered to a random sample of 142 patients with persistent asthma. Structural validity and internal consistency were assessed. For reproducibility analysis, 86 patients completed the instrument again 7 days later. Item-scale correlations and exploratory factor analysis were used to assess structural validity. Cronbach's alpha was used to test internal consistency, and the intra-class correlation coefficient was used for the analysis of reproducibility. All items of the Portuguese version of the mPEI were found to be equivalent to the original English version. There were strong item-scale correlations that confirmed construct validity, with a one component structure and good internal consistency (Cronbach's alpha >0.8) as well as high test-retest reliability (ICC=0.85). The mPEI showed sound psychometric properties for the evaluation of enablement in patients with asthma making it a reliable instrument for use in research and clinical practice in Portugal. Further studies are needed to confirm its responsiveness.

Development and validation of the computer technology literacy self-assessment scale for Taiwanese elementary school students.

PubMed

Chang, Chiung-Sui

2008-01-01

The purpose of this study was to describe the development and validation of an instrument to identify various dimensions of the computer technology literacy self-assessment scale (CTLS) for elementary school students. The instrument included five CTLS dimensions (subscales): the technology operation skills, the computer usages concepts, the attitudes toward computer technology, the learning with technology, and the Internet operation skills. Participants were 1,539 elementary school students in Taiwan. Data analysis indicated that the instrument developed in the study had satisfactory validity and reliability. Correlations analysis supported the legitimacy of using multiple dimensions in representing students' computer technology literacy. Significant differences were found between male and female students, and between grades on some CTLS dimensions. Suggestions are made for use of the instrument to examine complicated interplays between students' computer behaviors and their computer technology literacy.

Measurement properties of instruments that assess participation in young people with autism spectrum disorder: a systematic review.

PubMed

Lami, Francesca; Egberts, Kristine; Ure, Alexandra; Conroy, Rowena; Williams, Katrina

2018-03-01

To systematically review the measurement properties of instruments assessing participation in young people with autism spectrum disorder (ASD). A search was performed in MEDLINE, PsycINFO, and PubMed combining three constructs ('ASD', 'test of participation', 'measurement properties'). Results were restricted to articles including people aged 6 to 29 years. The 2539 identified articles were independently screened by two reviewers. For the included articles, data were extracted using standard forms and their risk of bias was assessed. Nine studies (8 cross-sectional) met the inclusion criteria, providing information on seven different instruments. The total sample included 634 participants, with sex available for 600 (males=494; females=106) and age available for 570, with mean age for these participants 140.58 months (SD=9.11; range=36-624). Included instruments were the school function assessment, vocational index, children's assessment of participation and enjoyment/preferences for activities of children, experience sampling method, Pediatric Evaluation of Disability Inventory, Computer Adaptive Test, adolescent and young adult activity card sort, and Patient-Reported Outcomes Measurement Information System parent-proxy peer relationships. Seven studies assessed reliability and validity; good properties were reported for half of the instruments considered. Most studies (n=6) had high risk of bias. Overall the quality of the evidence for each tool was limited. Validation of these instruments, or others that comprehensively assess participation, is needed. Future studies should follow recommended methodological standards. Seven instruments have been used to assess participation in young people with autism. One instrument, with excellent measurement properties in one study, does not comprehensively assess participation. Studies of three instruments that incorporate a more comprehensive assessment of participation have methodological limitations. Overall, limited evidence exists regarding measurement properties of participation assessments for young people with autism. © 2017 Mac Keith Press.

Health Service Quality Scale: Brazilian Portuguese translation, reliability and validity.

PubMed

Rocha, Luiz Roberto Martins; Veiga, Daniela Francescato; e Oliveira, Paulo Rocha; Song, Elaine Horibe; Ferreira, Lydia Masako

2013-01-17

The Health Service Quality Scale is a multidimensional hierarchical scale that is based on interdisciplinary approach. This instrument was specifically created for measuring health service quality based on marketing and health care concepts. The aim of this study was to translate and culturally adapt the Health Service Quality Scale into Brazilian Portuguese and to assess the validity and reliability of the Brazilian Portuguese version of the instrument. We conducted a cross-sectional, observational study, with public health system patients in a Brazilian university hospital. Validity was assessed using Pearson's correlation coefficient to measure the strength of the association between the Brazilian Portuguese version of the instrument and the SERVQUAL scale. Internal consistency was evaluated using Cronbach's alpha coefficient; the intraclass (ICC) and Pearson's correlation coefficients were used for test-retest reliability. One hundred and sixteen consecutive postoperative patients completed the questionnaire. Pearson's correlation coefficient for validity was 0.20. Cronbach's alpha for the first and second administrations of the final version of the instrument were 0.982 and 0.986, respectively. For test-retest reliability, Pearson's correlation coefficient was 0.89 and ICC was 0.90. The culturally adapted, Brazilian Portuguese version of the Health Service Quality Scale is a valid and reliable instrument to measure health service quality.

The development and psychometrical evaluation of a set of instruments to evaluate the effectiveness of diabetes patient education.

PubMed

Duprez, Veerle; De Pover, Marleen; De Spiegelaere, Marc; Beeckman, Dimitri

2014-02-01

To develop a set of psychometrically sound instruments to assess knowledge, self-management and self-efficacy of diabetic patients. Furthermore, a survey to evaluate the satisfaction about diabetes education for patients was developed and tested. Treatment and secondary prevention of diabetes require a complex combination of care components. Patients' education has been accepted to improve diabetes knowledge, self-management and self-efficacy. Psychometrically sound instruments are needed to measure these patient-centred outcomes. Psychometric instrument validation. The first phase included a systematic literature review to develop the instruments. Content validity was evaluated using a two-round Delphi procedure involving diabetes experts. The content validity of the instruments was excellent. In a second phase, a convenience sample of 188 diabetic patients in two hospitals in one specific care region in Belgium participated in the psychometric evaluation. The criterion-related validity and internal consistency reliability were evaluated. The study produced a 21-item knowledge instrument, reflecting knowledge about 'glycemic control' and 'medico-social management aspects'. The self-management instrument included 32 statements, reflecting 'treatment and compliance' and 'general lifestyle'. The self-efficacy instrument included 30 items, reflecting 'nutrition', 'treatment' and 'regimen'. The patient satisfaction survey included 36 items, reflecting satisfaction about the relationship among the diabetes specialist, the diabetes educator, podiatrist and dietician. An instrument set with sound psychometric characteristics was developed to assess knowledge, self-management and self-efficacy of diabetic patients. Future studies should focus on the association between the instrument outcomes and clinical patient outcomes. The current instrument can support the design of educational interventions and training programmes and reduce inconsistencies in the information that patients receive. Furthermore, the instruments can be used for benchmarking the quality of diabetic patient education. © 2013 Blackwell Publishing Ltd.

The Early Development Instrument: An Examination of Convergent and Discriminant Validity

ERIC Educational Resources Information Center

Hymel, Shelley; LeMare, Lucy; McKee, William

2011-01-01

The convergent and discriminant validity of the Early Development Instrument (EDI), a teacher-rated assessment of children's "school readiness", was investigated in a multicultural sample of 267 kindergarteners (53% male). Teachers evaluations on the EDI, both overall and in five domains (physical health/well-being, social competence,…

Validation of the diabetes, hypertension and hyperlipidemia (DHL) knowledge instrument in Malaysia

PubMed Central

2012-01-01

Background Patient's knowledge on diabetes, hypertension and hyperlipidaemia and its medications can be used as one of the outcome measures to assess the effectiveness of educational intervention. To date, no such instrument has been validated in Malaysia. Therefore, the aim of this study was to evaluate the validity and reliability of the Diabetes, Hypertension and Hyperlipidemia (DHL) knowledge instrument for assessing the knowledge of patients with type 2 diabetes in Malaysia. Methods A 28-item instrument which comprised of 5 domains: diabetes, hypertension, hyperlipidemia, medications and general issues was designed and tested. One point was given for every correct answer, whilst zero was given for incorrect answers. Scores ranged from 0 to 28, which were then converted into percentage. This was administered to 77 patients with type 2 diabetes in a tertiary hospital, who were on medication(s) for diabetes and who could understand English (patient group), and to 40 pharmacists (professional group). The DHL knowledge instrument was administered again to the patient group after one month. Excluded were patients less than 18 years old. Results Flesch reading ease was 60, which is satisfactory, while the mean difficulty factor(SD) was 0.74(0.21), indicating that DHL knowledge instrument was moderately easy. Internal consistency of the instrument was good, with Cronbach's α = 0.791. The test-retest scores showed no significant difference for 26 out of the 28 items, indicating that the questionnaire has achieved stable reliability. The overall mean(SD) knowledge scores was significantly different between the patient and professional groups [74.35(14.88) versus 93.84(6.47), p < 0.001]. This means that the DHL knowledge instrument could differentiate the knowledge levels of participants. The DHL knowledge instrument shows similar psychometric properties as other validated questionnaires. Conclusions The DHL knowledge instrument shows good promise to be adopted as an instrument for assessing diabetic patients' knowledge concerning their disease conditions and medications in Malaysia. PMID:22361093

Cultural adaptation: translatability assessment and linguistic validation of the patient-reported outcome instrument for irritable bowel syndrome with diarrhea.

PubMed

Delgado-Herrera, Leticia; Lasch, Kathryn; Popielnicki, Ana; Nishida, Akito; Arbuckle, Rob; Banderas, Benjamin; Zentner, Susan; Gagainis, Ingrid; Zeiher, Bernhardt

2016-01-01

Following a 2009 US Food and Drug Administration guidance, a new patient-reported outcome (PRO) instrument was developed to support end points in multinational clinical trials assessing irritable bowel syndrome with diarrhea (IBS-D) symptom severity. Our objective was to assess the translatability of the IBS-D PRO instrument into ten languages, and subsequently perform a cultural adaptation/linguistic validation of the questionnaire into Japanese and US Spanish. Translatability assessments of the US English version of the IBS-D PRO were performed by experienced PRO translators who were native speakers of each target language and currently residing in target-language countries. Languages were Chinese (People's Republic of China), Dutch (the Netherlands), French (Belgium), German (Germany), Japanese (Japan), Polish (Poland), Portuguese (Brazil), Russian (Russia), Spanish (Mexico), and Spanish (US). The project team assessed the instrument to identify potential linguistic and/or cultural adaptation issues. After the issues identified were resolved, the instrument was translated into Spanish (US) and Japanese through a process of two forward translations, one reconciled translation, and one backward translation. The project team reviewed the translated versions before the instruments were evaluated by cognitive debriefing interviews with samples of five Spanish (US) and five Japanese IBS-D patients. Linguistic and cultural adaptation concerns identified during the translatability assessment required minor revisions, mainly the presentation of dates/times and word structure. During the cognitive debriefing interviews, two of five Spanish respondents misunderstood the term "bowel movement" to mean only diarrhea in the Spanish version. Consequently, the term was changed from "movimiento intestinal" to "evacuaciones". None of the Japanese respondents identified issues with the Japanese version. The translatability of the IBS-D PRO instrument into ten target languages was confirmed, with only minor changes made to the translations of the instrument. The translation and linguistic validation into Spanish (US) and Japanese provide evidence that this instrument can be used in multinational trials and clinical settings.

Inflammatory bowel disease-specific health-related quality of life instruments: a systematic review of measurement properties.

PubMed

Chen, Xin-Lin; Zhong, Liang-Huan; Wen, Yi; Liu, Tian-Wen; Li, Xiao-Ying; Hou, Zheng-Kun; Hu, Yue; Mo, Chuan-Wei; Liu, Feng-Bin

2017-09-15

This review aims to critically appraise and compare the measurement properties of inflammatory bowel disease (IBD)-specific health-related quality of life instruments. Medline, EMBASE and ISI Web of Knowledge were searched from their inception to May 2016. IBD-specific instruments for patients with Crohn's disease, ulcerative colitis or IBD were enrolled. The basic characteristics and domains of the instruments were collected. The methodological quality of measurement properties and measurement properties of the instruments were assessed. Fifteen IBD-specific instruments were included, which included twelve instruments for adult IBD patients and three for paediatric IBD patients. All of the instruments were developed in North American and European countries. The following common domains were identified: IBD-related symptoms, physical, emotional and social domain. The methodological quality was satisfactory for content validity; fair in internal consistency, reliability, structural validity, hypotheses testing and criterion validity; and poor in measurement error, cross-cultural validity and responsiveness. For adult IBD patients, the IBDQ-32 and its short version (SIBDQ) had good measurement properties and were the most widely used worldwide. For paediatric IBD patients, the IMPACT-III had good measurement properties and had more translated versions. Most methodological quality should be promoted, especially measurement error, cross-cultural validity and responsiveness. The IBDQ-32 was the most widely used instrument with good reliability and validity, followed by the SIBDQ and IMPACT-III. Further validation studies are necessary to support the use of other instruments.

A Critical Review of Instruments Measuring Breastfeeding Attitudes, Knowledge, and Social Support.

PubMed

Casal, Corrine S; Lei, Ann; Young, Sera L; Tuthill, Emily L

2017-02-01

Breastfeeding provides beneficial health outcomes for infants and their mothers, and increasing its practice is a national priority in many countries. Despite increasing support to exclusively breastfeed, the prevalence at 6 months remains low. Breastfeeding behavior is influenced by a myriad of determinants, including breastfeeding attitudes, knowledge, and social support. Effective measurement of these determinants is critical to provide optimal support for women throughout the breastfeeding period. However, there are a multitude of available instruments measuring these constructs, which makes identification of an appropriate instrument challenging. Research aim: Our aim was to identify and critically examine the existing instruments measuring breastfeeding attitudes, knowledge, and social support. A total of 16 instruments was identified. Each instrument's purpose, theoretical underpinnings, and validity were analyzed. An overview, validation and adaptation for use in other settings was assessed for each instrument. Depth of reporting and validation testing differed greatly between instruments. Content, construct, and predictive validity were present for most but not all scales. When selecting and adapting instruments, attention should be paid to domains within the scale, number of items, and adaptation.

Physician Enabling Skills Questionnaire

PubMed Central

Hudon, Catherine; Lambert, Mireille; Almirall, José

2015-01-01

Abstract Objective To evaluate the reliability and validity of the newly developed Physician Enabling Skills Questionnaire (PESQ) by assessing its internal consistency, test-retest reliability, concurrent validity with patient-centred care, and predictive validity with patient activation and patient enablement. Design Validation study. Setting Saguenay, Que. Participants One hundred patients with at least 1 chronic disease who presented in a waiting room of a regional health centre family medicine unit. Main outcome measures Family physicians’ enabling skills, measured with the PESQ at 2 points in time (ie, while in the waiting room at the family medicine unit and 2 weeks later through a mail survey); patient-centred care, assessed with the Patient Perception of Patient-Centredness instrument; patient activation, assessed with the Patient Activation Measure; and patient enablement, assessed with the Patient Enablement Instrument. Results The internal consistency of the 6 subscales of the PESQ was adequate (Cronbach α = .69 to .92). The test-retest reliability was very good (r = 0.90; 95% CI 0.84 to 0.93). Concurrent validity with the Patient Perception of Patient-Centredness instrument was good (r = −0.67; 95% CI −0.78 to −0.53; P < .001). The PESQ accounts for 11% of the total variance with the Patient Activation Measure (r2 = 0.11; P = .002) and 19% of the variance with the Patient Enablement Instrument (r2 = 0.19; P < .001). Conclusion The newly developed PESQ presents good psychometric properties, allowing for its use in practice and research. PMID:26889507

An Integrative Review of Self-Efficacy Measurement Instruments in Youth with Type 1 Diabetes (T1DM)

PubMed Central

Rasbach, Lisa; Jenkins, Carolyn; Laffel, Lori

2014-01-01

Purpose The purpose of this study is to assess the extant literature on instruments used to measure self-efficacy in youth with type 1 diabetes (T1DM) and their caregivers and to critically evaluate these measurements. Methods An integrative review (2003–2013) was conducted searching PsycINFO, Cumulative Index to Nursing and Allied Health Literature (CINAHL), and U.S. National Library of Medicine PubMed service (PubMed) databases using key words diabetes, type 1 diabetes, and self-efficacy. The authors reviewed the resulting294 references for inclusion criteria of (a) sample of youth with T1DM or sample of caregivers of youth with T1DM, (b) description of the self-efficacy instrument as primary research, and (c) the instrument measured self-efficacy specifically related to diabetes management. Forty-five articles out of the initial 294 met criteria. Results Of the 45 articles, 10 different self-efficacy instruments were identified. The primary theoretical framework used was Bandura’s social cognitive theory and model of self-efficacy. Most participants were white middle class T1DM youth. Evaluations to assess validity often were not reported; however, a majority of studies reported high internal consistency of the instruments. Conclusions Sample homogeneity could limit the applicability of results to certain patient populations. Further psychometric analysis, including validity assessments, should be conducted in more diverse samples. Development of valid and reliable instruments for measuring self-efficacy that are sensitive to change across a wider caregiver base over time is necessary. While this review examined reliable and valid instruments used in research, future opportunities include evaluation of measuring self-efficacy in T1DM youth exposed to recent advances in diabetes management technologies. PMID:25216655

Development and Validation of a Symptom-Based Activity Index for Adults with Eosinophilic Esophagitis

PubMed Central

Schoepfer, Alain M.; Straumann, Alex; Panczak, Radoslaw; Coslovsky, Michael; Kuehni, Claudia E.; Maurer, Elisabeth; Haas, Nadine A.; Romero, Yvonne; Hirano, Ikuo; Alexander, Jeffrey A.; Gonsalves, Nirmala; Furuta, Glenn T.; Dellon, Evan S.; Leung, John; Collins, Margaret H.; Bussmann, Christian; Netzer, Peter; Gupta, Sandeep K.; Aceves, Seema S.; Chehade, Mirna; Moawad, Fouad J.; Enders, Felicity T.; Yost, Kathleen J.; Taft, Tiffany H.; Kern, Emily; Zwahlen, Marcel; Safroneeva, Ekaterina

2015-01-01

BACKGROUND & AIMS Standardized instruments are needed to assess the activity of eosinophilic esophagitis (EoE), to provide endpoints for clinical trials and observational studies. We aimed to develop and validate a patient-reported outcome (PRO) instrument and score, based on items that could account for variations in patients’ assessments of disease severity. We also evaluated relationships between patients’ assessment of disease severity and EoE-associated endoscopic, histologic, and laboratory findings. METHODS We collected information from 186 patients with EoE in Switzerland and the US (69.4% male; median age, 43 years) via surveys (n = 135), focus groups (n = 27), and semi-structured interviews (n = 24). Items were generated for the instruments to assess biologic activity based on physician input. Linear regression was used to quantify the extent to which variations in patient-reported disease characteristics could account for variations in patients’ assessment of EoE severity. The PRO instrument was prospectively used in 153 adult patients with EoE (72.5% male; median age, 38 years), and validated in an independent group of 120 patients with EoE (60.8% male; median age, 40.5 years). RESULTS Seven PRO factors that are used to assess characteristics of dysphagia, behavioral adaptations to living with dysphagia, and pain while swallowing accounted for 67% of the variation in patients’ assessment of disease severity. Based on statistical consideration and patient input, a 7-day recall period was selected. Highly active EoE, based on endoscopic and histologic findings, was associated with an increase in patient-assessed disease severity. In the validation study, the mean difference between patient assessment of EoE severity and PRO score was 0.13 (on a scale from 0 to 10). CONCLUSIONS We developed and validated an EoE scoring system based on 7 PRO items that assesses symptoms over a 7-day recall period. Clinicaltrials.gov number: NCT00939263. PMID:25160980

The N2N instrument to evaluate healthy work environments: an Italian validation.

PubMed

Palese, Alvisa; Dante, Angelo; Tonzar, Laura; Balboni, Bernardo

2014-02-01

The aims of the study were to (a) validate N2N Healthy Work Environment tool, (b) assess the healthiness of work environments as perceived by nurses themselves and (c) identify the factors associated with Italian nurses' perception of work environment healthiness. The linguistic and cultural adaptation of USA-N2N Healthy Work Environments was achieved through a process of forward/backward translation. Content validity was assessed by three expert nurses. The stability of the instrument was checked with a test/retest evaluation. The instrument psychometric properties, the confirmatory factor analysis as well the healthiness of the work environment and its determinant factors were evaluated with a sample of 294 nurses. The content and face validity of the N2N Healthy Work Environment instrument was confirmed. The instrument demonstrated good internal consistency (α of 0.82), excellent stability values (ρ > 0.70) and high levels of acceptability (response rate: 96.4 %). The confirmatory factor analysis has corroborated the existence of two factors as documented in the original instrument (Mays et al. in J Nurs Manag 19:18-26, 2011). Eighty-seven (29.6 %) nurses perceived the work environment where they work as "healthy". Working under a functional model of care delivery (χ(2) 24.856, p 0.000) and being responsible for one project or more (χ(2) 5.256, p 0.021) were associated with healthy environments. The instrument--valid and reliable, short in the number of items, easy to understand and based on international standards--allows a systematic assessment of the healthiness of the environment and might provide not only the opportunity to evaluate the effects of new organizational models and interventions, but also the possibility to activate a process of self-analysis and a process of ongoing review. The instrument can be used to systematically check the healthiness of Italian working environments, allowing for organizational diagnosis, targeted interventions and international comparisons.

Assessing the Validity of an Annual Survey for Measuring the Enacted Literacy Curriculum

ERIC Educational Resources Information Center

Camburn, Eric M.; Han, Seong Won; Sebastian, James

2017-01-01

Surveys are frequently used to inform consequential decisions about teachers, policies, and programs. Consequently, it is important to understand the validity of these instruments. This study assesses the validity of measures of instruction captured by an annual survey by comparing survey data with those of a validated daily log. The two…

Assessing Adolescent Mindfulness: Validation of an Adapted Mindful Attention Awareness Scale in Adolescent Normative and Psychiatric Populations

ERIC Educational Resources Information Center

Brown, Kirk Warren; West, Angela Marie; Loverich, Tamara M.; Biegel, Gina M.

2011-01-01

Interest in mindfulness-based interventions for children and adolescents is burgeoning, bringing with it the need for validated instruments to assess mindfulness in youths. The present studies were designed to validate among adolescents a measure of mindfulness previously validated for adults (e.g., Brown & Ryan, 2003), which we herein call…

In Defense of an Instrument-Based Approach to Validity

ERIC Educational Resources Information Center

Hood, S. Brian

2012-01-01

Paul E. Newton argues in favor of a conception of validity, viz, "the consensus definition of validity," according to which the extension of the predicate "is valid" is a subset of "assessment-based decision-making procedure[s], which [are] underwritten by an argument that the assessment procedure can be used to measure the attribute entailed by…

Development and evaluation of the content validity, practicability and feasibility of the Innovative dementia-oriented Assessment system for challenging behaviour in residents with dementia.

PubMed

Halek, Margareta; Holle, Daniela; Bartholomeyczik, Sabine

2017-08-14

One of the most difficult issues for care staff is the manifestation of challenging behaviour among residents with dementia. The first step in managing this type of behaviour is analysing its triggers. A structured assessment instrument can facilitate this process and may improve carers' management of the situation. This paper describes the development of an instrument designed for this purpose and an evaluation of its content validity and its feasibility and practicability in nursing homes. The development process and evaluation of the content validity were based on Lynn's methodology (1998). A literature review (steps 1 + 2) provided the theoretical framework for the instrument and for item formation. Ten experts (step 3) evaluated the first version of the instrument (the Innovative dementia-oriented Assessment (IdA®)) regarding its relevance, clarity, meaningfulness and completeness; content validity indices at the scale-level (S-CVI) and item-level (I-CVI) were calculated. Health care workers (step 4) evaluated the second version in a workshop. Finally, the instrument was introduced to 17 units in 11 nursing homes in a field study (step 5), and 60 care staff members assessed its practicability and feasibility. The IdA® used the need-driven dementia-compromised behaviour (NDB) model as a theoretical framework. The literature review and expert-based panel supported the content validity of the IdA®. At the item level, 77% of the ratings had a CVI greater than or equal to 0.78. The majority of the question-ratings (84%, n = 154) and answer-ratings (69%, n = 122) showed valid results, with none below 0.50. The health care workers confirmed the understandability, completeness and plausibility of the IdA®. Steps 3 and 4 led to further item clarification. The carers in the study considered the instrument helpful for reflecting challenging behaviour and beneficial for the care of residents with dementia. Negative ratings referred to the time required and the lack of effect on residents´ behaviour. There was strong evidence supporting the content validity of the IdA®. Despite the substantial length and time requirement, the instrument was considered helpful for analysing challenging behaviour. Thus, further research on the psychometric qualities, implementation aspects and effectiveness of the IdA® in understanding challenging behaviour is needed.

Validation of the "Security Needs Assessment Profile" for measuring the profiles of security needs of Chinese forensic psychiatric inpatients.

PubMed

Siu, B W M; Au-Yeung, C C Y; Chan, A W L; Chan, L S Y; Yuen, K K; Leung, H W; Yan, C K; Ng, K K; Lai, A C H; Davies, S; Collins, M

Mapping forensic psychiatric services with the security needs of patients is a salient step in service planning, audit and review. A valid and reliable instrument for measuring the security needs of Chinese forensic psychiatric inpatients was not yet available. This study aimed to develop and validate the Chinese version of the Security Needs Assessment Profile for measuring the profiles of security needs of Chinese forensic psychiatric inpatients. The Security Needs Assessment Profile by Davis was translated into Chinese. Its face validity, content validity, construct validity and internal consistency reliability were assessed by measuring the security needs of 98 Chinese forensic psychiatric inpatients. Principal factor analysis for construct validity provided a six-factor security needs model explaining 68.7% of the variance. Based on the Cronbach's alpha coefficient, the internal consistency reliability was rated as acceptable for procedural security (0.73), and fair for both physical security (0.62) and relational security (0.58). A significant sex difference (p=0.002) in total security score was found. The Chinese version of the Security Needs Assessment Profile is a valid and reliable instrument for assessing the security needs of Chinese forensic psychiatric inpatients. Copyright © 2017 Elsevier Ltd. All rights reserved.

The development and validation of the Youth Actuarial Care Needs Assessment Tool for Non-Offenders (Y-ACNAT-NO).

PubMed

Assink, Mark; van der Put, Claudia E; Oort, Frans J; Stams, Geert Jan J M

2015-03-04

In The Netherlands, police officers not only come into contact with juvenile offenders, but also with a large number of juveniles who were involved in a criminal offense, but not in the role of a suspect (i.e., juvenile non-offenders). Until now, no valid and reliable instrument was available that can be used by Dutch police officers for estimating the risk for future care needs of juvenile non-offenders. In the present study, the Youth Actuarial Care Needs Assessment Tool for Non-Offenders (Y-ACNAT-NO) was developed for predicting the risk for future care needs that consisted of (1) a future supervision order as imposed by a juvenile court judge and (2) future worrisome incidents involving child abuse, domestic violence/strife, and/or sexual offensive behavior at the juvenile's living address (i.e., problems in the child-rearing environment). Police records of 3,200 juveniles were retrieved from the Dutch police registration system after which the sample was randomly split in a construction (n = 1,549) and validation sample (n = 1,651). The Y-ACNAT-NO was developed by performing an Exhaustive CHAID analysis using the construction sample. The predictive validity of the instrument was examined in the validation sample by calculating several performance indicators that assess discrimination and calibration. The CHAID output yielded an instrument that consisted of six variables and eleven different risk groups. The risk for future care needs ranged from 0.06 in the lowest risk group to 0.83 in the highest risk group. The AUC value in the validation sample was .764 (95% CI [.743, .784]) and Sander's calibration score indicated an average assessment error of 3.74% in risk estimates per risk category. The Y-ACNAT-NO is the first instrument that can be used by Dutch police officers for estimating the risk for future care needs of juvenile non-offenders. The predictive validity of the Y-ACNAT-NO in terms of discrimination and calibration was sufficient to justify its use as an initial screening instrument when a decision is needed about referring a juvenile for further assessment of care needs.

Assessing learning outcomes in middle-division classical mechanics: The Colorado Classical Mechanics and Math Methods Instrument

NASA Astrophysics Data System (ADS)

Caballero, Marcos D.; Doughty, Leanne; Turnbull, Anna M.; Pepper, Rachel E.; Pollock, Steven J.

2017-06-01

Reliable and validated assessments of introductory physics have been instrumental in driving curricular and pedagogical reforms that lead to improved student learning. As part of an effort to systematically improve our sophomore-level classical mechanics and math methods course (CM 1) at CU Boulder, we have developed a tool to assess student learning of CM 1 concepts in the upper division. The Colorado Classical Mechanics and Math Methods Instrument (CCMI) builds on faculty consensus learning goals and systematic observations of student difficulties. The result is a 9-question open-ended post test that probes student learning in the first half of a two-semester classical mechanics and math methods sequence. In this paper, we describe the design and development of this instrument, its validation, and measurements made in classes at CU Boulder and elsewhere.

Coaching leadership: leaders' and followers' perception assessment questionnaires in nursing

PubMed Central

Cardoso, Maria Lúcia Alves Pereira; Ramos, Laís Helena; D'Innocenzo, Maria

2014-01-01

ABSTRACT Objective: To describe the development, content analysis, and reliability of two questionnaires to assess the perception of nurse leaders, nurse technicians, and licensed practical nurses – coached in the practice of leadership and the relation with the dimensions of the coaching process. Methods: This was a methodological study with a quantitative and qualitative approach, which had the goal of instrumentation in reference to the construction and validation of measuring instruments. The instrument proposition design was based on the literature on leadership, coaching, and assessment of psychometric properties, subjected to content validation as to clarity, relevance, and applicability in order to validate the propositions through the consensus of judges, using the Delphi technique, in 2010. The final version of the questionnaires was administered to 279 nurses and 608 nurse technicians and licensed practical nurses, at two university hospitals and two private hospitals. Results: The Cronbach's alpha value with all items of the self-perception instrument was very high (0.911). The team members' instrument of perception showed that for all determinants and for each dimension of the coaching process, Cronbach's overall alpha value (0.952) was considered quite high, pointing to a very strong consistency of the scale. Confirmatory analysis showed that the models were well adjusted. Conclusion: From the statistical validation we compared the possibility of reusing the questionnaires for other study samples, because there was evidence of reliability and applicability. PMID:24728249

EVA: laparoscopic instrument tracking based on Endoscopic Video Analysis for psychomotor skills assessment.

PubMed

Oropesa, Ignacio; Sánchez-González, Patricia; Chmarra, Magdalena K; Lamata, Pablo; Fernández, Alvaro; Sánchez-Margallo, Juan A; Jansen, Frank Willem; Dankelman, Jenny; Sánchez-Margallo, Francisco M; Gómez, Enrique J

2013-03-01

The EVA (Endoscopic Video Analysis) tracking system is a new system for extracting motions of laparoscopic instruments based on nonobtrusive video tracking. The feasibility of using EVA in laparoscopic settings has been tested in a box trainer setup. EVA makes use of an algorithm that employs information of the laparoscopic instrument's shaft edges in the image, the instrument's insertion point, and the camera's optical center to track the three-dimensional position of the instrument tip. A validation study of EVA comprised a comparison of the measurements achieved with EVA and the TrEndo tracking system. To this end, 42 participants (16 novices, 22 residents, and 4 experts) were asked to perform a peg transfer task in a box trainer. Ten motion-based metrics were used to assess their performance. Construct validation of the EVA has been obtained for seven motion-based metrics. Concurrent validation revealed that there is a strong correlation between the results obtained by EVA and the TrEndo for metrics, such as path length (ρ = 0.97), average speed (ρ = 0.94), or economy of volume (ρ = 0.85), proving the viability of EVA. EVA has been successfully validated in a box trainer setup, showing the potential of endoscopic video analysis to assess laparoscopic psychomotor skills. The results encourage further implementation of video tracking in training setups and image-guided surgery.

The Cognition Battery of the NIH Toolbox for Assessment of Neurological and Behavioral Function: Validation in an Adult Sample

PubMed Central

Weintraub, Sandra; Dikmen, Sureyya S.; Heaton, Robert K.; Tulsky, David S.; Zelazo, Philip David; Slotkin, Jerry; Carlozzi, Noelle E.; Bauer, Patricia J.; Wallner-Allen, Kathleen; Fox, Nathan; Havlik, Richard; Beaumont, Jennifer L.; Mungas, Dan; Manly, Jennifer J.; Moy, Claudia; Conway, Kevin; Edwards, Emmeline; Nowinski, Cindy J.; Gershon, Richard

2014-01-01

This paper introduces a special series on validity studies of the Cognition Battery (CB) from the U.S. National Institutes of Health Toolbox for the Assessment of Neurological and Behavioral Function (NIHTB) (R. C. Gershon et al., 2013) in an adult sample. This first paper in the series describes the sample, each of the seven instruments in the NIHTB-CB briefly, and the general approach to data analysis. Data are provided on test-retest reliability and practice effects, and raw scores (mean, standard deviation, range) are presented for each instrument and the gold standard instruments used to measure construct validity. Accompanying papers provide details on each instrument, including information about instrument development, psychometric properties, age and education effects on performance, and convergent and discriminant construct validity. One paper in the series is devoted to a factor analysis of the NIHTB-CB in adults and another describes the psychometric properties of three composite scores derived from the individual measures representing fluid and crystallized abilities and their combination. The NIHTB-CB is designed to provide a brief, comprehensive, common set of measures to allow comparisons among disparate studies and to improve scientific communication. PMID:24959840

The cognition battery of the NIH toolbox for assessment of neurological and behavioral function: validation in an adult sample.

PubMed

Weintraub, Sandra; Dikmen, Sureyya S; Heaton, Robert K; Tulsky, David S; Zelazo, Philip David; Slotkin, Jerry; Carlozzi, Noelle E; Bauer, Patricia J; Wallner-Allen, Kathleen; Fox, Nathan; Havlik, Richard; Beaumont, Jennifer L; Mungas, Dan; Manly, Jennifer J; Moy, Claudia; Conway, Kevin; Edwards, Emmeline; Nowinski, Cindy J; Gershon, Richard

2014-07-01

This study introduces a special series on validity studies of the Cognition Battery (CB) from the U.S. National Institutes of Health Toolbox for the Assessment of Neurological and Behavioral Function (NIHTB) (Gershon, Wagster et al., 2013) in an adult sample. This first study in the series describes the sample, each of the seven instruments in the NIHTB-CB briefly, and the general approach to data analysis. Data are provided on test-retest reliability and practice effects, and raw scores (mean, standard deviation, range) are presented for each instrument and the gold standard instruments used to measure construct validity. Accompanying papers provide details on each instrument, including information about instrument development, psychometric properties, age and education effects on performance, and convergent and discriminant construct validity. One study in the series is devoted to a factor analysis of the NIHTB-CB in adults and another describes the psychometric properties of three composite scores derived from the individual measures representing fluid and crystallized abilities and their combination. The NIHTB-CB is designed to provide a brief, comprehensive, common set of measures to allow comparisons among disparate studies and to improve scientific communication.

An Assessment Instrument to Measure Geospatial Thinking Expertise

ERIC Educational Resources Information Center

Huynh, Niem Tu; Sharpe, Bob

2013-01-01

Spatial thinking is fundamental to the practice and theory of geography, however there are few valid and reliable assessment methods in geography to measure student performance in spatial thinking. This article presents the development and evaluation of a geospatial thinking assessment instrument to measure participant understanding of spatial…

Relative validation of Block Kids Food Screener for dietary assessment in children and adolescents.

PubMed

Hunsberger, Monica; O'Malley, Jean; Block, Torin; Norris, Jean C

2015-04-01

Food frequency questionnaires (FFQs) are less time consuming and inexpensive instruments for collecting dietary intake when compared with 24-h dietary recalls or double-labelled water; however, the validation of FFQ is important as incorrect information may lead to biased conclusions about associations. Therefore, the relative validity of the Block Kids Food Screener (BKFS) developed for use with children was examined in a convenience sample of 99 youth recruited from the Portland, OR metropolitan area. Three 24-h dietary recalls served as the reference. The relative validity was analysed after natural log transformation of all variables except glycaemic index prior to correlation analysis. Daily cup equivalent totals from the BKFS and 'servings' from 24-h recalls were used to compute average daily intake of fruits, vegetables, potatoes, whole grains, legumes, meat/fish/poultry and dairy. Protein grams (g), total kcalories, glycaemic index (glucose reference), glycaemic load (glucose reference), total saturated fat (g) and added sugar (g) were also calculated by each instrument. The correlation between data obtained from the two instruments was corrected for the within-subject variation in food intake reported by the 24-h recalls using standard nutritional assessment methodology. The de-attenuated correlations in nutritional intake between the two dietary assessment instruments ranged from 0.526 for vegetables, to 0.878 for potatoes. The 24-h recall estimated higher levels of saturated fat and added sugar consumption, higher glycaemic loads and glycaemic indices; the de-attenuatted correlations of these measures ranged from 0.478 to 0.768. Assessment of Bland-Altman plots indicated no systematic difference between the two instruments for vegetable, dairy and meat/fish/poultry fat consumption. BKFS is a useful dietary assessment instrument for the nutrients and food groups it was designed to assess in children age 10-17 years. © 2012 Blackwell Publishing Ltd.

Clinical assessment of the physical activity pattern of chronic fatigue syndrome patients: a validation of three methods.

PubMed

Scheeres, Korine; Knoop, Hans; Meer, van der Jos; Bleijenberg, Gijs

2009-04-01

Effective treatment of chronic fatigue syndrome (CFS) with cognitive behavioural therapy (CBT) relies on a correct classification of so called 'fluctuating active' versus 'passive' patients. For successful treatment with CBT is it especially important to recognise the passive patients and give them a tailored treatment protocol. In the present study it was evaluated whether CFS patient's physical activity pattern can be assessed most accurately with the 'Activity Pattern Interview' (API), the International Physical Activity Questionnaire (IPAQ) or the CFS-Activity Questionnaire (CFS-AQ). The three instruments were validated compared to actometers. Actometers are until now the best and most objective instrument to measure physical activity, but they are too expensive and time consuming for most clinical practice settings. In total 226 CFS patients enrolled for CBT therapy answered the API at intake and filled in the two questionnaires. Directly after intake they wore the actometer for two weeks. Based on receiver operating characteristic (ROC) curves the validity of the three methods were assessed and compared. Both the API and the two questionnaires had an acceptable validity (0.64 to 0.71). None of the three instruments was significantly better than the others. The proportion of false predictions was rather high for all three instrument. The IPAQ had the highest proportion of correct passive predictions (sensitivity 70.1%). The validity of all three instruments appeared to be fair, and all showed rather high proportions of false classifications. Hence in fact none of the tested instruments could really be called satisfactory. Because the IPAQ showed to be the best in correctly predicting 'passive' CFS patients, which is most essentially related to treatment results, it was concluded that the IPAQ is the preferable alternative for an actometer when treating CFS patients in clinical practice.

The Borderline Syndrome Index: a validation study using the personality assessment schedule.

PubMed

Marlowe, M J; O'Neill-Byrne, K; Lowe-Ponsford, F; Watson, J P

1996-01-01

This study examines the validity and screening properties of the Borderline Syndrome Index--BSI (developed in the USA) for categories of the Personality Assessment Schedule--PAS (developed in the UK). Patients were recruited by case control sampling. Chance corrected agreement between instruments and screening properties of the BSI were calculated. The BSI proved a moderately sensitive but non-specific screen. Questionnaire scores were highly correlated with symptom measures. The results do not support the validity of the BSI or its use as a screening instrument. BSI scores may be distorted by current symptoms.

Using Confirmatory Factor Analysis to Validate the Chamberlin Affective Instrument for Mathematical Problem Solving with Academically Advanced Students

ERIC Educational Resources Information Center

Chamberlin, Scott A.; Moore, Alan D.; Parks, Kelly

2017-01-01

Background: Student affect plays a considerable role in mathematical problem solving performance, yet is rarely formally assessed. In this manuscript, an instrument and its properties are discussed to enable educational psychologists the opportunity to assess student affect. Aims: The study was conducted to norm the CAIMPS (instrument) with gifted…

Development and Validation of Mechanical Engineering Trade Skills Assessment Instrument for Sustainable Job Security in Yobe State

ERIC Educational Resources Information Center

Adamu, Gishua Garba; Dawha, Josphine Musa; Kamar, Tiamiyu Salihu

2015-01-01

Mechanical Engineering Trade Skills Assessment Instrument (METSAI) is aimed at determining the extent to which students have acquired practical skills before graduation that will enable them get employment for sustainable job security in Yobe state. The study employed instrumentation research design. The populations of the study were 23 mechanical…

Validity evidence and reliability of a simulated patient feedback instrument.

PubMed

Schlegel, Claudia; Woermann, Ulrich; Rethans, Jan-Joost; van der Vleuten, Cees

2012-01-27

In the training of healthcare professionals, one of the advantages of communication training with simulated patients (SPs) is the SP's ability to provide direct feedback to students after a simulated clinical encounter. The quality of SP feedback must be monitored, especially because it is well known that feedback can have a profound effect on student performance. Due to the current lack of valid and reliable instruments to assess the quality of SP feedback, our study examined the validity and reliability of one potential instrument, the 'modified Quality of Simulated Patient Feedback Form' (mQSF). Content validity of the mQSF was assessed by inviting experts in the area of simulated clinical encounters to rate the importance of the mQSF items. Moreover, generalizability theory was used to examine the reliability of the mQSF. Our data came from videotapes of clinical encounters between six simulated patients and six students and the ensuing feedback from the SPs to the students. Ten faculty members judged the SP feedback according to the items on the mQSF. Three weeks later, this procedure was repeated with the same faculty members and recordings. All but two items of the mQSF received importance ratings of > 2.5 on a four-point rating scale. A generalizability coefficient of 0.77 was established with two judges observing one encounter. The findings for content validity and reliability with two judges suggest that the mQSF is a valid and reliable instrument to assess the quality of feedback provided by simulated patients.

World Health Organization quality of life instrument-brief and Short Form-36 in patients with coronary artery disease: do they measure similar quality of life concepts?

PubMed

Cruz, Luciane Nascimento; Camey, Suzi Alves; Fleck, Marcelo Pio; Polanczyk, Carisi Anne

2009-10-01

This study aimed to assess the validity and reliability of World Health Organization Quality of Life Instrument (WHOQOL)-brief and SF-36 in patients with coronary artery disease (CAD). Considering that depression is known to be associated with bad outcomes in CAD and it is highly associated with poor Quality of Life (QOL), we tested the correlation between WHOQOL and SF-36 and an instrument to screen depressive symptoms. It is a cross-sectional survey conducted in 103 patients with documented CAD. QOL was measured through WHOQOL-brief and SF-36 and depressive symptoms were assessed by the Beck Depression Inventory (BDI). To evaluate convergent validity, the correlations between both QOL assessment instruments were examined; discriminant validity was assessed through BDI and QOL instruments correlations. Coefficient Cronbach's alpha was used to test reliability. Percentages of floor and ceiling effects were higher in SF-36 scores than the WHOQOL-brief ones. Although WHOQOL-brief showed a maximum of 1% of floor effect and 9% of ceiling effect, SF-36 presented 40 and 32%, respectively. Internal consistency reliability ranged from 0.65 to 0.85 for the WHOQOL-brief and 0.57 to 0.89 for the SF-36. Correlations between subscales of WHOQOL-brief and BDI ranged from -0.74 to -0.61 and correlations between subscales of SF-36 and BDI ranged from -0.68 to -0.26. SF-36 and WHOQOL-brief seem to be valid and consistent QOL measures in patients with CAD. Researchers should define the aims of their studies before choosing which instrument to use, because they appear to measure different constructs of QOL.

Development and initial validation of an instrument to assess stressors among South African sports coaches.

PubMed

Kubayi, Alliance; Toriola, Abel; Didymus, Faye

2018-06-01

The aim of this series of studies was to develop and initially validate an instrument to assess stressors among South African sports coaches. In study one, a preliminary pool of 45 items was developed based on existing literature and an expert panel was employed to assess the content validity and applicability of these items. In study two, the 32 items that were retained after study one were analysed using principal component analysis (PCA). The resultant factorial structure comprised four components: environmental stressors, performance stressors, task-related stressors, and athlete stressors. These four components were made up of 26 items and, together, the components and items comprised the provisional Stressors in Sports Coaching Questionnaire (SSCQ). The results show that the SSCQ demonstrates acceptable internal consistency (.73-.89). The findings provide preliminary evidence that SSCQ is a valid tool to assess stressors among South African sports coaches.

Ecological Validity and Clinical Utility of Patient-Reported Outcomes Measurement Information System (PROMIS®) instruments for detecting premenstrual symptoms of depression, anger, and fatigue

PubMed Central

Junghaenel, Doerte U.; Schneider, Stefan; Stone, Arthur A.; Christodoulou, Christopher; Broderick, Joan E.

2014-01-01

Objective This study examined the ecological validity and clinical utility of NIH Patient Reported-Outcomes Measurement Information System (PROMIS®) instruments for anger, depression, and fatigue in women with premenstrual symptoms. Methods One-hundred women completed daily diaries and weekly PROMIS assessments over 4 weeks. Weekly assessments were administered through Computerized Adaptive Testing (CAT). Weekly CATs and corresponding daily scores were compared to evaluate ecological validity. To test clinical utility, we examined if CATs could detect changes in symptom levels, if these changes mirrored those obtained from daily scores, and if CATs could identify clinically meaningful premenstrual symptom change. Results PROMIS CAT scores were higher in the pre-menstrual than the baseline (ps < .0001) and post-menstrual (ps < .0001) weeks. The correlations between CATs and aggregated daily scores ranged from .73 to .88 supporting ecological validity. Mean CAT scores showed systematic changes in accordance with the menstrual cycle and the magnitudes of the changes were similar to those obtained from the daily scores. Finally, Receiver Operating Characteristic (ROC) analyses demonstrated the ability of the CATs to discriminate between women with and without clinically meaningful premenstrual symptom change. Conclusions PROMIS CAT instruments for anger, depression, and fatigue demonstrated validity and utility in premenstrual symptom assessment. The results provide encouraging initial evidence of the utility of PROMIS instruments for the measurement of affective premenstrual symptoms. PMID:24630180

Factors affecting knowledge transfer from continuing professional education to clinical practice: Development and psychometric properties of a new instrument.

PubMed

Vasli, Parvaneh; Dehghan-Nayeri, Nahid; Khosravi, Laleh

2018-01-01

Despite the emphasis placed on the implementation of continuing professional education programs in Iran, researchers or practitioners have not developed an instrument for assessing the factors that affect the knowledge transfer from such programs to clinical practice. The aim of this study was to design and validate such instrument for the Iranian context. The research used a three-stage mix method. In the first stage, in-depth interviews with nurses and content analysis were conducted, after which themes were extracted from the data. In the second stage, the findings of the content analysis and literature review were examined, and preliminary instrument options were developed. In the third stage, qualitative content validity, face validity, content validity ratio, content validity index, and construct validity using exploratory factor analysis was conducted. The reliability of the instrument was measured before and after the determination of construct validity. Primary tool instrument initially comprised 53 items, and its content validity index was 0.86. In the multi-stage factor analysis, eight questions were excluded, thereby reducing 11 factors to five and finally, to four. The final instrument with 43 items consists of the following dimensions: structure and organizational climate, personal characteristics, nature and status of professionals, and nature of educational programs. Managers can use the Iranian instrument to identify factors affecting knowledge transfer of continuing professional education to clinical practice. Copyright © 2017. Published by Elsevier Ltd.

Reliability and Validity of Instruments for Assessing Perinatal Depression in African Settings: Systematic Review and Meta-Analysis

PubMed Central

Tsai, Alexander C.; Scott, Jennifer A.; Hung, Kristin J.; Zhu, Jennifer Q.; Matthews, Lynn T.; Psaros, Christina; Tomlinson, Mark

2013-01-01

Background A major barrier to improving perinatal mental health in Africa is the lack of locally validated tools for identifying probable cases of perinatal depression or for measuring changes in depression symptom severity. We systematically reviewed the evidence on the reliability and validity of instruments to assess perinatal depression in African settings. Methods and Findings Of 1,027 records identified through searching 7 electronic databases, we reviewed 126 full-text reports. We included 25 unique studies, which were disseminated in 26 journal articles and 1 doctoral dissertation. These enrolled 12,544 women living in nine different North and sub-Saharan African countries. Only three studies (12%) used instruments developed specifically for use in a given cultural setting. Most studies provided evidence of criterion-related validity (20 [80%]) or reliability (15 [60%]), while fewer studies provided evidence of construct validity, content validity, or internal structure. The Edinburgh postnatal depression scale (EPDS), assessed in 16 studies (64%), was the most frequently used instrument in our sample. Ten studies estimated the internal consistency of the EPDS (median estimated coefficient alpha, 0.84; interquartile range, 0.71-0.87). For the 14 studies that estimated sensitivity and specificity for the EPDS, we constructed 2 x 2 tables for each cut-off score. Using a bivariate random-effects model, we estimated a pooled sensitivity of 0.94 (95% confidence interval [CI], 0.68-0.99) and a pooled specificity of 0.77 (95% CI, 0.59-0.88) at a cut-off score of ≥9, with higher cut-off scores yielding greater specificity at the cost of lower sensitivity. Conclusions The EPDS can reliably and validly measure perinatal depression symptom severity or screen for probable postnatal depression in African countries, but more validation studies on other instruments are needed. In addition, more qualitative research is needed to adequately characterize local understandings of perinatal depression-like syndromes in different African contexts. PMID:24340036

Development and Application of a Novel Rasch-Based Methodology for Evaluating Multi-Tiered Assessment Instruments: Validation and Utilization of an Undergraduate Diagnostic Test of the Water Cycle

ERIC Educational Resources Information Center

Romine, William L.; Schaffer, Dane L.; Barrow, Lloyd

2015-01-01

We describe the development and validation of a three-tiered diagnostic test of the water cycle (DTWC) and use it to evaluate the impact of prior learning experiences on undergraduates' misconceptions. While most approaches to instrument validation take a positivist perspective using singular criteria such as reliability and fit with a measurement…

The importance of establishing reliability and validity of assessment instruments for mental health problems: An example from Somali children and adolescents living in three refugee camps in Ethiopia

PubMed Central

Hall, Brian J.; Puffer, Eve; Murray, Laura K.; Ismael, Abdulkadir; Bass, Judith K.; Sim, Amanda; Bolton, Paul A.

2014-01-01

Assessing mental health problems cross-culturally for children exposed to war and violence presents a number of unique challenges. One of the most important issues is the lack of validated symptom measures to assess these problems. The present study sought to evaluate the psychometric properties of two measures to assess mental health problems: the Achenbach Youth Self-Report and the Child Posttraumatic Stress Disorder Symptom Scale. We conducted a validity study in three refugee camps in Eastern Ethiopia in the outskirts of Jijiga, the capital of the Somali region. A total of 147 child and caregiver pairs were assessed, and scores obtained were submitted to rigorous psychometric evaluation. Excellent internal consistency reliability was obtained for symptom measures for children and their caregivers. Validation of study instruments based on local case definitions was obtained for the caregivers but not consistently for the children. Sensitivity and specificity of study measures were generally low, indicating that these scales would not perform adequately as screening instruments. Combined test-retest and inter-rater reliability was low for all scales. This study illustrates the need for validation and testing of existing measures cross-culturally. Methodological implications for future cross-cultural research studies in low- and middle-income countries are discussed. PMID:24955147

Validation of the dyspnea index in adolescents with exercise-induced paradoxical vocal fold motion.

PubMed

De Guzman, Vanessa; Ballif, Catherine L; Maurer, Rie; Hartnick, Christopher J; Raol, Nikhila

2014-09-01

Paradoxical vocal fold motion (PVFM) affects almost 1 million adolescents in the United States. However, to date, no disease-specific objective measure exists to assess symptom severity and response to treatment in adolescents with exercise-induced PVFM. To validate the Dyspnea Index (DI) quality-of-life instrument (previously validated for adults with breathing disorders) in children aged 12 to 18 years with exercise-induced PVFM and to determine the minimum significant DI change corresponding to patient-reported or caregiver-reported improvement or worsening of symptoms. A longitudinal study of 56 patients (age range, 12-18 years) diagnosed as having exercise-induced PVFM and their caregivers from February 1, 2013, to September 30, 2013, in an outpatient pediatric otolaryngology office practice. The DI was administered to patients and caregivers, with items modified to reflect the perspective of caregivers. Appropriate DI change was measured to reflect improvement or worsening of symptoms. Test-retest reliability was accomplished by having a subset of patients and caregivers complete the instrument twice within 2 weeks before therapy. Internal consistency was assessed by calculation of Cronbach α. Discriminant validity and convergent validity were determined by comparing DIs with assessment of global change in symptoms. The patient and caregiver mean (SD) DI changes were -12.9 (9.6) and -14.7 (9.3), respectively (P < .001 for both). Reliability was established by test-retest analysis with an intraclass correlation coefficient of 0.8 and by calculation of Cronbach α = 0.80, demonstrating internal consistency. Discriminant validity was determined by assessing for a significant DI change when patients globally perceived that a change existed after treatment (P < .001). A DI change of 8 or higher (P < .001) correlated with patient-reported significant change. Convergent validity was demonstrated by evaluating for significant DI change when no change was reported following treatment (P < .001). The DI appears to be a valid and reliable instrument to assess quality of life in exercise-induced pediatric PVFM. A DI change of 8 or higher seems significant. This instrument can serve as an objective tool to assess change in exercise-induced pediatric PVFM following speech therapy.

Reliability and Validity of Survey Instruments to Measure Work-Related Fatigue in the Emergency Medical Services Setting: A Systematic Review.

PubMed

Patterson, P Daniel; Weaver, Matthew D; Fabio, Anthony; Teasley, Ellen M; Renn, Megan L; Curtis, Brett R; Matthews, Margaret E; Kroemer, Andrew J; Xun, Xiaoshuang; Bizhanova, Zhadyra; Weiss, Patricia M; Sequeira, Denisse J; Coppler, Patrick J; Lang, Eddy S; Higgins, J Stephen

2018-02-15

This study sought to systematically search the literature to identify reliable and valid survey instruments for fatigue measurement in the Emergency Medical Services (EMS) occupational setting. A systematic review study design was used and searched six databases, including one website. The research question guiding the search was developed a priori and registered with the PROSPERO database of systematic reviews: "Are there reliable and valid instruments for measuring fatigue among EMS personnel?" (2016:CRD42016040097). The primary outcome of interest was criterion-related validity. Important outcomes of interest included reliability (e.g., internal consistency), and indicators of sensitivity and specificity. Members of the research team independently screened records from the databases. Full-text articles were evaluated by adapting the Bolster and Rourke system for categorizing findings of systematic reviews, and the rated data abstracted from the body of literature as favorable, unfavorable, mixed/inconclusive, or no impact. The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) methodology was used to evaluate the quality of evidence. The search strategy yielded 1,257 unique records. Thirty-four unique experimental and non-experimental studies were determined relevant following full-text review. Nineteen studies reported on the reliability and/or validity of ten different fatigue survey instruments. Eighteen different studies evaluated the reliability and/or validity of four different sleepiness survey instruments. None of the retained studies reported sensitivity or specificity. Evidence quality was rated as very low across all outcomes. In this systematic review, limited evidence of the reliability and validity of 14 different survey instruments to assess the fatigue and/or sleepiness status of EMS personnel and related shift worker groups was identified.

An Analysis and Validation of Vocational Special Needs Inservice Issues.

ERIC Educational Resources Information Center

Brown, James M.; And Others

This report presents the results of a four-year effort to develop a valid and reliable inservice needs assessment instrument. (The instrument was designed to identify competencies that should be the focus of inservice training for vocational educators who teach or provide services to special needs learners.) Chapter I is an introduction. Chapter…

The Importance of Multi-Group Validity Evidence in Gifted and Talented Identification and Research

ERIC Educational Resources Information Center

Peters, Scott J.

2011-01-01

Practitioners and researchers often review the validity evidence of an instrument before using it for student assessment or in the practice of diagnosing or identifying children with exceptionalities. However, few test manuals present data on instrument measurement equivalence/ invariance or differential item functioning. This information is…

The Reliability and Validity of a Performance Task for Evaluating Science Process Skills.

ERIC Educational Resources Information Center

Adams, Cheryll M.; Callahan, Carolyn M.

1995-01-01

The Diet Cola Test was designed as a process assessment of science aptitude in intermediate grade students. Investigations of the instrument's reliability and validity indicated that data did not support use of the instrument for identifying individual students' aptitude. However, results suggested the test's appropriateness for evaluating…

Development and Validation of the Basketball Offensive Game Performance Instrument

ERIC Educational Resources Information Center

Chen, Weiyun; Hendricks, Kristin; Zhu, Weimo

2013-01-01

The purpose of this study was to design and validate the Basketball Offensive Game Performance Instrument (BOGPI) that assesses an individual player's offensive game performance competency in basketball. Twelve physical education teacher education (PETE) students playing two 10-minute, 3 vs. 3 basketball games were videotaped at end of a…

Convergent and Divergent Validity of the Grammaticality and Utterance Length Instrument

ERIC Educational Resources Information Center

Castilla-Earls, Anny; Fulcher-Rood, Katrina

2018-01-01

Purpose: This feasibility study examines the convergent and divergent validity of the Grammaticality and Utterance Length Instrument (GLi), a tool designed to assess the grammaticality and average utterance length of a child's prerecorded story retell. Method: Three raters used the GLi to rate audio-recorded story retells from 100 English-speaking…

Quality of Work: Validation of a New Instrument in Three Languages

PubMed Central

Steffgen, Georges; Kohl, Diane; Reese, Gerhard; Happ, Christian; Sischka, Philipp

2015-01-01

Introduction and objective: A new instrument to measure quality of work was developed in three languages (German, French and Luxembourgish) and validated in a study of employees working in Luxembourg. Methods and results: A representative sample (n = 1529) was taken and exploratory factor analysis revealed a six-factor solution for the 21-item instrument (satisfaction and respect, mobbing, mental strain at work, cooperation, communication and feedback, and appraisal). Reliability analysis showed satisfying reliability for all six factors and the total questionnaire. In order to examine the construct validity of the new instrument, regression analyses were conducted to test whether the instrument predicted work characteristics’ influence on three components of well-being—burnout, psychological stress and maladaptive coping behaviors. Conclusion: The present validation offers a trilingual inventory for measuring quality of work that may be used, for example, as an assessment tool or for testing the effectiveness of interventions. PMID:26703634

Quality of Work: Validation of a New Instrument in Three Languages.

PubMed

Steffgen, Georges; Kohl, Diane; Reese, Gerhard; Happ, Christian; Sischka, Philipp

2015-11-26

A new instrument to measure quality of work was developed in three languages (German, French and Luxembourgish) and validated in a study of employees working in Luxembourg. A representative sample (n = 1529) was taken and exploratory factor analysis revealed a six-factor solution for the 21-item instrument (satisfaction and respect, mobbing, mental strain at work, cooperation, communication and feedback, and appraisal). Reliability analysis showed satisfying reliability for all six factors and the total questionnaire. In order to examine the construct validity of the new instrument, regression analyses were conducted to test whether the instrument predicted work characteristics’ influence on three components of well-being -- burnout, psychological stress and maladaptive coping behaviors. The present validation offers a trilingual inventory for measuring quality of work that may be used, for example, as an assessment tool or for testing the effectiveness of interventions.

Diagnosing paratonia in the demented elderly: reliability and validity of the Paratonia Assessment Instrument (PAI).

PubMed

Hobbelen, Johannes S M; Koopmans, Raymond T C M; Verhey, Frans R J; Habraken, Kitty M; de Bie, Rob A

2008-08-01

Paratonia is one of the associated movement disorders characteristic of dementia. The aim of this study was to develop an assessment tool (the Paratonia Assessment Instrument, PAI), based on the new consensus definition of paratonia. An additional aim was to investigate the reliability and validity of the PAI. A three-phase cross-sectional survey was conducted. In the first two phases, the PAI was developed and validated. In the third phase, the inter-observer reliability and feasibility of the instrument was tested. The original PAI consisted of five criteria that all needed to be met in order to make the diagnosis. On the basis of a qualitative analysis, one criterion was reformulated and another was removed. Following this, inter-observer reliability between the two assessors resulted in an improvement of Cohen's kappa from 0.532 in the initial phase to 0.677 in the second phase. This improvement was substantiated in the third phase by two independent assessors with Cohen's kappa ranging from 0.625 to 1. The PAI is a reliable and valid assessment tool for diagnosing paratonia in elderly people with dementia that can be applied easily in daily practice.

Pressure ulcers: development and psychometric evaluation of the attitude towards pressure ulcer prevention instrument (APuP).

PubMed

Beeckman, D; Defloor, T; Demarré, L; Van Hecke, A; Vanderwee, K

2010-11-01

Pressure ulcers continue to be a significant problem in hospitals, nursing homes and community care settings. Pressure ulcer incidence is widely accepted as an indicator for the quality of care. Negative attitudes towards pressure ulcer prevention may result in suboptimal preventive care. A reliable and valid instrument to assess attitudes towards pressure ulcer prevention is lacking. Development and psychometric evaluation of the Attitude towards Pressure ulcer Prevention instrument (APuP). Prospective psychometric instrument validation study. A literature review was performed to design the instrument. Content validity was evaluated by nine European pressure ulcer experts and five experts in psychometric instrument validation in a double Delphi procedure. A convenience sample of 258 nurses and 291 nursing students from Belgium and The Netherlands participated in order to evaluate construct validity and stability reliability of the instrument. The data were collected between February and May 2008. A factor analysis indicated the construct of a 13 item instrument in a five factor solution: (1) attitude towards personal competency to prevent pressure ulcers (three items); (2) attitude towards the priority of pressure ulcer prevention (three items); (3) attitude towards the impact of pressure ulcers (three items); (4) attitude towards personal responsibility in pressure ulcer prevention (two items); and (5) attitude towards confidence in the effectiveness of prevention (two items). This five factor solution accounted for 61.4% of the variance in responses related to attitudes towards pressure ulcer prevention. All items demonstrated factor loadings over 0.60. The instrument produced similar results during stability testing [ICC=0.88 (95% CI=0.84-0.91, P<0.001)]. For the total instrument, the internal consistency (Cronbachs alpha) was 0.79. The APuP is a psychometrically sound instrument that can be used to effectively assess attitudes towards pressure ulcer prevention in patient care, education, and research. In further research, the association between attitude, knowledge and clinical performance should be explored. Copyright 2010 Elsevier Ltd. All rights reserved.

Reliability and validity of the German version of the Utrecht Questionnaire for Outcome Assessment in Aesthetic Rhinoplasty (D-OAR).

PubMed

Spiekermann, Christoph; Rudack, Claudia; Stenner, Markus

2017-11-01

The outcome of aesthetic rhinoplasty is determined by the patient's subjective satisfaction with the nasal appearance which is difficult to assess. The Utrecht Questionnaire for Outcome Assessment in Aesthetic Rhinoplasty (OAR) is a brief and reliable instrument to assess the influence of the subjective nasal appearance on quality of life in patients undergoing aesthetic rhinoplasty. Preoperative application of this questionnaire reveals important aspects and possible disturbances of the body image which could be negative predictors concerning the result. On the other hand, it represents an appropriate tool to assess the postoperative outcome. The aim of this study was to determine the validity, reliability and responsiveness of the adapted German version of the OAR (D-OAR). The adaption of the OAR to German language was performed by a forward and backward translation process. Patients undergoing rhinoplasty were asked to complete the D-OAR preoperatively, 1, 3 and 12 months after procedure and healthy volunteers without any nasal complaints served as controls to test validity, reliability and responsiveness. An excellent internal consistency, a good test-retest reliability and good inter-item and item-total correlations demonstrated a good reliability of the D-OAR. The convincing validity of the adapted version was proven by an excellent discriminant and a sufficient content validity. Significant differences between pre- and postoperative D-OAR scores revealed a good responsiveness of the instrument. Hence, with a sufficient validity, reliability and sensitivity to changes, the D-OAR is a short and helpful instrument to assess the subjective perception of the nasal appearance in German patients.

Measurement properties of adult quality-of-life measurement instruments for eczema: a systematic review.

PubMed

Heinl, D; Prinsen, C A C; Deckert, S; Chalmers, J R; Drucker, A M; Ofenloch, R; Humphreys, R; Sach, T; Chamlin, S L; Schmitt, J; Apfelbacher, C

2016-03-01

The Harmonising Outcome Measures for Eczema (HOME) initiative has identified quality of life (QoL) as a core outcome domain to be evaluated in every eczema trial. It is unclear which of the existing QoL instruments is most appropriate for this domain. Thus, the aim of this review was to systematically assess the measurement properties of existing measurement instruments developed and/or validated for the measurement of QoL in adult eczema. We conducted a systematic literature search in PubMed and Embase identifying studies on measurement properties of adult eczema QoL instruments. For all eligible studies, we assessed the adequacy of the measurement properties and the methodological quality with the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. A best evidence synthesis summarizing findings from different studies was the basis to assign four degrees of recommendation (A-D). A total of 15 articles reporting on 17 instruments were included. No instrument fulfilled the criteria for category A. Six instruments were placed in category B, meaning that they have the potential to be recommended depending on the results of further validation studies. Three instruments had poor adequacy in at least one required adequacy criterion and were therefore put in category C. The remaining eight instruments were minimally validated and were thus placed in category D. Currently, no QoL instrument can be recommended for use in adult eczema. The Quality of Life Index for Atopic Dermatitis (QoLIAD) and the Dermatology Life Quality Index (DLQI) are recommended for further validation research. © 2015 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

Validation of the ENVISAT atmospheric chemistry instruments

NASA Astrophysics Data System (ADS)

Snoeij, P.; Koopman, R.; Attema, E.; Zehner, C.; Wursteisen, P.; Dehn, A.; de Laurentius, M.; Frerick, J.; Mantovani, R.; Saavedra de Miguel, L.

Three atmospheric-chemistry sensors form part of the ENVISAT payload that has been placed into orbit in March 2002. This paper presents the ENVISAT mission status and data policy, and reviews the end-to-end performance of the GOMOS, MIPAS and SCIAMACHY observation systems and will discuss the validation aspects of these instruments. In particular, for each instrument, the review addresses mission planning, in-orbit performance, calibration, data processor algorithms and configuration, reprocessing strategy, and product quality control assessment. An important part of the quality assessment is the Geophysical Validation. At the ACVT Validation workshop held in Frascati, Italy, from 3-7 May 2004, scientists and engineers presented analyses of the exhaustive series of tests that have been run on each of ENVISAT atmospheric chemistry sensors since the spacecraft was launched in March 2002. On the basis of workshop results it was decided that most of the data products provided by the ENVISAT atmospheric chemistry instruments are ready for operational delivery. Although the main validation phase for the atmospheric instruments of ENVISAT will be completed soon, ongoing validation products will continue throughout the lifetime of the ENVISAT mission. The long-term validation phase will: Provide assurance of data quality and accuracy for applications such as climate change research Investigate the fully representative range of geophysical conditions Investigate the fully representative range of seasonal cycles Perform long term monitoring for instrumental drifts and other artefacts Validate new products. This paper will also discuss the general status of the validation activities for GOMOS, MIPAS and SCIAMACHY. The main and long-term geophysical validation programme will be presented. The flight and ground-segment planning, configuration and performance characterization will be discussed. The evolution of each of the observation systems has been distinct during the mission history: the GOMOS instrument operation has undergone an important change, and its processing chain is subject of two upgrades. For MIPAS intervention on one of the on-board subsystems has proven necessary, and an important data processing improvement cycle has been completed through reconfiguration of the processing chain. SCIAMACHY operations have required only minor interventions, and the presentation will focus on the processing chain evolution.

Clinical evaluation of a new pressure ulcer risk assessment instrument, the Pressure Ulcer Risk Primary or Secondary Evaluation Tool (PURPOSE T).

PubMed

Coleman, Susanne; Smith, Isabelle L; McGinnis, Elizabeth; Keen, Justin; Muir, Delia; Wilson, Lyn; Stubbs, Nikki; Dealey, Carol; Brown, Sarah; Nelson, E Andrea; Nixon, Jane

2018-02-01

To test the psychometric properties and clinical usability of a new Pressure Ulcer Risk Assessment Instrument including inter-rater and test-retest reliability, convergent validity and data completeness. Methodological and practical limitations associated with traditional Pressure Ulcer Risk Assessment Instruments, prompted a programme to work to develop a new instrument, as part of the National Institute for Health Research funded, Pressure UlceR Programme Of reSEarch (RP-PG-0407-10056). Observational field test. For this clinical evaluation 230 patients were purposefully sampled across four broad levels of pressure ulcer risk with representation from four secondary care and four community NHS Trusts in England. Blinded and simultaneous paired (ward/community nurse and expert nurse) PURPOSE-T assessments were undertaken. Follow-up retest was undertaken by the expert nurse. Field notes of PURPOSE-T use were collected. Data were collected October 2012-January 2013. The clinical evaluation demonstrated "very good" (kappa) inter-rater and test-retest agreement for PURPOSE-T assessment decision overall. The percentage agreement for "problem/no problem" was over 75% for the main risk factors. Convergent validity demonstrated moderate to high associations with other measures of similar constructs. The PURPOSE-T evaluation facilitated the initial validation and clinical usability of the instrument and demonstrated that PURPOSE-T is suitable of use in clinical practice. Further study is needed to evaluate the impact of using the instrument on care processes and outcomes. © 2017 The Authors. Journal of Advanced Nursing Published by John Wiley & Sons Ltd.

Development and initial evaluation of an instrument to assess physiotherapists' clinical reasoning focused on clients' behavior change.

PubMed

Elvén, Maria; Hochwälder, Jacek; Dean, Elizabeth; Söderlund, Anne

2018-05-01

A systematically developed and evaluated instrument is needed to support investigations of physiotherapists' clinical reasoning integrated with the process of clients' behavior change. This study's aim was to develop an instrument to assess physiotherapy students' and physiotherapists' clinical reasoning focused on clients' activity-related behavior and behavior change, and initiate its evaluation, including feasibility and content validity. The study was conducted in three phases: 1) determination of instrument structure and item generation, based on a model, guidelines for assessing clinical reasoning, and existing measures; 2) cognitive interviews with five physiotherapy students to evaluate item understanding and feasibility; and 3) a Delphi process with 18 experts to evaluate content relevance. Phase 1 resulted in an instrument with four domains: Physiotherapist; Input from client; Functional behavioral analysis; and Strategies for behavior change. The instrument consists of case scenarios followed by items in which key features are identified, prioritized, or interpreted. Phase 2 resulted in revisions of problems and approval of feasibility. Phase 3 demonstrated high level of consensus regarding the instrument's content relevance. This feasible and content-validated instrument shows potential for use in investigations of physiotherapy students' and physiotherapists' clinical reasoning, however continued development and testing are needed.

The reliability and validity of the SF-8 with a conflict-affected population in northern Uganda.

PubMed

Roberts, Bayard; Browne, John; Ocaka, Kaducu Felix; Oyok, Thomas; Sondorp, Egbert

2008-12-02

The SF-8 is a health-related quality of life instrument that could provide a useful means of assessing general physical and mental health amongst populations affected by conflict. The purpose of this study was to test the validity and reliability of the SF-8 with a conflict-affected population in northern Uganda. A cross-sectional multi-staged, random cluster survey was conducted with 1206 adults in camps for internally displaced persons in Gulu and Amuru districts of northern Uganda. Data quality was assessed by analysing the number of incomplete responses to SF-8 items. Response distribution was analysed using aggregate endorsement frequency. Test-retest reliability was assessed in a separate smaller survey using the intraclass correlation test. Construct validity was measured using principal component analysis, and the Pearson Correlation test for item-summary score correlation and inter-instrument correlations. Known groups validity was assessed using a two sample t-test to evaluates the ability of the SF-8 to discriminate between groups known to have, and not have, physical and mental health problems. The SF-8 showed excellent data quality. It showed acceptable item response distribution based upon analysis of aggregate endorsement frequencies. Test-retest showed a good intraclass correlation of 0.61 for PCS and 0.68 for MCS. The principal component analysis indicated strong construct validity and concurred with the results of the validity tests by the SF-8 developers. The SF-8 also showed strong construct validity between the 8 items and PCS and MCS summary score, moderate inter-instrument validity, and strong known groups validity. This study provides evidence on the reliability and validity of the SF-8 amongst IDPs in northern Uganda.

The reliability and validity of the SF-8 with a conflict-affected population in northern Uganda

PubMed Central

Roberts, Bayard; Browne, John; Ocaka, Kaducu Felix; Oyok, Thomas; Sondorp, Egbert

2008-01-01

Background The SF-8 is a health-related quality of life instrument that could provide a useful means of assessing general physical and mental health amongst populations affected by conflict. The purpose of this study was to test the validity and reliability of the SF-8 with a conflict-affected population in northern Uganda. Methods A cross-sectional multi-staged, random cluster survey was conducted with 1206 adults in camps for internally displaced persons in Gulu and Amuru districts of northern Uganda. Data quality was assessed by analysing the number of incomplete responses to SF-8 items. Response distribution was analysed using aggregate endorsement frequency. Test-retest reliability was assessed in a separate smaller survey using the intraclass correlation test. Construct validity was measured using principal component analysis, and the Pearson Correlation test for item-summary score correlation and inter-instrument correlations. Known groups validity was assessed using a two sample t-test to evaluates the ability of the SF-8 to discriminate between groups known to have, and not have, physical and mental health problems. Results The SF-8 showed excellent data quality. It showed acceptable item response distribution based upon analysis of aggregate endorsement frequencies. Test-retest showed a good intraclass correlation of 0.61 for PCS and 0.68 for MCS. The principal component analysis indicated strong construct validity and concurred with the results of the validity tests by the SF-8 developers. The SF-8 also showed strong construct validity between the 8 items and PCS and MCS summary score, moderate inter-instrument validity, and strong known groups validity. Conclusion This study provides evidence on the reliability and validity of the SF-8 amongst IDPs in northern Uganda. PMID:19055716

Accumulation of Content Validation Evidence for the Critical Thinking Self-Assessment Scale.

PubMed

Nair, Girija Gopinathan; Hellsten, Laurie-Ann M; Stamler, Lynnette Leeseberg

2017-04-01

Critical thinking skills (CTS) are essential for nurses; assessing students' acquisition of these skills is a mandate of nursing curricula. This study aimed to develop a self-assessment instrument of critical thinking skills (Critical Thinking Self-Assessment Scale [CTSAS]) for students' self-monitoring. An initial pool of 196 items across 6 core cognitive skills and 16 subskills were generated using the American Philosophical Association definition of CTS. Experts' content review of the items and their ratings provided evidence of content relevance using the item-level content validity index (I-CVI) and Aiken's content validity coefficient (VIk). 115 items were retained (range of I-CVI values = .70 to .94 and range of VIk values = .69-.95; significant at p< .05). The CTSAS is the first CTS instrument designed specifically for self-assessment purposes.

Rater reliability and concurrent validity of the Keyboard Personal Computer Style instrument (K-PeCS).

PubMed

Baker, Nancy A; Cook, James R; Redfern, Mark S

2009-01-01

This paper describes the inter-rater and intra-rater reliability, and the concurrent validity of an observational instrument, the Keyboard Personal Computer Style instrument (K-PeCS), which assesses stereotypical postures and movements associated with computer keyboard use. Three trained raters independently rated the video clips of 45 computer keyboard users to ascertain inter-rater reliability, and then re-rated a sub-sample of 15 video clips to ascertain intra-rater reliability. Concurrent validity was assessed by comparing the ratings obtained using the K-PeCS to scores developed from a 3D motion analysis system. The overall K-PeCS had excellent reliability [inter-rater: intra-class correlation coefficients (ICC)=.90; intra-rater: ICC=.92]. Most individual items on the K-PeCS had from good to excellent reliability, although six items fell below ICC=.75. Those K-PeCS items that were assessed for concurrent validity compared favorably to the motion analysis data for all but two items. These results suggest that most items on the K-PeCS can be used to reliably document computer keyboarding style.

Assessing the reliability, predictive and construct validity of historical, clinical and risk management-20 (HCR-20) in Mexican psychiatric inpatients.

PubMed

Sada, Andrea; Robles-García, Rebeca; Martínez-López, Nicolás; Hernández-Ramírez, Rafael; Tovilla-Zarate, Carlos-Alfonso; López-Munguía, Fernando; Suárez-Alvarez, Enrique; Ayala, Xochitl; Fresán, Ana

2016-08-01

Assessing dangerousness to gauge the likelihood of future violent behaviour has become an integral part of clinical mental health practice in forensic and non-forensic psychiatric settings, one of the most effective instruments for this being the Historical, Clinical and Risk Management-20 (HCR-20). To examine the HCR-20 factor structure in Mexican psychiatric inpatients and to obtain its predictive validity and reliability for use in this population. In total, 225 patients diagnosed with psychotic, affective or personality disorders were included. The HCR-20 was applied at hospital admission and violent behaviours were assessed during psychiatric hospitalization using the Overt Aggression Scale (OAS). Construct validity, predictive validity and internal consistency were determined. Violent behaviour remains more severe in patients classified in the high-risk group during hospitalization. Fifteen items displayed adequate communalities in the original designated domains of the HCR-20 and internal consistency of the instruments was high. The HCR-20 is a suitable instrument for predicting violence risk in Mexican psychiatric inpatients.

Measuring primary care organizational capacity for diabetes care coordination: the Diabetes Care Coordination Readiness Assessment.

PubMed

Weeks, Douglas L; Polello, Jennifer M; Hansen, Daniel T; Keeney, Benjamin J; Conrad, Douglas A

2014-01-01

Not all primary care clinics are prepared to implement care coordination services for chronic conditions, such as diabetes. Understanding true capacity to coordinate care is an important first-step toward establishing effective and efficient care coordination. Yet, we could identify no diabetes-specific instruments to systematically assess readiness and/or status of primary care clinics to engage in diabetes care coordination. This report describes the development and initial validation of the Diabetes Care Coordination Readiness Assessment (DCCRA), which is intended to measure primary care clinic readiness to coordinate care for adult patients with diabetes. The instrument was developed through iterative item generation within a framework of five domains of care coordination: Organizational Capacity, Care Coordination, Clinical Management, Quality Improvement, and Technical Infrastructure. Validation data was collected on 39 primary care clinics. Content validity, inter-rater reliability, internal consistency, and construct validity of the 49-item instrument were assessed. Inter-rater agreement indices per item ranged from 0.50 to 1.0. Cronbach's alpha of the entire instrument was 0.964, and for the five domain scales ranged from 0.688 to 0.961. Clinics with existing care coordinators were rated as more ready to support care coordination than clinics without care coordinators for the entire DCCRA and for each domain, supporting construct validity. As providers increasingly attempt to adopt patient-centered approaches, introduction of the DCCRA is timely and appropriate for assisting clinics with identifying gaps in provision of care coordination services. The DCCRA's strengths include promising psychometric properties. A valid measure of diabetes care coordination readiness should be useful in diabetes program evaluation, assistance with quality improvement initiatives, and measurement of patient-centered care in research.

Development of an instrument for the evaluation of advanced life support performance.

PubMed

Peltonen, L-M; Peltonen, V; Salanterä, S; Tommila, M

2017-10-01

Assessing advanced life support (ALS) competence requires validated instruments. Existing instruments include aspects of technical skills (TS), non-technical skills (NTS) or both, but one instrument for detailed assessment that suits all resuscitation situations is lacking. This study aimed to develop an instrument for the evaluation of the overall ALS performance of the whole team. This instrument development study had four phases. First, we reviewed literature and resuscitation guidelines to explore items to include in the instrument. Thereafter, we interviewed resuscitation team professionals (n = 66), using the critical incident technique, to determine possible additional aspects associated with the performance of ALS. Second, we developed an instrument based on the findings. Third, we used an expert panel (n = 20) to assess the validity of the developed instrument. Finally, we revised the instrument based on the experts' comments and tested it with six experts who evaluated 22 video recorded resuscitations. The final version of the developed instrument had 69 items divided into adherence to guidelines (28 items), clinical decision-making (5 items), workload management (12 items), team behaviour (8 items), information management (6 items), patient integrity and consideration of laymen (4 items) and work routines (6 items). The Cronbach's α values were good, and strong correlations between the overall performance and the instrument were observed. The instrument may be useful for detailed assessment of the team's overall performance, but the numerous items make the use demanding. The instrument is still under development, and more research is needed to determine its psychometric properties. © 2017 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd.

Instruments Measuring Integrated Care: A Systematic Review of Measurement Properties.

PubMed

Bautista, Mary Ann C; Nurjono, Milawaty; Lim, Yee Wei; Dessers, Ezra; Vrijhoef, Hubertus Jm

2016-12-01

Policy Points: Investigations on systematic methodologies for measuring integrated care should coincide with the growing interest in this field of research. A systematic review of instruments provides insights into integrated care measurement, including setting the research agenda for validating available instruments and informing the decision to develop new ones. This study is the first systematic review of instruments measuring integrated care with an evidence synthesis of the measurement properties. We found 209 index instruments measuring different constructs related to integrated care; the strength of evidence on the adequacy of the majority of their measurement properties remained largely unassessed. Integrated care is an important strategy for increasing health system performance. Despite its growing significance, detailed evidence on the measurement properties of integrated care instruments remains vague and limited. Our systematic review aims to provide evidence on the state of the art in measuring integrated care. Our comprehensive systematic review framework builds on the Rainbow Model for Integrated Care (RMIC). We searched MEDLINE/PubMed for published articles on the measurement properties of instruments measuring integrated care and identified eligible articles using a standard set of selection criteria. We assessed the methodological quality of every validation study reported using the COSMIN checklist and extracted data on study and instrument characteristics. We also evaluated the measurement properties of each examined instrument per validation study and provided a best evidence synthesis on the adequacy of measurement properties of the index instruments. From the 300 eligible articles, we assessed the methodological quality of 379 validation studies from which we identified 209 index instruments measuring integrated care constructs. The majority of studies reported on instruments measuring constructs related to care integration (33%) and patient-centered care (49%); fewer studies measured care continuity/comprehensive care (15%) and care coordination/case management (3%). We mapped 84% of the measured constructs to the clinical integration domain of the RMIC, with fewer constructs related to the domains of professional (3.7%), organizational (3.4%), and functional (0.5%) integration. Only 8% of the instruments were mapped to a combination of domains; none were mapped exclusively to the system or normative integration domains. The majority of instruments were administered to either patients (60%) or health care providers (20%). Of the measurement properties, responsiveness (4%), measurement error (7%), and criterion (12%) and cross-cultural validity (14%) were less commonly reported. We found <50% of the validation studies to be of good or excellent quality for any of the measurement properties. Only a minority of index instruments showed strong evidence of positive findings for internal consistency (15%), content validity (19%), and structural validity (7%); with moderate evidence of positive findings for internal consistency (14%) and construct validity (14%). Our results suggest that the quality of measurement properties of instruments measuring integrated care is in need of improvement with the less-studied constructs and domains to become part of newly developed instruments. © 2016 Milbank Memorial Fund.

Instruments Measuring Integrated Care: A Systematic Review of Measurement Properties

PubMed Central

BAUTISTA, MARY ANN C.; NURJONO, MILAWATY; DESSERS, EZRA; VRIJHOEF, HUBERTUS JM

2016-01-01

Policy Points: Investigations on systematic methodologies for measuring integrated care should coincide with the growing interest in this field of research.A systematic review of instruments provides insights into integrated care measurement, including setting the research agenda for validating available instruments and informing the decision to develop new ones.This study is the first systematic review of instruments measuring integrated care with an evidence synthesis of the measurement properties.We found 209 index instruments measuring different constructs related to integrated care; the strength of evidence on the adequacy of the majority of their measurement properties remained largely unassessed. Context Integrated care is an important strategy for increasing health system performance. Despite its growing significance, detailed evidence on the measurement properties of integrated care instruments remains vague and limited. Our systematic review aims to provide evidence on the state of the art in measuring integrated care. Methods Our comprehensive systematic review framework builds on the Rainbow Model for Integrated Care (RMIC). We searched MEDLINE/PubMed for published articles on the measurement properties of instruments measuring integrated care and identified eligible articles using a standard set of selection criteria. We assessed the methodological quality of every validation study reported using the COSMIN checklist and extracted data on study and instrument characteristics. We also evaluated the measurement properties of each examined instrument per validation study and provided a best evidence synthesis on the adequacy of measurement properties of the index instruments. Findings From the 300 eligible articles, we assessed the methodological quality of 379 validation studies from which we identified 209 index instruments measuring integrated care constructs. The majority of studies reported on instruments measuring constructs related to care integration (33%) and patient‐centered care (49%); fewer studies measured care continuity/comprehensive care (15%) and care coordination/case management (3%). We mapped 84% of the measured constructs to the clinical integration domain of the RMIC, with fewer constructs related to the domains of professional (3.7%), organizational (3.4%), and functional (0.5%) integration. Only 8% of the instruments were mapped to a combination of domains; none were mapped exclusively to the system or normative integration domains. The majority of instruments were administered to either patients (60%) or health care providers (20%). Of the measurement properties, responsiveness (4%), measurement error (7%), and criterion (12%) and cross‐cultural validity (14%) were less commonly reported. We found <50% of the validation studies to be of good or excellent quality for any of the measurement properties. Only a minority of index instruments showed strong evidence of positive findings for internal consistency (15%), content validity (19%), and structural validity (7%); with moderate evidence of positive findings for internal consistency (14%) and construct validity (14%). Conclusions Our results suggest that the quality of measurement properties of instruments measuring integrated care is in need of improvement with the less‐studied constructs and domains to become part of newly developed instruments. PMID:27995711

The Nurses Self-Concept Instrument (NSCI): assessment of psychometric properties for Australian domestic and international student nurses.

PubMed

Angel, Elizabeth; Craven, Rhonda; Denson, Nida

2012-07-01

Professional self-concept is a critical driver of job satisfaction. In Australia, as international nursing enrolments rise, nursing is increasingly characterised by a professional body of international nurses who may differ from domestic Australian nurses in their nursing self-concept. At present, no psychometrically sound instrument for assessing nursing self-concept for Australian domestic and international nursing students is available. The purpose of this study was to: (1) develop an instrument (the Nurses' Self-Concept Instrument (NSCI)) to measure the professional self-concept of domestic and international nursing students in Australia, and (2) test the psychometric properties of this newly developed instrument. A literature review was conducted to generate the initial dimension and item pools to measure nurses' professional self-concept (NSCI). Two stakeholders examined the content and face validity of dimensions and items. Analysis was performed on data collected from 253 undergraduate nursing students in a large public university in Sydney, Australia, and consisted of domestic (n=218) and international (n=35) nursing students. Internal reliability was assessed using Cronbach's Alpha. Confirmatory factor analysis (CFA) was used to assess the construct validity of the NSCI. The resulting NSCI consisted of 14 items across four self-concept domains: care, leadership, staff relations, and knowledge. The CFA supported the hypothesised factor structure of the self-concept model. All reliabilities were acceptable for both domestic and international students (ranging from r=.78 to .93). The NSCI was shown to be a valid and reliable tool for assessing Australian domestic and international student nurses' professional self-concept. This instrument may also enable those responsible for recruitment of students into nursing courses to assess students' professional self-concept and implement appropriate strategies to foster the growth of lifelong career development. Copyright © 2012 Elsevier Ltd. All rights reserved.

Outcome instruments to assess scoliosis surgery.

PubMed

Bagó, Juan; Climent, Jose Ma; Pérez-Grueso, Francisco J S; Pellisé, Ferran

2013-03-01

To review and summarize the current knowledge regarding the outcome measures used to evaluate scoliosis surgery. Literature review. Outcome instruments should be tested to ensure that they have adequate metric characteristics: content and construct validity, reliability, and responsiveness. In the evaluation of scoliosis, generic instruments to assess health-related quality of life (HRQL) have been used, such as the SF-36 questionnaire and the EuroQol5D instrument. Nonetheless, it is preferable to use disease-specific instruments for this purpose, such as the SRS-22 Patient Questionnaire and the quality of life profile for spinal deformities (QLPSD). More recently, these generic and disease-specific instruments have been complemented with the use of super-specific instruments; i.e., those assessing a single aspect of the condition or specific populations with the condition. The patients' perception of their trunk deformity and body image has received particular attention, and several instruments are available to evaluate these aspects, such as the Walter-Reed Visual Assessment Scale (WRVAS), the Spinal Appearance Questionnaire (SAQ), and the Trunk Appearance Perception Scale (TAPS). The impacts of brace use can also be measured with specific scales, including the Bad Sobernheim Stress Questionnaire (BSSQ) and the Brace Questionnaire (BrQ). The available instruments to evaluate the treatment for non-idiopathic scoliosis have not been sufficiently validated and analyzed. Evaluation of scoliosis treatment should include the patient's perspective, which can be obtained with the use of patient-reported outcome measures.

A Study To Evaluate the Effectiveness of the Montana Functional Vision Assessment for Multihandicapped Persons.

ERIC Educational Resources Information Center

Marks, Ron

The study evaluated the validity, reliability, and effectiveness in providing needed information of the Montana Functional Vision Assessment (MFVA) Instrument by comparing it with the Texas Education Agency Functional Vision Assessment and field testing the instrument in evaluations of nine multihandicapped students (ages 3-11). Evaluation led to…

Measuring the youth bullying experience: a systematic review of the psychometric properties of available instruments.

PubMed

Vessey, Judith; Strout, Tania D; DiFazio, Rachel L; Walker, Allison

2014-12-01

Bullying is a significant problem in schools and measuring this concept remains problematic. The purposes of this study were to (1) identify the published self-report measures developed to assess youth bullying; (2) evaluate their psychometric properties and instrument characteristics; and (3) evaluate the quality of identified psychometric papers evaluating youth bullying measures. A systematic review of the literature was conducted using 4 electronic databases. Data extraction and appraisal of identified instruments were completed using a standardized method and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Thirty-one articles describing 27 self-report instruments were evaluated in our analysis. Quality assessments ranged from 18% to 91%, with 6 papers reaching or exceeding a quality score of 75%. Limited evidence supporting the reliability, validity, and responsiveness of existing youth bullying measures was identified. Evidence supporting the psychometric soundness of the instruments identified was limited. Many measures were in early development and additional evaluation is necessary to validate their psychometric properties. A pool of instruments possesses acceptable initial psychometric dependability for selected assessment purposes. These findings have significant implications for assessing youth bullying and designing and evaluating school-based interventions. © 2014, American School Health Association.

Validation of the content of the prevention protocol for early sepsis caused by Streptococcus agalactiaein newborns

PubMed Central

da Silva, Fabiana Alves; Vidal, Cláudia Fernanda de Lacerda; de Araújo, Ednaldo Cavalcante

2015-01-01

Abstract Objective: to validate the content of the prevention protocol for early sepsis caused by Streptococcus agalactiaein newborns. Method: a transversal, descriptive and methodological study, with a quantitative approach. The sample was composed of 15 judges, 8 obstetricians and 7 pediatricians. The validation occurred through the assessment of the content of the protocol by the judges that received the instrument for data collection - checklist - which contained 7 items that represent the requisites to be met by the protocol. The validation of the content was achieved by applying the Content Validity Index. Result: in the judging process, all the items that represented requirements considered by the protocol obtained concordance within the established level (Content Validity Index > 0.75). Of 7 items, 6 have obtained full concordance (Content Validity Index 1.0) and the feasibility item obtained a Content Validity Index of 0.93. The global assessment of the instruments obtained a Content Validity Index of 0.99. Conclusion: the validation of content that was done was an efficient tool for the adjustment of the protocol, according to the judgment of experienced professionals, which demonstrates the importance of conducting a previous validation of the instruments. It is expected that this study will serve as an incentive for the adoption of universal tracking by other institutions through validated protocols. PMID:26444165

Evaluation of the Performance of Routine Information System Management (PRISM) framework: evidence from Uganda.

PubMed

Hotchkiss, David R; Aqil, Anwer; Lippeveld, Theo; Mukooyo, Edward

2010-07-03

Sound policy, resource allocation and day-to-day management decisions in the health sector require timely information from routine health information systems (RHIS). In most low- and middle-income countries, the RHIS is viewed as being inadequate in providing quality data and continuous information that can be used to help improve health system performance. In addition, there is limited evidence on the effectiveness of RHIS strengthening interventions in improving data quality and use. The purpose of this study is to evaluate the usefulness of the newly developed Performance of Routine Information System Management (PRISM) framework, which consists of a conceptual framework and associated data collection and analysis tools to assess, design, strengthen and evaluate RHIS. The specific objectives of the study are: a) to assess the reliability and validity of the PRISM instruments and b) to assess the validity of the PRISM conceptual framework. Facility- and worker-level data were collected from 110 health care facilities in twelve districts in Uganda in 2004 and 2007 using records reviews, structured interviews and self-administered questionnaires. The analysis procedures include Cronbach's alpha to assess internal consistency of selected instruments, test-retest analysis to assess the reliability and sensitivity of the instruments, and bivariate and multivariate statistical techniques to assess validity of the PRISM instruments and conceptual framework. Cronbach's alpha analysis suggests high reliability (0.7 or greater) for the indices measuring a promotion of a culture of information, RHIS tasks self-efficacy and motivation. The study results also suggest that a promotion of a culture of information influences RHIS tasks self-efficacy, RHIS tasks competence and motivation, and that self-efficacy and the presence of RHIS staff have a direct influence on the use of RHIS information, a key aspect of RHIS performance. The study results provide some empirical support for the reliability and validity of the PRISM instruments and the validity of the PRISM conceptual framework, suggesting that the PRISM approach can be effectively used by RHIS policy makers and practitioners to assess the RHIS and evaluate RHIS strengthening interventions. However, additional studies with larger sample sizes are needed to further investigate the value of the PRISM instruments in exploring the linkages between RHIS data quality and use, and health systems performance.

[Validity of the Spanish version of the Hypersensitive Narcissism Scale (HSNS) in subjects attended at an Addiction Treatment Unit].

PubMed

Ripoll, Carmen; Salazar, José; Bobes, Julio

2010-01-01

Narcissistic personality is an important component of personality disorders which are prevalent in those presenting drug abuse or dependence. Assessment instruments usually consider self-esteem, narcissism and covert narcissism, but although Spanish versions of instruments for self-esteem and narcissism are available, there is no available test for covert narcissism. OBJECTIVE. To test the validity of the Spanish version of the Hypersensitive Narcissism Scale (HSNS) in individuals presenting drug abuse or dependence. In a sample of 79 outpatients, we assessed reliability by means of Cronbach's alpha and the intraclass correlation coefficient (ICC), construct validity through factor analysis, and concurrent validity by means of the correlation between the HSNS and measures of severity, disability, self-esteem, grandiose narcissism and personality disorders. Reliability of the HSNS total scale score was satisfactory (Cronbach's alpha = 0,73, ICC = 0,67), though some items would require further consideration. Factor analysis showed good construct validity with three factors compatible with the theory of covert narcissism. With regard to concurrent validity, covert narcissism (HSNS) correlated positively with open narcissism, severity and disability due to drug use, and negatively with self-esteem. Highest scores on the HSNS corresponded to borderline, narcissistic and passive-aggressive personality disorders. The Spanish version of the HSNS could be a valid instrument for the assessment of covert narcissism in those treated for drug abuse or dependence.

Assessing Clinical Reasoning (ASCLIRE): Instrument Development and Validation

ERIC Educational Resources Information Center

Kunina-Habenicht, Olga; Hautz, Wolf E.; Knigge, Michel; Spies, Claudia; Ahlers, Olaf

2015-01-01

Clinical reasoning is an essential competency in medical education. This study aimed at developing and validating a test to assess diagnostic accuracy, collected information, and diagnostic decision time in clinical reasoning. A norm-referenced computer-based test for the assessment of clinical reasoning (ASCLIRE) was developed, integrating the…

Validation of an instrument to assess evidence-based practice knowledge, attitudes, access, and confidence in the dental environment.

PubMed

Hendricson, William D; Rugh, John D; Hatch, John P; Stark, Debra L; Deahl, Thomas; Wallmann, Elizabeth R

2011-02-01

This article reports the validation of an assessment instrument designed to measure the outcomes of training in evidence-based practice (EBP) in the context of dentistry. Four EBP dimensions are measured by this instrument: 1) understanding of EBP concepts, 2) attitudes about EBP, 3) evidence-accessing methods, and 4) confidence in critical appraisal. The instrument-the Knowledge, Attitudes, Access, and Confidence Evaluation (KACE)-has four scales, with a total of thirty-five items: EBP knowledge (ten items), EBP attitudes (ten), accessing evidence (nine), and confidence (six). Four elements of validity were assessed: consistency of items within the KACE scales (extent to which items within a scale measure the same dimension), discrimination (capacity to detect differences between individuals with different training or experience), responsiveness (capacity to detect the effects of education on trainees), and test-retest reliability. Internal consistency of scales was assessed by analyzing responses of second-year dental students, dental residents, and dental faculty members using Cronbach coefficient alpha, a statistical measure of reliability. Discriminative validity was assessed by comparing KACE scores for the three groups. Responsiveness was assessed by comparing pre- and post-training responses for dental students and residents. To measure test-retest reliability, the full KACE was completed twice by a class of freshman dental students seventeen days apart, and the knowledge scale was completed twice by sixteen faculty members fourteen days apart. Item-to-scale consistency ranged from 0.21 to 0.78 for knowledge, 0.57 to 0.83 for attitude, 0.70 to 0.84 for accessing evidence, and 0.87 to 0.94 for confidence. For discrimination, ANOVA and post hoc testing by the Tukey-Kramer method revealed significant score differences among students, residents, and faculty members consistent with education and experience levels. For responsiveness to training, dental students and residents demonstrated statistically significant changes, in desired directions, from pre- to post-test. For the student test-retest, Pearson correlations for KACE scales were as follows: knowledge 0.66, attitudes 0.66, accessing evidence 0.74, and confidence 0.76. For the knowledge scale test-retest by faculty members, the Pearson correlation was 0.79. The construct validity of the KACE is equivalent to that of instruments that assess similar EBP dimensions in medicine. Item consistency for the knowledge scale was more variable than for other KACE scales, a finding also reported for medically oriented EBP instruments. We conclude that the KACE has good discriminative validity, responsiveness to training effects, and test-retest reliability.

Correlation of PROMIS Physical Function and Pain CAT Instruments With Oswestry Disability Index and Neck Disability Index in Spine Patients.

PubMed

Papuga, Mark O; Mesfin, Addisu; Molinari, Robert; Rubery, Paul T

2016-07-15

A prospective and retrospective cross-sectional cohort analysis. The aim of this study was to show that Patient-Reported Outcomes Measurement Information System (PROMIS) computer adaptive testing (CAT) assessments for physical function and pain interference can be efficiently collected in a standard office visit and to evaluate these scores with scores from previously validated Oswestry Disability Index (ODI) and Neck Disability Index (NDI) providing evidence of convergent validity for use in patients with spine pathology. Spinal surgery outcomes are highly variable, and substantial debate continues regarding the role and value of spine surgery. The routine collection of patient-based outcomes instruments in spine surgery patients may inform this debate. Traditionally, the inefficiency associated with collecting standard validated instruments has been a barrier to routine use in outpatient clinics. We utilized several CAT instruments available through PROMIS and correlated these with the results obtained using "gold standard" legacy outcomes measurement instruments. All measurements were collected at a routine clinical visit. The ODI and the NDI assessments were used as "gold standard" comparisons for patient-reported outcomes. PROMIS CAT instruments required 4.5 ± 1.8 questions and took 35 ± 16 seconds to complete, compared with ODI/NDI requiring 10 questions and taking 188 ± 85 seconds when administered electronically. Linear regression analysis of retrospective scores involving a primary back complaint revealed moderate to strong correlations between ODI and PROMIS physical function with r values ranging from 0.5846 to 0.8907 depending on the specific assessment and patient subsets examined. Routine collection of physical function outcome measures in clinical practice offers the ability to inform and improve patient care. We have shown that several PROMIS CAT instruments can be efficiently administered during routine clinical visits. The moderate to strong correlations found validate the utility of computer adaptive testing when compared with the gold standard "static" legacy assessments. 4.

Development of the Quality of Australian Nursing Documentation in Aged Care (QANDAC) instrument to assess paper-based and electronic resident records.

PubMed

Wang, Ning; Björvell, Catrin; Hailey, David; Yu, Ping

2014-12-01

To develop an Australian nursing documentation in aged care (Quality of Australian Nursing Documentation in Aged Care (QANDAC)) instrument to measure the quality of paper-based and electronic resident records. The instrument was based on the nursing process model and on three attributes of documentation quality identified in a systematic review. The development process involved five phases following approaches to designing criterion-referenced measures. The face and content validities and the inter-rater reliability of the instrument were estimated using a focus group approach and consensus model. The instrument contains 34 questions in three sections: completion of nursing history and assessment, description of care process and meeting the requirements of data entry. Estimates of the validity and inter-rater reliability of the instrument gave satisfactory results. The QANDAC instrument may be a useful audit tool for quality improvement and research in aged care documentation. © 2013 ACOTA.

Use of existing patient-reported outcome (PRO) instruments and their modification: the ISPOR Good Research Practices for Evaluating and Documenting Content Validity for the Use of Existing Instruments and Their Modification PRO Task Force Report.

PubMed

Rothman, Margaret; Burke, Laurie; Erickson, Pennifer; Leidy, Nancy Kline; Patrick, Donald L; Petrie, Charles D

2009-01-01

Patient-reported outcome (PRO) instruments are used to evaluate the effect of medical products on how patients feel or function. This article presents the results of an ISPOR task force convened to address good clinical research practices for the use of existing or modified PRO instruments to support medical product labeling claims. The focus of the article is on content validity, with specific reference to existing or modified PRO instruments, because of the importance of content validity in selecting or modifying an existing PRO instrument and the lack of consensus in the research community regarding best practices for establishing and documenting this measurement property. Topics addressed in the article include: definition and general description of content validity; PRO concept identification as the important first step in establishing content validity; instrument identification and the initial review process; key issues in qualitative methodology; and potential threats to content validity, with three case examples used to illustrate types of threats and how they might be resolved. A table of steps used to identify and evaluate an existing PRO instrument is provided, and figures are used to illustrate the meaning of content validity in relationship to instrument development and evaluation. RESULTS & RECOMMENDATIONS: Four important threats to content validity are identified: unclear conceptual match between the PRO instrument and the intended claim, lack of direct patient input into PRO item content from the target population in which the claim is desired, no evidence that the most relevant and important item content is contained in the instrument, and lack of documentation to support modifications to the PRO instrument. In some cases, careful review of the threats to content validity in a specific application may be reduced through additional well documented qualitative studies that specifically address the issue of concern. Published evidence of the content validity of a PRO instrument for an intended application is often limited. Such evidence is, however, important to evaluating the adequacy of a PRO instrument for the intended application. This article provides an overview of key issues involved in assessing and documenting content validity as it relates to using existing instruments in the drug approval process.

A systematic review of psychometric testing of instruments that measure intention to work with older people.

PubMed

Che, Chong Chin; Hairi, Noran Naqiah; Chong, Mei Chan

2017-09-01

To review systematically the psychometric properties of instruments used to measure intention to work with older people. Nursing students are part of the future healthcare workforce; thus, being aware of their intention to work with older people would give valuable insights to nursing education and practice. Despite a plethora of research on measuring intention to work with older people, a valid and reliable instrument has not been identified. A systematic literature review of evidence and psychometric properties. Eight database searches were conducted between 2006 - 2016. English articles were selected based on inclusion and exclusion criteria. The COSMIN checklist was used to assess instruments reporting a psychometric evaluation of validity and reliability. Of 41 studies identified for full text review, 36 met the inclusion criteria. Seven different types of instruments were identified for psychometric evaluation. Measures of reliability were reported in eight papers and validity in five papers. Evidence for each measurement property was limited, with each instrument demonstrating a lack of information on measurement properties. Based on the COSMIN checklist, the overall quality of the psychometric properties was rated as poor to good. No single instrument was found to be optimal for use. Studies of high methodological quality are needed to properly assess the measurement properties of the instruments that are currently available. Until such studies are available, we recommend using existing instruments with caution. © 2017 John Wiley & Sons Ltd.

A Diagnostic Assessment for Introductory Molecular and Cell Biology

PubMed Central

Wood, William B.; Martin, Jennifer M.; Guild, Nancy A.; Vicens, Quentin; Knight, Jennifer K.

2010-01-01

We have developed and validated a tool for assessing understanding of a selection of fundamental concepts and basic knowledge in undergraduate introductory molecular and cell biology, focusing on areas in which students often have misconceptions. This multiple-choice Introductory Molecular and Cell Biology Assessment (IMCA) instrument is designed for use as a pre- and posttest to measure student learning gains. To develop the assessment, we first worked with faculty to create a set of learning goals that targeted important concepts in the field and seemed likely to be emphasized by most instructors teaching these subjects. We interviewed students using open-ended questions to identify commonly held misconceptions, formulated multiple-choice questions that included these ideas as distracters, and reinterviewed students to establish validity of the instrument. The assessment was then evaluated by 25 biology experts and modified based on their suggestions. The complete revised assessment was administered to more than 1300 students at three institutions. Analysis of statistical parameters including item difficulty, item discrimination, and reliability provides evidence that the IMCA is a valid and reliable instrument with several potential uses in gauging student learning of key concepts in molecular and cell biology. PMID:21123692

Exploring the validity and reliability of a questionnaire for evaluating veterinary clinical teachers' supervisory skills during clinical rotations.

PubMed

Boerboom, T B B; Dolmans, D H J M; Jaarsma, A D C; Muijtjens, A M M; Van Beukelen, P; Scherpbier, A J J A

2011-01-01

Feedback to aid teachers in improving their teaching requires validated evaluation instruments. When implementing an evaluation instrument in a different context, it is important to collect validity evidence from multiple sources. We examined the validity and reliability of the Maastricht Clinical Teaching Questionnaire (MCTQ) as an instrument to evaluate individual clinical teachers during short clinical rotations in veterinary education. We examined four sources of validity evidence: (1) Content was examined based on theory of effective learning. (2) Response process was explored in a pilot study. (3) Internal structure was assessed by confirmatory factor analysis using 1086 student evaluations and reliability was examined utilizing generalizability analysis. (4) Relations with other relevant variables were examined by comparing factor scores with other outcomes. Content validity was supported by theory underlying the cognitive apprenticeship model on which the instrument is based. The pilot study resulted in an additional question about supervision time. A five-factor model showed a good fit with the data. Acceptable reliability was achievable with 10-12 questionnaires per teacher. Correlations between the factors and overall teacher judgement were strong. The MCTQ appears to be a valid and reliable instrument to evaluate clinical teachers' performance during short rotations.

Assessment of capabilities in persons with advanced stage of dementia: Validation of The Montessori Assessment System (MAS).

PubMed

Erkes, Jérôme; Camp, Cameron J; Raffard, Stéphane; Gély-Nargeot And, Marie-Christine; Bayard, Sophie

2017-01-01

This study evaluated the validity and reliability of the Montessori Assessment System. The Montessori Assessment System assesses preserved abilities in persons with moderate to severe dementia. In this respect, this instrument provides crucial information for the development of effective person-centered care plans. A total of 196 persons with a diagnosis of dementia in the moderate to severe stages of dementia were recruited in 10 long-term care facilities in France. All participants completed the Montessori Assessment System, the Clinical Dementia Rating Scale and/or the Mini Mental State Examination and the Severe Impairment Battery-short form. The internal consistency and temporal stability of the Montessori Assessment System were high. Additionally, good construct and divergent validity were demonstrated. Factor analysis showed a one-factor structure. The Montessori Assessment System demonstrated satisfactory psychometric properties while being a useful instrument to assess capabilities in persons with advanced stages of dementia and hence to develop person-centered plans of care.

Antecedents of Mathematics Self-Efficacy Beliefs for Middle and High School Students: An Instrument Validation Study

ERIC Educational Resources Information Center

Freed, Mark R.

2013-01-01

There were three primary purposes for this study. First, I investigated the psychometric properties of the Sources of Middle School Mathematics Self-Efficacy (SMMSE) scale (Usher, 2007; Usher & Pajares, 2009) with high school students. Validation for expanded use of the SMMSE scale was achieved by assessing the instrument's psychometric…

Development and Validation of Nature of Science Instrument for Elementary School Students

ERIC Educational Resources Information Center

Hacieminoglu, Esme; Yilmaz-Tüzün, Özgül; Ertepinar, Hamide

2014-01-01

The purposes of this study were to develop and validate an instrument for assessing elementary students' nature of science (NOS) views and to explain the elementary school students' NOS views, in terms of varying grade levels and gender. The sample included 782 students enrolled in sixth, seventh, and eighth grades. Exploratory factor analysis…

Pooled results from five validation studies of dietary self-report instruments using recovery biomarkers for potassium and sodium intake

USDA-ARS?s Scientific Manuscript database

We have pooled data from five large validation studies of dietary self-report instruments that used recovery biomarkers as referents to assess food frequency questionnaires (FFQs) and 24-hour recalls. We reported on total potassium and sodium intakes, their densities, and their ratio. Results were...

Development and Validation of an Instrument for Early Assessment of Management Potential in a Mid-Size Chemical Company

ERIC Educational Resources Information Center

Zehner, Robert L.; Holton, Elwood F., III

2004-01-01

This study reports on development and concurrent validation of a competency instrument to identify potential leaders in a mid-size chemical company. Four competencies were identified: courageous problem solving, perceived energy, networking, and perceived motivation. Four different comparison groups were examined in logistic regression analyses.…

Development and Validation of an Instrument for Assessing Attitudes of High School Students about Recycling

ERIC Educational Resources Information Center

Ugulu, Ilker

2015-01-01

Recycling and its applications are growing significantly due to the great potential for solving a range of environmental problems in society. Nevertheless, there are currently very few instruments that can provide valid and reliable data on students' attitudes toward recycling. In this regard, this article focuses on the development and validation…

Student Interest in Technology and Science (SITS) Survey: Development, Validation, and Use of a New Instrument

ERIC Educational Resources Information Center

Romine, William; Sadler, Troy D.; Presley, Morgan; Klosterman, Michelle L.

2014-01-01

This study presents the systematic development, validation, and use of a new instrument for measuring student interest in science and technology. The Student Interest in Technology and Science (SITS) survey is composed of 5 sub-sections assessing the following dimensions: interest in learning science, using technology to learn science, science…

Development and validation of a primary sclerosing cholangitis-specific patient-reported outcomes instrument: The PSC PRO.

PubMed

Younossi, Zobair M; Afendy, Arian; Stepanova, Maria; Racila, Andrei; Nader, Fatema; Gomel, Rachel; Safer, Ricky; Lenderking, William R; Skalicky, Anne; Kleinman, Leah; Myers, Robert P; Subramanian, G Mani; McHutchison, John G; Levy, Cynthia; Bowlus, Christopher L; Kowdley, Kris; Muir, Andrew J

2017-11-20

Primary sclerosing cholangitis (PSC) is a chronic liver disease associated with inflammation and biliary fibrosis that leads to cholangitis, cirrhosis, and impaired quality of life. Our objective was to develop and validate a PSC-specific patient-reported outcome (PRO) instrument. We developed a 42-item PSC PRO instrument that contains two modules (Symptoms and Impact of Symptoms) and conducted an external validation. Reliability and validity were evaluated using clinical data and a battery of other validated instruments. Test-retest reliability was assessed in a subgroup of patients who repeated the PSC PRO after the first administration. One hundred two PSC subjects (44 ± 13 years; 32% male, 74% employed, 39% with cirrhosis, 14% with a history of decompensated cirrhosis, 38% history of depression, and 68% with inflammatory bowel disease [IBD]) completed PSC PRO and other PRO instruments (Short Form 36 V2 [SF-36], Chronic Liver Disease Questionnaire [CLDQ], Primary Biliary Cholangitis - 40 [PBC-40], and five dimensions [5-D Itch]). PSC PRO demonstrated excellent internal consistency (Cronbach alphas, 0.84-0.94) and discriminant validity (41 of 42 items had the highest correlations with their own domains). There were good correlations between PSC PRO domains and relevant domains of SF-36, CLDQ, and PBC-40 (R = 0.69-0.90; all P < 0.0001), but lower (R = 0.31-0.60; P < 0.001) with 5-D Itch. Construct validity showed that PSC PRO can differentiate patients according to the presence and severity of cirrhosis and history of depression (P < 0.05), but not by IBD (P > 0.05). Test-retest reliability was assessed in 53 subjects who repeated PSC PRO within a median (interquartile range) of 37 (27-47) days. There was excellent reliability for most domains with intraclass correlations (0.71-0.88; all P < 0.001). PSC PRO is a self-administered disease-specific instrument developed according to U.S. Food and Drug Administration guidelines. This preliminary validation study suggests good psychometric properties. Further validation of the instrument in a larger and more diverse sample of PSC patients is needed. (Hepatology 2017). © 2017 by the American Association for the Study of Liver Diseases.

Preliminary development of the adolescent students' basic psychological needs at school scale.

PubMed

Tian, Lili; Han, Mengmeng; Huebner, E Scott

2014-04-01

The aim of the present study was to develop and provide evidence for the validity of a new measure of adolescent students' psychological need satisfaction at school, using a sample of Chinese students. We conducted four studies with four independent samples (total n = 1872). The first study aimed to develop items for the new instrument and to ascertain its factorial structure using exploratory factor analysis procedures. The second study aimed to examine the instrument's factorial structure using confirmatory factor analysis procedures as well as to assess its internal consistency reliability, convergent and divergent validity. The third study aimed to assess its measurement invariance across gender and age. The fourth study aimed to test its test-retest reliability over time and predictive validity. These preliminary results showed that the new instrument has promising psychometric properties. The potential contributions of the new instrument for future research and educational practices were discussed. Copyright © 2014 The Foundation for Professionals in Services for Adolescents. Published by Elsevier Ltd. All rights reserved.

Validity and reliability of the Behavior Problems Inventory, the Aberrant Behavior Checklist, and the Repetitive Behavior Scale-Revised among infants and toddlers at risk for intellectual or developmental disabilities: a multi-method assessment approach.

PubMed

Rojahn, Johannes; Schroeder, Stephen R; Mayo-Ortega, Liliana; Oyama-Ganiko, Rosao; LeBlanc, Judith; Marquis, Janet; Berke, Elizabeth

2013-05-01

Reliable and valid assessment of aberrant behaviors is essential in empirically verifying prevention and intervention for individuals with intellectual or developmental disabilities (IDD). Few instruments exist which assess behavior problems in infants. The current longitudinal study examined the performance of three behavior-rating scales for individuals with IDD that have been proven psychometrically sound in older populations: the Aberrant Behavior Checklist (ABC), the Behavior Problems Inventory (BPI-01), and the Repetitive Behavior Scale - Revised (RBS-R). Data were analyzed for 180 between six and 36 months old children at risk for IDD. Internal consistency (Cronbach's α) across the subscales of the three instruments was variable. Test-retest reliability of the three BPI-01 subscales ranged from .68 to .77 for frequency ratings and from .65 to .80 for severity ratings (intraclass correlation coefficients). Using a multitrait-multimethod matrix approach high levels of convergent and discriminant validity across the three instruments was found. As anticipated, there was considerable overlap in the information produced by the three instruments; however, each behavior-rating instrument also contributed unique information. Our findings support using all three scales in conjunction if possible. Copyright © 2013 Elsevier Ltd. All rights reserved.

Development and preliminary validation of a leadership competency instrument for existing and emerging allied health professional leaders.

PubMed

Ang, Hui-Gek; Koh, Jeremy Meng-Yeow; Lee, Jeffrey; Pua, Yong-Hao

2016-02-19

No instruments, to our knowledge, exist to assess leadership competency in existing and emerging allied health professional (AHP) leaders. This paper describes the development and preliminary exploration of the psychometric properties of a leadership competency instrument for existing and emerging AHP leaders and examines (i) its factor structure, (ii) its convergent validity with the Leadership Practices Inventory (LPI), and (iii) its discriminative validity in AHPs with different grades. During development, we included 25 items in the AHEAD (Aspiring leaders in Healthcare-Empowering individuals, Achieving excellence, Developing talents) instrument. A cross-sectional study was then conducted in 106 high-potential AHPs from Singapore General Hospital (34 men and 72 women) of different professional grades (49 principal-grade AHPs, 41 senior-grade AHPs, and 16 junior-grade AHPs) who completed both AHEAD and LPI instruments. Exploratory factor analysis was used to test the theoretical structure of AHEAD. Spearman correlation analysis was performed to evaluate the convergent validity of AHEAD with LPI. Using proportional odds regression models, we evaluated the association of grades of AHPs with AHEAD and LPI. To assess discriminative validity, the c-statistics - a measure of discrimination - were derived from these ordinal models. As theorized, factor analysis suggested a two-factor solution, where "skills" and "values" formed separate factors. Internal consistency of AHEAD was excellent (α-values > 0.88). Total and component AHEAD and LPI scores correlated moderately (Spearman ρ-values, 0.37 to 0.58). The c-index for discriminating between AHP grades was higher for AHEAD than for the LPI (0.76 vs. 0.65). The factorial structure of AHEAD was generally supported in our study. AHEAD showed convergent validity with the LPI and outperformed the LPI in terms of discriminative validity. These results provide initial evidence for the use of AHEAD to assess leadership competency in AHPs.

Measurement Properties of Questionnaires Measuring Continuity of Care: A Systematic Review

PubMed Central

Uijen, Annemarie A.; Heinst, Claire W.; Schellevis, Francois G.; van den Bosch, Wil J.H.M.; van de Laar, Floris A.; Terwee, Caroline B.; Schers, Henk J.

2012-01-01

Background Continuity of care is widely acknowledged as a core value in family medicine. In this systematic review, we aimed to identify the instruments measuring continuity of care and to assess the quality of their measurement properties. Methods We did a systematic review using the PubMed, Embase and PsycINFO databases, with an extensive search strategy including ‘continuity of care’, ‘coordination of care’, ‘integration of care’, ‘patient centered care’, ‘case management’ and its linguistic variations. We searched from 1995 to October 2011 and included articles describing the development and/or evaluation of the measurement properties of instruments measuring one or more dimensions of continuity of care (1) care from the same provider who knows and follows the patient (personal continuity), (2) communication and cooperation between care providers in one care setting (team continuity), and (3) communication and cooperation between care providers in different care settings (cross-boundary continuity). We assessed the methodological quality of the measurement properties of each instrument using the COSMIN checklist. Results We included 24 articles describing the development and/or evaluation of 21 instruments. Ten instruments measured all three dimensions of continuity of care. Instruments were developed for different groups of patients or providers. For most instruments, three or four of the six measurement properties were assessed (mostly internal consistency, content validity, structural validity and construct validity). Six instruments scored positive on the quality of at least three of six measurement properties. Conclusions Most included instruments have problems with either the number or quality of its assessed measurement properties or the ability to measure all three dimensions of continuity of care. Based on the results of this review, we recommend the use of one of the four most promising instruments, depending on the target population Diabetes Continuity of Care Questionnaire, Alberta Continuity of Services Scale-Mental Health, Heart Continuity of Care Questionnaire, and Nijmegen Continuity Questionnaire. PMID:22860100

Quality of prenatal care questionnaire: instrument development and testing.

PubMed

Heaman, Maureen I; Sword, Wendy A; Akhtar-Danesh, Noori; Bradford, Amanda; Tough, Suzanne; Janssen, Patricia A; Young, David C; Kingston, Dawn A; Hutton, Eileen K; Helewa, Michael E

2014-06-03

Utilization indices exist to measure quantity of prenatal care, but currently there is no published instrument to assess quality of prenatal care. The purpose of this study was to develop and test a new instrument, the Quality of Prenatal Care Questionnaire (QPCQ). Data for this instrument development study were collected in five Canadian cities. Items for the QPCQ were generated through interviews with 40 pregnant women and 40 health care providers and a review of prenatal care guidelines, followed by assessment of content validity and rating of importance of items. The preliminary 100-item QPCQ was administered to 422 postpartum women to conduct item reduction using exploratory factor analysis. The final 46-item version of the QPCQ was then administered to another 422 postpartum women to establish its construct validity, and internal consistency and test-retest reliability. Exploratory factor analysis reduced the QPCQ to 46 items, factored into 6 subscales, which subsequently were validated by confirmatory factor analysis. Construct validity was also demonstrated using a hypothesis testing approach; there was a significant positive association between women's ratings of the quality of prenatal care and their satisfaction with care (r = 0.81). Convergent validity was demonstrated by a significant positive correlation (r = 0.63) between the "Support and Respect" subscale of the QPCQ and the "Respectfulness/Emotional Support" subscale of the Prenatal Interpersonal Processes of Care instrument. The overall QPCQ had acceptable internal consistency reliability (Cronbach's alpha = 0.96), as did each of the subscales. The test-retest reliability result (Intra-class correlation coefficient = 0.88) indicated stability of the instrument on repeat administration approximately one week later. Temporal stability testing confirmed that women's ratings of their quality of prenatal care did not change as a result of giving birth or between the early postpartum period and 4 to 6 weeks postpartum. The QPCQ is a valid and reliable instrument that will be useful in future research as an outcome measure to compare quality of care across geographic regions, populations, and service delivery models, and to assess the relationship between quality of care and maternal and infant health outcomes.

Validity evidence and reliability of a simulated patient feedback instrument

PubMed Central

2012-01-01

Background In the training of healthcare professionals, one of the advantages of communication training with simulated patients (SPs) is the SP's ability to provide direct feedback to students after a simulated clinical encounter. The quality of SP feedback must be monitored, especially because it is well known that feedback can have a profound effect on student performance. Due to the current lack of valid and reliable instruments to assess the quality of SP feedback, our study examined the validity and reliability of one potential instrument, the 'modified Quality of Simulated Patient Feedback Form' (mQSF). Methods Content validity of the mQSF was assessed by inviting experts in the area of simulated clinical encounters to rate the importance of the mQSF items. Moreover, generalizability theory was used to examine the reliability of the mQSF. Our data came from videotapes of clinical encounters between six simulated patients and six students and the ensuing feedback from the SPs to the students. Ten faculty members judged the SP feedback according to the items on the mQSF. Three weeks later, this procedure was repeated with the same faculty members and recordings. Results All but two items of the mQSF received importance ratings of > 2.5 on a four-point rating scale. A generalizability coefficient of 0.77 was established with two judges observing one encounter. Conclusions The findings for content validity and reliability with two judges suggest that the mQSF is a valid and reliable instrument to assess the quality of feedback provided by simulated patients. PMID:22284898

Development and validation of a questionnaire assessing the perceived control in health care among older adults with care needs in the Netherlands.

PubMed

Claassens, L; Terwee, C B; Deeg, D J H; Broese van Groenou, M I; Widdershoven, G A M; Huisman, M

2016-04-01

In response to the increased emphasis placed on older people's self-reliance in many welfare societies, we aimed to develop and validate a measurement instrument, assessing perceived control in health care among older adults with care needs. The target group consists of older people who live (semi-)independently and use professional health care, with or without informal care. Phase I (development) of the study consisted of the construction of the instrument based on the input from a variety of stakeholders. Phase II (validation) entailed a quantitative study in a sample of 247 respondents selected from the Longitudinal Aging Study Amsterdam, to assess the instrument's construct validity (structural validity and hypotheses testing) and reliability (internal consistency). The questionnaire consists of 29 items, related to organizing professional care, communication with care professionals, health management in the home situation, planning (more) complex care in the future, and perceived support from the social network. Based on a factor analysis, we identified three subscales: (I.) 'perceived personal control in health care'; (II.) 'anticipated personal control regarding future health care'; and (III.) 'perceived support from the social network,' with internal consistencies varying from Cronbach's α = .71 to .90. Factor I was associated with mastery, self-efficacy, self-esteem (r = .31-.35) and factor III with social loneliness (r = -.42). Factor II correlated less strongly with mastery, self-efficacy, and self-esteem (r < .30). Our questionnaire revealed sufficient construct validity and internal consistency. The instrument provides a basis for further quantitative research regarding control, especially in relation to health care-related outcomes.

[Some critical remarks on standardised assessment instruments in nursing].

PubMed

Bartholomeyczik, Sabine

2007-08-01

The use of standardised instruments in nursing has rapidly grown and can be seen as a symptom of the necessary comprehensive nursing diagnostics. However, these instruments comprise the risk of misuse, if they are not critically evaluated. Published statements about tests of reliability and validity of an instrument are insufficient. First, the critical evaluation has to ask for the instrument's theoretical and content base: Is the instrument relevant for nursing, suitable for practice and leading to nursing actions? Two examples of well known instruments and different kinds of their utilization in nursing are discussed. Next, the instruments have to be questioned as "bodies with numbers". Studies on reliability and validity have to be as carefully evaluated as other empirical research. The sample, the suitability of agreement indicators (interraterreliability), kind and reason of tests have to be questioned. The same has to be done with tests of validity which comprise an even greater challenge. Methodological studies about these questions are missing; guidelines for test user qualifications need to be developed.

Concurrent validity and reliability of wireless instrumented insoles measuring postural balance and temporal gait parameters.

PubMed

Oerbekke, Michiel S; Stukstette, Mirelle J; Schütte, Kurt; de Bie, Rob A; Pisters, Martijn F; Vanwanseele, Benedicte

2017-01-01

The OpenGo seems promising to take gait analysis out of laboratory settings due to its capability of long-term measurements and mobility. However, the OpenGo's concurrent validity and reliability need to be assessed to determine if the instrument is suitable for validation in patient samples. Twenty healthy volunteers participated. Center of pressure data were collected under eyes open and closed conditions with participants performing unilateral stance trials on the gold standard (AMTI OR6-7 force plate) while wearing the OpenGo. Temporal gait data (stance time, gait cycle time, and cadence) were collected at a self-selected comfortable walking speed with participants performing test-retest trials on an instrumented treadmill while wearing the OpenGo. Validity was assessed using Bland-Altman plots. Reliability was assessed with Intraclass Correlation Coefficient (2,1) and smallest detectable changes were calculated. Negative means of differences were found in all measured parameters, illustrating lower scores for the OpenGo on average. The OpenGo showed negative upper limits of agreement in center of pressure parameters on the mediolateral axis. Temporal reliability ICCs ranged from 0.90-0.93. Smallest detectable changes for both stance times were 0.04 (left) and 0.05 (right) seconds, for gait cycle time 0.08s, and for cadence 4.5 steps per minute. The OpenGo is valid and reliable for the measurement of temporal gait parameters during walking. Measurements of center of pressure parameters during unilateral stance are not considered valid. The OpenGo seems a promising instrument for clinically screening and monitoring temporal gait parameters in patients, however validation in patient populations is needed. Copyright © 2016 Elsevier B.V. All rights reserved.

Measuring quality of life in cleft lip and palate patients: currently available patient-reported outcomes measures.

PubMed

Eckstein, Donna A; Wu, Rebecca L; Akinbiyi, Takintope; Silver, Lester; Taub, Peter J

2011-11-01

Patient-reported outcomes in cleft lip and palate treatment are critical for patient care. Traditional surgical outcomes focused on objective measures, such as photographs, anatomic measurements, morbidity, and mortality. Although these remain important, they leave many questions unanswered. Surveys that include aesthetics, speech, functionality, self-image, and quality of life provide more thorough outcomes assessment. It is vital that reliable, valid, and comprehensive questionnaires are available to craniofacial surgeons. The authors performed a literature review to identify questionnaires validated in cleft lip and palate patients. Qualifying instruments were assessed for adherence to guidelines for development and validation by the scientific advisory committee and for content. The authors identified 44 measures used in cleft lip and palate studies. After 15 ad hoc questionnaires, eight generic instruments, 11 psychiatric instruments, and one non-English language questionnaire were excluded, nine measures remained. Of these, four were never validated in the cleft population. Analysis revealed one craniofacial-specific measure (Youth Quality of Life-Facial Differences), two voice-related measures (Patient Voice-Related Quality of Life and Cleft Audit Protocol for Speech-Augmented), and two oral health-related measures (Child Oral Health Impact Profile and Child Oral Health Quality of Life). The Youth Quality of Life-Facial Differences, Child Oral Health Impact Profile, and Child Oral Health Quality of Life questionnaires were sufficiently validated. None was created specifically for clefts, resulting in content limitations. There is a lack of comprehensive, valid, and reliable questionnaires for cleft lip and palate surgery. For thorough assessment of satisfaction, further research to develop and validate cleft lip and palate surgery-specific instruments is needed.

Validation study of the Leeds Dyspepsia Questionnaire in a multi-ethnic Asian population.

PubMed

Mahadeva, Sanjiv; Chan, Wah-Kheong; Mohazmi, Mohammed; Sujarita, Ramanujam; Goh, Khean-Lee

2011-11-01

Outcome measures for clinical trials in dyspepsia require an assessment of symptom response. There is a lack of validated instruments assessing dyspepsia symptoms in the Asian region. We aimed to translate and validate the Leeds Dyspepsia Questionnaire (LDQ) in a multi-ethnic Asian population. A Malay and culturally adapted English version of the LDQ were developed according to established protocols. Psychometric evaluation was performed by assessing the validity, internal consistency, test-retest reliability and responsiveness of the instruments in both primary and secondary care patients. Between April and September 2010, both Malay (n=166) and Malaysian English (n=154) versions were assessed in primary and secondary care patients. Both language versions were found to be reliable (internal consistency was 0.80 and 0.74 (Cronbach's α) for Malay and English, respectively; spearman's correlation coefficient for test-retest reliability was 0.98 for both versions), valid (area under receiver operating curve for accuracy of diagnosing dyspepsia was 0.71 and 0.77 for Malay and English versions, respectively), discriminative (median LDQ score discriminated between primary and secondary care patients in Malay (11.0 vs 20.0, P<0.0001) and English (10.0 vs 14.0, P=0.001), and responsive (median LDQ score reduced after treatment in Malay (17.0 to 14.0, P=0.08) and English (18.0 to 11.0, P=0.008) to dyspepsia. The Malaysian versions of the LDQ are valid, reliable and responsive instruments for assessing symptoms in a multi-ethnic Asian population with dyspepsia. © 2011 Journal of Gastroenterology and Hepatology Foundation and Blackwell Publishing Asia Pty Ltd.

Validity of instruments to assess students' travel and pedestrian safety.

PubMed

Mendoza, Jason A; Watson, Kathy; Baranowski, Tom; Nicklas, Theresa A; Uscanga, Doris K; Hanfling, Marcus J

2010-05-18

Safe Routes to School (SRTS) programs are designed to make walking and bicycling to school safe and accessible for children. Despite their growing popularity, few validated measures exist for assessing important outcomes such as type of student transport or pedestrian safety behaviors. This research validated the SRTS school travel survey and a pedestrian safety behavior checklist. Fourth grade students completed a brief written survey on how they got to school that day with set responses. Test-retest reliability was obtained 3-4 hours apart. Convergent validity of the SRTS travel survey was assessed by comparison to parents' report. For the measure of pedestrian safety behavior, 10 research assistants observed 29 students at a school intersection for completion of 8 selected pedestrian safety behaviors. Reliability was determined in two ways: correlations between the research assistants' ratings to that of the Principal Investigator (PI) and intraclass correlations (ICC) across research assistant ratings. The SRTS travel survey had high test-retest reliability (kappa = 0.97, n = 96, p < 0.001) and convergent validity (kappa = 0.87, n = 81, p < 0.001). The pedestrian safety behavior checklist had moderate reliability across research assistants' ratings (ICC = 0.48) and moderate correlation with the PI (r = 0.55, p = < 0.01). When two raters simultaneously used the instrument, the ICC increased to 0.65. Overall percent agreement (91%), sensitivity (85%) and specificity (83%) were acceptable. These validated instruments can be used to assess SRTS programs. The pedestrian safety behavior checklist may benefit from further formative work.

Impact of syncope on quality of life: validation of a measure in patients undergoing tilt testing.

PubMed

Nave-Leal, Elisabete; Oliveira, Mário; Pais-Ribeiro, José; Santos, Sofia; Oliveira, Eunice; Alves, Teresa; Cruz Ferreira, Rui

2015-03-01

Recurrent syncope has a significant impact on quality of life. The development of measurement scales to assess this impact that are easy to use in clinical settings is crucial. The objective of the present study is a preliminary validation of the Impact of Syncope on Quality of Life questionnaire for the Portuguese population. The instrument underwent a process of translation, validation, analysis of cultural appropriateness and cognitive debriefing. A population of 39 patients with a history of recurrent syncope (>1 year) who underwent tilt testing, aged 52.1 ± 16.4 years (21-83), 43.5% male, most in active employment (n=18) or retired (n=13), constituted a convenience sample. The resulting Portuguese version is similar to the original, with 12 items in a single aggregate score, and underwent statistical validation, with assessment of reliability, validity and stability over time. With regard to reliability, the internal consistency of the scale is 0.9. Assessment of convergent and discriminant validity showed statistically significant results (p<0.01). Regarding stability over time, a test-retest of this instrument at six months after tilt testing with 22 patients of the sample who had not undergone any clinical intervention found no statistically significant changes in quality of life. The results indicate that this instrument is of value for assessing quality of life in patients with recurrent syncope in Portugal. Copyright © 2014 Sociedade Portuguesa de Cardiologia. Published by Elsevier España. All rights reserved.

[Assessing work-related stress: an Italian adaptation of the HSE Management Standards Work-Related Stress Indicator Tool].

PubMed

Marcatto, Francesco; D'Errico, Giuseppe; Di Blas, Lisa; Ferrante, Donatella

2011-01-01

The aim of this paper is to present a preliminary validation of an Italian adaptation of the HSE Management Standards Work-Related Stress Indicator Tool (IT), an instrument for assessing work-related stress at the organizational level, originally developed in Britain by the Health and Safety Executive. A scale that assesses the physical work environment has been added to the original version of the IT. 190 employees of the University of Trieste have been enrolled in the study. A confirmatory analysis showed a satisfactory fit of the eight-factors structure of the instrument. Further psychometric analysis showed adequate internal consistency of the IT scales and good criterion validity, as evidenced by the correlations with self-perception of stress, work satisfaction and motivation. In conclusion, the Indicator Tool proved to be a valid and reliable instrument for the assessment of work-related stress at the organizational level, and it is also compatible with the instructions provided by the Ministry of Labour and Social Policy (Circular letter 18/11/2010).

The Toronto outcome measure for craniofacial prosthetics: reliability and validity of a condition-specific quality-of-life instrument.

PubMed

Anderson, James D; Johnston, Dennis A; Haugh, Gil S; Kiat-Amnuay, Sudarat; Gettleman, Lawrence

2013-01-01

The purpose of this study was to refine the Toronto Outcome Measure for Craniofacial Prosthetics (TOMCP), present evidence for its reliability and validity, and use the instrument to explore differences in quality of life between prostheses made with chlorinated polyethylene (CPE) (experimental) and silicone (control). As part of a multicenter prospective controlled randomized double-blind single-crossover clinical trial of the two materials, the TOMCP was administered at the start and end of two 4-month study arms, during which 42 patients wore prostheses made from one material then the other. Reliability was assessed at the crossover. To determine validity of the TOMCP, the Linear Analogue Self-Assessment (LASA-12) and the Short-Form 8 (SF-8) were also administered with the TOMCP. The TOMCP was reduced by removing items that were unreliable, had poorly distributed answers, showed increased internal consistency after their removal, or were too highly correlated with more than one other item. The tests of reliability and validity were then repeated. Finally, the reduced instrument was used to test for differences in quality of life between prostheses made of the two materials. The item reduction tactics pared the 52-item instrument down to 27 items. The correlations of both TOMCP versions with the LASA-12 and the SF-8 were found to be statistically significant, providing evidence of the validity of the TOMCP. The instrument revealed significantly better quality of life with silicone rather than CPE prostheses. Both versions of the TOMCP were found to be reliable and valid. The instrument was able to show differences in quality of life between two materials.

Predictive Validity of a Student Self-Report Screener of Behavioral and Emotional Risk in an Urban High School

ERIC Educational Resources Information Center

Dowdy, Erin; Harrell-Williams, Leigh; Dever, Bridget V.; Furlong, Michael J.; Moore, Stephanie; Raines, Tara; Kamphaus, Randy W.

2016-01-01

Increasingly, schools are implementing school-based screening for risk of behavioral and emotional problems; hence, foundational evidence supporting the predictive validity of screening instruments is important to assess. This study examined the predictive validity of the Behavior Assessment System for Children-2 Behavioral and Emotional Screening…

Validity of a Self-Administered 3-Day Physical Activity Recall in Young Adults

ERIC Educational Resources Information Center

Han, Jennifer L.; Dinger, Mary K.

2009-01-01

Background: Most physical activity recall questionnaires assess activity over a 7-day period. However, questionnaires have been validated in adolescents and adults using shorter recall timeframes. Purpose: The purpose of this study was to assess the validity of a self-administered 3-day physical activity recall instrument (3DR) in young adults.…

Cross-cultural validation of the Children's Assessment of Participation and Enjoyment (CAPE) in Spain.

PubMed

Longo, E; Badia, M; Orgaz, B; Verdugo, M A

2014-03-01

Despite growing interest in the topic of participation, the construct has not yet been assessed in children and adolescents with and without cerebral palsy (CP) in Spain. As there are no available instruments to measure participation in leisure activities which have been adapted in this country, the goal of this study was to validate a Spanish version of the Children's Assessment of Participation and Enjoyment (CAPE). The sample comprised 199 children and adolescents with CP and 199 without CP, between 8 and 18 years of age, from seven regions in Spain. The adaptation of the original version of CAPE was carried out through translation and backward translation, and the validity of the instrument was analysed. Construct validity was assessed through the correlation of the diverse CAPE domains and the quality of life domains (KIDSCREEN questionnaire). Discriminant validity was established by comparing children and adolescents with CP and typically developing children and adolescents. For test-retest reliability, the children and adolescents with and without CP completed the CAPE questionnaire twice within 4 weeks. The correlations found between the CAPE domains and the quality of life domains show that the CAPE presents construct validity. The CAPE discriminated children and adolescents with CP from those without any disability in the results of participation. According to most CAPE domains, typically developing children and adolescents engage in a greater number of activities than children and adolescents with CP. Test-retest reliability for the Spanish version of CAPE was adequate. The study provides a valid instrument to assess the participation of children and adolescents with and without CP who live in Spain. © 2012 John Wiley & Sons Ltd.

Generalizability and Validity of a Mathematics Performance Assessment.

ERIC Educational Resources Information Center

Lane, Suzanne; And Others

1996-01-01

Evidence from test results of 3,604 sixth and seventh graders is provided for the generalizability and validity of the Quantitative Understanding: Amplifying Student Achievement and Reasoning (QUASAR) Cognitive Assessment Instrument, which is designed to measure program outcomes and growth in mathematics. (SLD)

Ecological validity and clinical utility of Patient-Reported Outcomes Measurement Information System (PROMIS®) instruments for detecting premenstrual symptoms of depression, anger, and fatigue.

PubMed

Junghaenel, Doerte U; Schneider, Stefan; Stone, Arthur A; Christodoulou, Christopher; Broderick, Joan E

2014-04-01

This study examined the ecological validity and clinical utility of NIH Patient Reported-Outcomes Measurement Information System (PROMIS®) instruments for anger, depression, and fatigue in women with premenstrual symptoms. One-hundred women completed daily diaries and weekly PROMIS assessments over 4weeks. Weekly assessments were administered through Computerized Adaptive Testing (CAT). Weekly CATs and corresponding daily scores were compared to evaluate ecological validity. To test clinical utility, we examined if CATs could detect changes in symptom levels, if these changes mirrored those obtained from daily scores, and if CATs could identify clinically meaningful premenstrual symptom change. PROMIS CAT scores were higher in the pre-menstrual than the baseline (ps<.0001) and post-menstrual (ps<.0001) weeks. The correlations between CATs and aggregated daily scores ranged from .73 to .88 supporting ecological validity. Mean CAT scores showed systematic changes in accordance with the menstrual cycle and the magnitudes of the changes were similar to those obtained from the daily scores. Finally, Receiver Operating Characteristic (ROC) analyses demonstrated the ability of the CATs to discriminate between women with and without clinically meaningful premenstrual symptom change. PROMIS CAT instruments for anger, depression, and fatigue demonstrated validity and utility in premenstrual symptom assessment. The results provide encouraging initial evidence of the utility of PROMIS instruments for the measurement of affective premenstrual symptoms. Copyright © 2014 Elsevier Inc. All rights reserved.

Adaptation and Validation of a Burnout Inventory in a Survey of the Staff of a Correctional Institution in Bulgaria.

PubMed

Harizanova, Stanislava N; Mateva, Nonka G; Tarnovska, Tanya Ch

2016-12-01

Burnout syndrome is a phenomenon that seems to be studied globally in relation to all types of populations. The staff in the system of correctional institutions in Bulgaria, however, is oddly left out of this tendency. There is no standardized model in Bulgaria that can be used to detect possible susceptibility to professional burnout. The methods available at present only register the irreversible changes that have already set in the functioning of the individual. V. Boyko's method for burnout assessment allows clinicians to use individual approach to patients and affords easy comparability of results with data from other psychodiagnostic instruments. Adaptation of the assessment instruments to fit the specificities of a study population (linguistic, ethno-cultural, etc.) is obligatory so that the instrument could be correctly used and yield valid results. Validation is one of the most frequently used technique to achieve this. The aim of the present study was to adapt and validate V. Boyko's burnout inventory for diagnosing burnout and assessment of the severity of the burnout syndrome in correctional officers. We conducted a pilot study with 50 officers working in the Plovdiv Regional Correction Facility by test-retest survey performed at an interval of 2 to 4 months. All participants completed the adapted questionnaire translated into Bulgarian voluntarily and anonymously. Statistical analysis was performed using SPSS v.17. We found a mild-to-strong statistically significant correlation (P<0.01) across all subscales between the most frequently used questionnaire for assessing the burnout syndrome, the Maslach Burnout Inventory, and the tool we propose here. The high Cronbach's α coefficient (α=0.94) and Spearman-Brown coefficient (rsb=0.86), and the low mean between-item correlation (r=0.30) demonstrated the instrument's good reliability and validity. With the validation herein presented we offer a highly reliable Bulgarian variant of Boyko's method for burnout assessment and research.

Assessing Probabilistic Reasoning in Verbal-Numerical and Graphical-Pictorial Formats: An Evaluation of the Psychometric Properties of an Instrument

ERIC Educational Resources Information Center

Agus, Mirian; Penna, Maria Pietronilla; Peró-Cebollero, Maribel; Guàrdia-Olmos, Joan

2016-01-01

Research on the graphical facilitation of probabilistic reasoning has been characterised by the effort expended to identify valid assessment tools. The authors developed an assessment instrument to compare reasoning performances when problems were presented in verbal-numerical and graphical-pictorial formats. A sample of undergraduate psychology…

Assessment of Work Performance (AWP)--development of an instrument.

PubMed

Sandqvist, Jan L; Törnquist, Kristina B; Henriksson, Chris M

2006-01-01

Adequate work assessments are a matter of importance both for individuals and society [5,29,31,38,40,46,52]. However, there is a lack of adequate and reliable instruments for use in work rehabilitation [14,15,20,21,31,44]. The purpose of this study was to develop and evaluate an observation instrument for assessing work performance, the AWP (Assessment of Work Performance). The purpose of the 14-item instrument is to assess the individual's observable working skills in three different areas: motor skills, process skills, and communication and interaction skills. This article describes the development and results of preliminary testing of the AWP. The testing indicates a satisfactory face validity and utility for the AWP and supports further research and testing of the instrument.

Cultural adaptation: translatability assessment and linguistic validation of the patient-reported outcome instrument for irritable bowel syndrome with diarrhea

PubMed Central

Delgado-Herrera, Leticia; Lasch, Kathryn; Popielnicki, Ana; Nishida, Akito; Arbuckle, Rob; Banderas, Benjamin; Zentner, Susan; Gagainis, Ingrid; Zeiher, Bernhardt

2016-01-01

Background and objective Following a 2009 US Food and Drug Administration guidance, a new patient-reported outcome (PRO) instrument was developed to support end points in multinational clinical trials assessing irritable bowel syndrome with diarrhea (IBS-D) symptom severity. Our objective was to assess the translatability of the IBS-D PRO instrument into ten languages, and subsequently perform a cultural adaptation/linguistic validation of the questionnaire into Japanese and US Spanish. Materials and methods Translatability assessments of the US English version of the IBS-D PRO were performed by experienced PRO translators who were native speakers of each target language and currently residing in target-language countries. Languages were Chinese (People’s Republic of China), Dutch (the Netherlands), French (Belgium), German (Germany), Japanese (Japan), Polish (Poland), Portuguese (Brazil), Russian (Russia), Spanish (Mexico), and Spanish (US). The project team assessed the instrument to identify potential linguistic and/or cultural adaptation issues. After the issues identified were resolved, the instrument was translated into Spanish (US) and Japanese through a process of two forward translations, one reconciled translation, and one backward translation. The project team reviewed the translated versions before the instruments were evaluated by cognitive debriefing interviews with samples of five Spanish (US) and five Japanese IBS-D patients. Results Linguistic and cultural adaptation concerns identified during the translatability assessment required minor revisions, mainly the presentation of dates/times and word structure. During the cognitive debriefing interviews, two of five Spanish respondents misunderstood the term “bowel movement” to mean only diarrhea in the Spanish version. Consequently, the term was changed from “movimiento intestinal” to “evacuaciones”. None of the Japanese respondents identified issues with the Japanese version. Conclusion The translatability of the IBS-D PRO instrument into ten target languages was confirmed, with only minor changes made to the translations of the instrument. The translation and linguistic validation into Spanish (US) and Japanese provide evidence that this instrument can be used in multinational trials and clinical settings. PMID:27382346

Cross-cultural adaptation, validity, and reliability of the Parenting Styles and Dimensions Questionnaire - Short Version (PSDQ) for use in Brazil.

PubMed

Oliveira, Thaís D; Costa, Danielle de S; Albuquerque, Maicon R; Malloy-Diniz, Leandro F; Miranda, Débora M; de Paula, Jonas J

2018-06-11

The Parenting Styles and Dimensions Questionnaire (PSDQ) is used worldwide to assess three styles (authoritative, authoritarian, and permissive) and seven dimensions of parenting. In this study, we adapted the short version of the PSDQ for use in Brazil and investigated its validity and reliability. Participants were 451 mothers of children aged 3 to 18 years, though sample size varied with analyses. The translation and adaptation of the PSDQ followed a rigorous methodological approach. Then, we investigated the content, criterion, and construct validity of the adapted instrument. The scale content validity index (S-CVI) was considered adequate (0.97). There was evidence of internal validity, with the PSDQ dimensions showing strong correlations with their higher-order parenting styles. Confirmatory factor analysis endorsed the three-factor, second-order solution (i.e., three styles consisting of seven dimensions). The PSDQ showed convergent validity with the validated Brazilian version of the Parenting Styles Inventory (Inventário de Estilos Parentais - IEP), as well as external validity, as it was associated with several instruments measuring sociodemographic and behavioral/emotional-problem variables. The PSDQ is an effective and reliable psychometric instrument to assess childrearing strategies according to Baumrind's model of parenting styles.

Neighborhood Quality and Attachment: Validation of the Revised Residential Environment Assessment Tool.

PubMed

Poortinga, Wouter; Calve, Tatiana; Jones, Nikki; Lannon, Simon; Rees, Tabitha; Rodgers, Sarah E; Lyons, Ronan A; Johnson, Rhodri

2017-04-01

Various studies have shown that neighborhood quality is linked to neighborhood attachment and satisfaction. However, most have relied upon residents' own perceptions rather than independent observations of the neighborhood environment. This study examines the reliability and validity of the revised Residential Environment Assessment Tool (REAT 2.0), an audit instrument covering both public and private spaces of the neighborhood environment. The research shows that REAT 2.0 is a reliable, easy-to-use instrument and that most underlying constructs can be validated against residents' own neighborhood perceptions. The convergent validity of the instrument, which was tested against digital map data, can be improved for a number of miscellaneous urban form items. The research further found that neighborhood attachment was significantly associated with the overall REAT 2.0 score. This association can mainly be attributed to the property-level neighborhood quality and natural elements components. The research demonstrates the importance of private spaces in the outlook of the neighborhood environment.

Updating the OMERACT filter: implications of filter 2.0 to select outcome instruments through assessment of "truth": content, face, and construct validity.

PubMed

Tugwell, Peter; Boers, Maarten; D'Agostino, Maria-Antonietta; Beaton, Dorcas; Boonen, Annelies; Bingham, Clifton O; Choy, Ernest; Conaghan, Philip G; Dougados, Maxime; Duarte, Catia; Furst, Daniel E; Guillemin, Francis; Gossec, Laure; Heiberg, Turid; van der Heijde, Désirée M; Hewlett, Sarah; Kirwan, John R; Kvien, Tore K; Landewé, Robert B; Mease, Philip J; Østergaard, Mikkel; Simon, Lee; Singh, Jasvinder A; Strand, Vibeke; Wells, George

2014-05-01

The Outcome Measures in Rheumatology (OMERACT) Filter provides guidelines for the development and validation of outcome measures for use in clinical research. The "Truth" section of the OMERACT Filter requires that criteria be met to demonstrate that the outcome instrument meets the criteria for content, face, and construct validity. Discussion groups critically reviewed a variety of ways in which case studies of current OMERACT Working Groups complied with the Truth component of the Filter and what issues remained to be resolved. The case studies showed that there is broad agreement on criteria for meeting the Truth criteria through demonstration of content, face, and construct validity; however, several issues were identified that the Filter Working Group will need to address. These issues will require resolution to reach consensus on how Truth will be assessed for the proposed Filter 2.0 framework, for instruments to be endorsed by OMERACT.

Assessing the Learning Environment for Medical Students: An Evaluation of a Novel Survey Instrument in Four Medical Schools.

PubMed

Pololi, Linda H; Evans, Arthur T; Nickell, Leslie; Reboli, Annette C; Coplit, Lisa D; Stuber, Margaret L; Vasiliou, Vasilia; Civian, Janet T; Brennan, Robert T

2017-06-01

A practical, reliable, and valid instrument is needed to measure the impact of the learning environment on medical students' well-being and educational experience and to meet medical school accreditation requirements. From 2012 to 2015, medical students were surveyed at the end of their first, second, and third year of studies at four medical schools. The survey assessed students' perceptions of the following nine dimensions of the school culture: vitality, self-efficacy, institutional support, relationships/inclusion, values alignment, ethical/moral distress, work-life integration, gender equity, and ethnic minority equity. The internal reliability of each of the nine dimensions was measured. Construct validity was evaluated by assessing relationships predicted by our conceptual model and prior research. Assessment was made of whether the measurements were sensitive to differences over time and across institutions. Six hundred and eighty-six students completed the survey (49 % women; 9 % underrepresented minorities), with a response rate of 89 % (range over the student cohorts 72-100 %). Internal consistency of each dimension was high (Cronbach's α 0.71-0.86). The instrument was able to detect significant differences in the learning environment across institutions and over time. Construct validity was supported by demonstrating several relationships predicted by our conceptual model. The C-Change Medical Student Survey is a practical, reliable, and valid instrument for assessing the learning environment of medical students. Because it is sensitive to changes over time and differences across institution, results could potentially be used to facilitate and monitor improvements in the learning environment of medical students.

Instrument Motion Metrics for Laparoscopic Skills Assessment in Virtual Reality and Augmented Reality.

PubMed

Fransson, Boel A; Chen, Chi-Ya; Noyes, Julie A; Ragle, Claude A

2016-11-01

To determine the construct and concurrent validity of instrument motion metrics for laparoscopic skills assessment in virtual reality and augmented reality simulators. Evaluation study. Veterinarian students (novice, n = 14) and veterinarians (experienced, n = 11) with no or variable laparoscopic experience. Participants' minimally invasive surgery (MIS) experience was determined by hospital records of MIS procedures performed in the Teaching Hospital. Basic laparoscopic skills were assessed by 5 tasks using a physical box trainer. Each participant completed 2 tasks for assessments in each type of simulator (virtual reality: bowel handling and cutting; augmented reality: object positioning and a pericardial window model). Motion metrics such as instrument path length, angle or drift, and economy of motion of each simulator were recorded. None of the motion metrics in a virtual reality simulator showed correlation with experience, or to the basic laparoscopic skills score. All metrics in augmented reality were significantly correlated with experience (time, instrument path, and economy of movement), except for the hand dominance metric. The basic laparoscopic skills score was correlated to all performance metrics in augmented reality. The augmented reality motion metrics differed between American College of Veterinary Surgeons diplomates and residents, whereas basic laparoscopic skills score and virtual reality metrics did not. Our results provide construct validity and concurrent validity for motion analysis metrics for an augmented reality system, whereas a virtual reality system was validated only for the time score. © Copyright 2016 by The American College of Veterinary Surgeons.

CODEMamb - an observational communication behavior assessment tool for use in ambulatory dementia care.

PubMed

Knebel, Maren; Haberstroh, Julia; Kümmel, Anne; Pantel, Johannes; Schröder, Johannes

2016-12-01

Communication improves well-being and quality of life for both people with dementia and their professional and family caregivers. Individualized communication, as required in informed consent procedures and psychosocial interventions, can improve quality of life, especially in ambulatory settings. However, few valid and reliable instruments exist that enable communication to be assessed and communication and behavioral resources to be identified. We, therefore, extended and adapted the newly developed observational instrument CODEM for use in ambulatory settings (CODEM amb ). Reliability and validity of the new instrument were studied in a total of 171 patients, whereby principal component analysis revealed three important factors: relationship aspects, verbal communication behavior and nonverbal communication behavior. CODEM amb [Formula: see text]s internal consistency, interrater and retest reliability were satisfactory to excellent. Convergent validity indices, as shown by examining correlations with similar but not identical constructs (CERAD-NP verbal subscales), were medium-high, while the divergent validity index (constructional praxis) was relatively low. The relationship to peer-rating remained nonsignificant. Criterion validity was investigated in groups of patients in accordance with their cognitive status. As expected, verbal communication abilities deteriorate faster than the relationship aspects of communication as the disease progresses. In summary, CODEM amb is a reliable and valid instrument that can be used to collect important information with the ultimate aim of supporting communication with people with dementia.

Validation of the Dutch Eating Behaviour Questionnaire (DEBQ) among Maltese women.

PubMed

Dutton, Elaine; Dovey, Terence M

2016-12-01

The main aim of this study was to assess the dimensional structure of the Maltese version of the Dutch Eating Behaviour Questionnaire (DEBQ) and evaluate the instrument's validity and reliability among Maltese women (N = 586). Exploratory factor analysis reflected the theoretical structure of three factors; emotional, restrained and external eating which was supported by a Confirmatory Factor analysis. Minor issues with specific items in the Emotional and External eating scale were identified and discussed. Criterion-related validity was ascertained through correlations with the EAT-26. The study also assessed the DEBQ's predictive value in differentiating between BMI groups and between dieters and weight maintainers. The results suggest that the Maltese DEBQ is a psychometrically valid and reliable instrument for assessing eating behaviours with women in the Maltese community. The study also highlights the critical role of Emotional and Restrained eating in dieting and overweight Maltese women. Copyright © 2016 Elsevier Ltd. All rights reserved.

[An instrument in Spanish to evaluate the performance of clinical teachers by students].

PubMed

Bitran, Marcela; Mena, Beltrán; Riquelme, Arnoldo; Padilla, Oslando; Sánchez, Ignacio; Moreno, Rodrigo

2010-06-01

The modernization of clinical teaching has called for the creation of faculty development programs, and the design of suitable instruments to evaluate clinical teachers' performance. To report the development and validation of an instrument in Spanish designed to measure the students' perceptions of their clinical teachers' performance and to provide them with feedback to improve their teaching practices. In a process that included the active participation of authorities, professors in charge of courses and internships, clinical teachers, students and medical education experts, we developed a 30-item questionnaire called MEDUC30 to evaluate the performance of clinical teachers by their students. The internal validity was assessed by factor analysis of 5214 evaluations of 265 teachers, gathered from 2004 to 2007. The reliability was measured with the Cronbach's alpha coefficient and the generalizability coefficient (g). MEDUC30 had good content and construct validity. Its internal structure was compatible with four factors: patient-centered teaching, teaching skills, assessment skills and learning climate, and it proved to be consistent with the structure anticipated by the theory. The scores were highly reliable (Cronbach's alpha: 0.97); five evaluations per teacher were sufficient to reach a reliability coefficient (g) of 0.8. MEDUC30 is a valid, reliable and useful instrument to evaluate the performance of clinical teachers. To our knowledge, this is the first instrument in Spanish for which solid validity and reliability evidences have been reported. We hope that MEDUC30 will be used to improve medical education in Spanish-speaking medical schools, providing teachers a specific feedback upon which to improve their pedagogical practice, and authorities with valuable information for the assessment of their faculty.

Validation of the Portuguese self-administered computerised 24-hour dietary recall among second-, third- and fourth-grade children

USDA-ARS?s Scientific Manuscript database

Current methods for assessing children's dietary intake, such as interviewer-administered 24-h dietary recall (24-h DR), are time consuming and resource intensive. Self-administered instruments offer a low-cost diet assessment method for use with children. The present study assessed the validity of ...

Is It Safe? Reliability and Validity of Structured versus Unstructured Child Safety Judgments

ERIC Educational Resources Information Center

Bartelink, Cora; de Kwaadsteniet, Leontien; ten Berge, Ingrid J.; Witteman, Cilia L. M.

2017-01-01

Background: The LIRIK, an instrument for the assessment of child safety and risk, is designed to improve assessments by guiding professionals through a structured evaluation of relevant signs, risk factors, and protective factors. Objective: We aimed to assess the interrater agreement and the predictive validity of professionals' judgments made…

Content Validity for the VVSymQ® Instrument: A New Patient-Reported Outcome Measure for the Assessment of Varicose Veins Symptoms.

PubMed

Paty, Jean; Elash, Celeste A; Turner-Bowker, Diane M

2017-02-01

Varicose veins are common and can impact patients' quality of life, but consensus regarding the evaluation of varicose vein symptoms is lacking and existing measures have limitations. This research aimed to develop and establish the content validity of a new electronic patient-reported outcome (PRO) measure, the VVSymQ ® instrument, to assess symptoms of superficial venous insufficiency (varicose veins) in clinical trials. The development of the VVSymQ ® instrument began with qualitative interviews with patients based on the symptom domain of the VEINES-QOL/Sym, an existing PRO instrument for chronic venous disorders of the leg. Three phases of qualitative research were conducted to examine the relevance and importance of the symptoms to patients with varicose veins, and the patients' ability to understand and use the VVSymQ ® instrument. The development included evaluating questions that had 1-week and 24-h recall periods, and paper and electronic versions of the new instrument. Five symptoms (heaviness, achiness, swelling, throbbing, and itching [HASTI™]) were consistently reported by patients across all sources of qualitative data. The final version of the VVSymQ ® instrument queries patients on the HASTI™ symptoms using a 24-h recall period and a 6-point duration-based response scale ranging from "None of the time" to "All of the time," and is administered daily via an electronic diary. Cognitive interviews demonstrated varicose vein patients' understanding of and their ability to use the final version of the VVSymQ ® instrument. Content validity was established for the VVSymQ ® instrument, which assesses the five HASTI™ symptoms of varicose veins daily via an electronic diary and has promise for use in research and practice.

[Design and validation of an instrument to assess families at risk for health problems].

PubMed

Puschel, Klaus; Repetto, Paula; Solar, María Olga; Soto, Gabriela; González, Karla

2012-04-01

There is a paucity of screening instruments with a high clinical predictive value to identify families at risk and therefore, develop focused interventions in primary care. To develop an easy to apply screening instrument with a high clinical predictive value to identify families with a higher health vulnerability. In the first stage of the study an instrument with a high content validity was designed through a review of existent instruments, qualitative interviews with families and expert opinions following a Delphi approach of three rounds. In the second stage, concurrent validity was tested through a comparative analysis between the pilot instrument and a family clinical interview conducted to 300 families randomly selected from a population registered at a primary care clinic in Santiago. The sampling was blocked based on the presence of diabetes, depression, child asthma, behavioral disorders, presence of an older person or the lack of previous conditions among family members. The third stage, was directed to test the clinical predictive validity of the instrument by comparing the baseline vulnerability obtained by the instrument and the change in clinical status and health related quality of life perceptions of the family members after nine months of follow-up. The final SALUFAM instrument included 13 items and had a high internal consistency (Cronbach's alpha: 0.821), high test re-test reproducibility (Pearson correlation: 0.84) and a high clinical predictive value for clinical deterioration (Odds ratio: 1.826; 95% confidence intervals: 1.101-3.029). SALUFAM instrument is applicable, replicable, has a high content validity, concurrent validity and clinical predictive value.

Development of a Chinese medicine assessment measure: an interdisciplinary approach using the delphi method.

PubMed

Schnyer, Rosa N; Conboy, Lisa A; Jacobson, Eric; McKnight, Patrick; Goddard, Thomas; Moscatelli, Francesca; Legedza, Anna T R; Kerr, Catherine; Kaptchuk, Ted J; Wayne, Peter M

2005-12-01

The diagnostic framework and clinical reasoning process in Chinese medicine emphasizes the contextual and qualitative nature of a patient's illness. Chinese medicine assessment data may help interpret clinical outcomes. As part of a study aimed at assessing the validity and improving the inter-rater reliability of the Chinese diagnostic process, a structured assessment instrument was developed for use in clinical trials of acupuncture and other Chinese medical therapies. To foster collaboration and maximize resources and information, an interdisciplinary advisory team was assembled. Under the guidance of two group process facilitators, and in order to establish whether the assessment instrument was consistent with accepted Chinese medicine diagnostic categories (face validity) and included the full range of each concept's meaning (content validity), a panel of Traditional Chinese Medicine (TCM) expert clinicians was convened and their responses were organized using the Delphi process, an iterative, anonymous, idea-generating and consensus-building process. An aggregate rating measure was obtained by taking the mean of mean ratings for each question across all 10 experts. Over three rounds, the overall rating increased from 7.4 (SD = 1.3) in Round 1 to 9.1 (SD = 0.5) in Round 3. The level of agreement among clinicians was measured by a decrease in SD. The final instrument TEAMSI-TCM (Traditional East Asian Medicine Structured Interview, TCM version) uses the pattern differentiation model characteristic of TCM. This modular, dynamic version was specifically designed to assess women, with a focus on gynecologic conditions; with modifications it can be adapted for use with other populations and conditions. TEAMSI-TCM is a prescriptive instrument that guides clinicians to use the proper indicators, combine them in a systematic manner, and generate conclusions. In conjunction with treatment manualization and training it may serve to increase inter-rater reliability and inter-trial reproducibility in Chinese medicine clinical trials. Testing of the validity and reliability of this instrument currently is underway.

Screening for frailty in community-dwelling elderly subjects: Predictive validity of the modified SEGA instrument.

PubMed

Oubaya, N; Dramé, M; Novella, J-L; Quignard, E; Cunin, C; Jolly, D; Mahmoudi, R

2017-11-01

To study the capacity of the SEGAm instrument to predict loss of independence among elderly community-dwelling subjects. The study was performed in four French departments (Ardennes, Marne, Meurthe-et-Moselle, Meuse). Subjects aged 65 years or more, living at home, who could read and understand French, with a degree of autonomy corresponding to groups 5 or 6 in the AGGIR autonomy evaluation scale were included. Assessment included demographic characteristics, comprehensive geriatric assessment, and the SEGAm instrument at baseline. Subjects had follow-up visits at home at 6 and 12 months. During follow-up, vital status and level of independence were recorded. Logistic regression was used to study predictive validity of the SEGAm instrument. Among the 116 subjects with complete follow-up, 84 (72.4%) were classed as not very frail at baseline, 23 (19.8%) as frail, and 9 (7.8%) as very frail; 63 (54.3%) suffered loss of at least one ADL or IADL at 12 months. By multivariable analysis, frailty status at baseline was significantly associated with loss of independence during the 12 months of follow-up (OR=4.52, 95% CI=1.40-14.68; p=0.01). We previously validated the SEGAm instrument in terms of feasibility, acceptability, internal structure validity, reliability, and discriminant validity. This instrument appears to be a suitable tool for screening frailty among community-dwelling elderly subjects, and could be used as a basis to plan early targeted interventions for subjects at risk of adverse outcome. Copyright © 2017 Elsevier B.V. All rights reserved.

Psychometric properties of the Chinese version of the attitudes towards cardiopulmonary resuscitation with defibrillation (ACPRD-C) among female hospital nurses in Taiwan.

PubMed

Lin, Hsing-Long; Lin, Mei-Hsiang; Ho, Chao-Chung; Fu, Chin-Hua; Koo, Malcolm

2017-07-01

Nurses are often the first responders to in-hospital cardiac emergencies. A positive attitude towards cardiopulmonary resuscitation with defibrillation may contribute to early cardiopulmonary resuscitation and rapid defibrillation, which are associated with enhanced long-term survival. The aim of this study was to translate and adapt the 31-item attitudes towards cardiopulmonary resuscitation with defibrillation and the national resuscitation guidelines (ACPRD) instrument into Chinese and to evaluate its psychometric properties in a sample of Taiwanese hospital nurses. The ACPRD instrument was translated into Chinese using professional translation services. Content validity index based on five experts to refine the translated instrument. The final instrument was applied to a sample of 290 female nurses, recruited from a regional hospital in southern Taiwan, to assess its internal consistency, factor structure, and discriminative validity. The Chinese ACPRD instrument showed good internal consistency (Cronbach's alpha=0.87). Seven factors emerged from the factor analysis. The instrument showed good discriminative validity and were able to differentiate the attitudes of nurses with more experience of defibrillation or cardiopulmonary resuscitation from those with less experience. Nurses working in emergency ward or intensive care unit also showed significantly higher overall scores compared to those working in other units. The Chinese ACPRD demonstrated adequate content validity, internal consistency, sensible factor structure, and good discriminative validity. Among Chinese-speaking nurses, it may be used as a tool for assessing the effectiveness of educational programs that aim to improve their confidence in performing cardiopulmonary resuscitation with defibrillation. Copyright © 2017 Elsevier Ltd. All rights reserved.

Mathematics Screening: The Development and Pilot Study of a Mathematics Screening Instrument for K-2 Grades

ERIC Educational Resources Information Center

Anderson, Stephen A.

2010-01-01

This paper summarizes an action research project to develop a math screening instrument that would be effective (valid and reliable) and efficient (time for administration). An instrument was developed after review of the mathematics assessment and mathematics disabilities literature. The instrument was administered to kindergarten, first, and…

Validity of Guatemalan Mother's Self-Reported Breast-Feeding Practices of 3-Month-Old Infants.

PubMed

Mazariegos, Monica; Slater, Christine; Ramirez-Zea, Manuel

2016-12-01

Breast-feeding practices (BFPs) can be assessed by interviewing the mother about current feeding practices and with a 24-hour recall. It is crucial to establish the accuracy of these methods, which are commonly used by public health decision makers to design health policies aimed at increasing exclusive breast-feeding rates. We aimed to validate 2 self-report BFP instruments using the dose-to-mother deuterium oxide turnover technique (DMDOT) as the reference method. Breast-feeding practices were assessed by interviewing the mother about current feeding practices and with a 24-hour recall in 36 Guatemalan mother-infant pairs. The validity of these instruments was assessed using DMDOT as the reference method. Both self-report instruments overestimated exclusively breast-fed (EBF) infants. Infants classified as EBF were 50% by the reported current feeding practice, 61% by the 24-hour recall, and only 36% using DMDOT. Sensitivity to detect EBF infants from the mother's self-report was 92% (95% CI: 62%-99%) while from the 24-hour recall was 100% (95% CI: 72%-100%, P < .01). However, specificity for both instruments was low, at 74% (95% CI: 51%-89%) for reported current feeding practice and at 61% (95% CI: 39%-79%) for the 24-hour recall (P < .01). Both reported current feeding practice and the 24-hour recall instruments overestimated exclusive breast-feeding. Nevertheless, the use of reported current feeding practice provided more accurate data to assess BFPs in a public health setting. Furthermore, population-based surveys should consider the overestimation of exclusive breast-feeding caused when using these BFP instruments. © The Author(s) 2016.

The Ostomy-Q: Development and Psychometric Validation of an Instrument to Evaluate Outcomes Associated with Ostomy Appliances.

PubMed

Nafees, Beenish; Rasmussen, Mikkel; LLoyd, Andrew

2017-01-01

Using an ostomy appliance can affect many aspects of a person's health-related quality of life (HRQL). A 2-part, descrip- tive study was designed to develop and validate an instrument to assess quality-of-life outcomes related to ostomy ap- pliance use. Study inclusion/exclusion criteria stipulated participants should be 18 to 85 years of age, have an ileostomy or colostomy, used an appliance for a minimum of 3 months without assistance, and able to complete an online survey. All participants provided sociodemographic and clinical information. In phase 1, a literature search was conducted and existing instruments used to measure HRQL in persons with an ostomy were assessed. Subsequently, the Ostomy-Q, a 23-item, Likert-response type questionnaire, divided into 4 domains (Discreetness, Comfort, Confidence, and Social Life), was developed based on published evidence and existing ostomy-related HRQL tools. Seven (7) participants re- cruited from a manufacturer user panel took part in exploratory/cognitive qualitative interviews to refine the new quality- of-life questionnaire. In phase 2, the instrument was tested to assess item variability and conceptual structure, item-total correlation, internal consistency, test-retest reliability, sensitivity, and minimal important difference (MID) in an online validation study among 200 participants from the manufacturer's user panel (equally divided by gender, 125 [62.5%] >50 years old, 128 [64%] with an ileostomy). This exercise also included completion of the Stoma Quality of Life Question- naire and 2 domains from the Ostomy Adjustment Inventory-23 to assess convergent validity. Eighty-two (82) participants recompleted these study instruments 2 weeks later to assess test-retest reliability. Sociodemographic and clinical data were assessed using descriptive statistics; Cronbach's alpha was used for internal consistency (minimum 0.70), principle component analysis for item variability/conceptual structure, and item-total correlation; intraclass correlation coefficient was used for test-retest reliability; and standard error of measurement was applied to MID. All domains demonstrated good internal consistency (between 0.69 and 0.78). All scales showed stability, with a minimum intraclass correlation coefficient of 0.743 (P <.001). The Ostomy-Q showed good convergent validity with other instruments to which it was compared (P <.01). In this study, the Ostomy-Q was found to be a reliable and valid outcome measure that can enhance understanding of the impact of ostomy appliances on users. Some items for social relationships and discreetness may need more exploring in the future with other patient groups.

Assessing the capacity of ministries of health to use research in decision-making: conceptual framework and tool.

PubMed

Rodríguez, Daniela C; Hoe, Connie; Dale, Elina M; Rahman, M Hafizur; Akhter, Sadika; Hafeez, Assad; Irava, Wayne; Rajbangshi, Preety; Roman, Tamlyn; Ţîrdea, Marcela; Yamout, Rouham; Peters, David H

2017-08-01

The capacity to demand and use research is critical for governments if they are to develop policies that are informed by evidence. Existing tools designed to assess how government officials use evidence in decision-making have significant limitations for low- and middle-income countries (LMICs); they are rarely tested in LMICs and focus only on individual capacity. This paper introduces an instrument that was developed to assess Ministry of Health (MoH) capacity to demand and use research evidence for decision-making, which was tested for reliability and validity in eight LMICs (Bangladesh, Fiji, India, Lebanon, Moldova, Pakistan, South Africa, Zambia). Instrument development was based on a new conceptual framework that addresses individual, organisational and systems capacities, and items were drawn from existing instruments and a literature review. After initial item development and pre-testing to address face validity and item phrasing, the instrument was reduced to 54 items for further validation and item reduction. In-country study teams interviewed a systematic sample of 203 MoH officials. Exploratory factor analysis was used in addition to standard reliability and validity measures to further assess the items. Thirty items divided between two factors representing organisational and individual capacity constructs were identified. South Africa and Zambia demonstrated the highest level of organisational capacity to use research, whereas Pakistan and Bangladesh were the lowest two. In contrast, individual capacity was highest in Pakistan, followed by South Africa, whereas Bangladesh and Lebanon were the lowest. The framework and related instrument represent a new opportunity for MoHs to identify ways to understand and improve capacities to incorporate research evidence in decision-making, as well as to provide a basis for tracking change.

Assessment for Learning Instrumentation in Higher Education

ERIC Educational Resources Information Center

Akib, Erwin; Ghafar, Mohamad Najib Abdul

2015-01-01

This study explains assessment for learning instrumentation, especially in higher education. The population of this study was 100 lecturers of the Muhammadiyah Makassar University, Indonesia. A total of 50 items from six construct were analyzed and used to determine the reliability and validity of the questionnaire. The result shows that the…

Measuring Students' Emotions in the Early Years: The Achievement Emotions Questionnaire-Elementary School (AEQ-ES)

ERIC Educational Resources Information Center

Lichtenfeld, Stephanie; Pekrun, Reinhard; Stupnisky, Robert H.; Reiss, Kristina; Murayama, Kou

2012-01-01

This article reports about the development and validation of a measurement instrument assessing elementary school students' achievement emotions (Achievement Emotions Questionnaire-Elementary School, AEQ-ES). Specifically, the instrument assesses students' enjoyment, anxiety, and boredom pertaining to three types of academic settings (i.e.,…

Dimensions of Family Functioning: Perspectives of Low-Income African American Single Parent Families

ERIC Educational Resources Information Center

Mccreary, Linda L.; Dancy, Barbara L.

2004-01-01

Family functioning is influenced by socio-economic status, culture, family structure, and developmental stage, and is assessed primarily using instruments developed for middle-income European American two-parent families. These instruments may not validly assess low-income African American single-parent families. This qualitative study was…

Problems in developing an instrument for the rapid assessment of personality status.

PubMed

Van Horn, E; Manley, C; Leddy, D; Cicchetti, D; Tyrer, P

2000-06-01

To assess the validity of a quick assessment instrument (10 minutes) for assessing personality status, the Rapid Personality Assessment Schedule (PAS-R). The PAS-R was evaluated in psychotic patients recruited in one of the centres involved in a multicentre randomised controlled trial of intensive vs standard case management (the UK700 case management trial). Patients were assessed using both a full version of the PAS (PAS-I - ICD version) and the PAS-R. The weighted kappa statistic was used to gauge the (criterion-related) validity of the PAS-R using the PAS-I as the gold standard. Both measure code personality status using a four-point rating of severity in addition to recording individual categories of personality disorder. One hundred fifty-five (77%) of 201 patients recruited were assessed with both instruments. The weighted kappa statistic was 0.31, suggesting only moderate agreement between the PAS-I and PAS-R instruments under the four-point rating format, and 0.39 for the dichotomous personality disorder/no disorder separation. The sensitivity (64%) and specificity (82%) of the PAS-R in predicting PAS-I personality disorder were as satisfactory as for other screening instruments but still somewhat disappointing, and the PAS-R had an overall diagnostic accuracy of 78%. The PAS-R is a quick and rough method of detecting personality abnormality but is not a substitute for a fuller assessment.

Instruments to assess the perception of physicians in the decision-making process of specific clinical encounters: a systematic review

PubMed Central

Légaré, France; Moher, David; Elwyn, Glyn; LeBlanc, Annie; Gravel, Karine

2007-01-01

Background The measurement of processes and outcomes that reflect the complexity of the decision-making process within specific clinical encounters is an important area of research to pursue. A systematic review was conducted to identify instruments that assess the perception physicians have of the decision-making process within specific clinical encounters. Methods For every year available up until April 2007, PubMed, PsycINFO, Current Contents, Dissertation Abstracts and Sociological Abstracts were searched for original studies in English or French. Reference lists from retrieved studies were also consulted. Studies were included if they reported a self-administered instrument evaluating physicians' perceptions of the decision-making process within specific clinical encounters, contained sufficient description to permit critical appraisal and presented quantitative results based on administering the instrument. Two individuals independently assessed the eligibility of the instruments and abstracted information on their conceptual underpinnings, main evaluation domain, development, format, reliability, validity and responsiveness. They also assessed the quality of the studies that reported on the development of the instruments with a modified version of STARD. Results Out of 3431 records identified and screened for evaluation, 26 potentially relevant instruments were assessed; 11 met the inclusion criteria. Five instruments were published before 1995. Among those published after 1995, five offered a corresponding patient version. Overall, the main evaluation domains were: satisfaction with the clinical encounter (n = 2), mutual understanding between health professional and patient (n = 2), mental workload (n = 1), frustration with the clinical encounter (n = 1), nurse-physician collaboration (n = 1), perceptions of communication competence (n = 2), degree of comfort with a decision (n = 1) and information on medication (n = 1). For most instruments (n = 10), some reliability and validity criteria were reported in French or English. Overall, the mean number of items on the modified version of STARD was 12.4 (range: 2 to 18). Conclusion This systematic review provides a critical appraisal and repository of instruments that assess the perception physicians have of the decision-making process within specific clinical encounters. More research is needed to pursue the validation of the existing instruments and the development of patient versions. This will help researchers capture the complexity of the decision-making process within specific clinical encounters. PMID:17937801

Development and validation of an instrument to measure nurse educator perceived confidence in clinical teaching.

PubMed

Nguyen, Van N B; Forbes, Helen; Mohebbi, Mohammadreza; Duke, Maxine

2017-12-01

Teaching nursing in clinical environments is considered complex and multi-faceted. Little is known about the role of the clinical nurse educator, specifically the challenges related to transition from clinician, or in some cases, from newly-graduated nurse to that of clinical nurse educator, as occurs in developing countries. Confidence in the clinical educator role has been associated with successful transition and the development of role competence. There is currently no valid and reliable instrument to measure clinical nurse educator confidence. This study was conducted to develop and psychometrically test an instrument to measure perceived confidence among clinical nurse educators. A multi-phase, multi-setting survey design was used. A total of 468 surveys were distributed, and 363 were returned. Data were analyzed using exploratory and confirmatory factor analyses. The instrument was successfully tested and modified in phase 1, and factorial validity was subsequently confirmed in phase 2. There was strong evidence of internal consistency, reliability, content, and convergent validity of the Clinical Nurse Educator Skill Acquisition Assessment instrument. The resulting instrument is applicable in similar contexts due to its rigorous development and validation process. © 2017 The Authors. Nursing & Health Sciences published by John Wiley & Sons Australia, Ltd.

Developing an assessment of fire-setting to guide treatment in secure settings: the St Andrew's Fire and Arson Risk Instrument (SAFARI).

PubMed

Long, Clive G; Banyard, Ellen; Fulton, Barbara; Hollin, Clive R

2014-09-01

Arson and fire-setting are highly prevalent among patients in secure psychiatric settings but there is an absence of valid and reliable assessment instruments and no evidence of a significant approach to intervention. To develop a semi-structured interview assessment specifically for fire-setting to augment structured assessments of risk and need. The extant literature was used to frame interview questions relating to the antecedents, behaviour and consequences necessary to formulate a functional analysis. Questions also covered readiness to change, fire-setting self-efficacy, the probability of future fire-setting, barriers to change, and understanding of fire-setting behaviour. The assessment concludes with indications for assessment and a treatment action plan. The inventory was piloted with a sample of women in secure care and was assessed for comprehensibility, reliability and validity. Staff rated the St Andrews Fire and Risk Instrument (SAFARI) as acceptable to patients and easy to administer. SAFARI was found to be comprehensible by over 95% of the general population, to have good acceptance, high internal reliability, substantial test-retest reliability and validity. SAFARI helps to provide a clear explanation of fire-setting in terms of the complex interplay of antecedents and consequences and facilitates the design of an individually tailored treatment programme in sympathy with a cognitive-behavioural approach. Further studies are needed to verify the reliability and validity of SAFARI with male populations and across settings.

Assessing South China (Guangzhou) High School Students' Views on Nature of Science: A Validation Study

ERIC Educational Resources Information Center

Deng, Feng; Chai, Ching Sing; Tsai, Chin-Chung; Lin, Tzung-Jin

2014-01-01

Research on students' views on nature of science (VNOS) in Asian countries such as China is notably lacking. This study aimed to develop and validate an instrument to measure South China high school students' VNOS. Based on the previously acquired qualitative data, the instrument included seven VNOS dimensions which reflect the crucial aspects of…

The Development and Validation of the GMOAS, an Instrument Measuring Secondary School Students' Attitudes towards Genetically Modified Organisms

ERIC Educational Resources Information Center

Herodotou, Christothea; Kyza, Eleni A.; Nicolaidou, Iolie; Hadjichambis, Andreas; Kafouris, Dimitris; Terzian, Freda

2012-01-01

Genetically modified organisms (GMOs) is a rapidly evolving area of scientific innovation and an issue receiving global attention. Attempts to devise usable instruments that assess people's attitudes towards this innovation have been rare and non-systematic. The aim of this paper is to present the development and validation of the genetically…

Validity and reliability of wii fit balance board for the assessment of balance of healthy young adults and the elderly.

PubMed

Chang, Wen-Dien; Chang, Wan-Yi; Lee, Chia-Lun; Feng, Chi-Yen

2013-10-01

[Purpose] Balance is an integral part of human ability. The smart balance master system (SBM) is a balance test instrument with good reliability and validity, but it is expensive. Therefore, we modified a Wii Fit balance board, which is a convenient balance assessment tool, and analyzed its reliability and validity. [Subjects and Methods] We recruited 20 healthy young adults and 20 elderly people, and administered 3 balance tests. The correlation coefficient and intraclass correlation of both instruments were analyzed. [Results] There were no statistically significant differences in the 3 tests between the Wii Fit balance board and the SBM. The Wii Fit balance board had a good intraclass correlation (0.86-0.99) for the elderly people and positive correlations (r = 0.58-0.86) with the SBM. [Conclusions] The Wii Fit balance board is a balance assessment tool with good reliability and high validity for elderly people, and we recommend it as an alternative tool for assessing balance ability.

Psychometric evaluation of the Swedish adaptation of the Inventory for Assessing the Process of Cultural Competence Among Healthcare Professionals--Revised (IAPCC-R).

PubMed

Olt, Helen; Jirwe, Maria; Gustavsson, Petter; Emami, Azita

2010-01-01

The purpose of this study was to describe the translation, adaption, and psychometric evaluation process in relation to validity and reliability of the Swedish version of the instrument, Inventory for Assessing The Process of Cultural Competence Among Healthcare Professionals-Revised (IAPCC-R) following the translation, adaptation, and psychometric evaluation process. Validity tests were conducted on the response processes (N = 15), the content (N = 7), and the internal structure of the instrument (N = 334). Reliability (alpha = .65 for the total scale varying between -.01 and .65 for the different subscales) was evaluated in terms of internal consistency. Results indicated weak validity and reliability though it is difficult to conclude whether this is related to adaptation issues or the original construction.The testing of the response process identified problems in relation to respondents' conceptualization of cultural competence. The test of the content identified a weak correspondence between the items and the underlying model. In addition, a confirmatory factor analysis did not confirm the proposed structure of the instrument. This study concludes that this instrument is not valid and reliable for use with a Swedish population of practicing nurses or nursing students.

True communication skills assessment in interdepartmental OSCE stations: Standard setting using the MAAS-Global and EduG.

PubMed

Setyonugroho, Winny; Kropmans, Thomas; Murphy, Ruth; Hayes, Peter; van Dalen, Jan; Kennedy, Kieran M

2018-01-01

Comparing outcome of clinical skills assessment is challenging. This study proposes reliable and valid comparison of communication skills (1) assessment as practiced in Objective Structured Clinical Examinations (2). The aim of the present study is to compare CS assessment, as standardized according to the MAAS Global, between stations in a single undergraduate medical year. An OSCE delivered in an Irish undergraduate curriculum was studied. We chose the MAAS-Global as an internationally recognized and validated instrument to calibrate the OSCE station items. The MAAS-Global proportion is the percentage of station checklist items that can be considered as 'true' CS. The reliability of the OSCE was calculated with G-Theory analysis and nested ANOVA was used to compare mean scores of all years. MAAS-Global scores in psychiatry stations were significantly higher than those in other disciplines (p<0.03) and above the initial pass mark of 50%. The higher students' scores in psychiatry stations were related to higher MAAS-Global proportions when compared to the general practice stations. Comparison of outcome measurements, using the MAAS Global as a standardization instrument, between interdisciplinary station checklists was valid and reliable. The MAAS-Global was used as a single validated instrument and is suggested as gold standard. Copyright © 2017. Published by Elsevier B.V.

Is the Scale for Measuring Motivational Interviewing Skills a valid and reliable instrument for measuring the primary care professionals motivational skills?: EVEM study protocol.

PubMed

Pérula, Luis Á; Campiñez, Manuel; Bosch, Josep M; Barragán Brun, Nieves; Arboniés, Juan C; Bóveda Fontán, Julia; Martín Alvarez, Remedios; Prados, Jose A; Martín-Rioboó, Enrique; Massons, Josep; Criado, Margarita; Fernández, José Á; Parras, Juan M; Ruiz-Moral, Roger; Novo, Jesús M

2012-11-22

Lifestyle is one of the main determinants of people's health. It is essential to find the most effective prevention strategies to be used to encourage behavioral changes in their patients. Many theories are available that explain change or adherence to specific health behaviors in subjects. In this sense the named Motivational Interviewing has increasingly gained relevance. Few well-validated instruments are available for measuring doctors' communication skills, and more specifically the Motivational Interviewing. The hypothesis of this study is that the Scale for Measuring Motivational Interviewing Skills (EVEM questionnaire) is a valid and reliable instrument for measuring the primary care professionals skills to get behavior change in patients. To test the hypothesis we have designed a prospective, observational, multi-center study to validate a measuring instrument. - Thirty-two primary care centers in Spain. -Sampling and Size: a) face and consensual validity: A group composed of 15 experts in Motivational Interviewing. b) Assessment of the psychometric properties of the scale; 50 physician- patient encounters will be videoed; a total of 162 interviews will be conducted with six standardized patients, and another 200 interviews will be conducted with 50 real patients (n=362). Four physicians will be specially trained to assess 30 interviews randomly selected to test the scale reproducibility. -Measurements for to test the hypothesis: a) Face validity: development of a draft questionnaire based on a theoretical model, by using Delphi-type methodology with experts. b) Scale psychometric properties: intraobservers will evaluate video recorded interviews: content-scalability validity (Exploratory Factor Analysis), internal consistency (Cronbach alpha), intra-/inter-observer reliability (Kappa index, intraclass correlation coefficient, Bland & Altman methodology), generalizability, construct validity and sensitivity to change (Pearson product-moment correlation coefficient). The verification of the hypothesis that EVEM is a valid and reliable tool for assessing motivational interviewing would be a major breakthrough in the current theoretical and practical knowledge, as it could be used to assess if the providers put into practice a patient centered communication style and can be used both for training or researching purposes. TRIALS REGISTRATION Dislip-EM study: NCT01282190 (ClinicalTrials.gov).

Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project.

PubMed

2003-02-01

International interest in clinical practice guidelines has never been greater but many published guidelines do not meet the basic quality requirements. There have been renewed calls for validated criteria to assess the quality of guidelines. To develop and validate an international instrument for assessing the quality of the process and reporting of clinical practice guideline development. The instrument was developed through a multi-staged process of item generation, selection and scaling, field testing, and refinement procedures. 100 guidelines selected from 11 participating countries were evaluated independently by 194 appraisers with the instrument. Following refinement the instrument was further field tested on three guidelines per country by a new set of 70 appraisers. The final version of the instrument contained 23 items grouped into six quality domains with a 4 point Likert scale to score each item (scope and purpose, stakeholder involvement, rigour of development, clarity and presentation, applicability, editorial independence). 95% of appraisers found the instrument useful for assessing guidelines. Reliability was acceptable for most domains (Cronbach's alpha 0.64-0.88). Guidelines produced as part of an established guideline programme had significantly higher scores on editorial independence and, after the publication of a national policy, had significantly higher quality scores on rigour of development (p<0.005). Guidelines with technical documentation had higher scores on that domain (p<0.0001). This is the first time an appraisal instrument for clinical practice guidelines has been developed and tested internationally. The instrument is sensitive to differences in important aspects of guidelines and can be used consistently and easily by a wide range of professionals from different backgrounds. The adoption of common standards should improve the consistency and quality of the reporting of guideline development worldwide and provide a framework to encourage international comparison of clinical practice guidelines.

Validation of the Dementia Care Assessment Packet-Instrumental Activities of Daily Living

PubMed Central

Lee, Seok Bum; Park, Jeong Ran; Yoo, Jeong-Hwa; Park, Joon Hyuk; Lee, Jung Jae; Yoon, Jong Chul; Jhoo, Jin Hyeong; Lee, Dong Young; Woo, Jong Inn; Han, Ji Won; Huh, Yoonseok; Kim, Tae Hui

2013-01-01

Objective We aimed to evaluate the psychometric properties of the IADL measure included in the Dementia Care Assessment Packet (DCAP-IADL) in dementia patients. Methods The study involved 112 dementia patients and 546 controls. The DCAP-IADL was scored in two ways: observed score (OS) and predicted score (PS). The reliability of the DCAP-IADL was evaluated by testing its internal consistency, inter-rater reliability and test-retest reliability. Discriminant validity was evaluated by comparing the mean OS and PS between dementia patients and controls by ANCOVA. Pearson or Spearman correlation analysis was performed with other instruments to assess concurrent validity. Receiver operating characteristics curve analysis was performed to examine diagnostic accuracy. Results Chronbach's α coefficients of the DCAP-IADL were above 0.7. The values in dementia patients were much higher (OS=0.917, PS=0.927), indicating excellent degrees of internal consistency. Inter-rater reliabilities and test-retest reliabilities were statistically significant (p<0.05). PS exhibited higher reliabilities than OS. The mean OS and PS of dementia patients were significantly higher than those of the non-demented group after controlling for age, sex and education level. The DCAP-IADL was significantly correlated with other IADL instruments and MMSE-KC (p<0.001). Areas under the curves of the DCAP-IADL were above 0.9. Conclusion The DCAP-IADL is a reliable and valid instrument for evaluating instrumental ability of daily living for the elderly, and may also be useful for screening dementia. Moreover, administering PS may enable the DCAP-IADL to overcome the differences in gender, culture and life style that hinders accurate evaluation of the elderly in previous IADL instruments. PMID:24302946

Prioritisation of patients on waiting lists for hip and knee arthroplasties and cataract surgery: Instruments validation

PubMed Central

Allepuz, Alejandro; Espallargues, Mireia; Moharra, Montse; Comas, Mercè; Pons, Joan MV

2008-01-01

Background Prioritisation instruments were developed for patients on waiting list for hip and knee arthroplasties (AI) and cataract surgery (CI). The aim of the study was to assess their convergent and discriminant validity and inter-observer reliability. Methods Multicentre validation study which included orthopaedic surgeons and ophthalmologists from 10 hospitals. Participating doctors were asked to include all eligible patients placed in the waiting list for the procedures under study during the medical visit. Doctors assessed patients' priority through a visual analogue scale (VAS) and administered the prioritisation instrument. Information on socio-demographic data and health-related quality of life (HRQOL) (HUI3, EQ-5D, WOMAC and VF-14) was obtained through a telephone interview with patients. The correlation coefficients between the prioritisation instrument score and VAS and HRQOL were calculated. For the reliability study a self-administered questionnaire, which included hypothetic patients' scenarios, was sent via postal mail to the doctors. The priority of these scenarios was assessed through the prioritisation instrument. The intraclass correlation coefficient (ICC) between doctors was calculated. Results Correlations with VAS were strong for the AI (0.64, CI95%: 0.59–0.68) and for the CI (0.65, CI95%: 0.62–0.69), and moderate between the WOMAC and the AI (0.39, CI95%: 0.33–0.45) and the VF-14 and the CI (0.38, IC95%: 0.33–0.43). The results of the discriminant analysis were in general as expected. Inter-observer reliability was 0.79 (CI95%: 0.64–0.94) for the AI, and 0.79 (CI95%: 0.63–0.95) for the CI. Conclusion The results show acceptable validity and reliability of the prioritisation instruments in establishing priority for surgery. PMID:18397519

Quality of life instruments for economic evaluations in health and social care for older people: a systematic review.

PubMed

Makai, Peter; Brouwer, Werner B F; Koopmanschap, Marc A; Stolk, Elly A; Nieboer, Anna P

2014-02-01

Gaining health may not be the main goal of healthcare services aimed at older people, which may (also) seek to improve wellbeing. This emphasizes the need of finding appropriate outcome measures for economic evaluation of such services, particularly in long-term care, capturing more than only health-related quality of life (HrQol). This review assesses the usefulness of HrQol and wellbeing instruments for economic evaluations specifically aimed at older people, focusing on generic and preference-based questionnaires measuring wellbeing in particular. We systematically searched six databases and extracted instruments used to assess HrQol and wellbeing outcomes. Instruments were compared based on their usefulness for economic evaluation of services aimed at older people (dimensions measured, availability of utility scores, extent of validation). We identified 487 articles using 34 generic instruments: 22 wellbeing (two of which were preference-based) and 11 HrQol instruments. While standard HrQol instruments measure physical, social and psychological dimensions, wellbeing instruments contain additional dimensions such as purpose in life and achievement, security, and freedom. We found four promising wellbeing instruments for inclusion in economic evaluation: Ferrans and Powers QLI and the WHO-Qol OLD, ICECAP-O and the ASCOT. Ferrans and Powers QLI and the WHO-Qol OLD are widely validated but lack preference-weights while for ICECAP-O and the ASCOT preference-weights are available, but are less widely validated. Until preference-weights are available for the first two instruments, the ICECAP-O and the ASCOT currently appear to be the most useful instruments for economic evaluations in services aimed at older people. Their limitations are that (1) health dimensions may be captured only partially and (2) the instruments require further validation. Therefore, we currently recommend using the ICECAP-O or the ASCOT alongside the EQ-5D or SF-6D when evaluating interventions aimed at older people. Copyright © 2013 Elsevier Ltd. All rights reserved.

Movie for the Assessment of Social Cognition (MASC): Spanish validation.

PubMed

Lahera, G; Boada, L; Pousa, E; Mirapeix, I; Morón-Nozaleda, G; Marinas, L; Gisbert, L; Pamiàs, M; Parellada, M

2014-08-01

We present the Spanish validation of the "Movie for the Assessment of Social Cognition" instrument (MASC-SP). We recruited 22 adolescents and young adults with Asperger syndrome and 26 participants with typical development. The MASC-SP and three other social cognition instruments (Ekman Pictures of Facial Affect test, Reading the Mind in the Eyes Test, and Happé's Strange Stories) were administered to both groups. Individuals with Asperger syndrome had significantly lower scores in all measures of social cognition. The MASC-SP showed strong correlations with all three measures and relative independence of general cognitive functions. Internal consistency was optimal (0.86) and the test-retest was good. The MASC-SP is an ecologically valid and useful tool for assessing social cognition in the Spanish population.

Transcultural adaptation to Spanish of the instrument "Effectiveness of Auditory Rehabilitation" for the assessment of quality of life in patients using hearing aids.

PubMed

Cardemil, Felipe; Esquivel, Patricia; Aguayo, Lorena; Barría, Tamara; Fuente, Adrian; Carvajal, Rocío; Fromín, Rose; Villalobos, Iván; Yueh, Bevan

2013-01-01

It is becoming increasingly important to have reliable and valid questionnaires. This becomes especially important when evaluating hearing loss. the "Effectiveness of Auditory Rehabilitation" (EAR) questionnaire for the Spanish-speaking population. This instrument assesses quality of life and hearing aspects in patients using hearing aids. Cross-sectional validation study. A cultural adaptation through the use of English to Spanish translations and re-translations was carried out. The validity and reliability of the newly adapted instrument were evaluated. A total of 69 individuals (44 older adults and 25 younger adults) were examined. The pure-tone averages (PTA, 500, 1,000 and 2,000 Hz) were 47.3 dB HL and 47.1 dB HL for the left and right ears, respectively. The mean maximum speech discrimination in silence for monosyllables were 83.3% and 82.9% for the left and right ears, respectively. Internal consistency presented Cronbach alpha values of 0.85 and 0.77 for the internal and external dimensions, respectively. The intraclass correlation coefficients were 0.80 for the internal module and 0.85 for the external module. Construct validity reported a correlation coefficient of 0.71 at baseline and 0.76 at 3 months after the initial assessment for the internal module, and 0.62 at baseline and 0.74 at 3 months after the initial assessment for the external module. The size effects were 1.3 and 1.1 for the internal and external modules, respectively. The Spanish version of the EAR questionnaire seems to be a reliable and valid instrument. The evaluation of audiological aspects, as well as aspects relating to aesthetics and comfort are the main strengths of this instrument. Finally, the EAR scale is more sensitive to change than other scales. Copyright © 2013 Elsevier España, S.L. All rights reserved.

Do short courses in evidence based medicine improve knowledge and skills? Validation of Berlin questionnaire and before and after study of courses in evidence based medicine

PubMed Central

Fritsche, L; Greenhalgh, T; Falck-Ytter, Y; Neumayer, H-H; Kunz, R

2002-01-01

Objective To develop and validate an instrument for measuring knowledge and skills in evidence based medicine and to investigate whether short courses in evidence based medicine lead to a meaningful increase in knowledge and skills. Design Development and validation of an assessment instrument and before and after study. Setting Various postgraduate short courses in evidence based medicine in Germany. Participants The instrument was validated with experts in evidence based medicine, postgraduate doctors, and medical students. The effect of courses was assessed by postgraduate doctors from medical and surgical backgrounds. Intervention Intensive 3 day courses in evidence based medicine delivered through tutor facilitated small groups. Main outcome measure Increase in knowledge and skills. Results The questionnaire distinguished reliably between groups with different expertise in evidence based medicine. Experts attained a threefold higher average score than students. Postgraduates who had not attended a course performed better than students but significantly worse than experts. Knowledge and skills in evidence based medicine increased after the course by 57% (mean score before course 6.3 (SD 2.9) v 9.9 (SD 2.8), P<0.001). No difference was found among experts or students in absence of an intervention. Conclusions The instrument reliably assessed knowledge and skills in evidence based medicine. An intensive 3 day course in evidence based medicine led to a significant increase in knowledge and skills. What is already known on this topicNumerous observational studies have investigated the impact of teaching evidence based medicine to healthcare professionals, with conflicting resultsMost of the studies were of poor methodological qualityWhat this study addsAn instrument assessing basic knowledge and skills required for practising evidence based medicine was developed and validatedAn intensive 3 day course on evidence based medicine for doctors from various backgrounds and training level led to a clinically meaningful improvement of knowledge and skills PMID:12468485

Measurement of predictive validity in violence risk assessment studies: a second-order systematic review.

PubMed

Singh, Jay P; Desmarais, Sarah L; Van Dorn, Richard A

2013-01-01

The objective of the present review was to examine how predictive validity is analyzed and reported in studies of instruments used to assess violence risk. We reviewed 47 predictive validity studies published between 1990 and 2011 of 25 instruments that were included in two recent systematic reviews. Although all studies reported receiver operating characteristic curve analyses and the area under the curve (AUC) performance indicator, this methodology was defined inconsistently and findings often were misinterpreted. In addition, there was between-study variation in benchmarks used to determine whether AUCs were small, moderate, or large in magnitude. Though virtually all of the included instruments were designed to produce categorical estimates of risk - through the use of either actuarial risk bins or structured professional judgments - only a minority of studies calculated performance indicators for these categorical estimates. In addition to AUCs, other performance indicators, such as correlation coefficients, were reported in 60% of studies, but were infrequently defined or interpreted. An investigation of sources of heterogeneity did not reveal significant variation in reporting practices as a function of risk assessment approach (actuarial vs. structured professional judgment), study authorship, geographic location, type of journal (general vs. specialized audience), sample size, or year of publication. Findings suggest a need for standardization of predictive validity reporting to improve comparison across studies and instruments. Copyright © 2013 John Wiley & Sons, Ltd.

Validation of educational assessments: a primer for simulation and beyond.

PubMed

Cook, David A; Hatala, Rose

2016-01-01

Simulation plays a vital role in health professions assessment. This review provides a primer on assessment validation for educators and education researchers. We focus on simulation-based assessment of health professionals, but the principles apply broadly to other assessment approaches and topics. Validation refers to the process of collecting validity evidence to evaluate the appropriateness of the interpretations, uses, and decisions based on assessment results. Contemporary frameworks view validity as a hypothesis, and validity evidence is collected to support or refute the validity hypothesis (i.e., that the proposed interpretations and decisions are defensible). In validation, the educator or researcher defines the proposed interpretations and decisions, identifies and prioritizes the most questionable assumptions in making these interpretations and decisions (the "interpretation-use argument"), empirically tests those assumptions using existing or newly-collected evidence, and then summarizes the evidence as a coherent "validity argument." A framework proposed by Messick identifies potential evidence sources: content, response process, internal structure, relationships with other variables, and consequences. Another framework proposed by Kane identifies key inferences in generating useful interpretations: scoring, generalization, extrapolation, and implications/decision. We propose an eight-step approach to validation that applies to either framework: Define the construct and proposed interpretation, make explicit the intended decision(s), define the interpretation-use argument and prioritize needed validity evidence, identify candidate instruments and/or create/adapt a new instrument, appraise existing evidence and collect new evidence as needed, keep track of practical issues, formulate the validity argument, and make a judgment: does the evidence support the intended use? Rigorous validation first prioritizes and then empirically evaluates key assumptions in the interpretation and use of assessment scores. Validation science would be improved by more explicit articulation and prioritization of the interpretation-use argument, greater use of formal validation frameworks, and more evidence informing the consequences and implications of assessment.

Development and validation of an instrument to assess the prescribing readiness of medical students in Malaysia.

PubMed

Lai, Pauline Siew Mei; Sim, Si Mui; Chua, Siew Siang; Tan, Choo Hock; Ng, Chirk Jenn; Achike, Francis Ifejika; Teng, Cheong Lieng

2015-09-21

Prescribing incompetence is an important factor that contributes to prescribing error, and this is often due to inadequate training during medical schools. We therefore aimed to develop and validate an instrument to assess the prescribing readiness of medical students (PROMS) in Malaysia. The PROMS comprised of 26 items with four domains: undergraduate learning opportunities; hands-on clinical skills practice; information gathering behaviour; and factors affecting the learning of prescribing skills. The first three domains were adapted from an existing questionnaire, while items from the last domain were formulated based on findings from a nominal group discussion. Face and content validity was determined by an expert panel, pilot tested in a class of final year (Year 5) medical students, and assessed using the Flesch reading ease. To assess the reliability of the PROMS, the internal consistency and test-retest (at baseline and 2 weeks later) were assessed using the Wilcoxon Signed Ranks test and Spearman's rho. The discriminative validity of the PROMS was assessed using the Mann-Whitney U-test (to assess if the PROMS could discriminate between final year medical students from a public and a private university). A total of 119 medical students were recruited. Flesch reading ease was 46.9, indicating that the instrument was suitable for use in participants undergoing tertiary education. The overall Cronbach alpha value of the PROMS was 0.695, which was satisfactory. Test-retest showed no difference for 25/26 items, indicating that our instrument was reliable. Responses from the public and private university final year medical students were significantly different in 10/26 items, indicating that the PROMS was able to discriminate between these two groups. Medical students from the private university reported fewer learning opportunities and hands-on practice compared to those from the public university. On the other hand, medical students from the private university reported more frequent use of both web based and non-web-based resources compared to their public university counterparts. The PROMS instrument was found to be a reliable and valid tool for assessing medical students' readiness to prescribe in Malaysia. It may also inform on the adequacy of medical programmes in training prescribing skills.

Measuring the quality of motivational interviewing in primary health care encounters: The development and validation of the motivational interviewing assessment scale (MIAS).

PubMed

Campiñez Navarro, Manuel; Pérula de Torres, Luis Ángel; Bosch Fontcuberta, Josep M; Barragán Brun, Nieves; Arbonies Ortiz, Juan Carlos; Novo Rodríguez, Jesús Manuel; Bóveda Fontán, Julia; Martín Alvarez, Remedios; Prados Castillejo, Jose Antonio; Rivas Doutreleau, Gabriela Renée; Domingo Peña, Carmen; Castro Moreno, Jaime Jesús; Romero Rodríguez, Esperanza María

2016-09-01

Motivational interviewing (MI) is a collaborative, goal-oriented method to help patients change behaviour. Tools that are often used to measure MI are the motivational interviewing skills code' (MISC), the 'motivational interviewing treatment integrity' (MITI) and the 'behaviour change counselling index' (BECCI). The first two instruments have not been designed to be used in primary healthcare (PHC) settings. The BECCI actually is time-consuming. The motivational interviewing assessment scale (MIAS, 'EVEM' in Spanish) was developed to measure MI in PHC encounters as an alternative to the previous instruments. To validate MIAS as an instrument to assess the quality of MI in PHC settings. (a) Sixteen experts in MI participated in the design, face and consensus validity, using a Delphi-type methodology. (b) 27 PHC centres located in Spain. four experts in MI tested its psychometric properties with 332 video recordings coming from the Dislip-EM study (consultations provided by 37 practitioners). dimensionality, internal consistency, reliability (intra-class correlation coefficient-ICC), sensitivity to change and convergent validity with the BECCI scale. A 14-item scale was obtained after the validation process. Factor analysis: two factors explained 76.6% of the total variance. Internal consistency, α = 0.99. Reliability: intra-rater ICC = 0.96; inter-rater ICC = 0.97. Sensitivity to change: means before and after training were 23.63 versus 38.57 (P < 0.001). Spearman's coefficient between the MIAS and the BECCI scale was 0.98 (P < 0.001). The MIAS is a consistent and reliable instrument to assess the use of MI in PHC settings. [Box: see text].

Validation of an instrument to assess visual ability in children with visual impairment in China.

PubMed

Huang, Jinhai; Khadka, Jyoti; Gao, Rongrong; Zhang, Sifang; Dong, Wenpeng; Bao, Fangjun; Chen, Haisi; Wang, Qinmei; Chen, Hao; Pesudovs, Konrad

2017-04-01

To validate a visual ability instrument for school-aged children with visual impairment in China by translating, culturally adopting and Rasch scaling the Cardiff Visual Ability Questionnaire for Children (CVAQC). The 25-item CVAQC was translated into Mandarin using a standard protocol. The translated version (CVAQC-CN) was subjected to cognitive testing to ensure a proper cultural adaptation of its content. Then, the CVAQC-CN was interviewer-administered to 114 school-aged children and young people with visual impairment. Rasch analysis was carried out to assess its psychometric properties. The correlation between the CVAQC-CN visual ability scores and clinical measure of vision (visual acuity; VA and contrast sensitivity, CS) were assessed using Spearman's r. Based on cultural adaptation exercise, cognitive testing, missing data and Rasch metrics-based iterative item removal, three items were removed from the original 25. The 22-item CVAQC-CN demonstrated excellent measurement precision (person separation index, 3.08), content validity (item separation, 10.09) and item reliability (0.99). Moreover, the CVAQC-CN was unidimensional and had no item bias. The person-item map indicated good targeting of item difficulty to person ability. The CVAQC-CN had moderate correlations between CS (-0.53, p<0.00001) and VA (0.726, p<0.00001), respectively, indicating its validity. The 22-item CVAQC-CN is a psychometrically robust and valid instrument to measure visual ability in children with visual impairment in China. The instrument can be used as a clinical and research outcome measure to assess the change in visual ability after low vision rehabilitation intervention. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.

Recommendations on the most suitable quality-of-life measurement instruments for bariatric and body contouring surgery: a systematic review.

PubMed

de Vries, C E E; Kalff, M C; Prinsen, C A C; Coulman, K D; den Haan, C; Welbourn, R; Blazeby, J M; Morton, J M; van Wagensveld, B A

2018-06-08

The objective of this study is to systematically assess the quality of existing patient-reported outcome measures developed and/or validated for Quality of Life measurement in bariatric surgery (BS) and body contouring surgery (BCS). We conducted a systematic literature search in PubMed, EMBASE, PsycINFO, CINAHL, Cochrane Database Systematic Reviews and CENTRAL identifying studies on measurement properties of BS and BCS Quality of Life instruments. For all eligible studies, we evaluated the methodological quality of the studies by using the COnsensus-based Standards for the selection of health Measurement INstruments checklist and the quality of the measurement instruments by applying quality criteria. Four degrees of recommendation were assigned to validated instruments (A-D). Out of 4,354 articles, a total of 26 articles describing 24 instruments were included. No instrument met all requirements (category A). Seven instruments have the potential to be recommended depending on further validation studies (category B). Of these seven, the BODY-Q has the strongest evidence for content validity in BS and BCS. Two instruments had poor quality in at least one required quality criterion (category C). Fifteen instruments were minimally validated (category D). The BODY-Q, developed for BS and BCS, possessed the strongest evidence for quality of measurement properties and has the potential to be recommended in future clinical trials. © 2018 The Authors. Obesity Reviews published by John Wiley & Sons Ltd on behalf of World Obesity Federation.

Exploring the use of the Dementia Management Strategies Scale in caregivers of persons with dementia in Singapore.

PubMed

Tan, Louisa; Yap, Philip; Ng, Wai Yee; Luo, Nan

2013-01-01

Well-being in persons with dementia (PWD) depends much on the quality and type of care received. The Dementia Management Strategies Scale (DMSS) is a useful instrument to appraise care styles of caregivers. The present study expanded on previous research by refining and establishing the scale's content validity and psychometric properties in the Singapore context. Five family caregivers and four dementia care professionals (nurse, occupational therapist, social worker and doctor) reviewed the DMSS for content validity. Two hundred and forty-six family caregivers completed questionnaires which assessed caregiver and patient characteristics, and dementia management strategies with DMSS. Internal consistency reliability was assessed and construct validity was evaluated through Pearson's correlation with extant instruments. Eight items from the 28-item DMSS were omitted after content review as they were deemed inappropriate in our socio-cultural setting. A factor analysis with Varimax rotation confirmed a two-factor structure (positive and negative dimensions) for the revised DMSS (rDMSS). The two subscales showed good internal consistency (Cronbach's alpha .89 and .87). Moderate to strong correlations (.35-.53) with the scales, Zarit Burden Instrument, Revised Memory and Behavioural Problems Checklist, General Health Questionnaire, Short Sense of Competence Scale, Gains in Alzheimer's Care Instrument and Positive Aspects of Caregiving established convergent and divergent construct validity of rDMSS. The shortened 20-item rDMSS is a psychometrically valid instrument which can serve as a measure of dementia care strategy from the perspective of the caregiver in Singapore.

Development and content validation of performance assessments for endoscopic third ventriculostomy.

PubMed

Breimer, Gerben E; Haji, Faizal A; Hoving, Eelco W; Drake, James M

2015-08-01

This study aims to develop and establish the content validity of multiple expert rating instruments to assess performance in endoscopic third ventriculostomy (ETV), collectively called the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT). The important aspects of ETV were identified through a review of current literature, ETV videos, and discussion with neurosurgeons, fellows, and residents. Three assessment measures were subsequently developed: a procedure-specific checklist (CL), a CL of surgical errors, and a global rating scale (GRS). Neurosurgeons from various countries, all identified as experts in ETV, were then invited to participate in a modified Delphi survey to establish the content validity of these instruments. In each Delphi round, experts rated their agreement including each procedural step, error, and GRS item in the respective instruments on a 5-point Likert scale. Seventeen experts agreed to participate in the study and completed all Delphi rounds. After item generation, a total of 27 procedural CL items, 26 error CL items, and 9 GRS items were posed to Delphi panelists for rating. An additional 17 procedural CL items, 12 error CL items, and 1 GRS item were added by panelists. After three rounds, strong consensus (>80% agreement) was achieved on 35 procedural CL items, 29 error CL items, and 10 GRS items. Moderate consensus (50-80% agreement) was achieved on an additional 7 procedural CL items and 1 error CL item. The final procedural and error checklist contained 42 and 30 items, respectively (divided into setup, exposure, navigation, ventriculostomy, and closure). The final GRS contained 10 items. We have established the content validity of three ETV assessment measures by iterative consensus of an international expert panel. Each measure provides unique assessment information and thus can be used individually or in combination, depending on the characteristics of the learner and the purpose of the assessment. These instruments must now be evaluated in both the simulated and operative settings, to determine their construct validity and reliability. Ultimately, the measures contained in the NEVAT may prove suitable for formative assessment during ETV training and potentially as summative assessment measures during certification.

The diagnostic efficiency of the extended German Brøset Violence Checklist to assess the risk of violence.

PubMed

Rechenmacher, Josef; Müller, Gerhard; Abderhalden, Christoph; Schulc, Eva

2014-01-01

The prevention of aggression and violence of patients is part of the challenge for the psychiatric inpatient care. Resources needed are a systematic risk assessment and taking preventive measures according to the risk. The extended Brøset Violence Checklist (BVC-CH) is an assessment instrument for the short-term assessment of the risk of violence for physical attacks toward medical staff and other patients. Until now, the instrument was only validated in the context of the development phase of the instrument. The aim of this study was to investigate how valid the BVC-CH scale is for adult psychiatry in acute inpatient care facilities. In a prospective cohort study, 232 consecutively admitted patients were assessed using the BVC-CH. The calculation of the predictive values was based on a contingency table. The discriminatory power of the instrument and the determination of the cutoff point were done using the receiver operating characteristic (ROC) curve analysis. Physical attacks were registered with the Staff Observation of Aggression Scale-Revised (SOAS-R). The sensitivity was 58.8% and the specificity was 96.8% by a cutoff point of > or = 7. By choosing a cutoff point of > or = 6, the sensitivity was 64.7% and the specificity was 95.1%. A value of .93 was determined for the area under the curve receiver operating characteristic (AUC(ROC)). Overall, the BVC-CH is a valid instrument for the short-term prediction of physical attacks. Further research of the BVC-CH is recommended but in particular for the cutoff point.

The Quantitative Reasoning for College Science (QuaRCS) Assessment: Emerging Themes from 5 Years of Data

NASA Astrophysics Data System (ADS)

Follette, Katherine; Dokter, Erin; Buxner, Sanlyn

2018-01-01

The Quantitative Reasoning for College Science (QuaRCS) Assessment is a validated assessment instrument that was designed to measure changes in students' quantitative reasoning skills, attitudes toward mathematics, and ability to accurately assess their own quantitative abilities. It has been administered to more than 5,000 students at a variety of institutions at the start and end of a semester of general education college science instruction. I will begin by briefly summarizing our published work surrounding validation of the instrument and identification of underlying attitudinal factors (composite variables identified via factor analysis) that predict 50% of the variation in students' scores on the assessment. I will then discuss more recent unpublished work, including: (1) Development and validation of an abbreviated version of the assessment (The QuaRCS Light), which results in marked improvements in students' ability to maintain a high effort level throughout the assessment and has broad implications for quantitative reasoning assessments in general, and (2) Our efforts to revise the attitudinal portion of the assessment to better assess math anxiety level, another key factor in student performance on numerical assessments.

Supporting Educational Researchers and Practitioners in Their Work in Education: Assessing the Verbal Reasoning Skills of Five- to Six-Year-Old Children

ERIC Educational Resources Information Center

Säre, Egle; Luik, Piret; Fisher, Robert

2016-01-01

The purpose of this study was to design an instrument for five- to six-year-old children to help measure their verbal reasoning skills and assess the validity and reliability of the resulting instrument. For this purpose, the researchers have created the Younger Children Verbal Reasoning Test (YCVR-test) and a control instrument, which have been…

Development and Validation of the Scan of Postgraduate Educational Environment Domains (SPEED): A Brief Instrument to Assess the Educational Environment in Postgraduate Medical Education

PubMed Central

Schönrock-Adema, Johanna; Visscher, Maartje; Raat, A. N. Janet; Brand, Paul L. P.

2015-01-01

Introduction Current instruments to evaluate the postgraduate medical educational environment lack theoretical frameworks and are relatively long, which may reduce response rates. We aimed to develop and validate a brief instrument that, based on a solid theoretical framework for educational environments, solicits resident feedback to screen the postgraduate medical educational environment quality. Methods Stepwise, we developed a screening instrument, using existing instruments to assess educational environment quality and adopting a theoretical framework that defines three educational environment domains: content, atmosphere and organization. First, items from relevant existing instruments were collected and, after deleting duplicates and items not specifically addressing educational environment, grouped into the three domains. In a Delphi procedure, the item list was reduced to a set of items considered most important and comprehensively covering the three domains. These items were triangulated against the results of semi-structured interviews with 26 residents from three teaching hospitals to achieve face validity. This draft version of the Scan of Postgraduate Educational Environment Domains (SPEED) was administered to residents in a general and university hospital and further reduced and validated based on the data collected. Results Two hundred twenty-three residents completed the 43-item draft SPEED. We used half of the dataset for item reduction, and the other half for validating the resulting SPEED (15 items, 5 per domain). Internal consistencies were high. Correlations between domain scores in the draft and brief versions of SPEED were high (>0.85) and highly significant (p<0.001). Domain score variance of the draft instrument was explained for ≥80% by the items representing the domains in the final SPEED. Conclusions The SPEED comprehensively covers the three educational environment domains defined in the theoretical framework. Because of its validity and brevity, the SPEED is promising as useful and easily applicable tool to regularly screen educational environment quality in postgraduate medical education. PMID:26413836

Development and validation of an instrument to assess future orientation and resilience in adolescence.

PubMed

Di Maggio, Ilaria; Ginevra, Maria Cristina; Nota, Laura; Soresi, Salvatore

2016-08-01

The study is aimed at providing the development and initial validation of the Design My Future (DMF), which may be administered in career counseling and research activities to assess adolescents' future orientation and resilience. Two studies with two independent samples of Italian adolescents were conducted to examine psychometric requisites of DMF. Specifically, in the first study, after developing items and examined the content validity, the factorial structure, reliability and discriminant validity of the DMF were tested. In the second study, the measurement invariance across gender, conducing a sequence of nested CFA models, was evaluated. Results showed good psychometric support for the instrument with Italian adolescents. Copyright © 2016 The Foundation for Professionals in Services for Adolescents. Published by Elsevier Ltd. All rights reserved.

Assessing Learning Outcomes in Middle-Division Classical Mechanics: The Colorado Classical Mechanics and Math Methods Instrument

ERIC Educational Resources Information Center

Caballero, Marcos D.; Doughty, Leanne; Turnbull, Anna M.; Pepper, Rachel E.; Pollock, Steven J.

2017-01-01

Reliable and validated assessments of introductory physics have been instrumental in driving curricular and pedagogical reforms that lead to improved student learning. As part of an effort to systematically improve our sophomore-level classical mechanics and math methods course (CM 1) at CU Boulder, we have developed a tool to assess student…

Developing an ICT-Literacy Task-Based Assessment Instrument: The Findings on the Final Testing Phase

ERIC Educational Resources Information Center

Mat-jizat, Jessnor Elmy

2013-01-01

This paper reports the findings of a study which seeks to identify the information and communications technology (ICT) literacy levels of trainee teachers, by investigating their ICT proficiency using a task-bask assessment instrument. The Delphi technique was used as a primary validation method for the new assessment tool and the ICT literacy…

The Development of a Web-Based Assessment System to Identify Students' Misconception Automatically on Linear Kinematics with a Four-Tier Instrument Test

ERIC Educational Resources Information Center

Pujayanto, Pujayanto; Budiharti, Rini; Adhitama, Egy; Nuraini, Niken Rizky Amalia; Putri, Hanung Vernanda

2018-01-01

This research proposes the development of a web-based assessment system to identify students' misconception. The system, named WAS (web-based assessment system), can identify students' misconception profile on linear kinematics automatically after the student has finished the test. The test instrument was developed and validated. Items were…

Assessment of the wish to hasten death in patients with advanced disease: A systematic review of measurement instruments.

PubMed

Bellido-Pérez, Mercedes; Monforte-Royo, Cristina; Tomás-Sábado, Joaquín; Porta-Sales, Josep; Balaguer, Albert

2017-06-01

Patients with advanced conditions may present a wish to hasten death. Assessing this wish is complex due to the nature of the phenomenon and the difficulty of conceptualising it. To identify and analyse existing instruments for assessing the wish to hasten death and to rate their reported psychometric properties. Systematic review based on PRISMA guidelines. The COnsensus-based Standards for the selection of health Measurement INstruments checklist was used to evaluate the methodological quality of validation studies and the measurement properties of the instrument described. The CINAHL, PsycINFO, Pubmed and Web of Science databases were searched from inception to November 2015. A total of 50 articles involving assessment of the wish to hasten death were included. Eight concerned instrument validation and were evaluated using COnsensus-based Standards for the selection of health Measurement INstruments criteria. They reported data for between two and seven measurement properties, with ratings between fair and excellent. Of the seven instruments identified, the Desire for Death Rating Scale or the Schedule of Attitudes toward Hastened Death feature in 48 of the 50 articles. The Schedule of Attitudes toward Hastened Death is the most widely used and is the instrument whose psychometric properties have been most often analysed. Versions of the Schedule of Attitudes toward Hastened Death are available in five languages other than the original English. This systematic review has analysed existing instruments for assessing the wish to hasten death. It has also explored the methodological quality of studies that have examined the measurement properties of these instruments and offers ratings of the reported properties. These results will be useful to clinicians and researchers with an interest in a phenomenon of considerable relevance to advanced patients.

Assessment of the wish to hasten death in patients with advanced disease: A systematic review of measurement instruments

PubMed Central

Bellido-Pérez, Mercedes; Monforte-Royo, Cristina; Tomás-Sábado, Joaquín; Porta-Sales, Josep; Balaguer, Albert

2016-01-01

Background: Patients with advanced conditions may present a wish to hasten death. Assessing this wish is complex due to the nature of the phenomenon and the difficulty of conceptualising it. Aim: To identify and analyse existing instruments for assessing the wish to hasten death and to rate their reported psychometric properties. Design: Systematic review based on PRISMA guidelines. The COnsensus-based Standards for the selection of health Measurement INstruments checklist was used to evaluate the methodological quality of validation studies and the measurement properties of the instrument described. Data sources: The CINAHL, PsycINFO, Pubmed and Web of Science databases were searched from inception to November 2015. Results: A total of 50 articles involving assessment of the wish to hasten death were included. Eight concerned instrument validation and were evaluated using COnsensus-based Standards for the selection of health Measurement INstruments criteria. They reported data for between two and seven measurement properties, with ratings between fair and excellent. Of the seven instruments identified, the Desire for Death Rating Scale or the Schedule of Attitudes toward Hastened Death feature in 48 of the 50 articles. The Schedule of Attitudes toward Hastened Death is the most widely used and is the instrument whose psychometric properties have been most often analysed. Versions of the Schedule of Attitudes toward Hastened Death are available in five languages other than the original English. Conclusion: This systematic review has analysed existing instruments for assessing the wish to hasten death. It has also explored the methodological quality of studies that have examined the measurement properties of these instruments and offers ratings of the reported properties. These results will be useful to clinicians and researchers with an interest in a phenomenon of considerable relevance to advanced patients. PMID:28124578

Construct Validation--Community College Instructional Development Inventory

ERIC Educational Resources Information Center

Xiong, Soua; Delgado, Nexi; Wood, J. Luke; Harris, Frank, III

2017-01-01

This white paper describes the construct validation of the Community College Instructional Development Inventory (CC-IDI). The CC-IDI is an institutional assessment tool designed to inform professional development programming for instructional faculty. The instrument was developed to serve as a standardized assessment tool to determine the…

Validity of instruments to assess students' travel and pedestrian safety

USDA-ARS?s Scientific Manuscript database

Safe Routes to School (SRTS) programs are designed to make walking and bicycling to school,safe and accessible for children. Despite their growing popularity, few validated measures exist for assessing important outcomes such as type of student transport or pedestrian safety behaviors. This research...

[Validation of a scale to assess the labour quality of life in public hospitals from Tlaxcala].

PubMed

Hernández-Vicente, Irma Alejandra; Lumbreras-Guzmán, Marivel; Méndez-Hernández, Pablo; Rojas-Lima, Elodia; Cervantes-Rodríguez, Margarita; Juárez-Flores, Clara Arlina

2017-01-01

To validate a scale for assessing the labour quality of life in public hospitals (LQL-PH) from Tlaxcala, Mexico. The instrument was validated among 669 health workers from six hospitals from the Ministry of Health of Tlaxcala, Mexico. Content validity was by inquiry to experts, construct validity by factor analysis, criterion validity by comparing with other scales, and reliability with Cronbach's Alpha. The factor analysis uncovered four dimensions: "individual welfare", "conditions and labour environment", "organization", and "well-being accomplished by the work"; reliability was 0.921. Workers who perceibed better LQL-PH were: under 50 years old, with temporary contract, with less seniority in job, with work schedule at daytime of weekends, and those with academic degree. LQL-PH showed to be an instrument phsycometrically valid and reliable. It's recommendable to prove this scale in other public and private health institutions, as well as its relationship with key health care indicators of labour performance and management.

Investigating the Validity and Reliability of the Vanderbilt Assessment of Leadership in Education

ERIC Educational Resources Information Center

Porter, Andrew C.; Polikoff, Morgan S.; Goldring, Ellen B.; Murphy, Joseph; Elliott, Stephen N.; May, Henry

2010-01-01

The Vanderbilt Assessment of Leadership in Education (VAL-ED) is a multirater assessment of principals' learning-centered leadership. The instrument was developed based on the Standards for Educational and Psychological Testing. In this article, we report on the validity and reliability evidence for the VAL-ED accumulated in a national field…

Comparative Validity of the Shedler and Westen Assessment Procedure-200

ERIC Educational Resources Information Center

Mullins-Sweatt, Stephanie N.; Widiger, Thomas A.

2008-01-01

A predominant dimensional model of general personality structure is the five-factor model (FFM). Quite a number of alternative instruments have been developed to assess the domains of the FFM. The current study compares the validity of 2 alternative versions of the Shedler and Westen Assessment Procedure (SWAP-200) FFM scales, 1 that was developed…

Assessing the Culture of Residency Using the C - Change Resident Survey: Validity Evidence in 34 U.S. Residency Programs.

PubMed

Pololi, Linda H; Evans, Arthur T; Civian, Janet T; Shea, Sandy; Brennan, Robert T

2017-07-01

A practical instrument is needed to reliably measure the clinical learning environment and professionalism for residents. To develop and present evidence of validity of an instrument to assess the culture of residency programs and the clinical learning environment. During 2014-2015, we surveyed residents using the C - Change Resident Survey to assess residents' perceptions of the culture in their programs. Residents in all years of training in 34 programs in internal medicine, pediatrics, and general surgery in 14 geographically diverse public and private academic health systems. The C - Change Resident Survey assessed residents' perceptions of 13 dimensions of the culture: Vitality, Self-Efficacy, Institutional Support, Relationships/Inclusion, Values Alignment, Ethical/Moral Distress, Respect, Mentoring, Work-Life Integration, Gender Equity, Racial/Ethnic Minority Equity, and self-assessed Competencies. We measured the internal reliability of each of the 13 dimensions and evaluated response process, content validity, and construct-related evidence validity by assessing relationships predicted by our conceptual model and prior research. We also assessed whether the measurements were sensitive to differences in specialty and across institutions. A total of 1708 residents completed the survey [internal medicine: n = 956, pediatrics: n = 411, general surgery: n = 311 (51% women; 16% underrepresented in medicine minority)], with a response rate of 70% (range across programs, 51-87%). Internal consistency of each dimension was high (Cronbach α: 0.73-0.90). The instrument was able to detect significant differences in the learning environment across programs and sites. Evidence of validity was supported by a good response process and the demonstration of several relationships predicted by our conceptual model. The C - Change Resident Survey assesses the clinical learning environment for residents, and we encourage further study of validity in different contexts. Results could be used to facilitate and monitor improvements in the clinical learning environment and resident well-being.

The Validity of a New Structured Assessment of Gastrointestinal Symptoms Scale (SAGIS) for Evaluating Symptoms in the Clinical Setting.

PubMed

Koloski, N A; Jones, M; Hammer, J; von Wulffen, M; Shah, A; Hoelz, H; Kutyla, M; Burger, D; Martin, N; Gurusamy, S R; Talley, N J; Holtmann, G

2017-08-01

The clinical assessments of patients with gastrointestinal symptoms can be time-consuming, and the symptoms captured during the consultation may be influenced by a variety of patient and non-patient factors. To facilitate standardized symptom assessment in the routine clinical setting, we developed the Structured Assessment of Gastrointestinal Symptom (SAGIS) instrument to precisely characterize symptoms in a routine clinical setting. We aimed to validate SAGIS including its reliability, construct and discriminant validity, and utility in the clinical setting. Development of the SAGIS consisted of initial interviews with patients referred for the diagnostic work-up of digestive symptoms and relevant complaints identified. The final instrument consisted of 22 items as well as questions on extra intestinal symptoms and was given to 1120 consecutive patients attending a gastroenterology clinic randomly split into derivation (n = 596) and validation datasets (n = 551). Discriminant validity along with test-retest reliability was assessed. The time taken to perform a clinical assessment with and without the SAGIS was recorded along with doctor satisfaction with this tool. Exploratory factor analysis conducted on the derivation sample suggested five symptom constructs labeled as abdominal pain/discomfort (seven items), gastroesophageal reflux disease/regurgitation symptoms (four items), nausea/vomiting (three items), diarrhea/incontinence (five items), and difficult defecation and constipation (2 items). Confirmatory factor analysis conducted on the validation sample supported the initially developed five-factor measurement model ([Formula: see text], p < 0.0001, χ 2 /df = 4.6, CFI = 0.90, TLI = 0.88, RMSEA = 0.08). All symptom groups demonstrated differentiation between disease groups. The SAGIS was shown to be reliable over time and resulted in a 38% reduction of the time required for clinical assessment. The SAGIS instrument has excellent psychometric properties and supports the clinical assessment of and symptom-based categorization of patients with a wide spectrum of gastrointestinal symptoms.

Development and exploratory analysis of the Neurorehabilitation Program Styles Survey.

PubMed

McCorkel, Beth A; Glueckauf, Robert L; Ecklund-Johnson, Eric P; Tomusk, Allison B; Trexler, Lance E; Diller, Leonard

2003-01-01

To develop a survey instrument that assesses implementation of key components of outpatient neurorehabilitation programs and test the capacity of this instrument to differentiate between rehabilitation approaches. The Neurorehabilitation Program Styles Survey (NPSS) was administered to 18 outpatient facilities: 10 specialized and 8 discipline-specific outpatient neurorehabilitation programs. Scores were compared between types of programs using independent samples t tests. The NPSS showed good reliability and contrasted groups validity, significantly differentiating between types of programs. The NPSS holds considerable promise as a tool for distinguishing among different types of brain injury programs, and for assessing the differential effectiveness of specialized versus discipline-specific outpatient brain rehabilitation programs. Future research on the NPSS will assess the stability of the instrument over time, its content validity, and capacity to differentiate the full continuum of neurorehabilitation programs.

Evaluative frailty index for physical activity (EFIP): a reliable and valid instrument to measure changes in level of frailty.

PubMed

de Vries, Nienke M; Staal, J Bart; Olde Rikkert, Marcel G M; Nijhuis-van der Sanden, Maria W G

2013-04-01

Physical activity is assumed to be important in the prevention and treatment of frailty. It is unclear, however, to what extent frailty can be influenced because instruments designed to assess frailty have not been validated as evaluative outcome instruments in clinical practice. The aims of this study were: (1) to develop a frailty index (i.e., the evaluative frailty index for physical activity [EFIP]) based on the method of deficit accumulation and (2) to test the clinimetric properties of the EFIP. The content of the EFIP was determined using a written Delphi procedure. Intrarater reliability, interrater reliability, and construct validity were determined in an observational study (n=24). Intrarater reliability and interrater reliability were calculated using Cohen kappa and intraclass correlation coefficients (ICCs). Construct validity was determined by correlating the score on the EFIP with those on the timed "up & go" test (TUG), the performance-oriented mobility assessment (POMA), and the Cumulative Illness Rating Scale for Geriatrics (CIRS-G). Fifty items were included in the EFIP. Interrater reliability (Cohen kappa=0.72, ICC=.96) and intrarater reliability (Cohen kappa=0.77 and 0.80, ICC=.93 and .98) were good. As expected, a fair to moderate correlation with the TUG, POMA, and CIRS-G was found (.61, -.70, and .66, respectively). Reliability and validity of the EFIP have been tested in a small sample. These and other clinimetric properties, such as responsiveness, will be assessed or reassessed in a larger study population. The EFIP is a reliable and valid instrument to evaluate the effect of physical activity on frailty in research and in clinical practice.

Development of a radiology faculty appraisal instrument by using critical incident interviewing.

PubMed

Collins, J; Albanese, M A; Thakor, S K; Propeck, P A; Scanlan, K A

1997-12-01

To develop a valid and reliable radiology faculty appraisal instrument based on scientific methods. Fifteen radiology residents participated in critical incident interviewing. During a 1-hour interview, a resident was asked to describe five incidents each of effective and ineffective faculty behavior. Two investigators independently listened to the tape-recorded interviews, and two different investigators sorted the incidents into broad categories. A faculty appraisal instrument was developed by listing similar incidents under broad categories. A five-point rating scale was applied to each item. Content validity was assessed by resident and faculty critique of the appraisal instrument. A total of 168 incidents of faculty behavior were generated. The frequency with which similar incidents were reported was recorded. The most common behaviors reported were related to staff expertise and teaching. Interjudge reliability was good, as determined by computing K indices of agreement (overall K = 0.59). There was good agreement regarding instrument content validity among residents but not among faculty. Residents supported the use of the new appraisal instrument, but further tests of validity and reliability and faculty acceptance of the instrument will determine its usefulness as a tool for monitoring faculty teaching performance and making decisions regarding faculty promotion.

Translation, cross-cultural adaptation and validation of an HIV/AIDS knowledge and attitudinal instrument.

PubMed

Zometa, Carlos S; Dedrick, Robert; Knox, Michael D; Westhoff, Wayne; Siri, Rodrigo Simán; Debaldo, Ann

2007-06-01

An instrument developed in the United States by the Centers for Disease Control and Prevention to assess HIV/AIDS knowledge and four attitudinal dimensions (Peer Pressure, Abstinence, Drug Use, and Threat of HIV Infection) and an instrument developed by Basen-Engquist et al. (1999) to measure abstinence and condom use were translated, cross-culturally adapted, and validated for use with Spanish-speaking high school students in El Salvador. A back-translation of the English version was cross-culturally adapted using two different review panels and pilot-tested with Salvadorian students. An expert panel established content validity, and confirmatory factor analysis provided support for construct validity. Results indicated that the methodology was successful in cross-culturally adapting the instrument developed by the Centers for Disease Control and Prevention and the instrument developed by Basen-Engquist et al. The psychometric properties of the knowledge section were acceptable and there was partial support for the four-factor attitudinal model underlying the CDC instrument and the two-factor model underlying the Basen-Engquist et al. instrument. Additional studies with Spanish-speaking populations (either in the United States or Latin America) are needed to evaluate the generalizability of the present results.

Development of a clinical feeding assessment scale for very young infants in South Africa

PubMed Central

2016-01-01

Background There is a need for validated neonatal feeding assessment instruments in South Africa. A locally developed instrument may contribute to standardised evaluation procedures of high-risk neonates and address needs in resource constrained developing settings. Objective The aim of the study was to develop and validate the content of a clinical feeding assessment scale to diagnose oropharyngeal dysphagia (OPD) in neonates. Method The Neonatal Feeding Assessment Scale (NFAS) was developed using the Delphi method. Five international and South African speech-language therapists (SLTs) formed the expert panel, participating in two rounds of electronic questionnaires to develop and validate the content of the NFAS. Results All participants agreed on the need for the development of a valid clinical feeding assessment instrument to use with the neonatal population. The initial NFAS consisted of 240 items across 8 sections, and after the Delphi process was implemented, the final format was reduced to 211 items across 6 sections. The final format of the NFAS is scored using a binary scoring system guiding the clinician to diagnose the presence or absence of OPD. All members agreed on the format, the scoring system and the feeding constructs addressed in the revised final format of the NFAS. Conclusion The Delphi method and the diverse clinical and research experience of participants could be integrated to develop the NFAS which may be used in clinical practice in South Africa or similar developing contexts. Because of demographically different work settings marked by developed versus developing contexts, participants did not have the same expectations of a clinical dysphagia assessment. The international participants contributed to evidence-based content development. Local participants considered the contextual challenges of South African SLTs entering the field with basic competencies in neonatal dysphagia management, thereby justifying a comprehensive clinical instrument. The NFAS is aimed at clinicians working in Neonatal Intensive Care Units where they manage large caseloads of high-risk neonates. Further validation of the NFAS is recommended to determine its criterion validity in comparison with a widely accepted standard such as the modified barium swallow study. PMID:27796101

Valid and reliable authentic assessment of culminating student performance in the biomedical sciences.

PubMed

Oh, Deborah M; Kim, Joshua M; Garcia, Raymond E; Krilowicz, Beverly L

2005-06-01

There is increasing pressure, both from institutions central to the national scientific mission and from regional and national accrediting agencies, on natural sciences faculty to move beyond course examinations as measures of student performance and to instead develop and use reliable and valid authentic assessment measures for both individual courses and for degree-granting programs. We report here on a capstone course developed by two natural sciences departments, Biological Sciences and Chemistry/Biochemistry, which engages students in an important culminating experience, requiring synthesis of skills and knowledge developed throughout the program while providing the departments with important assessment information for use in program improvement. The student work products produced in the course, a written grant proposal, and an oral summary of the proposal, provide a rich source of data regarding student performance on an authentic assessment task. The validity and reliability of the instruments and the resulting student performance data were demonstrated by collaborative review by content experts and a variety of statistical measures of interrater reliability, including percentage agreement, intraclass correlations, and generalizability coefficients. The high interrater reliability reported when the assessment instruments were used for the first time by a group of external evaluators suggests that the assessment process and instruments reported here will be easily adopted by other natural science faculty.

A qualitative approach to assessing work ability.

PubMed

Tengland, Per-Anders

2013-01-01

We often need to be able to assess the extent to which individuals have or lack work ability. For this we need instruments. Most of the instruments available have flaws. They either lack validity or they use roundabout methods when collecting information about the individual's work ability. The aim of this paper is to present a conceptual model for constructing a questionnaire that can be used for assessing work ability. The methods used are philosophical, i.e. analytical and deductive. A conceptual theory is provided, and based on the resulting definition of the concept of "work ability" conclusions are drawn regarding how to assess work ability. When constructing quantitative instruments, we can increase validity through using a more strict definition of work ability. However, such an approach will only solve some of the problems noted above. The proposal is, instead, to create a qualitative questionnaire, founded on a definition of "work ability", which focuses on the concrete problems concerning the work ability of the individual. Finally, a sketch of such an instrument is provided, with questions covering all the relevant aspects of work ability. The qualitative questionnaire proposed is believed to be superior to more traditional (quantitative) instruments for assessing a person's work ability, as well as for finding solutions to her problems concerning work ability.

QNOTE: an instrument for measuring the quality of EHR clinical notes.

PubMed

Burke, Harry B; Hoang, Albert; Becher, Dorothy; Fontelo, Paul; Liu, Fang; Stephens, Mark; Pangaro, Louis N; Sessums, Laura L; O'Malley, Patrick; Baxi, Nancy S; Bunt, Christopher W; Capaldi, Vincent F; Chen, Julie M; Cooper, Barbara A; Djuric, David A; Hodge, Joshua A; Kane, Shawn; Magee, Charles; Makary, Zizette R; Mallory, Renee M; Miller, Thomas; Saperstein, Adam; Servey, Jessica; Gimbel, Ronald W

2014-01-01

The outpatient clinical note documents the clinician's information collection, problem assessment, and patient management, yet there is currently no validated instrument to measure the quality of the electronic clinical note. This study evaluated the validity of the QNOTE instrument, which assesses 12 elements in the clinical note, for measuring the quality of clinical notes. It also compared its performance with a global instrument that assesses the clinical note as a whole. Retrospective multicenter blinded study of the clinical notes of 100 outpatients with type 2 diabetes mellitus who had been seen in clinic on at least three occasions. The 300 notes were rated by eight general internal medicine and eight family medicine practicing physicians. The QNOTE instrument scored the quality of the note as the sum of a set of 12 note element scores, and its inter-rater agreement was measured by the intraclass correlation coefficient. The Global instrument scored the note in its entirety, and its inter-rater agreement was measured by the Fleiss κ. The overall QNOTE inter-rater agreement was 0.82 (CI 0.80 to 0.84), and its note quality score was 65 (CI 64 to 66). The Global inter-rater agreement was 0.24 (CI 0.19 to 0.29), and its note quality score was 52 (CI 49 to 55). The QNOTE quality scores were consistent, and the overall QNOTE score was significantly higher than the overall Global score (p=0.04). We found the QNOTE to be a valid instrument for evaluating the quality of electronic clinical notes, and its performance was superior to that of the Global instrument. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Patient Evaluation of Emotional Comfort Experienced (PEECE): developing and testing a measurement instrument

PubMed Central

Lester, L; Bulsara, C; Petterson, A; Bennett, K; Allen, E; Joske, D

2017-01-01

Objectives The Patient Evaluation of Emotional Comfort Experienced (PEECE) is a 12-item questionnaire which measures the mental well-being state of emotional comfort in patients. The instrument was developed using previous qualitative work and published literature. Design Instrument development. Setting Acute Care Public Hospital, Western Australia. Participants Sample of 374 patients. Interventions A multidisciplinary expert panel assessed the face and content validity of the instrument and following a pilot study, the psychometric properties of the instrument were explored. Main outcome measures Exploratory and confirmatory factor analysis assessed the underlying dimensions of the PEECE instrument; Cronbach's α was used to determine the reliability; κ was used for test–retest reliability of the ordinal items. Results 2 factors were identified in the instrument and named ‘positive emotions’ and ‘perceived meaning’. A greater proportion of male patients were found to report positive emotions compared with female patients. The instrument was found to be feasible, reliable and valid for use with inpatients and outpatients. Conclusions PEECE was found to be a feasible instrument for use with inpatient and outpatients, being easily understood and completed. Further psychometric testing is recommended. PMID:28122833

Developing an Acceptability Assessment of Preventive Dental Treatments

PubMed Central

Hyde, Susan; Gansky, Stuart A.; Gonzalez-Vargas, Maria J.; Husting, Sheila R.; Cheng, Nancy F.; Millstein, Susan G.; Adams, Sally H.

2012-01-01

Objectives Early childhood caries (ECC) is very prevalent among young Hispanic children. ECC is amenable to a variety of preventive procedures, yet many Hispanic families underutilize dental services. Acceptability research may assist in health care planning and resource allocation by identifying patient preferences among efficacious treatments with the goal of improving their utilization. The purposes of this study were (a) to develop a culturally competent acceptability assessment instrument, directed toward the caregivers of young Hispanic children, for five preventive dental treatments for ECC and (b) to test the instrument's reliability and validity. Methods An instrument of five standard treatments known to prevent ECC was developed, translated, reviewed by focus groups, and pilot tested, then tested for reliability. The instrument included illustrated cards, brief video clips, and samples of the treatments and was culturally appropriate for low-income Hispanic caregivers. In addition to determining the acceptability of the five treatments individually, the treatments were also presented as paired comparisons. Results Focus groups and debriefing interviews following the pilot tests established that the instrument has good face validity. The illustrated cards, product samples, and video demonstrations of the five treatments resulted in an instrument possessing good content validity. The instrument has good to excellent test–retest reliability, with identical time 1–time 2 responses for each of the five treatments 92 percent of the time (range 87 to 97 percent), and the same treatment of the paired comparisons preferred 75 percent of the time (range 61 to 90 percent). Conclusions The acceptability instrument described is reliable and valid and may be useful in program planning efforts to identify and increase the utilization of preferred ECC preventive treatments for target populations. PMID:18662256

Developing an acceptability assessment of preventive dental treatments.

PubMed

Hyde, Susan; Gansky, Stuart A; Gonzalez-Vargas, Maria J; Husting, Sheila R; Cheng, Nancy F; Millstein, Susan G; Adams, Sally H

2009-01-01

Early childhood caries (ECC) is very prevalent among young Hispanic children. ECC is amenable to a variety of preventive procedures, yet many Hispanic families underutilize dental services. Acceptability research may assist in health care planning and resource allocation by identifying patient preferences among efficacious treatments with the goal of improving their utilization. The purposes of this study were (a) to develop a culturally competent acceptability assessment instrument, directed toward the caregivers of young Hispanic children, for five preventive dental treatments for ECC and (b) to test the instrument's reliability and validity. An instrument of five standard treatments known to prevent ECC was developed, translated, reviewed by focus groups, and pilot tested, then tested for reliability The instrument included illustrated cards, brief video clips, and samples of the treatments and was culturally appropriate for low-income Hispanic caregivers. In addition to determining the acceptability of the five treatments individually, the treatments were also presented as paired comparisons. Focus groups and debriefing interviews following the pilot tests established that the instrument has good face validity. The illustrated cards, product samples, and video demonstrations of the five treatments resulted in an instrument possessing good content validity. The instrument has good to excellent test-retest reliability, with identical time 1-time 2 responses for each of the five treatments 92 percent of the time (range 87 to 97 percent), and the same treatment of the paired comparisons preferred 75 percent of the time (range 61 to 90 percent). The acceptability instrument described is reliable and valid and may be useful in program planning efforts to identify and increase the utilization of preferred ECC preventive treatments for target populations.

Development and analysis of an instrument to assess student understanding of GOB chemistry knowledge relevant to clinical nursing practice.

PubMed

Brown, Corina E; Hyslop, Richard M; Barbera, Jack

2015-01-01

The General, Organic, and Biological Chemistry Knowledge Assessment (GOB-CKA) is a multiple-choice instrument designed to assess students' understanding of the chemistry topics deemed important to clinical nursing practice. This manuscript describes the development process of the individual items along with a psychometric evaluation of the final version of the items and instrument. In developing items for the GOB-CKA, essential topics were identified through a series of expert interviews (with practicing nurses, nurse educators, and GOB chemistry instructors) and confirmed through a national survey. Individual items were tested in qualitative studies with students from the target population for clarity and wording. Data from pilot and beta studies were used to evaluate each item and narrow the total item count to 45. A psychometric analysis performed on data from the 45-item final version was used to provide evidence of validity and reliability. The final version of the instrument has a Cronbach's alpha value of 0.76. Feedback from an expert panel provided evidence of face and content validity. Convergent validity was estimated by comparing the results from the GOB-CKA with the General-Organic-Biochemistry Exam (Form 2007) of the American Chemical Society. Instructors who wish to use the GOB-CKA for teaching and research may contact the corresponding author for a copy of the instrument. © 2014 Wiley Periodicals, Inc.

Development of an Instrument to Assess Parent-College Child Communication Regarding Alcohol Use Behaviors

ERIC Educational Resources Information Center

Chaney, Beth H.; Cremeens, Jennifer

2013-01-01

Background: Past research suggests that parent-child communication can serve as protective factors to reduce alcohol misuse among college-aged children. Purpose: This article presents the methodology used and preliminary findings for developing and validating an instrument to assess parent-college student communication regarding alcohol use.…

In-Training Assessment Using Direct Observation of Single-Patient Encounters: A Literature Review

ERIC Educational Resources Information Center

Pelgrim, E. A. M.; Kramer, A. W. M.; Mokkink, H. G. A.; van den Elsen, L.; Grol, R. P. T. M.; van der Vleuten, C. P. M.

2011-01-01

We reviewed the literature on instruments for work-based assessment in single clinical encounters, such as the mini-clinical evaluation exercise (mini-CEX), and examined differences between these instruments in characteristics and feasibility, reliability, validity and educational effect. A PubMed search of the literature published before 8…

An Empirical Validation of the Instrument: Student Perceptions of Teaching Effectiveness.

ERIC Educational Resources Information Center

West, Sandra S.; Denton, Jon J.

An assessment of the Student Perception of Teaching Effectiveness instrument (SPTE) is presented. The SPTE was developed to assess teaching interns' performance from the student's viewpoint. The Texas Teacher Appraisal System was selected as the content base for the development of the SPTE. The Texas system covers instructional strategies,…

The Development and Validation of an Instrument to Monitor the Implementation of Social Constructivist Learning Environments in Grade 9 Science Classrooms in South Africa

ERIC Educational Resources Information Center

Luckay, Melanie B.; Laugksch, Rudiger C.

2015-01-01

This article describes the development and validation of an instrument that can be used to assess students' perceptions of their learning environment as a means of monitoring and guiding changes toward social constructivist learning environments. The study used a mixed-method approach with priority given to the quantitative data collection. During…

The Development and Validation of an Instrument to Assess Teacher Invitations and Teacher Effectiveness as Reported by Students in a Technical and General Post-Secondary Educational Setting.

ERIC Educational Resources Information Center

Inglis, Sandra Cheldelin

An instrument to measure student reports of perceived teacher invitations and of teacher behaviors traditionally considered effective was developed and validated. Invitational (I-type) factors and effective (E-type) factors were correlated with academic achievement. Scale items were suggested by the literature, and then rated and categorized by…

The Development of a Secondary-Level Solo Wind Instrument Performance Rubric Using the Multifaceted Rasch Partial Credit Measurement Model

ERIC Educational Resources Information Center

Wesolowski, Brian C.; Amend, Ross M.; Barnstead, Thomas S.; Edwards, Andrew S.; Everhart, Matthew; Goins, Quentin R.; Grogan, Robert J., III; Herceg, Amanda M.; Jenkins, S. Ira; Johns, Paul M.; McCarver, Christopher J.; Schaps, Robin E.; Sorrell, Gary W.; Williams, Jonathan D.

2017-01-01

The purpose of this study was to describe the development of a valid and reliable rubric to assess secondary-level solo instrumental music performance based on principles of invariant measurement. The research questions that guided this study included (1) What is the psychometric quality (i.e., validity, reliability, and precision) of a scale…

Translation, Validation, and Reliability of the Dutch Late-Life Function and Disability Instrument Computer Adaptive Test.

PubMed

Arensman, Remco M; Pisters, Martijn F; de Man-van Ginkel, Janneke M; Schuurmans, Marieke J; Jette, Alan M; de Bie, Rob A

2016-09-01

Adequate and user-friendly instruments for assessing physical function and disability in older adults are vital for estimating and predicting health care needs in clinical practice. The Late-Life Function and Disability Instrument Computer Adaptive Test (LLFDI-CAT) is a promising instrument for assessing physical function and disability in gerontology research and clinical practice. The aims of this study were: (1) to translate the LLFDI-CAT to the Dutch language and (2) to investigate its validity and reliability in a sample of older adults who spoke Dutch and dwelled in the community. For the assessment of validity of the LLFDI-CAT, a cross-sectional design was used. To assess reliability, measurement of the LLFDI-CAT was repeated in the same sample. The item bank of the LLFDI-CAT was translated with a forward-backward procedure. A sample of 54 older adults completed the LLFDI-CAT, World Health Organization Disability Assessment Schedule 2.0, RAND 36-Item Short-Form Health Survey physical functioning scale (10 items), and 10-Meter Walk Test. The LLFDI-CAT was repeated in 2 to 8 days (mean=4.5 days). Pearson's r and the intraclass correlation coefficient (ICC) (2,1) were calculated to assess validity, group-level reliability, and participant-level reliability. A correlation of .74 for the LLFDI-CAT function scale and the RAND 36-Item Short-Form Health Survey physical functioning scale (10 items) was found. The correlations of the LLFDI-CAT disability scale with the World Health Organization Disability Assessment Schedule 2.0 and the 10-Meter Walk Test were -.57 and -.53, respectively. The ICC (2,1) of the LLFDI-CAT function scale was .84, with a group-level reliability score of .85. The ICC (2,1) of the LLFDI-CAT disability scale was .76, with a group-level reliability score of .81. The high percentage of women in the study and the exclusion of older adults with recent joint replacement or hospitalization limit the generalizability of the results. The Dutch LLFDI-CAT showed strong validity and high reliability when used to assess physical function and disability in older adults dwelling in the community. © 2016 American Physical Therapy Association.

Schedule for personality assessment from notes and documents (SPAN-DOC): Preliminary validation, links to the ICD-11 classification of personality disorder, and use in eating disorders.

PubMed

Kim, Youl-Ri; Tyrer, Peter; Lee, Hong-Seock; Kim, Sung-Gon; Connan, Frances; Kinnaird, Emma; Olajide, Kike; Crawford, Mike

2016-05-01

The underlying core of personality is insufficiently assessed by any single instrument. This has led to the development of instruments adapted for written records in the assessment of personality disorder. To test the construct validity and inter-rater reliability of a new personality assessment method. This study (four parts) assessed the construct validity of the Schedule for Personality Assessment from Notes and Documents (SPAN-DOC), a dimensional assessment from clinical records. We examined inter-rater reliability using case vignettes (Part 1) and convergent validity in three ways: by comparison with NEO Five-Factor Inventory in 130 Korean patients (Part 2), with agreed ICD-11 personality severity levels in two populations (Part 3) and determining its use in assessing the personality status in 90 British patients with eating disorders (Part 4). Internal consistency (alpha = .90) and inter-rater reliability (intraclass correlation coefficient ≥ .88) were satisfactory. Each factor in the five-factor model of personality was correlated with conceptually valid SPAN-DOC variables. The SPAN-DOC domain traits in those with eating disorders were categorized into 3 clusters: self-aggrandisement, emotionally unstable, and anxious/dependent. This study provides preliminary support for the usefulness of SPAN-DOC in the assessment of personality disorder. Copyright © 2016 John Wiley & Sons, Ltd. Copyright © 2016 John Wiley & Sons, Ltd.

Validity of instruments to assess students' travel and pedestrian safety

PubMed Central

2010-01-01

Background Safe Routes to School (SRTS) programs are designed to make walking and bicycling to school safe and accessible for children. Despite their growing popularity, few validated measures exist for assessing important outcomes such as type of student transport or pedestrian safety behaviors. This research validated the SRTS school travel survey and a pedestrian safety behavior checklist. Methods Fourth grade students completed a brief written survey on how they got to school that day with set responses. Test-retest reliability was obtained 3-4 hours apart. Convergent validity of the SRTS travel survey was assessed by comparison to parents' report. For the measure of pedestrian safety behavior, 10 research assistants observed 29 students at a school intersection for completion of 8 selected pedestrian safety behaviors. Reliability was determined in two ways: correlations between the research assistants' ratings to that of the Principal Investigator (PI) and intraclass correlations (ICC) across research assistant ratings. Results The SRTS travel survey had high test-retest reliability (κ = 0.97, n = 96, p < 0.001) and convergent validity (κ = 0.87, n = 81, p < 0.001). The pedestrian safety behavior checklist had moderate reliability across research assistants' ratings (ICC = 0.48) and moderate correlation with the PI (r = 0.55, p =< 0.01). When two raters simultaneously used the instrument, the ICC increased to 0.65. Overall percent agreement (91%), sensitivity (85%) and specificity (83%) were acceptable. Conclusions These validated instruments can be used to assess SRTS programs. The pedestrian safety behavior checklist may benefit from further formative work. PMID:20482778

Development and initial validation of the Parental PELICAN Questionnaire (PaPEQu)--an instrument to assess parental experiences and needs during their child's end-of-life care.

PubMed

Zimmermann, Karin; Cignacco, Eva; Eskola, Katri; Engberg, Sandra; Ramelet, Anne-Sylvie; Von der Weid, Nicolas; Bergstraesser, Eva

2015-12-01

To develop and test the Parental PELICAN Questionnaire, an instrument to retrospectively assess parental experiences and needs during their child's end-of-life care. To offer appropriate care for dying children, healthcare professionals need to understand the illness experience from the family perspective. A questionnaire specific to the end-of-life experiences and needs of parents losing a child is needed to evaluate the perceived quality of paediatric end-of-life care. This is an instrument development study applying mixed methods based on recommendations for questionnaire design and validation. The Parental PELICAN Questionnaire was developed in four phases between August 2012-March 2014: phase 1: item generation; phase 2: validity testing; phase 3: translation; phase 4: pilot testing. Psychometric properties were assessed after applying the Parental PELICAN Questionnaire in a sample of 224 bereaved parents in April 2014. Validity testing covered the evidence based on tests of content, internal structure and relations to other variables. The Parental PELICAN Questionnaire consists of approximately 90 items in four slightly different versions accounting for particularities of the four diagnostic groups. The questionnaire's items were structured according to six quality domains described in the literature. Evidence of initial validity and reliability could be demonstrated with the involvement of healthcare professionals and bereaved parents. The Parental PELICAN Questionnaire holds promise as a measure to assess parental experiences and needs and is applicable to a broad range of paediatric specialties and settings. Future validation is needed to evaluate its suitability in different cultures. © 2015 John Wiley & Sons Ltd.

Development and validation of Dutch version of Lasater Clinical Judgment Rubric in hospital practice: An instrument design study.

PubMed

Vreugdenhil, Jettie; Spek, Bea

2018-03-01

Clinical reasoning in patient care is a skill that cannot be observed directly. So far, no reliable, valid instrument exists for the assessment of nursing students' clinical reasoning skills in hospital practice. Lasater's clinical judgment rubric (LCJR), based on Tanner's model "Thinking like a nurse" has been tested, mainly in academic simulation settings. The aim is to develop a Dutch version of the LCJR (D-LCJR) and to test its psychometric properties when used in a hospital traineeship context. A mixed-model approach was used to develop and to validate the instrument. Ten dedicated educational units in a university hospital. A well-mixed group of 52 nursing students, nurse coaches and nurse educators. A Delphi panel developed the D-LCJR. Students' clinical reasoning skills were assessed "live" by nurse coaches, nurse educators and students who rated themselves. The psychometric properties tested during the assessment process are reliability, reproducibility, content validity and construct validity by testing two hypothesis: 1) a positive correlation between assessed and self-reported sum scores (convergent validity) and 2) a linear relation between experience and sum score (clinical validity). The obtained D-LCJR was found to be internally consistent, Cronbach's alpha 0.93. The rubric is also reproducible with intraclass correlations between 0.69 and 0.78. Experts judged it to be content valid. The two hypothesis were both tested significant, supporting evidence for construct validity. The translated and modified LCJR, is a promising tool for the evaluation of nursing students' development in clinical reasoning in hospital traineeships, by students, nurse coaches and nurse educators. More evidence on construct validity is necessary, in particular for students at the end of their hospital traineeship. Based on our research, the D-LCJR applied in hospital traineeships is a usable and reliable tool. Copyright © 2017 Elsevier Ltd. All rights reserved.

Design and validation of the Health Professionals' Attitudes Toward the Homeless Inventory (HPATHI).

PubMed

Buck, David S; Monteiro, F Marconi; Kneuper, Suzanne; Rochon, Donna; Clark, Dana L; Melillo, Allegra; Volk, Robert J

2005-01-10

Recent literature has called for humanistic care of patients and for medical schools to begin incorporating humanism into medical education. To assess the attitudes of health-care professionals toward homeless patients and to demonstrate how those attitudes might impact optimal care, we developed and validated a new survey instrument, the Health Professional Attitudes Toward the Homeless Inventory (HPATHI). An instrument that measures providers' attitudes toward the homeless could offer meaningful information for the design and implementation of educational activities that foster more compassionate homeless health care. Our intention was to describe the process of designing and validating the new instrument and to discuss the usefulness of the instrument for assessing the impact of educational experiences that involve working directly with the homeless on the attitudes, interest, and confidence of medical students and other health-care professionals. The study consisted of three phases: identifying items for the instrument; pilot testing the initial instrument with a group of 72 third-year medical students; and modifying and administering the instrument in its revised form to 160 health-care professionals and third-year medical students. The instrument was analyzed for reliability and validity throughout the process. A 19-item version of the HPATHI had good internal consistency with a Cronbach's alpha of 0.88 and a test-retest reliability coefficient of 0.69. The HPATHI showed good concurrent validity, and respondents with more than one year of experience with homeless patients scored significantly higher than did those with less experience. Factor analysis yielded three subscales: Personal Advocacy, Social Advocacy, and Cynicism. The HPATHI demonstrated strong reliability for the total scale and satisfactory test-retest reliability. Extreme group comparisons suggested that experience with the homeless rather than medical training itself could affect health-care professionals' attitudes toward the homeless. This could have implications for the evaluation of medical school curricula.

Cultural adaptation and psychometric properties of the family questionnaire in a Brazilian sample of relatives of schizophrenia outpatients.

PubMed

Zanetti, Ana C G; Wiedemann, Georg; Dantas, Rosana A S; Hayashida, Miyeko; de Azevedo-Marques, João M; Galera, Sueli A F

2013-06-01

To evaluate the internal reliability and validity of the Brazilian Portuguese version of the Family Questionnaire among families of schizophrenia outpatients. The main studies about the family environment of schizophrenia patients are related to the concept of Expressed Emotion. There is currently no instrument to evaluate this concept in Brazil that is easily applicable and comparable with studies from other countries. Methodological and cross-sectional research design. A convenience sample of 130 relatives of schizophrenia outpatients was selected. The translation and cultural adaptation of the instrument involved experts in mental health and experts in the German language and included back translation, semantic evaluation of items and pretesting of the instrument with 30 relatives of schizophrenia outpatients. The psychometric properties of the instrument were studied with another 100 relatives, which fulfilled the requirements for the Brazilian Portuguese version of the instrument. The psychometric properties of the instrument were assessed by construct validity (using an analysis of its key components, comparisons between distinct groups-convergent validity with the Antonovsky's Sense of Coherence Scale) and reliability (checking the internal consistency of its items and its test-retest reproducibility). The analysis of main components confirmed dimensionality patterns that were comparable between the original and adapted versions. In two domains of the instrument, critical comments and emotional over-involvement had moderate and significant correlations, respectively, with Antonovsky's Sense of Coherence Scale, appropriate values of Cronbach's alpha and strong and significant correlations, respectively, in test-retest reproducibility. We observed significant differences between distinct groups of parents in the category of emotional over-involvement. We conclude that the Portuguese-adapted version of the Family Questionnaire is valid and reliable for the study sample. This study provided evidence that the Family Questionnaire is a reliable and valid instrument for assessing expressed emotion. It is easy and practical to use and is acceptable for use in a Brazilian cultural population. © 2012 Blackwell Publishing Ltd.

The Development and Validation of an Instrument to Measure Preservice Teachers' Self-Efficacy in Regard to The Teaching of Science as Inquiry

NASA Astrophysics Data System (ADS)

Dira Smolleck, Lori; Zembal-Saul, Carla; Yoder, Edgar P.

2006-06-01

The purpose of this study was to develop, validate, and establish the reliability of an instrument that measures preservice teachers' self-efficacy in regard to the teaching of science as inquiry. The instrument, Teaching Science as Inquiry (TSI), is based upon the work of Bandura (1977, 1981, 1982, 1986, 1989, 1995, 1997), Riggs (1988), and Enochs and Riggs (1990). Self-efficacy in regard to the teaching of science as inquiry was measured through the use of a 69-item Likert-type scale instrument designed by the author of the study. Based on the standardized development processes used and the associated evidence, the TSI appears to be a content and construct valid instrument with high internal reliability for use with preservice elementary teachers to assess self-efficacy beliefs in regard to the teaching of science as inquiry.

The Development and Validation of an Instrument to Measure Preservice Teachers' Self-Efficacy in Regard to The Teaching of Science as Inquiry

NASA Astrophysics Data System (ADS)

Smolleck, Lori Dira; Zembal-Saul, Carla; Yoder, Edgar P.

2006-06-01

The purpose of this study was to develop, validate, and establish the reliability of an instrument that measures preservice teachers' self-efficacy in regard to the teaching of science as inquiry. The instrument, Teaching Science as Inquiry (TSI), is based upon the work of Bandura (1977, 1981, 1982, 1986, 1989, 1995, 1997), Riggs (1988), and Enochs and Riggs (1990). Self-efficacy in regard to the teaching of science as inquiry was measured through the use of a 69-item Likert-type scale instrument designed by the author of the study. Based on the standardized development processes used and the associated evidence, the TSI appears to be a content and construct valid instrument with high internal reliability for use with preservice elementary teachers to assess self-efficacy beliefs in regard to the teaching of science as inquiry.

Using the Rasch measurement model to design a report writing assessment instrument.

PubMed

Carlson, Wayne R

2013-01-01

This paper describes how the Rasch measurement model was used to develop an assessment instrument designed to measure student ability to write law enforcement incident and investigative reports. The ability to write reports is a requirement of all law enforcement recruits in the state of Michigan and is a part of the state's mandatory basic training curriculum, which is promulgated by the Michigan Commission on Law Enforcement Standards (MCOLES). Recently, MCOLES conducted research to modernize its training and testing in the area of report writing. A structured validation process was used, which included: a) an examination of the job tasks of a patrol officer, b) input from content experts, c) a review of the professional research, and d) the creation of an instrument to measure student competency. The Rasch model addressed several measurement principles that were central to construct validity, which were particularly useful for assessing student performances. Based on the results of the report writing validation project, the state established a legitimate connectivity between the report writing standard and the essential job functions of a patrol officer in Michigan. The project also produced an authentic instrument for measuring minimum levels of report writing competency, which generated results that are valid for inferences of student ability. Ultimately, the state of Michigan must ensure the safety of its citizens by licensing only those patrol officers who possess a minimum level of core competency. Maintaining the validity and reliability of both the training and testing processes can ensure that the system for producing such candidates functions as intended.

Quality of life assessment in cosmetics: specificity and interest of the international BeautyQol instrument.

PubMed

Beresniak, Ariel; Auray, Jean-Paul; Duru, Gérard; Aractingi, Selim; Krueger, Gerald G; Talarico, Sergio; Tsutani, Kiichiro; Dupont, Danielle; de Linares, Yolaine

2015-09-01

The wide use of cosmetics and their perceived benefits upon well-being imply objective descriptions of their effects upon the different dimensions contributing to the quality of life (QoL). Such a goal pleas for using relevant and validated scientific instruments with robust measurement methods. This paper discusses the interest of the new validated questionnaire BeautyQoL specifically designed to assess the effect of cosmetic products on physical appearance and QoL. After conducting a review of skin appearance and QoL, three phases of the international codevelopment have been carried out in the following sequence: semi-directed interviews (Phase 1), acceptability study (Phase 2), and validation study (Phase 3). Data collection and validation process have been carried out in 16 languages. This review confirms that QoL instruments developed in dermatology are not suitable to assess cosmetic products, mainly because of their lack of sensitivity. General acceptability of BeautyQol was very good. Forty-two questions have been structured in five dimensions that explained 76.7% of the total variance: Social Life, Self-confidence, Mood, Vitality, and Attractiveness. Cronbach's alpha coefficients are between 0.932 and 0.978, confirming the good internal consistency of the results. The BeautyQol questionnaire is the first international instrument specific to cosmetic products and physical appearance that has been validated in 16 languages and could be used in a number of clinical trials and descriptive studies to demonstrate the added value of these products on the QoL. © 2015 Wiley Periodicals, Inc.

Physical activity measurement instruments for children with cerebral palsy: a systematic review.

PubMed

Capio, Catherine M; Sit, Cindy H P; Abernethy, Bruce; Rotor, Esmerita R

2010-10-01

this paper is a systematic review of physical activity measurement instruments for field-based studies involving children with cerebral palsy (CP). database searches using PubMed Central, MEDLINE, CINAHL Plus, PsycINFO, EMBASE, Cochrane Library, and PEDro located 12 research papers, identifying seven instruments that met the inclusion criteria of (1) having been developed for children aged 0 to 18 years, (2) having been used to evaluate a physical activity dimension, and (3) having been used in a field-based study involving children with CP. The instruments reviewed were the Activities Scale for Kids - Performance version (ASKp), the Canada Fitness Survey, the Children's Assessment of Participation and Enjoyment/Preferences for Activities of Children (CAPE/PAC), the Compendium of Physical Activities, the Physical Activity Questionnaire - Adolescents (PAQ-A), StepWatch, and the Uptimer. Second-round searches yielded 11 more papers, providing reliability and validity evidence for the instruments. the instruments measure physical activity frequency, mode, domain, and duration. Although most instruments demonstrated adequate reliability and validity, only the ASKp and CAPE/PAC have established reliability and validity for children with physical disabilities; the Uptimer has established concurrent validity. No instrument measuring intensity in free-living has been validated or found reliable for children with CP. the findings suggest that further studies are needed to examine the methodological properties of physical activity measurement in children with CP. Combining subjective and objective instruments is recommended to achieve better understanding of physical activity participation.

Development and Psychometric Properties of a Standardized Assessment for Adults Who Are Deaf-Blind

ERIC Educational Resources Information Center

Dalby, Dawn M.; Hirdes, John P.; Stolee, Paul; Strong, J. Graham; Poss, Jeff; Tjam, Erin Y.; Bowman, Lindsay; Ashworth, Melody

2009-01-01

The internal consistency and validity of the interRAI Community Health Assessment and Deafblind Supplement were tested with 182 persons with deaf-blindness. All subscales demonstrated good to excellent internal consistency, and expected associations provided evidence of convergent validity. This instrument can facilitate standardized service…

The Measure of Adolescent Heterosocial Competence: Development and Initial Validation

ERIC Educational Resources Information Center

Grover, Rachel L.; Nangle, Douglas W.; Zeff, Karen R.

2005-01-01

We developed and began construct validation of the Measure of Adolescent Heterosocial Competence (MAHC), a self-report instrument assessing the ability to negotiate effectively a range of challenging other-sex social interactions. Development followed the Goldfried and D'Zurilla (1969) behavioral-analytic model for assessing competence.…

Measuring Emotions in Students' Learning and Performance: The Achievement Emotions Questionnaire (AEQ)

ERIC Educational Resources Information Center

Pekrun, Reinhard; Goetz, Thomas; Frenzel, Anne C.; Barchfeld, Petra; Perry, Raymond P.

2011-01-01

Aside from test anxiety scales, measurement instruments assessing students' achievement emotions are largely lacking. This article reports on the construction, reliability, internal validity, and external validity of the Achievement Emotions Questionnaire (AEQ) which is designed to assess various achievement emotions experienced by students in…

Behavioral and Emotional Rating Scale: Two Studies of Convergent Validity

ERIC Educational Resources Information Center

Trout, Alexandra L.; Ryan, Joseph B.; La Vigne, Steven P.; Epstein, Michael H.

2003-01-01

As the field of mental health services shifts its focus to early intervention, the need to develop valid and reliable measures of young children's behavioral functioning is clear. Traditional assessment instruments have focused on deficits, problems, and pathologies to the exclusion of strengths and competencies. However, assessing child strengths…

Assessing Meritorious Teacher Performance: A Differential Validity Study.

ERIC Educational Resources Information Center

Ellett, Chad D; Capie, William

The Teacher Assessment and Development System (TADS) - Meritorious Teacher Program (MTP) FORM instrument is used in the Dade County Public Schools, Miami, Florida, to evaluate teachers. Its validity for decisions concerning merit pay for master teachers was examined in this study. Specifically, its ability to discriminate between high performing…

Instruments and Scoring Guide of the Experiential Education Evaluation Project.

ERIC Educational Resources Information Center

Conrad, Dan; Hedin, Diane

As a result of the Experiential Education Evaluation Project the publication identifies instruments used to measure and assess experiential learning programs. The following information is given for each instrument: rationale for its inclusion in the study; precise issues or outcomes designed to measure, validity and reliability data; and…

Illness Adaptation: Clarifying the Concept and Validating a Scale.

ERIC Educational Resources Information Center

Young, Rosalie F.; Kahana, Eva

Traditionally, coping and adaptation have been considered synonymous in individual's responses to illness and other stressful situations. The Illness Adaptation Scale (IAS) is a 12-item instrument which was designed to assess adaptational outcomes in illness situations as well as four coping modes (instrumental-self oriented, instrumental-other…

The cross-cultural equivalence of participation instruments: a systematic review.

PubMed

Stevelink, S A M; van Brakel, W H

2013-07-01

Concepts such as health-related quality of life, disability and participation may differ across cultures. Consequently, when assessing such a concept using a measure developed elsewhere, it is important to test its cultural equivalence. Previous research suggested a lack of cultural equivalence testing in several areas of measurement. This paper reviews the process of cross-cultural equivalence testing of instruments to measure participation in society. An existing cultural equivalence framework was adapted and used to assess participation instruments on five categories of equivalence: conceptual, item, semantic, measurement and operational equivalence. For each category, several aspects were rated, resulting in an overall category rating of 'minimal/none', 'partial' or 'extensive'. The best possible overall study rating was five 'extensive' ratings. Articles were included if the instruments focussed explicitly on measuring 'participation' and were theoretically grounded in the ICIDH(-2) or ICF. Cross-validation articles were only included if it concerned an adaptation of an instrument developed in a high or middle-income country to a low-income country or vice versa. Eight cross-cultural validation studies were included in which five participation instruments were tested (Impact on Participation and Autonomy, London Handicap Scale, Perceived Impact and Problem Profile, Craig Handicap Assessment Reporting Technique, Participation Scale). Of these eight studies, only three received at least two 'extensive' ratings for the different categories of equivalence. The majority of the cultural equivalence ratings given were 'partial' and 'minimal/none'. The majority of the 'none/minimal' ratings were given for item and measurement equivalence. The cross-cultural equivalence testing of the participation instruments included leaves much to be desired. A detailed checklist is proposed for designing a cross-validation study. Once a study has been conducted, the checklist can be used to ensure comprehensive reporting of the validation (equivalence) testing process and its results. • Participation instruments are often used in a different cultural setting than initial developed for. • The conceptualization of participation may vary across cultures. Therefore, cultural equivalence – the extent to which an instrument is equally suitable for use in two or more cultures – is an important concept to address. • This review showed that the process of cultural equivalence testing of the included participation instruments was often addressed insufficiently. • Clinicians should be aware that application of participations instruments in a different culture than initially developed for needs prior testing of cultural validity in the next context.

Empirically Examining the Risk of Intimate Partner Violence: The Revised Domestic Violence Screening Instrument (DVSI-R)

PubMed Central

Grant, Stephen R

2006-01-01

SYNOPSIS Objective This study extends recent research on assessing the risk of intimate partner violence by determining the concurrent and predictive validity of a revised version of the Domestic Violence Screening Instrument (DVSI-R) and whether evidence of such validity is sustained independent of perpetrator demographic characteristics and forms of intimate violence. The analyses highlight violent incidents involving multiple victims as an indicator of “severe” violence. Previous research did not address these issues. Methods Data were analyzed on 14,970 assessments conducted in the State of Connecticut from September 1, 2004 through May 2, 2005. Hierarchical regression and receiver operating characteristic analyses were used to address the objectives of this research. Results The empirical findings support the concurrent and predictive validity of the DVSI-R and show that it is robust in its applicability. The findings further show that incidents involving multiple victims are highly associated with DVSI-R risk scores and recidivistic violence. Conclusion Validating and demonstrating the robustness of a risk assessment instrument is only a first step in preventing violence involving intimate partners or others in family or family-like relationships. The challenge is to train professionals responsible for addressing the problem of such violence to link valid risk assessments to well-crafted strategies of supervision and treatment so that the victimized or other potential victims are protected and perpetrators are held accountable for their actions. PMID:16827441

Translation and validation of the assistive technology device predisposition assessment in Greek in order to assess satisfaction with use of the selected assistive device.

PubMed

Koumpouros, Yiannis; Papageorgiou, Effie; Karavasili, Alexandra; Alexopoulou, Despoina

2017-07-01

To examine the Assistive Technology Device Predisposition Assessment scale and provide evidence of validity and reliability of the Greek version. We translated and adapted the original instrument in Greek according to the most well-known guidelines recommendations. Field test studies were conducted in a rehabilitation hospital to validate the appropriateness of the final results. Ratings of the different items were statistically analyzed. We recruited 115 subjects who were administered the Form E of the original questionnaire. The experimental analysis conducted revealed a three subscales structure: (i) Adaptability, (ii) Fit to Use, and (iii) Socializing. According to the results of our study the three subscales measure different constructs. Reliability measures (ICC = 0.981, Pearson's correlation = 0.963, Cronbach's α = 0.701) yielded high values. Test-retest outcome showed great stability. This is the first study, at least to the knowledge of the authors, which focuses merely on measuring the satisfaction of the users from the used assistive device, while exploring the Assistive Technology Device Predisposition Assessment - Device Form in such depth. According to the results, it is a stable, valid and reliable instrument and applicable to the Greek population. Thus, it can be used to measure the satisfaction of patients with assistive devices. Implications for Rehabilitation The paper explores the cultural adaptability and applicability of ATD PA - Device Form. ATD PA - Device Form can be used to assess user satisfaction by the selected assistive device. ATD PA - Device Form is a valid and reliable instrument in measuring users' satisfaction in Greekreality.

Instrument for assessing mobile technology acceptability in diabetes self-management: a validation and reliability study.

PubMed

Frandes, Mirela; Deiac, Anca V; Timar, Bogdan; Lungeanu, Diana

2017-01-01

Nowadays, mobile technologies are part of everyday life, but the lack of instruments to assess their acceptability for the management of chronic diseases makes their actual adoption for this purpose slow. The objective of this study was to develop a survey instrument for assessing patients' attitude toward and intention to use mobile technology for diabetes mellitus (DM) self-management, as well as to identify sociodemographic characteristics and quality of life factors that affect them. We first conducted the documentation and instrument design phases, which were subsequently followed by the pilot study and instrument validation. Afterward, the instrument was administered 103 patients (median age: 37 years; range: 18-65 years) diagnosed with type 1 or type 2 DM, who accepted to participate in the study. The reliability and construct validity were assessed by computing Cronbach's alpha and using factor analysis, respectively. The instrument included statements about the actual use of electronic devices for DM management, interaction between patient and physician, attitude toward using mobile technology, and quality of life evaluation. Cronbach's alpha was 0.9 for attitude toward using mobile technology and 0.97 for attitude toward using mobile device applications for DM self-management. Younger patients (Spearman's ρ =-0.429; P <0.001) with better glycemic control (Spearman's ρ =-0.322; P <0.001) and higher education level (Kendall's τ =0.51; P <0.001) had significantly more favorable attitude toward using mobile assistive applications for DM control. Moreover, patients with a higher quality of life presented a significantly more positive attitude toward using modern technology (Spearman's ρ =0.466; P <0.001). The instrument showed good reliability and internal consistency, making it suitable for measuring the acceptability of mobile technology for DM self-management. Additionally, we found that even if most of the patients showed positive attitude toward mobile applications, only a moderate level of intention to indeed use them was observed. Moreover, the study indicated that barriers were truthfulness and easiness to use.

Measuring Nurses' Value, Implementation, and Knowledge of Evidence-Based Practice: Further Psychometric Testing of the Quick-EBP-VIK Survey.

PubMed

Connor, Linda; Paul, Fiona; McCabe, Margaret; Ziniel, Sonja

2017-02-01

The Quick-EBP-VIK is a new instrument for measuring nurses' value, implementation, and knowledge of EBP. Psychometric testing was conducted in two parts. Part 1 describes the tool development and validity testing which resulted in the development of a 25-item survey after receiving ≥0.80 Item-Level Content Validity Index for both clarity and relevance. Part 2 describes psychometric testing was necessary to assess additional types of validity and reliability. The purpose of this paper is to further describe the psychometric testing of the Quick-EBP-VIK survey instrument. This descriptive study was designed to assess test-retest reliability, internal consistency and construct validity via a web-based survey. The survey instrument was e-mailed to all nurses at the study hospital. Nurses who responded to the first survey (Wave 1) received another e-mail invitation to complete the survey instrument again (Wave 2) for the purpose of assessing the test-retest reliability of the instrument. A total of 1,177 deliverable e-mails were sent to all nursing staff at one free standing pediatric hospital with Magnet ® designation in the northeast. A total of 382 nurses returned completed surveys, indicating a 32.5% response rate for Wave 1. A total of 131 nurses responded to Wave 2 indicating a response rate of 34.3%. The intraclass correlation coefficients for the items included in the final instrument ranged from 0.43 to 0.80 and were deemed sufficient. These represent a sufficient intraclass correlation coefficient. The Cronbach's Alpha values for each of the three domains are all higher than 0.7 indicating that the items of each of the measurement dimension are internally consistent. However, the composite reliability of the third domain was slightly lower than 0.7 when using Raykov's Rho. The Quick-EBP-VIK instrument has gone through rigorous comprehensive testing and has demonstrated good psychometric properties. © 2016 Sigma Theta Tau International.

Development and Validation of a Practical Instrument for Injury Prevention: The Occupational Safety and Health Monitoring and Assessment Tool (OSH-MAT).

PubMed

Sun, Yi; Arning, Martin; Bochmann, Frank; Börger, Jutta; Heitmann, Thomas

2018-06-01

The Occupational Safety and Health Monitoring and Assessment Tool (OSH-MAT) is a practical instrument that is currently used in the German woodworking and metalworking industries to monitor safety conditions at workplaces. The 12-item scoring system has three subscales rating technical, organizational, and personnel-related conditions in a company. Each item has a rating value ranging from 1 to 9, with higher values indicating higher standard of safety conditions. The reliability of this instrument was evaluated in a cross-sectional survey among 128 companies and its validity among 30,514 companies. The inter-rater reliability of the instrument was examined independently and simultaneously by two well-trained safety engineers. Agreement between the double ratings was quantified by the intraclass correlation coefficient and absolute agreement of the rating values. The content validity of the OSH-MAT was evaluated by quantifying the association between OSH-MAT values and 5-year average injury rates by Poisson regression analysis adjusted for the size of the companies and industrial sectors. The construct validity of OSH-MAT was examined by principle component factor analysis. Our analysis indicated good to very good inter-rater reliability (intraclass correlation coefficient = 0.64-0.74) of OSH-MAT values with an absolute agreement of between 72% and 81%. Factor analysis identified three component subscales that met exactly the structure theory of this instrument. The Poisson regression analysis demonstrated a statistically significant exposure-response relationship between OSH-MAT values and the 5-year average injury rates. These analyses indicate that OSH-MAT is a valid and reliable instrument that can be used effectively to monitor safety conditions at workplaces.

Psychometric evaluation of a new instrument to measure disease self-management of the early stage chronic kidney disease patients.

PubMed

Lin, Chiu-Chu; Wu, Chia-Chen; Wu, Li-Min; Chen, Hsing-Mei; Chang, Shu-Chen

2013-04-01

This study aims to develop a valid and reliable chronic kidney disease self-management instrument (CKD-SM) for assessing early stage chronic kidney disease patients' self-management behaviours. Enhancing early stage chronic kidney disease patients' self-management plays a key role in delaying the progression of chronic kidney disease. Healthcare provider understanding of early stage chronic kidney disease patients' self-management behaviours can help develop effective interventions. A valid and reliable instrument for measuring chronic kidney disease patients' self-management behaviours is needed. A cross-sectional descriptive study collected data for principal components analysis with oblique rotation. Mandarin- or Taiwanese-speaking adults with chronic kidney disease (n=252) from two medical centres and one regional hospital in Southern Taiwan completed the CKD-SM. Construct validity was evaluated by exploratory factor analysis. Internal consistency and test-retest reliability were estimated by Cronbach's alpha and Pearson correlation coefficients. Four factors were extracted and labelled self-integration, problem-solving, seeking social support and adherence to recommended regimen. The four factors accounted for 60.51% of the total variance. Each factor showed acceptable internal reliability with Cronbach's alpha from 0.77-0.92. The test-retest correlations for the CKD-SM was 0.72. The psychometric quality of the CKD-SM instrument was satisfactory. Research to conduct a confirmatory factor analysis to further validate this new instrument's construct validity is recommended. The CKD-SM instrument is useful for clinicians who wish to identify the problems with self-management among chronic kidney disease patients early. Self-management assessment will be helpful to develop intervention tailored to the needs of the chronic kidney disease population. © 2013 Blackwell Publishing Ltd.

Assessing Reliability and Validity of the "GroPromo" Audit Tool for Evaluation of Grocery Store Marketing and Promotional Environments

ERIC Educational Resources Information Center

Kerr, Jacqueline; Sallis, James F.; Bromby, Erica; Glanz, Karen

2012-01-01

Objective: To evaluate reliability and validity of a new tool for assessing the placement and promotional environment in grocery stores. Methods: Trained observers used the "GroPromo" instrument in 40 stores to code the placement of 7 products in 9 locations within a store, along with other promotional characteristics. To test construct validity,…

Using wound care algorithms: a content validation study.

PubMed

Beitz, J M; van Rijswijk, L

1999-09-01

Valid and reliable heuristic devices facilitating optimal wound care are lacking. The objectives of this study were to establish content validation data for a set of wound care algorithms, to identify their associated strengths and weaknesses, and to gain insight into the wound care decision-making process. Forty-four registered nurse wound care experts were surveyed and interviewed at national and regional educational meetings. Using a cross-sectional study design and an 83-item, 4-point Likert-type scale, this purposive sample was asked to quantify the degree of validity of the algorithms' decisions and components. Participants' comments were tape-recorded, transcribed, and themes were derived. On a scale of 1 to 4, the mean score of the entire instrument was 3.47 (SD +/- 0.87), the instrument's Content Validity Index was 0.86, and the individual Content Validity Index of 34 of 44 participants was > 0.8. Item scores were lower for those related to packing deep wounds (P < .001). No other significant differences were observed. Qualitative data analysis revealed themes of difficulty associated with wound assessment and care issues, that is, the absence of valid and reliable definitions. The wound care algorithms studied proved valid. However, the lack of valid and reliable wound assessment and care definitions hinders optimal use of these instruments. Further research documenting their clinical use is warranted. Research-based practice recommendations should direct the development of future valid and reliable algorithms designed to help nurses provide optimal wound care.

Guideline for translation and national validation of the Quality of Life in Hand Eczema Questionnaire (QOLHEQ).

PubMed

Oosterhaven, Jart A F; Schuttelaar, Marie L A; Apfelbacher, Christian; Diepgen, Thomas L; Ofenloch, Robert F

2017-08-01

There is a need for well-developed and validated questionnaires to measure patient reported outcomes. The Quality of Life in Hand Eczema Questionnaire (QOLHEQ) is such a validated instrument measuring disease-specific health-related quality of life in hand eczema patients. A re-validation of measurement properties is required before an instrument is used in a new population. With the objective of arriving at a guideline for translation and national validation of the QOLHEQ, we have developed the design of a reference study on how to adequately assess measurement properties of the QOLHEQ based on interdisciplinary discussions and current standards. We present a step-by-step guideline to assess translation (including cross-cultural adaptation), scale structure, validity, reproducibility, responsiveness, and interpretability. We describe which outcomes should be reported for each measurement property, and give advice on how to calculate these. It is also specified which sample size is needed, how to deal with missing data, and which cutoff values should be applied for the measurement properties assessed during the validation process. In conclusion, this guideline, presenting a reference validation study for the QOLHEQ, creates the possibility to harmonize the national validation of the various language versions of the QOLHEQ. © 2017 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

Cross-cultural French adaptation and validation of the Impact On Family Scale (IOFS).

PubMed

Boudas, Raphaël; Jégu, Jérémie; Grollemund, Bruno; Quentel, Elvire; Danion-Grilliat, Anne; Velten, Michel

2013-04-23

The IOFS (Impact On Family Scale) questionnaire is a useful instrument to assess the impact of chronic childhood conditions on general family quality of life. As this instrument was not validated in French, we proposed to translate, adapt and validate the IOFS questionnaire for clinical and research use in French-speaking populations. The sample studied comprised French-speaking parents with a child presenting a cleft lip or cleft lip and palate, aged 6 to 12 years and treated in the University Hospital of Strasbourg, France. The 15-item version of the IOFS was translated into French and then sent to the parents by post. The structure of the measure was studied using Exploratory Factor Analysis (EFA), internal consistency was assessed using Cronbach's alpha coefficient and test-retest reliability was studied by calculating the Intraclass Correlation Coefficient (ICC). The French version of the IOFS questionnaire exhibited very good psychometric properties. For practitioners, this instrument will facilitate the assessment of the impact of chronic childhood conditions on quality of life among French-speaking families.

Validation of VARK learning modalities questionnaire using Rasch analysis

NASA Astrophysics Data System (ADS)

Fitkov-Norris, E. D.; Yeghiazarian, A.

2015-02-01

This article discusses the application of Rasch analysis to assess the internal validity of a four sub-scale VARK (Visual, Auditory, Read/Write and Kinaesthetic) learning styles instrument. The results from the analysis show that the Rasch model fits the majority of the VARK questionnaire data and the sample data support the internal validity of the four sub-constructs at 1% level of significance for all but one item. While this suggests that the instrument could potentially be used as a predictor for a person's learning preference orientation, further analysis is necessary to confirm the invariability of the instrument across different user groups across factors such as gender, age, educational and cultural background.

Developing and validating an instrument for measuring mobile computing self-efficacy.

PubMed

Wang, Yi-Shun; Wang, Hsiu-Yuan

2008-08-01

IT-related self-efficacy has been found to have a critical influence on system use. However, traditional measures of computer self-efficacy and Internet-related self-efficacy are perceived to be inapplicable in the context of mobile computing and commerce because they are targeted primarily at either desktop computer or wire-based technology contexts. Based on previous research, this study develops and validates a multidimensional instrument for measuring mobile computing self-efficacy (MCSE). This empirically validated instrument will be useful to researchers in developing and testing the theories of mobile user behavior, and to practitioners in assessing the mobile computing self-efficacy of users and promoting the use of mobile commerce systems.

Development and validation of an instrument to assess perceived social influence on health behaviors

PubMed Central

HOLT, CHERYL L.; CLARK, EDDIE M.; ROTH, DAVID L.; CROWTHER, MARTHA; KOHLER, CONNIE; FOUAD, MONA; FOUSHEE, RUSTY; LEE, PATRICIA A.; SOUTHWARD, PENNY L.

2012-01-01

Assessment of social influence on health behavior is often approached through a situational context. The current study adapted an existing, theory-based instrument from another content domain to assess Perceived Social Influence on Health Behavior (PSI-HB) among African Americans, using an individual difference approach. The adapted instrument was found to have high internal reliability (α = .81–.84) and acceptable testretest reliability (r = .68–.85). A measurement model revealed a three-factor structure and supported the theoretical underpinnings. Scores were predictive of health behaviors, particularly among women. Future research using the new instrument may have applied value assessing social influence in the context of health interventions. PMID:20522506

Frailty Assessment in Heart Failure: an Overview of the Multi-domain Approach.

PubMed

McDonagh, Julee; Ferguson, Caleb; Newton, Phillip J

2018-02-01

The study aims (1) to provide a contemporary description of frailty assessment in heart failure and (2) to provide an overview of multi-domain frailty assessment in heart failure. Frailty assessment is an important predictive measure for mortality and hospitalisation in individuals with heart failure. To date, there are no frailty assessment instruments validated for use in heart failure. This has resulted in significant heterogeneity between studies regarding the assessment of frailty. The most common frailty assessment instrument used in heart failure is the Frailty Phenotype which focuses on five physical domains of frailty; the appropriateness a purely physical measure of frailty in individuals with heart failure who frequently experience decreased exercise tolerance and shortness of breath is yet to be determined. A limited number of studies have approached frailty assessment using a multi-domain view which may be more clinically relevant in heart failure. There remains a lack of consensus regarding frailty assessment and an absence of a validated instrument in heart failure. Despite this, frailty continues to be assessed frequently, primarily for research purposes, using predominantly physical frailty measures. A more multidimensional view of frailty assessment using a multi-domain approach will likely be more sensitive to identifying at risk patients.

Validation of cross-cultural child mental health and psychosocial research instruments: adapting the Depression Self-Rating Scale and Child PTSD Symptom Scale in Nepal

PubMed Central

2011-01-01

Background The lack of culturally adapted and validated instruments for child mental health and psychosocial support in low and middle-income countries is a barrier to assessing prevalence of mental health problems, evaluating interventions, and determining program cost-effectiveness. Alternative procedures are needed to validate instruments in these settings. Methods Six criteria are proposed to evaluate cross-cultural validity of child mental health instruments: (i) purpose of instrument, (ii) construct measured, (iii) contents of construct, (iv) local idioms employed, (v) structure of response sets, and (vi) comparison with other measurable phenomena. These criteria are applied to transcultural translation and alternative validation for the Depression Self-Rating Scale (DSRS) and Child PTSD Symptom Scale (CPSS) in Nepal, which recently suffered a decade of war including conscription of child soldiers and widespread displacement of youth. Transcultural translation was conducted with Nepali mental health professionals and six focus groups with children (n = 64) aged 11-15 years old. Because of the lack of child mental health professionals in Nepal, a psychosocial counselor performed an alternative validation procedure using psychosocial functioning as a criterion for intervention. The validation sample was 162 children (11-14 years old). The Kiddie-Schedule for Affective Disorders and Schizophrenia (K-SADS) and Global Assessment of Psychosocial Disability (GAPD) were used to derive indication for treatment as the external criterion. Results The instruments displayed moderate to good psychometric properties: DSRS (area under the curve (AUC) = 0.82, sensitivity = 0.71, specificity = 0.81, cutoff score ≥ 14); CPSS (AUC = 0.77, sensitivity = 0.68, specificity = 0.73, cutoff score ≥ 20). The DSRS items with significant discriminant validity were "having energy to complete daily activities" (DSRS.7), "feeling that life is not worth living" (DSRS.10), and "feeling lonely" (DSRS.15). The CPSS items with significant discriminant validity were nightmares (CPSS.2), flashbacks (CPSS.3), traumatic amnesia (CPSS.8), feelings of a foreshortened future (CPSS.12), and easily irritated at small matters (CPSS.14). Conclusions Transcultural translation and alternative validation feasibly can be performed in low clinical resource settings through task-shifting the validation process to trained mental health paraprofessionals using structured interviews. This process is helpful to evaluate cost-effectiveness of psychosocial interventions. PMID:21816045

Validity and reliability of an instrument for assessing case analyses in bioengineering ethics education.

PubMed

Goldin, Ilya M; Pinkus, Rosa Lynn; Ashley, Kevin

2015-06-01

Assessment in ethics education faces a challenge. From the perspectives of teachers, students, and third-party evaluators like the Accreditation Board for Engineering and Technology and the National Institutes of Health, assessment of student performance is essential. Because of the complexity of ethical case analysis, however, it is difficult to formulate assessment criteria, and to recognize when students fulfill them. Improvement in students' moral reasoning skills can serve as the focus of assessment. In previous work, Rosa Lynn Pinkus and Claire Gloeckner developed a novel instrument for assessing moral reasoning skills in bioengineering ethics. In this paper, we compare that approach to existing assessment techniques, and evaluate its validity and reliability. We find that it is sensitive to knowledge gain and that independent coders agree on how to apply it.

Measuring leprosy-related stigma - a pilot study to validate a toolkit of instruments.

PubMed

Rensen, Carin; Bandyopadhyay, Sudhakar; Gopal, Pala K; Van Brakel, Wim H

2011-01-01

Stigma negatively affects the quality of life of leprosy-affected people. Instruments are needed to assess levels of stigma and to monitor and evaluate stigma reduction interventions. We conducted a validation study of such instruments in Tamil Nadu and West Bengal, India. Four instruments were tested in a 'Community Based Rehabilitation' (CBR) setting, the Participation Scale, Internalised Scale of Mental Illness (ISMI) adapted for leprosy-affected persons, Explanatory Model Interview Catalogue (EMIC) for leprosy-affected and non-affected persons and the General Self-Efficacy (GSE) Scale. We evaluated the following components of validity, construct validity, internal consistency, test-retest reproducibility and reliability to distinguish between groups. Construct validity was tested by correlating instrument scores and by triangulating quantitative and qualitative findings. Reliability was evaluated by comparing levels of stigma among people affected by leprosy and community controls, and among affected people living in CBR project areas and those in non-CBR areas. For the Participation, ISMI and EMIC scores significant differences were observed between those affected by leprosy and those not affected (p = 0.0001), and between affected persons in the CBR and Control group (p < 0.05). The internal consistency of the instruments measured with Cronbach's α ranged from 0.83 to 0.96 and was very good for all instruments. Test-retest reproducibility coefficients were 0.80 for the Participation score, 0.70 for the EMIC score, 0.62 for the ISMI score and 0.50 for the GSE score. The construct validity of all instruments was confirmed. The Participation and EMIC Scales met all validity criteria, but test-retest reproducibility of the ISMI and GSE Scales needs further evaluation with a shorter test-retest interval and longer training and additional adaptations for the latter.

Development of a Standardized Kalamazoo Communication Skills Assessment Tool for Radiologists: Validation, Multisource Reliability, and Lessons Learned.

PubMed

Brown, Stephen D; Rider, Elizabeth A; Jamieson, Katherine; Meyer, Elaine C; Callahan, Michael J; DeBenedectis, Carolynn M; Bixby, Sarah D; Walters, Michele; Forman, Sara F; Varrin, Pamela H; Forbes, Peter; Roussin, Christopher J

2017-08-01

The purpose of this study was to develop and test a standardized communication skills assessment instrument for radiology. The Delphi method was used to validate the Kalamazoo Communication Skills Assessment instrument for radiology by revising and achieving consensus on the 43 items of the preexisting instrument among an interdisciplinary team of experts consisting of five radiologists and four nonradiologists (two men, seven women). Reviewers assessed the applicability of the instrument to evaluation of conversations between radiology trainees and trained actors portraying concerned parents in enactments about bad news, radiation risks, and diagnostic errors that were video recorded during a communication workshop. Interrater reliability was assessed by use of the revised instrument to rate a series of enactments between trainees and actors video recorded in a hospital-based simulator center. Eight raters evaluated each of seven different video-recorded interactions between physicians and parent-actors. The final instrument contained 43 items. After three review rounds, 42 of 43 (98%) items had an average rating of relevant or very relevant for bad news conversations. All items were rated as relevant or very relevant for conversations about error disclosure and radiation risk. Reliability and rater agreement measures were moderate. The intraclass correlation coefficient range was 0.07-0.58; mean, 0.30; SD, 0.13; and median, 0.30. The range of weighted kappa values was 0.03-0.47; mean, 0.23; SD, 0.12; and median, 0.22. Ratings varied significantly among conversations (χ 2 6 = 1186; p < 0.0001) and varied significantly by viewing order, rater type, and rater sex. The adapted communication skills assessment instrument is highly relevant for radiology, having moderate interrater reliability. These findings have important implications for assessing the relational competencies of radiology trainees.

Translation, validation, and cultural adaptation of the Rhinosinusitis Disability Index and the Chronic Sinusitis Survey into Arabic.

PubMed

Aldrees, Turki; Almubarak, Zaid; Hassouneh, Basil; Albosaily, Ahamed; Aloulah, Mohammad; Almasoud, Mai; Alsaleh, Saad

2018-01-01

Disease-specific quality of life instruments assess the impact of chronic rhinosinusitis on patients' quality of life (QoL). To the extent of our knowledge, there are no Arabic versions of two instruments-the Rhinosinusitis Disability Index (RSDI) and the Chronic Sinusitis Survey (CSS). Develop an Arabic-validated version of both instruments, thus allowing its use among the Arabic population. Prospective cross-sectional study for instrument validation. Tertiary university hospital. This study was conducted between September 2015 and October 2016. We followed the international comprehensive guidelines for translation and cross-cultural adaptation of QoL instruments. Test-retest reliability, discriminant validity, and responsiveness ability of both the RSDI and CSS Arabic versions. 124. The sample comprised 75 patients diagnosed with chronic rhinosinusitis and 49 healthy control subjects. The Arabic version of both instruments showed high internal consistency (Cronbach's alpha: RSDI=0.97, CSS=.88) and the ability to differentiate between diseased and healthy volunteers (P less than .0001). The translated versions also detected significant change in response to an intervention (P less than .0001). These Arabic validated versions of the RSDI and CSS can be used for both clinical and research purposes. This study was performed in only one tertiary hospital. None.

Development and Validation of an Instrument Measuring Theory-Based Determinants of Monitoring Obesogenic Behaviors of Pre-Schoolers among Hispanic Mothers.

PubMed

Branscum, Paul; Lora, Karina R

2016-06-02

Public health interventions are greatly needed for obesity prevention, and planning for such strategies should include community participation. The study's purpose was to develop and validate a theory-based instrument with low-income, Hispanic mothers of preschoolers, to assess theory-based determinants of maternal monitoring of child's consumption of fruits and vegetables and sugar-sweetened beverages (SSB). Nine focus groups with mothers were conducted to determine nutrition-related behaviors that mothers found as most obesogenic for their children. Next, behaviors were operationally defined and rated for importance and changeability. Two behaviors were selected for investigation (fruits and vegetable and SSB). Twenty semi-structured interviews with mothers were conducted next to develop culturally appropriate items for the instrument. Afterwards, face and content validity were established using a panel of six experts. Finally, the instrument was tested with a sample of 238 mothers. Psychometric properties evaluated included construct validity (using the maximum likelihood extraction method of factor analysis), and internal consistency reliability (Cronbach's alpha). Results suggested that all scales on the instrument were valid and reliable, except for the autonomy scales. Researchers and community planners working with Hispanic families can use this instrument to measure theory-based determinants of parenting behaviors related to preschoolers' consumption of fruits and vegetables, and SSB.

[Assessment of Functioning when Conducting Occupational Capacity Evaluations--What is "Evidence-Based"?].

PubMed

Canela, Carlos; Schleifer, Roman; Dube, Anish; Hengartner, Michael P; Ebner, Gerhard; Seifritz, Erich; Liebrenz, Michael

2016-03-01

Occupational capacity evaluations have previously been subject to criticism for lacking in quality and consistency. To the authors' knowledge, there is no clear consensus on the best way to formally assess functioning within capacity evaluations. In this review we investigated different instruments that are used to assess functioning in occupational capacity evaluations. Systematic review of the literature. Though several instruments that assess functional capacity were found in our search, a specific validated instrument assessing occupational capacity as part of a larger psychiatric evaluation was not found. The limitations of the existing instruments on assessing functional capacity are discussed. Medical experts relying on instruments to conduct functional capacity evaluations should be cognizant of their limitations. The findings call for the development and use of an instrument specifically designed to assess the functional and occupational capacity of psychiatric patients, which is also likely to improve the quality of these reports. © Georg Thieme Verlag KG Stuttgart · New York.

Validation of an instrument to evaluate health promotion at schools

PubMed Central

Pinto, Raquel Oliveira; Pattussi, Marcos Pascoal; Fontoura, Larissa do Prado; Poletto, Simone; Grapiglia, Valenca Lemes; Balbinot, Alexandre Didó; Teixeira, Vanessa Andina; Horta, Rogério Lessa

2016-01-01

ABSTRACT OBJECTIVE To validate an instrument designed to assess health promotion in the school environment. METHODS A questionnaire, based on guidelines from the World Health Organization and in line with the Brazilian school health context, was developed to validate the research instrument. There were 60 items in the instrument that included 40 questions for the school manager and 20 items with direct observations made by the interviewer. The items’ content validation was performed using the Delphi technique, with the instrument being applied in 53 schools from two medium-sized cities in the South region of Brazil. Reliability (Cronbach’s alpha and split-half) and validity (principal component analysis) analyses were performed. RESULTS The final instrument remained composed of 28 items, distributed into three dimensions: pedagogical, structural and relational. The resulting components showed good factorial loads (> 0.4) and acceptable reliability (> 0.6) for most items. The pedagogical dimension identifies educational activities regarding drugs and sexuality, violence and prejudice, auto care and peace and quality of life. The structural dimension is comprised of access, sanitary structure, and conservation and equipment. The relational dimension includes relationships within the school and with the community. CONCLUSIONS The proposed instrument presents satisfactory validity and reliability values, which include aspects relevant to promote health in schools. Its use allows the description of the health promotion conditions to which students from each educational institution are exposed. Because this instrument includes items directly observed by the investigator, it should only be used during periods when there are full and regular activities at the school in question. PMID:26982958

Validity aspects of the patient feedback questionnaire on consultation skills (PFC), a promising learning instrument in medical education.

PubMed

Reinders, Marcel E; Blankenstein, Annette H; Knol, Dirk L; de Vet, Henrica C W; van Marwijk, Harm W J

2009-08-01

A focus on the communicator competency is considered to be an important requirement to help physicians to acquire consultation skills. A feedback questionnaire, in which patients assess consultation skills might be a useful learning tool. An existing questionnaire on patient perception of patient-centeredness (PPPC) was adapted to cover the 'communicator' items in the competency profile. We assessed the face and content validity, the construct validity and the internal consistency of this new patient feedback on consultation skills (PFC) questionnaire. We assessed the face validity of the PFC by interviewing patients and general practice trainees (GPTs) during the developmental process. The content validity was determined by experts (n=10). First-year GPTs (23) collected 222 PFCs, from which the data were used to assess the construct validity (factor analysis), internal consistency, response rates and ceiling effects. The PFC adequately covers the corresponding 'communicator' competency (face and content validity). Factor analysis showed a one-dimensional construct. The internal consistency was high (Cronbach's alpha 0.89). For the single items, the response rate varied from 89.2% to 100%; the maximum score (ceiling effect) varied from 45.5% to 89.2%. The PFC appears to be a valid, internally consistent instrument. The PFC may be a valuable learning tool with which GPTs, other physicians and medical students can acquire feedback from patients regarding their consultation skills.

Validation of the "Quality of Life in School" instrument in Canadian elementary school students.

PubMed

Ghotra, Satvinder; McIsaac, Jessie-Lee D; Kirk, Sara F L; Kuhle, Stefan

2016-01-01

Background. School is an integral component of the life of a child, and thus quality of school life is an important part of the overall quality of life experienced by a child. There are a few instruments available to measure the quality of school life but they are often not available in English, or they are not appropriate for use alongside other instruments in a survey of young children. The Quality of Life in School (QoLS) instrument is a short, self-report measure to assess elementary school students' perception of their quality of school life in four domains. The instrument was developed in Israel and has been validated among Hebrew-speaking children. The aim of the current study was to evaluate the psychometric properties of the QoLS measure in Canadian elementary school children. Methods. A total of 629 children attending grades 4-6 were recruited in a population-based cross-sectional study. The QoLS measure was administered to participating children by trained research assistants. In addition, their socio-demographic details and academic data were also obtained. The psychometric testing included exploratory factor analysis and reliability estimation using internal consistency (Cronbach's Alpha). Construct validity was investigated using the known groups comparisons for discriminative validity and via convergent validity. Results. A four-factor structure was generated explaining 39% of the total variance in the model. The results showed good internal consistency and acceptable floor and ceiling effects. Cronbach's Alpha ranged from 0.75 to 0.93. Known groups comparisons showed that the QoLS measure discriminated well between subgroups on the basis of gender, grade, and academic achievement, thus providing evidence of construct validity. The convergent validity was also appropriate with all the four domains demonstrating moderate to strong correlations to each other and to the total QoLS score. Conclusions. QoLS appears to be a valid and reliable measure for quality of school life assessment in young Canadian children.

Validation of a combined health literacy and numeracy instrument for patients with type 2 diabetes.

PubMed

Luo, Huabin; Patil, Shivajirao P; Wu, Qiang; Bell, Ronny A; Cummings, Doyle M; Adams, Alyssa D; Hambidge, Bertha; Craven, Kay; Gao, Fei

2018-05-20

This study aimed to validate a new consolidated measure of health literacy and numeracy (health literacy scale [HLS] plus the subjective numeracy scale [SNS]) in patients with type 2 diabetes (T2DM). A convenience sample (N = 102) of patients with T2DM was recruited from an academic family medicine center in the southeastern US between September-December 2017. Participants completed a questionnaire that included the composite HLS/SNS (22 questions) and a commonly used objective measure of health literacy-S-TOFHLA (40 questions). Internal reliability of the HLS/SNS was assessed using Cronbach's alpha. Criterion and construct validity was assessed against the S-TOFHLA. The composite HLS/SNS had good internal reliability (Cronbach's alpha = 0.83). A confirmatory factor analysis revealed there were four factors in the new instrument. Model fit indices showed good model-data fit (RMSEA = 0.08). The Spearman's rank order correlation coefficient between the HLS/SNS and the S-TOFHLA was 0.45 (p < 0.01). Our study suggests that the composite HLS/SNS is a reliable, valid instrument. Published by Elsevier B.V.

Validation of a psychometric instrument to assess motivation in veterinary bachelor students.

PubMed

Vandeweerd, Jean-Michel; Dugdale, Alexandra; Romainville, Marc

2014-01-01

There are indications that motivation correlates with better performance for those studying veterinary medicine. To assess objectively whether motivation profiles influence both veterinary students' attitudes towards educational interventions and their academic success and whether changes in curriculum can affect students' motivation, there is need for an instrument that can provide a valid measurement of the strength of motivation for the study of veterinary medicine. Our objectives were to design and validate a questionnaire that can be used as a psychometric scale to capture the motivation profiles of veterinary students. Question items were obtained from semi-structured interviews with students and from a review of the relevant literature. Each item was scored on a 5-point scale. The preliminary instrument was trialed on a cohort of 450 students. Responses were subjected to reliability and principal component analysis. A 14-item scale was designed, within which two factors explained 53.4% of the variance among the items. The scale had good face, content, and construct validities as well as a good internal consistency (Cronbach's alpha=.88).

The Shedler-Westen Assessment Procedure (SWAP): Evaluating Psychometric Questions about Its Reliability, Validity, and Impact of Its Fixed Score Distribution

ERIC Educational Resources Information Center

Blagov, Pavel S.; Bi, Wu; Shedler, Jonathan; Westen, Drew

2012-01-01

The Shedler-Westen Assessment Procedure (SWAP) is a personality assessment instrument designed for use by expert clinical assessors. Critics have raised questions about its psychometrics, most notably its validity across observers and situations, the impact of its fixed score distribution on research findings, and its test-retest reliability. We…

Measuring Classroom Assessment Practice Using Instructional Artifacts: A Validation Study of the QAS Notebook

ERIC Educational Resources Information Center

Martinez, Jose Felipe; Borko, Hilda; Stecher, Brian; Luskin, Rebecca; Kloser, Matt

2012-01-01

We report the results of a pilot validation study of the Quality Assessment in Science Notebook, a portfolio-like instrument for measuring teacher assessment practices in middle school science classrooms. A statewide sample of 42 teachers collected 2 notebooks during the school year, corresponding to science topics taught in the fall and spring.…

Investigating the Quality of the School Technology Needs Assessment (STNA) 3.0: A Validity and Reliability Study

ERIC Educational Resources Information Center

Corn, Jenifer O.

2010-01-01

Schools and districts should use a well-designed needs assessment to inform important decisions about a range of technology program areas. Presently, there is a lack of valid and reliable instruments available and accessible to schools to effectively assess their educational needs to better design and evaluate their projects and initiatives. The…

Validity and reliability of a self-report instrument to assess social support and physical environmental correlates of physical activity in adolescents

PubMed Central

2012-01-01

Background The purpose of this study was to examine the internal consistency, test-retest reliability, construct validity and predictive validity of a new German self-report instrument to assess the influence of social support and the physical environment on physical activity in adolescents. Methods Based on theoretical consideration, the short scales on social support and physical environment were developed and cross-validated in two independent study samples of 9 to 17 year-old girls and boys. The longitudinal sample of Study I (n = 196) was recruited from a German comprehensive school, and subjects in this study completed the questionnaire twice with a between-test interval of seven days. Cronbach’s alphas were computed to determine the internal consistency of the factors. Test-retest reliability of the latent factors was assessed using intra-class coefficients. Factorial validity of the scales was assessed using principle components analysis. Construct validity was determined using a cross-validation technique by performing confirmatory factor analysis with the independent nationwide cross-sectional sample of Study II (n = 430). Correlations between factors and three measures of physical activity (objectively measured moderate-to-vigorous physical activity (MVPA), self-reported habitual MVPA and self-reported recent MVPA) were calculated to determine the predictive validity of the instrument. Results Construct validity of the social support scale (two factors: parental support and peer support) and the physical environment scale (four factors: convenience, public recreation facilities, safety and private sport providers) was shown. Both scales had moderate test-retest reliability. The factors of the social support scale also had good internal consistency and predictive validity. Internal consistency and predictive validity of the physical environment scale were low to acceptable. Conclusions The results of this study indicate moderate to good reliability and construct validity of the social support scale and physical environment scale. Predictive validity was only confirmed for the social support scale but not for the physical environment scale. Hence, it remains unclear if a person’s physical environment has a direct or an indirect effect on physical activity behavior or a moderation function. PMID:22928865

Validity and reliability of a self-report instrument to assess social support and physical environmental correlates of physical activity in adolescents.

PubMed

Reimers, Anne K; Jekauc, Darko; Mess, Filip; Mewes, Nadine; Woll, Alexander

2012-08-29

The purpose of this study was to examine the internal consistency, test-retest reliability, construct validity and predictive validity of a new German self-report instrument to assess the influence of social support and the physical environment on physical activity in adolescents. Based on theoretical consideration, the short scales on social support and physical environment were developed and cross-validated in two independent study samples of 9 to 17 year-old girls and boys. The longitudinal sample of Study I (n = 196) was recruited from a German comprehensive school, and subjects in this study completed the questionnaire twice with a between-test interval of seven days. Cronbach's alphas were computed to determine the internal consistency of the factors. Test-retest reliability of the latent factors was assessed using intra-class coefficients. Factorial validity of the scales was assessed using principle components analysis. Construct validity was determined using a cross-validation technique by performing confirmatory factor analysis with the independent nationwide cross-sectional sample of Study II (n = 430). Correlations between factors and three measures of physical activity (objectively measured moderate-to-vigorous physical activity (MVPA), self-reported habitual MVPA and self-reported recent MVPA) were calculated to determine the predictive validity of the instrument. Construct validity of the social support scale (two factors: parental support and peer support) and the physical environment scale (four factors: convenience, public recreation facilities, safety and private sport providers) was shown. Both scales had moderate test-retest reliability. The factors of the social support scale also had good internal consistency and predictive validity. Internal consistency and predictive validity of the physical environment scale were low to acceptable. The results of this study indicate moderate to good reliability and construct validity of the social support scale and physical environment scale. Predictive validity was only confirmed for the social support scale but not for the physical environment scale. Hence, it remains unclear if a person's physical environment has a direct or an indirect effect on physical activity behavior or a moderation function.

Brief screening for co-occurring disorders among women entering substance abuse treatment.

PubMed

Lincoln, Alisa K; Liebschutz, Jane M; Chernoff, Miriam; Nguyen, Dana; Amaro, Hortensia

2006-09-07

Despite the importance of identifying co-occurring psychiatric disorders in substance abuse treatment programs, there are few appropriate and validated instruments available to substance abuse treatment staff to conduct brief screen for these conditions. This paper describes the development, implementation and validation of a brief screening instrument for mental health diagnoses and trauma among a diverse sample of Black, Hispanic and White women in substance abuse treatment. With input from clinicians and consumers, we adapted longer existing validated instruments into a 14 question screen covering demographics, mental health symptoms and physical and sexual violence exposure. All women entering treatment (methadone, residential and out-patient) at five treatment sites were screened at intake (N = 374). Eighty nine percent reported a history of interpersonal violence, and 70% reported a history of sexual assault. Eighty-eight percent reported mental health symptoms in the last 30 days. The screening questions administered to 88 female clients were validated against in-depth psychiatric diagnostic assessments by trained mental health clinicians. We estimated measures of predictive validity, including sensitivity, specificity and predictive values positive and negative. Screening items were examined multiple ways to assess utility. The screen is a useful and valid proxy for PTSD but not for other mental illness. Substance abuse treatment programs should incorporate violence exposure questions into clinical use as a matter of policy. More work is needed to develop brief screening tools measures for front-line treatment staff to accurately assess other mental health needs of women entering substance abuse treatment.

Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews.

PubMed

Shea, Beverley J; Grimshaw, Jeremy M; Wells, George A; Boers, Maarten; Andersson, Neil; Hamel, Candyce; Porter, Ashley C; Tugwell, Peter; Moher, David; Bouter, Lex M

2007-02-15

Our objective was to develop an instrument to assess the methodological quality of systematic reviews, building upon previous tools, empirical evidence and expert consensus. A 37-item assessment tool was formed by combining 1) the enhanced Overview Quality Assessment Questionnaire (OQAQ), 2) a checklist created by Sacks, and 3) three additional items recently judged to be of methodological importance. This tool was applied to 99 paper-based and 52 electronic systematic reviews. Exploratory factor analysis was used to identify underlying components. The results were considered by methodological experts using a nominal group technique aimed at item reduction and design of an assessment tool with face and content validity. The factor analysis identified 11 components. From each component, one item was selected by the nominal group. The resulting instrument was judged to have face and content validity. A measurement tool for the 'assessment of multiple systematic reviews' (AMSTAR) was developed. The tool consists of 11 items and has good face and content validity for measuring the methodological quality of systematic reviews. Additional studies are needed with a focus on the reproducibility and construct validity of AMSTAR, before strong recommendations can be made on its use.

Validation of online psychometric instruments for common mental health disorders: a systematic review.

PubMed

van Ballegooijen, Wouter; Riper, Heleen; Cuijpers, Pim; van Oppen, Patricia; Smit, Johannes H

2016-02-25

Online questionnaires for measuring common mental health disorders such as depression and anxiety disorders are increasingly used. The psychometrics of several pen-and-paper questionnaires have been re-examined for online use and new online instruments have been developed and tested for validity as well. This study aims to review and synthesise the literature on this subject and provide a framework for future research. We searched Medline and PsycINFO for psychometric studies on online instruments for common mental health disorders and extracted the psychometric data. Studies were coded and assessed for quality by independent raters. We included 56 studies on 62 online instruments. For common instruments such as the CES-D, MADRS-S and HADS there is mounting evidence for adequate psychometric properties. Further results are scattered over different instruments and different psychometric characteristics. Few studies included patient populations. We found at least one online measure for each of the included mental health disorders and symptoms. A small number of online questionnaires have been studied thoroughly. This study provides an overview of online instruments to refer to when choosing an instrument for assessing common mental health disorders online, and can structure future psychometric research.

Assessing Protective Factors for Violence Risk in U.K. General Mental Health Services Using the Structured Assessment of Protective Factors.

PubMed

Haines, Alina; Brown, Andrew; Javaid, Syed Fahad; Khan, Fayyaz; Noblett, Steve; Omodunbi, Oladipupo; Sadiq, Khurram; Zaman, Wahid; Whittington, Richard

2017-12-01

Violence risk assessment and management are key tasks in mental health services and should be guided by validated instruments covering both risk and protective factors. This article is part of an international effort to validate the Structured Assessment of Protective Factors (SAPROF) for violence. The SAPROF, Historical, Clinical, Risk Management-20 (HCR-20) and the Psychopathy Checklist-Screening Version (PCL-SV) were administered in a sample of 261 patients in U.K. forensic, general inpatient, and community mental health settings. There was significant variation between these groups on SAPROF scores with fewer protective factors in the forensic group. The prospective validity of the SAPROF for nonviolence in the general inpatient and community samples was moderate (area under the curve [AUC] = .60). Adoption of the SAPROF or similar instruments as a supplement to risk-focused assessments has the potential to improve awareness of protective factors and enhance therapeutic engagement in a range of mental health services.

Assessing burden in families of critical care patients.

PubMed

Kentish-Barnes, Nancy; Lemiale, Virginie; Chaize, Marine; Pochard, Frédéric; Azoulay, Elie

2009-10-01

To provide critical care clinicians with information on validated instruments for assessing burden in families of critical care patients. PubMed (1979-2009). We included all quantitative studies that used a validated instrument to evaluate the prevalence of, and risk factors for, burden on families. We extracted the descriptions of the instruments used and the main results. Family burden after critical illness can be detected reliably and requires preventive strategies and specific treatments. Using simple face-to-face interviews, intensivists can learn to detect poor comprehension and its determinants. Instruments for detecting symptoms of anxiety, depression, or stress can be used reliably even by physicians with no psychiatric training. For some symptoms, the evaluation should take place at a distance from intensive care unit discharge or death. Experience with families of patients who died in the intensive care unit and data from the literature have prompted studies of bereaved family members and the development of interventions aimed at decreasing guilt and preventing complicated grief. We believe that burden on families should be assessed routinely. In clinical studies, using markers for burden measured by validated tools may provide further evidence that effective communication and efforts to detect and to prevent symptoms of stress, anxiety, or depression provide valuable benefits to families.

Antisocial personality disorder and psychopathy in women: a literature review on the reliability and validity of assessment instruments.

PubMed

Dolan, Mairead; Völlm, Birgit

2009-01-01

Crime rates are low in women compared to men. The two disorders most commonly associated with offending behaviour, antisocial personality disorder (ASPD) and psychopathy, are also less prevalent in female samples. However, developments in forensic psychiatry have often ignored gender, and the utility of constructs such as psychopathy and their assessment instruments in female samples remains unclear. This article presents a review of studies looking at rates of ASPD and psychopathy and on the reliability and validity of assessment instruments of these disorders in women. Gender differences in symptom patterns will be considered. The literature seems to suggest that DSM-IV criteria for ASPD may lead to an underestimation of the prevalence of the disorder in women due to the requirement of childhood conduct disorder symptoms. The Psychopathy Checklist-Revised (PCL-R) is a valid and reliable instrument to identify psychopathy in women but there are gender differences in the factor structure and item loadings on this measure. Research to date seems to suggest a three-factor model may be most strongly supported in females. Preliminary evidence suggests the PCL-R may have some value in predicting future offending while the PCL:SV may be useful in predicting institutional violence. Clinical implications are discussed.

A Methodological Review of the Assessment of Humanism in Medical Students.

PubMed

Buck, Era; Holden, Mark; Szauter, Karen

2015-11-01

Humanism is a complex construct that defies simplistic measurement. How educators measure humanism shapes understanding and implications for learners. This systematic review sought to address the following questions: How do medical educators assess humanism in medical students, and how does the measurement impact the understanding of humanism in undergraduate medical education (UME)? Using the IECARES (integrity, excellence, compassion, altruism, respect, empathy, and service) Gold Foundation framework, a search of English literature databases from 2000 to 2013 on assessment of humanism in medical students revealed more than 900 articles, of which 155 met criteria for analysis. Using descriptive statistics, articles and assessments were analyzed for construct measured, study design, assessment method, instrument type, perspective/source of assessment, student level, validity evidence, and national context. Of 202 assessments reported in 155 articles, 162 (80%) used surveys; 164 (81%) used student self-reports. One hundred nine articles (70%) included only one humanism construct. Empathy was the most prevalent construct present in 96 (62%); 49 (51%) of those used a single instrument. One hundred fifteen (74%) used exclusively quantitative data; only 48 (31%) used a longitudinal design. Construct underrepresentation was identified as a threat to validity in half of the assessments. Articles included 34 countries; 87 (56%) were from North America. Assessment of humanism in UME incorporates a limited scope of a complex construct, often relying on single quantitative measures from self-reported survey instruments. This highlights the need for multiple methods, perspectives, and longitudinal designs to strengthen the validity of humanism assessments.

Development and evaluation of the Korean Health Literacy Instrument.

PubMed

Kang, Soo Jin; Lee, Tae Wha; Paasche-Orlow, Michael K; Kim, Gwang Suk; Won, Hee Kwan

2014-01-01

The purpose of this study is to develop and validate the Korean Health Literacy Instrument, which measures the capacity to understand and use health-related information and make informed health decisions in Korean adults. In Phase 1, 33 initial items were generated to measure functional, interactive, and critical health literacy with prose, document, and numeracy tasks. These items included content from health promotion, disease management, and health navigation contexts. Content validity assessment was conducted by an expert panel, and 11 items were excluded. In Phase 2, the 22 remaining items were administered to a convenience sample of 292 adults from community and clinical settings. Exploratory factor and item difficulty and discrimination analyses were conducted and four items with low discrimination were deleted. In Phase 3, the remaining 18 items were administered to a convenience sample of 315 adults 40-64 years of age from community and clinical settings. A confirmatory factor analysis was performed to test the construct validity of the instrument. The Korean Health Literacy Instrument has a range of 0 to 18. The mean score in our validation study was 11.98. The instrument exhibited an internal consistency reliability coefficient of 0.82, and a test-retest reliability of 0.89. The instrument is suitable for screening individuals who have limited health literacy skills. Future studies are needed to further define the psychometric properties and predictive validity of the Korean Health Literacy Instrument.

Translation and validation of the Arab version of the Late-Life Function and Disability Instrument: a cross sectional study.

PubMed

Elboim-Gabyzon, Michal; Agmon, Maayan; Azaiza, Faisal; Laufer, Yocheved

2015-04-24

The Late-Life Function and Disability Instrument (LLFDI) provides a comprehensive, reliable, and valid assessment of physical function and disability in community-dwelling adults. There does not appear to be a validated, comprehensive instrument for assessing function and disability in Arabic. The objective of the present study was to translate and culturally adapt the LLFDI to Arabic, and to determine its test-retest reliability and validity. The LLFDI was translated to Arabic through a forward and backward translation process, and approved by a bilingual committee of experts. Sixty-one (26 male and 35 female) Arabic speaking, healthy, older adults, ages 65-88, living in northern Israel participated in the study. To determine test-retest reliability, the questionnaire was administered twice to 41 subjects with a 6 to 8day interval. Construct validity was examined by correlating the LLFDI responses with the 10-item physical function (PF-10) subscales of the General Health Survey (SF-36), with the physical component of SF-36 (SF-36 PCS), and with two performance measures, the Berg Balance Scale (BBS) and Time Up and Go (TUG) test. Additionally, gender and fall related differences in the LLFDI were also examined. Internal consistency (Cronbach's alpha) was good to excellent (0.77 to 0.97). Test-retest agreement was good to very good (function component: 0.86-0.93, disability component: 0.77-0.93). Correlation with the SF-36 PCS and PF-10 was moderate to strong for both LLFDI components (function, r = 0.53-0.65 and r = 0.57-0.63, and LLFDI disability, r = 0.57-0.76 and 0.53-0.73, respectively). Significant, moderate-to-strong correlations between the LLFDI and BBS (r = 0.73-0.87) and a significant, moderate, negative correlation between LLFDI and TUG test (r = -0.59- -0.68) were noted. The standard error of measure was 6-12%, and the smallest real difference was 18-33%. Discriminative validity for both gender and fall status were also demonstrated. The Arabic version of the LLFDI is a highly reliable and valid instrument for assessing function and disability in community dwelling, Arab older adults. The translated instrument has a discriminative ability between genders and between fallers and non-fallers. The translated instrument may be used in clinical settings and for research purposes.

The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study.

PubMed

Mokkink, Lidwine B; Terwee, Caroline B; Patrick, Donald L; Alonso, Jordi; Stratford, Paul W; Knol, Dirk L; Bouter, Lex M; de Vet, Henrica C W

2010-05-01

Aim of the COSMIN study (COnsensus-based Standards for the selection of health status Measurement INstruments) was to develop a consensus-based checklist to evaluate the methodological quality of studies on measurement properties. We present the COSMIN checklist and the agreement of the panel on the items of the checklist. A four-round Delphi study was performed with international experts (psychologists, epidemiologists, statisticians and clinicians). Of the 91 invited experts, 57 agreed to participate (63%). Panel members were asked to rate their (dis)agreement with each proposal on a five-point scale. Consensus was considered to be reached when at least 67% of the panel members indicated 'agree' or 'strongly agree'. Consensus was reached on the inclusion of the following measurement properties: internal consistency, reliability, measurement error, content validity (including face validity), construct validity (including structural validity, hypotheses testing and cross-cultural validity), criterion validity, responsiveness, and interpretability. The latter was not considered a measurement property. The panel also reached consensus on how these properties should be assessed. The resulting COSMIN checklist could be useful when selecting a measurement instrument, peer-reviewing a manuscript, designing or reporting a study on measurement properties, or for educational purposes.

Validity and reliability of Patient-Reported Outcomes Measurement Information System (PROMIS) Instruments in Osteoarthritis

PubMed Central

Broderick, Joan E.; Schneider, Stefan; Junghaenel, Doerte U.; Schwartz, Joseph E.; Stone, Arthur A.

2013-01-01

Objective Evaluation of known group validity, ecological validity, and test-retest reliability of four domain instruments from the Patient Reported Outcomes Measurement System (PROMIS) in osteoarthritis (OA) patients. Methods Recruitment of an osteoarthritis sample and a comparison general population (GP) through an Internet survey panel. Pain intensity, pain interference, physical functioning, and fatigue were assessed for 4 consecutive weeks with PROMIS short forms on a daily basis and compared with same-domain Computer Adaptive Test (CAT) instruments that use a 7-day recall. Known group validity (comparison of OA and GP), ecological validity (comparison of aggregated daily measures with CATs), and test-retest reliability were evaluated. Results The recruited samples matched (age, sex, race, ethnicity) the demographic characteristics of the U.S. sample for arthritis and the 2009 Census for the GP. Compliance with repeated measurements was excellent: > 95%. Known group validity for CATs was demonstrated with large effect sizes (pain intensity: 1.42, pain interference: 1.25, and fatigue: .85). Ecological validity was also established through high correlations between aggregated daily measures and weekly CATs (≥ .86). Test-retest validity (7-day) was very good (≥ .80). Conclusion PROMIS CAT instruments demonstrated known group and ecological validity in a comparison of osteoarthritis patients with a general population sample. Adequate test-retest reliability was also observed. These data provide encouraging initial data on the utility of these PROMIS instruments for clinical and research outcomes in osteoarthritis patients. PMID:23592494

Short version of the "instrument for assessment of stress in nursing students" in the Brazilian reality.

PubMed

Costa, Ana Lúcia Siqueira; Silva, Rodrigo Marques da; Mussi, Fernanda Carneiro; Serrano, Patrícia Maria; Graziano, Eliane da Silva; Batista, Karla de Melo

2018-01-08

validate a short version of the Instrument for assessment of stress in nursing students in the Brazilian reality. Methodological study conducted with 1047 nursing students from five Brazilian institutions, who answered the 30 items initially distributed in eight domains. Data were analyzed in the R Statistical Package and in the latent variable analysis, using exploratory and confirmatory factor analyses, Cronbach's alpha and item-total correlation. The short version of the instrument had 19 items distributed into four domains: Environment, Professional Training, Theoretical Activities and Performance of Practical Activities. The confirmatory analysis showed absolute and parsimony fit to the proposed model with satisfactory residual levels. Alpha values ​​per factor ranged from 0.736 (Environment) to 0.842 (Performance of Practical Activities). The short version of the instrument has construct validity and reliability for application to Brazilian nursing undergraduates at any stage of the course.

Psychometric properties of the Symptom Status Questionnaire-Heart Failure.

PubMed

Heo, Seongkum; Moser, Debra K; Pressler, Susan J; Dunbar, Sandra B; Mudd-Martin, Gia; Lennie, Terry A

2015-01-01

Many patients with heart failure (HF) experience physical symptoms, poor health-related quality of life (HRQOL), and high rates of hospitalization. Physical symptoms are associated with HRQOL and are major antecedents of hospitalization. However, reliable and valid physical symptom instruments have not been established. Therefore, this study examined the psychometric properties of the Symptom Status Questionnaire-Heart Failure (SSQ-HF) in patients with HF. Data on symptoms using the SSQ-HF were collected from 249 patients (aged 61 years, 67% male, 45% in New York Heart Association functional class III/IV). Internal consistency reliability was assessed using Cronbach's α. Item homogeneity was assessed using item-total and interitem correlations. Construct validity was assessed using factor analysis and testing hypotheses on known relationships. Data on depressive symptoms (Beck Depression Inventory II), HRQOL (Minnesota Living With Heart Failure Questionnaire), and event-free survival were collected to test known relationships. Internal consistency reliability was supported: Cronbach's α was .80. Item-total correlation coefficients and interitem correlation coefficients were acceptable. Factor analysis supported the construct validity of the instrument. More severe symptoms were associated with more depressive symptoms, poorer HRQOL, and more risk for hospitalization, emergency department visit, or death, controlling for covariates. The findings of this study support the reliability and validity of the SSQ-HF. Clinicians and researchers can use this instrument to assess physical symptoms in patients with HF.

Validation and reliability of the Turkish Utian Quality-of-Life Scale in postmenopausal women.

PubMed

Abay, Halime; Kaplan, Sena

2016-04-01

There are a limited number of menopause-specific quality-of-life scales for the Turkish population. This study was conducted to evaluate the validity and reliability of the Turkish Utian Quality-of-Life Scale in postmenopausal women. The study group was comprised of 250 postmenopausal women who applied to a training and research hospital's menopause clinic in Turkey. A survey form and the Turkish Utian quality-of-Life Scale were used to collect data, and the Turkish version of Short Form-36 was used to evaluate reliability with an equivalent form. Language-validity, content-validity, and construct-validity methods were used to assess the validity of the scale, and Cronbach's α coefficient calculation and the equivalent-form reliability methods were used to assess the reliability of the scale. The Turkish Utian Quality-of-Life Scale was determined to be a valid and reliable instrument for measuring the quality of life of postmenopausal women. Confirmatory factor analysis demonstrates that the instrument fits well with 23 items and a four-factor model. The Cronbach's α coefficient for the quality-of-life domains were as follows: 0.88 overall, 0.79 health, 0.78 emotional, 0.76 sexual, and 0.75 occupational. Reliability of the instrument was confirmed through significant correlations between scores on the Turkish version of the Utian Quality-of-Life Scale and the Turkish version of the Short Form-36 (r = 0.745, P < 0.001). This research emphasizes that the Turkish Utian Quality-of-Life Scale is reliable and valid in postmenopausal women-it is a useful instrument for measuring quality of life during menopause.

Assessing therapy-relevant cognitive capacities in young people: development and psychometric evaluation of the self-reflection and insight scale for youth.

PubMed

Sauter, Floor M; Heyne, David; Blöte, Anke W; van Widenfelt, Brigit M; Westenberg, P Michiel

2010-05-01

The effectiveness of cognitive-behaviour therapy with young people may be influenced by a young person's capacity for self-reflection and insight. Clinicians who assess clients' proficiencies in these cognitive capacities can better tailor cognitive and behavioural techniques to the client, facilitating engagement and enhancing treatment outcome. It is therefore important that sound instruments for assessing self-reflection and insight in young people are available. The aim of the current study was to translate and adapt the Self-Reflection and Insight Scale (SRIS) for use with a child and adolescent population (Study 1), and to evaluate the psychometric properties of the resulting measure, the Self-Reflection and Insight Scale for Youth (SRIS-Y; Study 2). In Study 1 (n=145), the comprehensibility of the SRIS-Y was assessed in a community sample of children and adolescents. Study 2 (n=215) then explored the reliability and structural, convergent, and divergent validity of the SRIS-Y. The SRIS-Y was found to be comprehensible to young people, and had good reliability and structural validity. It appears that the SRIS-Y is a sound instrument for assessing therapy-relevant cognitive capacities in young people, of potential benefit in both research and clinical contexts. Future research foci include the predictive validity of the instrument.

Measuring the Perception of the Teachers' Autonomy-Supportive Behavior in Physical Education: Development and Initial Validation of a Multi-Dimensional Instrument

ERIC Educational Resources Information Center

Tilga, Henri; Hein, Vello; Koka, Andre

2017-01-01

This research aimed to develop and validate an instrument to assess the students' perceptions of the teachers' autonomy-supportive behavior by the multi-dimensional scale (Multi-Dimensional Perceived Autonomy Support Scale for Physical Education). The participants were 1,476 students aged 12- to 15-years-old. In Study 1, a pool of 37 items was…

[image omitted] Review of five instruments for the assessment of Asperger's Disorder in adults.

PubMed

Stoesz, Brenda M; Montgomery, Janine M; Smart, Sherri L; Hellsten, Laurie-Ann M

2011-04-01

Due to the recent inclusion of Asperger's Disorder (AD) in the Diagnostic and Statistical Manual of Mental Disorders (APA, 1994) and the International Classification of Diseases (WHO, 1993), concerns regarding diagnosis of AD, particularly in adults, are emerging. Many existing instruments used to assess Pervasive Developmental Disorder are more appropriate for identifying Autistic Disorder (AU) in children, and their usefulness for assessing AD in adults is questionable. We describe and critically review five instruments created specifically for identifying AD in adults. Overall, the normative information provided is limited and evidence of the reliability and validity for each instrument is relatively poor. Further research and development is required before we would recommend one instrument over another for the assessment of AD in adults.

Systematic review of the psychometric properties and theoretical grounding of instruments evaluating self-care in people with type 2 diabetes mellitus.

PubMed

Caro-Bautista, Jorge; Martín-Santos, Francisco Javier; Morales-Asencio, Jose Miguel

2014-06-01

To determine the psychometric properties and theoretical grounding of instruments that evaluate self-care behaviour or barriers in people with type 2 diabetes. There are many instruments designed to evaluate self-care behaviour or barriers in this population, but knowledge about their psychometric validation processes is lacking. Systematic review. We conducted a search for psychometric or validation studies published between January 1990-December 2012. We carried out searches in Pubmed, CINAHL, PsycINFO, ProQuolid, BibliPRO and Google SCHOLAR to identify instruments that evaluated self-care behaviours or barriers to diabetes self-care. We conducted a systematic review with the following inclusion criteria: Psychometric or clinimetric validation studies that included patients with type 2 diabetes (exclusively or partially) and which analysed self-care behaviour or barriers to self-care and proxies like self-efficacy or empowerment, from a multidimensional approach. Language: Spanish or English. Two authors independently assessed the quality of the studies and extracted data using Terwee's proposed criteria: psychometrics properties, dimensionality, theoretical ground and population used for validation through each included instrument. Sixteen instruments achieved the inclusion criteria for the review. We detected important methodological flaws in many of the selected instruments. Only the Self-management Profile for Type 2 Diabetes and Problem Areas in Diabetes Scale met half of Terwee's quality criteria. There are no instruments for identifying self-care behaviours or barriers elaborated with a strong validation process. Further research should be carried out to provide patients, clinicians and researchers with valid and reliable instruments that are methodologically solid and theoretically grounded. © 2013 John Wiley & Sons Ltd.

Development and Validation of the Biobanking Attitudes and Knowledge Survey (BANKS)

PubMed Central

Wells, Kristen J.; Arevalo, Mariana; Meade, Cathy D.; Gwede, Clement K.; Quinn, Gwendolyn P.; Luque, John S.; Miguel, Gloria San; Watson, Dale; Phillips, Rebecca; Reyes, Carmen; Romo, Margarita; West, Jim; Jacobsen, Paul B.

2014-01-01

Background No validated multi-scale instruments exist that measure community members’ views on biobanking and biospecimen donation. This study describes the development and psychometric properties of the English-language BANKS (Biobanking Attitudes aNd Knowledge Survey). Methods The BANKS was created by item generation through review of scientific literature, focus groups with community members, and input from a community advisory board. Items were refined through cognitive interviews. Content validity was assessed through an expert panel review. Psychometric properties of the BANKS were assessed in a sample of 85 community members. Results The final BANKS includes 3 scales: Attitudes, Knowledge, and Self-Efficacy; as well as 3 single items, which evaluated receptivity and intention to donate a biospecimen for research. Cronbach's alpha coefficients for two scales that use Likert response format indicated high internal consistency (Attitudes: α=.88; Self-Efficacy: α=.95). Content validity indices were moderate, ranging from 0.69 to 0.89. Intention to donate blood and intention to donate urine were positively correlated with attitudes, knowledge, self-efficacy, and receptivity to learning more about biobanking (p's range from .029 to <.001). Conclusions The final BANKS shows evidence of satisfactory reliability and validity, is easy to administer, and is a promising tool to inform biospecimen research. Additional studies should be conducted with larger samples considering biospecimen donation to further assess the instrument's reliability and validity. Impact A valid and reliable instrument measuring community members’ views about biobanking may help researchers evaluate relevant communication interventions to enhance understanding, intention, and actual biospecimen donation. A Spanish-language BANKS is under development. PMID:24609846

Validation and Exploration of Instruments for Assessing Public Knowledge of and Attitudes toward Nanotechnology

ERIC Educational Resources Information Center

Lin, Shu-Fen; Lin, Huann-shyang; Wu, Yi-ying

2013-01-01

The purposes of this study were to develop instruments that assess public knowledge of nanotechnology (PKNT), public attitudes toward nanotechnology (PANT) and conduct a pilot study for exploring the relationship between PKNT and PANT. The PKNT test was composed of six scales involving major nanotechnology concepts, including size and scale,…

Assessment of Symptoms in Adult Survivors of Incest: A Factor Analytic Study of the Responses to Childhood Incest Questionnaire.

ERIC Educational Resources Information Center

Edwards, Patrick W.; Donaldson, Mary Ann

1989-01-01

A study of the construction and factor validity of the Response to Child Incest Questionnaire, a self-report instrument for assessing commonly reported symptoms of adult survivors of incest, is reported. The instrument's usefulness as a pre- and post-treatment measure and further research needs are discussed. (MSE)

Technological Pedagogical Content Knowledge (TPACK): The Development and Validation of an Assessment Instrument for Preservice Teachers

ERIC Educational Resources Information Center

Schmidt, Denise A.; Baran, Evrim; Thompson, Ann D.; Mishra, Punya; Koehler, Matthew J.; Shin, Tae S.

2009-01-01

Based in Shulman's idea of Pedagogical Content Knowledge, Technological Pedagogical Content Knowledge (TPACK) has emerged as a useful frame for describing and understanding the goals for technology use in preservice teacher education. This paper addresses the need for a survey instrument designed to assess TPACK for preservice teachers. The paper…

Predictive Validity of Self-Report Questionnaires in the Assessment of Autism Spectrum Disorders in Adults

ERIC Educational Resources Information Center

Sizoo, Bram B.; Horwitz, E. H.; Teunisse, J. P.; Kan, C. C.; Vissers, C. T. W. M.; Forceville, E. J. M.; Van Voorst, A. J. P.; Geurts, H. M.

2015-01-01

While various screening instruments for autism spectrum disorders are widely used in diagnostic assessments, their psychometric properties have not been simultaneously evaluated in the outpatient setting where these instruments are used most. In this study, we tested the Ritvo Autism Asperger Diagnostic Scale-Revised and two short versions of the…

Assessment of the elderly's functional ability to manage their medication: a systematic literature review.

PubMed

Advinha, Ana Margarida; Lopes, Manuel José; de Oliveira-Martins, Sofia

2017-02-01

Background The evaluation of the elderly's ability to manage medication through the use of a validated tool can be a significant step in identifying inabilities and needs, with the objective of increasing their self-care skills, and promoting successful aging. Aim of the review To identify studies assessing the elderly's functional ability to manage their own medication. Method For the search strategy, the PICO method was used: P-Population (elderly), I-Instruments (tools for assessing medication management ability), C-Context (community) and O-Outcomes (functional ability to manage medication). The final search query was run in MEDLINE/PubMed, CINAHL Plus, ISI Web of Science and Scopus. The whole process was developed according to the PRISMA statement. Results The search retrieved 8051 records. In each screening stage, the selection criteria were applied to eliminate records where at least one of the exclusion criteria was verified. At the end of this selection, we obtained a total of 18 papers (17 studies). The results allow the conclusion to be drawn that studies use several different instruments, most of them not validated. The authors agree that medication management abilities decrease as cognitive impairment increases, even if a lot of studies assess only the physical dimension. DRUGS was the instrument most often used. Conclusion Older adults' ability to manage their medication should be assessed using tools specifically built and validate for the purpose. DRUGS (which uses the real regimen taken by the elderly) was the most widely used assessment instrument in the screened studies.

An Instrument to Measure Maturity of Integrated Care: A First Validation Study

PubMed Central

2018-01-01

Introduction: Lessons captured from interviews with 12 European regions are represented in a new instrument, the B3-Maturity Model (B3-MM). B3-MM aims to assess maturity along 12 dimensions reflecting the various aspects that need to be managed in order to deliver integrated care. The objective of the study was to test the content validity of B3-MM as part of SCIROCCO (Scaling Integrated Care into Context), a European Union funded project. Methods: A literature review was conducted to compare B3-MM’s 12 dimensions and their measurement scales with existing measures and instruments that focus on assessing the development of integrated care. Subsequently, a three-round survey conducted through a Delphi study with international experts in the field of integrated care was performed to test the relevance of: 1) the dimensions, 2) the maturity indicators and 3) the assessment scale used in B3-MM. Results: The 11 articles included in the literature review confirmed all the dimensions described in the original version of B3-MM. The Delphi study rounds resulted in various phrasing amendments of indicators and assessment scale. Full agreement among the experts on the relevance of the 12 B3-MM dimensions, their indicators, and assessment scale was reached after the third Delphi round. Conclusion and discussion: The B3-MM dimensions, maturity indicators and assessment scale showed satisfactory content validity. While the B3-MM is a unique instrument based on existing knowledge and experiences of regions in integrated care, further testing is needed to explore other measurement properties of B3-MM. PMID:29588644

Development of an Instrument to Assess Prospective Elementary Teacher Self-Efficacy Beliefs about Equitable Science Teaching and Learning (SEBEST)

NASA Astrophysics Data System (ADS)

Ritter, Jennifer M.; Boone, William J.; Rubba, Peter A.

2001-06-01

This paper presents an overview of the procedures used to develop and validate an instrument to measure the self-efficacy beliefs of prospective elementary teachers about equitable science teaching and learning. The instrument, titled the SEBEST, was based on the work of Ashton and Webb (1986a, 1986b) and Bandura (1977, 1986). It was modeled after the Science Teaching Efficacy Belief Instrument (STEBI) (Riggs, 1988) and the Science Teaching Efficacy Belief Instrument for Prospective Teachers (STEBI-B) (Enochs & Riggs, 1990). Based on the standardized development procedures used and associated evidence, the SEBEST appears to be a content and construct valid instrument, with high internal reliability qualities. "Most probable response" plots are introduced and used to bring meaning to SEBEST raw scores.

Bem Sex Role Inventory Validation in the International Mobility in Aging Study.

PubMed

Ahmed, Tamer; Vafaei, Afshin; Belanger, Emmanuelle; Phillips, Susan P; Zunzunegui, Maria-Victoria

2016-09-01

This study investigated the measurement structure of the Bem Sex Role Inventory (BSRI) with different factor analysis methods. Most previous studies on validity applied exploratory factor analysis (EFA) to examine the BSRI. We aimed to assess the psychometric properties and construct validity of the 12-item short-form BSRI in a sample administered to 1,995 older adults from wave 1 of the International Mobility in Aging Study (IMIAS). We used Cronbach's alpha to assess internal consistency reliability and confirmatory factor analysis (CFA) to assess psychometric properties. EFA revealed a three-factor model, further confirmed by CFA and compared with the original two-factor structure model. Results revealed that a two-factor solution (instrumentality-expressiveness) has satisfactory construct validity and superior fit to data compared to the three-factor solution. The two-factor solution confirms expected gender differences in older adults. The 12-item BSRI provides a brief, psychometrically sound, and reliable instrument in international samples of older adults.

Validation of learning assessments: A primer.

PubMed

Peeters, Michael J; Martin, Beth A

2017-09-01

The Accreditation Council for Pharmacy Education's Standards 2016 has placed greater emphasis on validating educational assessments. In this paper, we describe validity, reliability, and validation principles, drawing attention to the conceptual change that highlights one validity with multiple evidence sources; to this end, we recommend abandoning historical (confusing) terminology associated with the term validity. Further, we describe and apply Kane's framework (scoring, generalization, extrapolation, and implications) for the process of validation, with its inferences and conclusions from varied uses of assessment instruments by different colleges and schools of pharmacy. We then offer five practical recommendations that can improve reporting of validation evidence in pharmacy education literature. We describe application of these recommendations, including examples of validation evidence in the context of pharmacy education. After reading this article, the reader should be able to understand the current concept of validation, and use a framework as they validate and communicate their own institution's learning assessments. Copyright © 2017 Elsevier Inc. All rights reserved.

[Diagnostics of work motivation (DIAMO): optimization and construct validity].

PubMed

Ranft, Andreas; Fiedler, Rolf; Greitemann, Bernhard; Heuft, Gereon

2009-01-01

Faced with increasing cost pressure of the social insurance system the carriers of rehabilitation programs focus on the efficacy of their measures. The diagnostic instrument for work motivation (DIAMO) has been developed to assess the influence of job-related motivation on the rehabilitation outcome. The inner structure of the instrument was validated and optimized in a cohort of medical rehabilitation patients (n = 422). Construct validity was further tested by using established instruments. Ten scales related to self-image, intention of action and goodness of fit show good psychometric qualities (Cronbachs alpha: 0.72 - 0.86). The constructs correlate moderately-to-strongly with personality-oriented scales while correlation with disease-related contents is low. The DIAMO is a generic and not disease oriented instrument. It would be expected to facilitate the development of vocational interventions to increase the rehabilitation outcome.

Psychometric properties of the Aberrant Behavior Checklist, the Anxiety, Depression and Mood Scale, the Assessment of Dual Diagnosis and the Social Performance Survey Schedule in adults with intellectual disabilities.

PubMed

Rojahn, Johannes; Rowe, Ellen W; Kasdan, Shana; Moore, Linda; van Ingen, Daniel J

2011-01-01

Progress in clinical research and in empirically supported interventions in the area of psychopathology in intellectual disabilities (ID) depends on high-quality assessment instruments. To this end, psychometric properties of four instruments were examined: the Aberrant Behavior Checklist (ABC), the Assessment of Dual Diagnosis (ADD), the Anxiety, Depression and Mood Scale (ADAMS), and the Social Performance Survey Schedule (SPSS). Data were collected in two community-based groups of adults with mild to profound ID (n = 263). Subscale reliability (internal consistency) ranged from fair to excellent for the ABC, the ADAMS, and the SPSS (mean coefficient α across ABC subscales was .87 (ranging from fair to excellent), the ADAMS subscales .83 (ranging from fair to good), and the SPSS subscales .91 (range from good to excellent). The ADD subscales had generally lower reliability scores with a mean of .59 (ranging from unacceptable to good). Convergent and discriminant validity was determined by bivariate Spearman ρ correlations between subscales of one instrument and the subscales of the other three instruments. For the most part, all four instruments showed solid convergent and discriminant validity. To examine the factorial validity, Confirmatory Factor Analyses (CFA) were attempted with the inter-item covariance matrix of each instrument. Generally, the data did not show good fits with the measurement models for the SPSS, ABC, or the ADAMS (CFA analyses with the ADD would not converge). However, most of the items on these three instruments had significant loadings on their respective factors. Copyright © 2011 Elsevier Ltd. All rights reserved.

Psychometric Issues in the Assessment of Clinical Competencies.

ERIC Educational Resources Information Center

Scofield, Michael E.; Yoxtheimer, Linda L.

1983-01-01

Reviewed 149 studies which measured counselor or therapist competencies and examined them for evidence of the reliability and validity associated with the assessment instruments. Only 43 percent of the measurements included reliability data computed at the time they were used, and only 12 percent were accompanied by evidence of their validity.…

Assessing Students' Emotional Competence in Higher Education: Development and Validation of the Widener Emotional Learning Scale

ERIC Educational Resources Information Center

Wang, Ning; Young, Thomas; Wilhite, Stephen C.; Marczyk, Geoffrey

2011-01-01

This article reports the development and validation studies of the Widener Emotional Learning Scale (WELS), a self-report measure, for assessing students' social and emotional competence in higher education. Conceptual specifications, item development, psychometric properties, and factor structure of the instrument are reported in the article. The…

Validation of Scale of Commitment to Democratic Values among Secondary Students

ERIC Educational Resources Information Center

Gafoor, K. Abdul

2015-01-01

This study reports development of a reliable and valid instrument for assessing the commitment to democratic values among secondary school students in Kerala from 57 likert type statements originally developed in 2007 by Gafoor and Thushara to assess commitment to nine values avowed in the Indian Constitution. Nine separate maximum likelihood…

Reliability, Validity and Utility of a Multiple Intelligences Assessment for Career Planning.

ERIC Educational Resources Information Center

Shearer, C. Branton

"The Multiple Intelligences Developmental Assessment Scales" (MIDAS) is a self- (or other-) completed instrument which is based upon the theory of multiple intelligences. The validity, reliability, and utility data regarding the MIDAS are reported here. The measure consists of 7 main scales and 24 subscales which summarize a person's intellectual…

Validity and Reliability of Wii Fit Balance Board for the Assessment of Balance of Healthy Young Adults and the Elderly

PubMed Central

Chang, Wen-Dien; Chang, Wan-Yi; Lee, Chia-Lun; Feng, Chi-Yen

2013-01-01

[Purpose] Balance is an integral part of human ability. The smart balance master system (SBM) is a balance test instrument with good reliability and validity, but it is expensive. Therefore, we modified a Wii Fit balance board, which is a convenient balance assessment tool, and analyzed its reliability and validity. [Subjects and Methods] We recruited 20 healthy young adults and 20 elderly people, and administered 3 balance tests. The correlation coefficient and intraclass correlation of both instruments were analyzed. [Results] There were no statistically significant differences in the 3 tests between the Wii Fit balance board and the SBM. The Wii Fit balance board had a good intraclass correlation (0.86–0.99) for the elderly people and positive correlations (r = 0.58–0.86) with the SBM. [Conclusions] The Wii Fit balance board is a balance assessment tool with good reliability and high validity for elderly people, and we recommend it as an alternative tool for assessing balance ability. PMID:24259769

Development and Validation of the Blended Learning Environment Instrument (BLEI) in Higher Education

ERIC Educational Resources Information Center

Aljahni, Areej; Al-Begain, Khalid; Skinner, Heather

2014-01-01

Part of ongoing research into the efficacy of blended learning in higher education within the Kingdom of Saudi Arabia (KSA). The need for, and development of, a Blended Learning Environment Instrument (BLEI) are explained. This new instrument assesses student perceptions across five core aspects of blended learning environments: Infrastructure,…

Measurement properties of quality-of-life measurement instruments for infants, children and adolescents with eczema: a systematic review.

PubMed

Heinl, D; Prinsen, C A C; Sach, T; Drucker, A M; Ofenloch, R; Flohr, C; Apfelbacher, C

2017-04-01

Quality of life (QoL) is one of the core outcome domains identified by the Harmonising Outcome Measures for Eczema (HOME) initiative to be assessed in every eczema trial. There is uncertainty about the most appropriate QoL instrument to measure this domain in infants, children and adolescents. To systematically evaluate the measurement properties of existing measurement instruments developed and/or validated for the measurement of QoL in infants, children and adolescents with eczema. A systematic literature search in PubMed and Embase, complemented by a thorough hand search of reference lists, retrieved studies on measurement properties of eczema QoL instruments for infants, children and adolescents. For all eligible studies, we judged the adequacy of the measurement properties and the methodological study quality with the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. Results from different studies were summarized in a best-evidence synthesis and formed the basis to assign four degrees of recommendation. Seventeen articles, three of which were found by hand search, were included. These 17 articles reported on 24 instruments. No instrument can be recommended for use in all eczema trials because none fulfilled all required adequacy criteria. With adequate internal consistency, reliability and hypothesis testing, the U.S. version of the Childhood Atopic Dermatitis Impact Scale (CADIS), a proxy-reported instrument, has the potential to be recommended depending on the results of further validation studies. All other instruments, including all self-reported ones, lacked significant validation data. Currently, no QoL instrument for infants, children and adolescents with eczema can be highly recommended. Future validation research should primarily focus on the CADIS, but also attempt to broaden the evidence base for the validity of self-reported instruments. © 2016 British Association of Dermatologists.

Development of a questionnaire to measure quality of life in adolescents with food allergy: the FAQL-teen.

PubMed

Resnick, Elena S; Pieretti, Mariah M; Maloney, Jennifer; Noone, Sally; Muñoz-Furlong, Anne; Sicherer, Scott H

2010-11-01

Living with food allergies affects quality of life (QOL) and may be particularly problematic for teenagers. To develop a validated food allergy QOL assessment tool for US adolescents (FAQL-teen). Initial items were developed through expert opinion, literature review, and adolescent focus groups, resulting in an 88-question impact assessment questionnaire. This questionnaire was completed by 52 adolescents for effect scoring; final instrument questions were determined through analysis of effect scores. The final 17-item instrument was completed by 203 participants aged 13 to 19 years via an Internet link on the Food Allergy & Anaphylaxis Network Web site and via paper surveys distributed at a Food Allergy & Anaphylaxis Network conference. Items were scored on a 7-point Likert scale: 0 corresponded to "not troubled/limited," 3 to "moderately troubled/limited," and 6 to "extremely troubled/limited." Areas most troubling included limitations on social activities (score, 2.7), not being able to eat what others were eating (score, 2.7), and limited choice of restaurants (score, 3.9). Instrument validation steps showed strong internal validity (Cronbach α = .9). The instrument discriminated by disease severity: adolescents with a history of anaphylaxis had significantly lower QOL (higher scores) than did those without a history of anaphylaxis (P = .003). While developing a food allergy QOL assessment tool for US adolescents (FAQL-teen), we identified multiple social and emotional concerns that could be targeted for adolescent counseling. This instrument is internally valid and has the ability to discriminate, making it a useful tool in adolescent food allergy studies. Copyright © 2010 American College of Allergy, Asthma & Immunology. Published by Elsevier Inc. All rights reserved.

Objective assessment based on motion-related metrics and technical performance in laparoscopic suturing.

PubMed

Sánchez-Margallo, Juan A; Sánchez-Margallo, Francisco M; Oropesa, Ignacio; Enciso, Silvia; Gómez, Enrique J

2017-02-01

The aim of this study is to present the construct and concurrent validity of a motion-tracking method of laparoscopic instruments based on an optical pose tracker and determine its feasibility as an objective assessment tool of psychomotor skills during laparoscopic suturing. A group of novice ([Formula: see text] laparoscopic procedures), intermediate (11-100 laparoscopic procedures) and experienced ([Formula: see text] laparoscopic procedures) surgeons performed three intracorporeal sutures on an ex vivo porcine stomach. Motion analysis metrics were recorded using the proposed tracking method, which employs an optical pose tracker to determine the laparoscopic instruments' position. Construct validation was measured for all 10 metrics across the three groups and between pairs of groups. Concurrent validation was measured against a previously validated suturing checklist. Checklists were completed by two independent surgeons over blinded video recordings of the task. Eighteen novices, 15 intermediates and 11 experienced surgeons took part in this study. Execution time and path length travelled by the laparoscopic dissector presented construct validity. Experienced surgeons required significantly less time ([Formula: see text]), travelled less distance using both laparoscopic instruments ([Formula: see text]) and made more efficient use of the work space ([Formula: see text]) compared with novice and intermediate surgeons. Concurrent validation showed strong correlation between both the execution time and path length and the checklist score ([Formula: see text] and [Formula: see text], [Formula: see text]). The suturing performance was successfully assessed by the motion analysis method. Construct and concurrent validity of the motion-based assessment method has been demonstrated for the execution time and path length metrics. This study demonstrates the efficacy of the presented method for objective evaluation of psychomotor skills in laparoscopic suturing. However, this method does not take into account the quality of the suture. Thus, future works will focus on developing new methods combining motion analysis and qualitative outcome evaluation to provide a complete performance assessment to trainees.

Translating and validating a Training Needs Assessment tool into Greek

PubMed Central

Markaki, Adelais; Antonakis, Nikos; Hicks, Carolyn M; Lionis, Christos

2007-01-01

Background The translation and cultural adaptation of widely accepted, psychometrically tested tools is regarded as an essential component of effective human resource management in the primary care arena. The Training Needs Assessment (TNA) is a widely used, valid instrument, designed to measure professional development needs of health care professionals, especially in primary health care. This study aims to describe the translation, adaptation and validation of the TNA questionnaire into Greek language and discuss possibilities of its use in primary care settings. Methods A modified version of the English self-administered questionnaire consisting of 30 items was used. Internationally recommended methodology, mandating forward translation, backward translation, reconciliation and pretesting steps, was followed. Tool validation included assessing item internal consistency, using the alpha coefficient of Cronbach. Reproducibility (test – retest reliability) was measured by the kappa correlation coefficient. Criterion validity was calculated for selected parts of the questionnaire by correlating respondents' research experience with relevant research item scores. An exploratory factor analysis highlighted how the items group together, using a Varimax (oblique) rotation and subsequent Cronbach's alpha assessment. Results The psychometric properties of the Greek version of the TNA questionnaire for nursing staff employed in primary care were good. Internal consistency of the instrument was very good, Cronbach's alpha was found to be 0.985 (p < 0.001) and Kappa coefficient for reproducibility was found to be 0.928 (p < 0.0001). Significant positive correlations were found between respondents' current performance levels on each of the research items and amount of research involvement, indicating good criterion validity in the areas tested. Factor analysis revealed seven factors with eigenvalues of > 1.0, KMO (Kaiser-Meyer-Olkin) measure of sampling adequacy = 0.680 and Bartlett's test of sphericity, p < 0.001. Conclusion The translated and adapted Greek version is comparable with the original English instrument in terms of validity and reliability and it is suitable to assess professional development needs of nursing staff in Greek primary care settings. PMID:17474989

Conceptualizing and assessing improvement capability: a review

PubMed Central

Boaden, Ruth; Walshe, Kieran

2017-01-01

Abstract Purpose The literature is reviewed to examine how ‘improvement capability’ is conceptualized and assessed and to identify future areas for research. Data sources An iterative and systematic search of the literature was carried out across all sectors including healthcare. The search was limited to literature written in English. Data extraction The study identifies and analyses 70 instruments and frameworks for assessing or measuring improvement capability. Information about the source of the instruments, the sectors in which they were developed or used, the measurement constructs or domains they employ, and how they were tested was extracted. Results of data synthesis The instruments and framework constructs are very heterogeneous, demonstrating the ambiguity of improvement capability as a concept, and the difficulties involved in its operationalisation. Two-thirds of the instruments and frameworks have been subject to tests of reliability and half to tests of validity. Many instruments have little apparent theoretical basis and do not seem to have been used widely. Conclusion The assessment and development of improvement capability needs clearer and more consistent conceptual and terminological definition, used consistently across disciplines and sectors. There is scope to learn from existing instruments and frameworks, and this study proposes a synthetic framework of eight dimensions of improvement capability. Future instruments need robust testing for reliability and validity. This study contributes to practice and research by presenting the first review of the literature on improvement capability across all sectors including healthcare. PMID:28992146

Validation of an Instrument for Assessing Conceptual Change with Respect to the Theory of Evolution by Secondary Biology Students

NASA Astrophysics Data System (ADS)

Goff, Kevin David

This pilot study evaluated the validity of a new quantitative, closed-response instrument for assessing student conceptual change regarding the theory of evolution. The instrument has two distinguishing design features. First, it is designed not only to gauge student mastery of the scientific model of evolution, but also to elicit a trio of deeply intuitive tendencies that are known to compromise many students' understanding: the projection of intentional agency, teleological directionality, and immutable essences onto biological phenomena. Second, in addition to a section of conventional multiple choice questions, the instrument contains a series of items where students may simultaneously endorse both scientifically normative propositions and intuitively appealing yet unscientific propositions, without having to choose between them. These features allow for the hypothesized possibility that the three intuitions are partly innate, themselves products of cognitive evolution in our hominin ancestors, and thus may continue to inform students' thinking even after instruction and conceptual change. The test was piloted with 340 high school students from diverse schools and communities. Confirmatory factor analysis and other statistical methods provided evidence that the instrument already has strong potential for validly distinguishing students who hold a correct scientific understanding from those who do not, but that revision and retesting are needed to render it valid for gauging students' adherence to intuitive misconceptions. Ultimately the instrument holds promise as a tool for classroom intervention studies by conceptual change researchers, for diagnostic testing and data gathering by instructional leaders, and for provoking classroom dialogue and debate by science teachers.

Validation of the Juhnke-Balkin Life Balance Inventory

ERIC Educational Resources Information Center

Davis, R. J.; Balkin, Richard S.; Juhnke, Gerald A.

2014-01-01

Life balance is an important construct within the counseling profession. A validation study utilizing exploratory factor analysis and multiple regression was conducted on the Juhnke-Balkin Life Balance Inventory. Results from the study serve as evidence of validity for an assessment instrument designed to measure life balance.

Psychometric Properties of the Chinese Version of the Occupational Fatigue Exhaustion/Recovery Scale: A Test in a Nursing Population.

PubMed

Fang, Jin-Bo; Zhou, Chun-Fen; Huang, Jing; Qiu, Chang-Jian

2018-06-01

The Occupational Fatigue Exhaustion/Recovery Scale (OFER) was designed to assess occupational fatigue in nurses. Although the original English version of this instrument has shown high degrees of reliability and validity, a Chinese version of this scale has yet to be verified. The aim of this study was to evaluate the psychometric properties of the OFER in a population of Chinese nurses. The scale was translated using translation and back-translation. The validities and reliabilities were evaluated on 923 qualified participants using content validity index, concurrent validity, factorial validity, internal consistency reliability, and test-retest reliability. The content validity index for the OFER was .92. The correlation coefficients between the scores of the OFER subscales and the criteria in this study (varying from -.498 to .705) verified that the OFER has acceptable concurrent validity. Principal component analysis and confirmatory factor analysis revealed that three factors correspond to the structure of the original instrument and that recovery mediates the relationship between acute and chronic fatigue. The Cronbach's alpha for the chronic fatigue, acute fatigue, and intershift recovery subscales were .83, .85, and .86, respectively. Test-retest reliabilities with correlation coefficients from .61 to .78 were found in the three subscales. OFER is a reliable and valid instrument for assessing work-related fatigue in Chinese nurses. However, further improvement of the acute fatigue subscale is recommended. The OFER has the potential to elicit information that is useful for assessing fatigue in nurses in China. Furthermore, as it differentiates between acute and chronic fatigue, OFER may be an effective tool for guiding the development and implementation of various, related intervention measures.

Psychometric validation of the PROQOL-HIV questionnaire, a new health-related quality of life instrument-specific to HIV disease.

PubMed

Duracinsky, Martin; Lalanne, Christophe; Le Coeur, Sophie; Herrmann, Susan; Berzins, Baiba; Armstrong, Andrew Richard; Lau, Joseph Tak Fai; Fournier, Isabelle; Chassany, Olivier

2012-04-15

This study reports the psychometric validation of a new HIV/AIDS-specific health-related quality of life (HRQL) questionnaire, the Patient Reported Outcomes Quality of Life-HIV. The instrument was developed simultaneously across Europe, North and South America, Africa, Asia, and Australia to assess multidimensional quality of life impairments in the era of highly active antiretroviral therapy. A cross-sectional study was performed in 8 countries. The pilot 70-item questionnaire was co-administered with the HIV symptoms index, the EQ-5D and Medical Outcomes Study-HIV questionnaires. Demographic and biomedical data were collected. After item analysis and reduction, convergent discriminant concurrent validity and known-group validity were examined. Internal consistency and reliability scores were assessed using Cronbach alpha and intraclass correlation. The final sample of 791 patients was composed of 64% males (median age: 41 years, HIV diagnosis = 5 years), 13.8% were treatment naive. Item reduction yielded a 43-item form surveying 8 dimensions and 1 global health item that showed good convergent and discriminant validity and reliability (98% scaling success; Cronbach alphas 0.77-0.89). Correlations with EQ-5D and Medical Outcomes Study-HIV complied with concurrent validity expectations; likewise, correlations against the number of self-reported symptoms and depression showed good support for criterion validity. A test-retest study on French patients (n = 34) showed temporal stability (intraclass correlation coefficient = 0.86). Significant and meaningful differences of HRQL scores between countries were found. The Patient Reported Outcomes Quality of Life-HIV questionnaire is a valid and reliable instrument for assessing HRQL specific to HIV disease in different cultures and healthcare systems.

Overview of classical test theory and item response theory for the quantitative assessment of items in developing patient-reported outcomes measures.

PubMed

Cappelleri, Joseph C; Jason Lundy, J; Hays, Ron D

2014-05-01

The US Food and Drug Administration's guidance for industry document on patient-reported outcomes (PRO) defines content validity as "the extent to which the instrument measures the concept of interest" (FDA, 2009, p. 12). According to Strauss and Smith (2009), construct validity "is now generally viewed as a unifying form of validity for psychological measurements, subsuming both content and criterion validity" (p. 7). Hence, both qualitative and quantitative information are essential in evaluating the validity of measures. We review classical test theory and item response theory (IRT) approaches to evaluating PRO measures, including frequency of responses to each category of the items in a multi-item scale, the distribution of scale scores, floor and ceiling effects, the relationship between item response options and the total score, and the extent to which hypothesized "difficulty" (severity) order of items is represented by observed responses. If a researcher has few qualitative data and wants to get preliminary information about the content validity of the instrument, then descriptive assessments using classical test theory should be the first step. As the sample size grows during subsequent stages of instrument development, confidence in the numerical estimates from Rasch and other IRT models (as well as those of classical test theory) would also grow. Classical test theory and IRT can be useful in providing a quantitative assessment of items and scales during the content-validity phase of PRO-measure development. Depending on the particular type of measure and the specific circumstances, the classical test theory and/or the IRT should be considered to help maximize the content validity of PRO measures. Copyright © 2014 Elsevier HS Journals, Inc. All rights reserved.

Quality-of-life measures for use within care homes: a systematic review of their measurement properties.

PubMed

Aspden, Trefor; Bradshaw, Siobhan A; Playford, E Diane; Riazi, Afsane

2014-09-01

the aims of this review were (i) to identify quality-of-life (QoL) measures which have had their measurement properties validated in people residing in care homes or nursing homes, and to critically compare and summarise these instruments and (ii) to make recommendations for measurement instruments. bibliographic databases PsycINFO, PubMed, Cochrane, CINAHL and Embase were searched for articles evaluating measurement properties of QoL instruments in people residing in care homes. Methodological quality of studies was assessed using the consensus-based standards for the selection of health measurement instruments checklist. Measurement properties of instruments were appraised using a systematic checklist. the search strategy resulted in 3252 unique citations, of which 15 articles were included in this review. These articles assessed 13 instruments, 8 of which were dementia or Alzheimer specific instruments. The QUALIDEM, a dementia-specific observational instrument, had the widest array of information available on its measurement properties, which were mostly satisfactory. Most measurement instruments lacked information on hypotheses testing and content validity. Information on responsiveness and measurement error was not available for any instrument. for people with dementia living in care homes, the QUALIDEM is recommended for measuring QoL. For residents without dementia, we recommend Kane et al.'s Psychosocial Quality of Life Domains questionnaire. Studies of higher methodological quality, assessing a wider range of measurement properties are needed to allow a more fully informed choice of QoL instrument. © The Author 2014. Published by Oxford University Press on behalf of the British Geriatrics Society. All rights reserved. For Permissions, please email: journals.permissions@oup.com.

Self-administered health literacy instruments for people with diabetes: systematic review of measurement properties.

PubMed

Lee, Eun-Hyun; Kim, Chun-Ja; Lee, Jiyeon; Moon, Seung Hei

2017-09-01

The aims of this study were to identify all available self-administered instruments measuring health literacy in people with diabetes and to determine the current instrument that is the most appropriate for applying to this population in both practice and research. A systematic review of measurement properties. MEDLINE, EMBASE and CINAHL electronic databases from their inception up to 28 March 2016. The methodological quality of each included study was assessed using the COnsensus-based Standards for the selection of health Measurement INstruments checklist. The reported results for measurement properties in the studies were assessed according to Terwee's quality criteria. Thirteen self-administered instruments measuring health literacy in people with diabetes were identified, of which six (44%) were diabetes-specific instruments. The instruments that covered the broadest contents of health literacy were the Health Literacy Scale and Health Literacy Questionnaire. The (test-retest) reliability, measurement error and responsiveness were not evaluated for any instrument, while internal consistency and hypothesis testing validity were the most frequently assessed measurement properties. With the current evidence, the Health Literacy Scale may be the most appropriate instrument for patients with diabetes in practice and research. However, the structural validity of this scale needs to be further established, particularly in other language versions. It is also recommended to use the Diabetes Numeracy Test-15 along with the Health Literacy Scale to complement the lack of numeracy measures in the Health Literacy Scale. © 2017 John Wiley & Sons Ltd.

Development of Depression Profile: a new psychometric instrument to selectively evaluate depressive symptoms based on the neurocircuitry theory.

PubMed

Faludi, Gábor; Gonda, Xenia; Kliment, Edit; Bekes, Vera; Mészáros, Veronika; Oláh, Attila

2010-06-01

Although we have several self-report instruments available to assess depression, they yield a composite score and thus do not allow for the differential examination of major symptom clusters associated with depression. However, such an instrument would be a useful tool in subtyping depression and selecting the most appropriate pharmacotherapy for each patient. The neurocircuitry theory describes the biochemical and neuroanatomic background associated with the major symptoms of depression. Based on the neurocircuitry theory, our team has developed a new instrument, the Depression Profile, to selectively assess depressive symptom clusters associated with different neurotransmitter systems and neuroanatomic structures. The aim of our study was to investigate the psychometric characteristics of Depression Profile. 339 patients consecutively admitted with DSM-IV major depression in our hospital completed the Depression Profile in the first two weeks of their hospitalisation. 81 patients in an adult outpatient unit also completed the Zung Self-rating Depression Scale. Internal consistency of Depression Profile was tested with item analysis. The external validity of Depression Profile against the Zung Self-rating Depression Scale was tested using Pearson correlations. The internal consistency of Depression Profile proved to be excellent. The Cronbach alpha values of the scales met the expectable minimum level derived from the number of items in the scales. In testing for convergent validity, all Pearson correlation coefficients between Depression profile subscales and the Zung Self-rating Depression Scale were significant and moderate to high which indicates the good external validity of our instrument. The initial psychometric evaluation of Depression Profile indicates that our instrument has good reliability and internal and external validity. The instrument also proved to be useful in clinical work to aid the choice of medications and determine the subtype of depressive episodes. Further studies, possibly with biochemical and neuroimaging methodology are needed to validate the 9 main symptom clusters of the Depression Profile subscales with respect to their neuroanatomical and neurochemical bases.

Cross-cultural adaptation and validation of the neonatal/infant Braden Q risk assessment scale.

PubMed

de Lima, Edson Luiz; de Brito, Maria José Azevedo; de Souza, Diba Maria Sebba Tosta; Salomé, Geraldo Magela; Ferreira, Lydia Masako

2016-02-01

To translate into Brazilian Portuguese and cross-culturally adapt the Neonatal/Infant Braden Q Risk Assessment Scale (Neonatal/Infant Braden Q Scale), and test the psychometric properties, reproducibility and validity of the instrument. There is a lack of studies on the development of pressure ulcers in children, especially in neonates. Thirty professionals participated in the cross-cultural adaptation of the Brazilian-Portuguese version of the scale. Fifty neonates of both sexes were assessed between July 2013 and June 2014. Reliability and reproducibility were tested in 20 neonates and construct validity was measured by correlating the Neonatal/Infant Braden Q Scale with the Braden Q Risk Assessment Scale (Braden Q Scale). Discriminant validity was assessed by comparing the scores of neonates with and without ulcers. The scale showed inter-rater reliability (ICC = 0.98; P < 0.001) and intra-rater reliability (ICC = 0.79; P < 0.001). A strong correlation was found between the Neonatal/Infant Braden Q Scale and Braden Q Scale (r = 0.96; P < 0.001). The cross-culturally adapted Brazilian version of the Neonatal/Infant Braden Q Scale is a reliable instrument, showing face, content and construct validity. Copyright © 2015 Tissue Viability Society. Published by Elsevier Ltd. All rights reserved.

Development and testing of the questionnaire CEC-61: Knowledge about cervical cancer in Chilean adolescents.

PubMed

Urrutia, María Teresa; Gajardo, Macarena; Padilla, Oslando

2017-05-22

Despite a clear association between human papillomavirus and cervical cancer, knowledge in adolescent populations regarding the disease and methods for its detection and prevention is deficient. The aim of this study was to develop and test a new questionnaire concerning knowledge on cervical cancer. An instrument was developed and validated to measure knowledge in 226 Chilean adolescents between April and June 2011. Content validity, construct validity, and reliability analysis of the instrument were performed. The new, validated instrument, called CEC-61 (Conocimientos en Cancer Cérvicouterino-61 items/Knowledge in Cervical Cancer-61 items), contains nine factors and 61 items. The new questionnaire explained 81% of the variance with a reliability of 0.96. The assessment of knowledge with a valid and reliable instrument is the first step in creating interventions for a population and to encourage appropriate preventive behavior. CEC-61 is highly reliable and has a clear factorial structure to evaluate knowledge in nine domains related to cervical cancer disease, cervical cancer risk, papilloma virus infection, the Papanicolaou test, and the papilloma virus vaccine.

Is the Scale for Measuring Motivational Interviewing Skills a valid and reliable instrument for measuring the primary care professionals motivational skills?: EVEM study protocol

PubMed Central

2012-01-01

Background Lifestyle is one of the main determinants of people’s health. It is essential to find the most effective prevention strategies to be used to encourage behavioral changes in their patients. Many theories are available that explain change or adherence to specific health behaviors in subjects. In this sense the named Motivational Interviewing has increasingly gained relevance. Few well-validated instruments are available for measuring doctors’ communication skills, and more specifically the Motivational Interviewing. Methods/Design The hypothesis of this study is that the Scale for Measuring Motivational Interviewing Skills (EVEM questionnaire) is a valid and reliable instrument for measuring the primary care professionals skills to get behavior change in patients. To test the hypothesis we have designed a prospective, observational, multi-center study to validate a measuring instrument. –Scope: Thirty-two primary care centers in Spain. -Sampling and Size: a) face and consensual validity: A group composed of 15 experts in Motivational Interviewing. b) Assessment of the psychometric properties of the scale; 50 physician- patient encounters will be videoed; a total of 162 interviews will be conducted with six standardized patients, and another 200 interviews will be conducted with 50 real patients (n=362). Four physicians will be specially trained to assess 30 interviews randomly selected to test the scale reproducibility. -Measurements for to test the hypothesis: a) Face validity: development of a draft questionnaire based on a theoretical model, by using Delphi-type methodology with experts. b) Scale psychometric properties: intraobservers will evaluate video recorded interviews: content-scalability validity (Exploratory Factor Analysis), internal consistency (Cronbach alpha), intra-/inter-observer reliability (Kappa index, intraclass correlation coefficient, Bland & Altman methodology), generalizability, construct validity and sensitivity to change (Pearson product–moment correlation coefficient). Discussion The verification of the hypothesis that EVEM is a valid and reliable tool for assessing motivational interviewing would be a major breakthrough in the current theoretical and practical knowledge, as it could be used to assess if the providers put into practice a patient centered communication style and can be used both for training or researching purposes. Trials registration Dislip-EM study NCT01282190 (ClinicalTrials.gov) PMID:23173902

Validity of Students Worksheet Based Problem-Based Learning for 9th Grade Junior High School in living organism Inheritance and Food Biotechnology.

NASA Astrophysics Data System (ADS)

Jefriadi, J.; Ahda, Y.; Sumarmin, R.

2018-04-01

Based on preliminary research of students worksheet used by teachers has several disadvantages such as students worksheet arranged directly drove learners conduct an investigation without preceded by directing learners to a problem or provide stimulation, student's worksheet not provide a concrete imageand presentation activities on the students worksheet not refer to any one learning models curicullum recommended. To address problems Reviews these students then developed a worksheet based on problem-based learning. This is a research development that using Ploom models. The phases are preliminary research, development and assessment. The instruments used in data collection that includes pieces of observation/interviews, instrument self-evaluation, instruments validity. The results of the validation expert on student worksheets get a valid result the average value 80,1%. Validity of students worksheet based problem-based learning for 9th grade junior high school in living organism inheritance and food biotechnology get valid category.

Assessment of sexual difficulties associated with multi-modal treatment for cervical or endometrial cancer: A systematic review of measurement instruments.

PubMed

White, Isabella D; Sangha, Amrit; Lucas, Grace; Wiseman, Theresa

2016-12-01

Practitioners and researchers require an outcome measure that accurately identifies the range of common treatment-induced changes in sexual function and well-being experienced by women after cervical or endometrial cancer. This systematic review critically appraised the measurement properties and clinical utility of instruments validated for the measurement of female sexual dysfunction (FSD) in this clinical population. A bibliographic database search for questionnaire development or validation papers was completed and methodological quality and measurement properties of selected studies rated using the Consensus-based Standards for the selection of health Measurement Instrument (COSMIN) checklist. 738 articles were screened, 13 articles retrieved for full text assessment and 7 studies excluded, resulting in evaluation of 6 papers; 2 QoL and 4 female sexual morbidity measures. Five of the six instruments omitted one or more dimension of female sexual function and only one instrument explicitly measured distress associated with sexual changes as per DSM V (APA 2013) diagnostic criteria. None of the papers reported measurement error, responsiveness data was available for only two instruments, three papers failed to report on criterion validity, and test-retest reliability reporting was inconsistent. Heterosexual penile-vaginal intercourse remains the dominant sexual activity focus for sexual morbidity PROMS terminology and instruments lack explicit reference to solo or non-coital sexual expression or validation in a non-heterosexual sample. Four out of six instruments included mediating treatment or illness items such as vaginal changes, menopause or altered body image. Findings suggest that the Female Sexual Function Index (FSFI) remains the most robust sexual morbidity outcome measure, for research or clinical use, in sexually active women treated for cervical or endometrial cancer. Development of an instrument that measures sexual dysfunction in women who are infrequently/not sexually active due to treatment consequences is still required to identify women in need of sexual rehabilitation. Copyright © 2016 Elsevier Inc. All rights reserved.

The impact of sub-skills and item content on students' skills with regard to the control-of-variables strategy

NASA Astrophysics Data System (ADS)

Schwichow, Martin; Christoph, Simon; Boone, William J.; Härtig, Hendrik

2016-01-01

The so-called control-of-variables strategy (CVS) incorporates the important scientific reasoning skills of designing controlled experiments and interpreting experimental outcomes. As CVS is a prominent component of science standards appropriate assessment instruments are required to measure these scientific reasoning skills and to evaluate the impact of instruction on CVS development. A detailed review of existing CVS instruments suggests that they utilize different, and only a few of the four, critical CVS sub-skills in the item development. This study presents a new CVS assessment instrument (CVS Inventory, CVSI) and investigates the validity of student measures derived from this instrument utilizing Rasch analyses. The results indicate that the CVSI produces reliable and valid student measures with regard to CVS. Furthermore, the results show that the item difficulty depends on the CVS sub-skills utilized in item development, but not on the item content. Accordingly, previous instruments that are restricted to a few CVS sub-skills tend to over- or underestimate students' CVS skills. In addition, these results indicate that students are able to use CVS as a domain general strategy in multiple content areas. Consequences for science instruction and assessment are discussed.