Neuropsychiatric Clinical Trials: Should They Accommodate Real-World Practices or Set Standards for Clinical Practices?

PubMed Central

Becker, Robert E.; Greig, Nigel H.

2012-01-01

The fundamental tenet of Evidence-Based Medicine (EBM) is to “integrate the best research evidence with clinical expertise and patient values,”1(p1) a commitment accepted in neuropsychiatry.2,3 The EBM group recognizes various factors that undermine the quality and use of evidence generated in research, “three limitations…to science and medicine-shortage of coherent evidence, difficulties applying evidence in care, and barriers to quality practice-and further impediments to EBM practice-practitioners lacking skills evaluating evidence sources, having limited time, and being unaware of support for EBM working, thus failing to follow its practices.”1(p7) Other risks to validity are less widely acknowledged. Clinical trials (CTs), especially randomized controlled trials (RCTs), and summary reviews of results from more than 1 RCT provide EBM’s gold standard sources for sound evidence.1(pp105-144) Sackett et al 1 and other authors suggest subjecting RCTs and reviews of RCTs to specific tests of validity before the practitioner uses the evidence. We recently compiled additional threats to validity of the neuropsychiatric evidence base,4,5 a list already incomplete in view of recent concerns with industry influence evidenced by ghost authorships 6 and selective reporting.7,8 Each of the factors we compiled potentially affects the reliability and therefore the validity of the RCT evidence base, is not addressed systematically in EBM guidance on how to develop and use the research literature, and potentially impacts neuropsychiatric research by allowing drugs to fail because of the factor functioning as a methodological weakness in clinical studies.5 In this article, we (1) cull from the literature factors that methodologically put clinical research and the evidence base at risk, (2) uncover assumptions that may account for these factors going unnoticed as risks to medicine’s evidence base, and (3) suggest steps to increase the effectiveness of neuropsychiatric drug developments, CTs, and validity and use of the evidence base for practitioners. Specifically, we provide evidence that problems of unreliability caused by human errors and biases currently undermine the validity of psychiatric research. We suggest revisions of some assumptions behind research methods and practices as part of an effort to protect research from these errors and biases.4 PMID:19142109

Linking simulation-based educational assessments and patient-related outcomes: a systematic review and meta-analysis.

PubMed

Brydges, Ryan; Hatala, Rose; Zendejas, Benjamin; Erwin, Patricia J; Cook, David A

2015-02-01

To examine the evidence supporting the use of simulation-based assessments as surrogates for patient-related outcomes assessed in the workplace. The authors systematically searched MEDLINE, EMBASE, Scopus, and key journals through February 26, 2013. They included original studies that assessed health professionals and trainees using simulation and then linked those scores with patient-related outcomes assessed in the workplace. Two reviewers independently extracted information on participants, tasks, validity evidence, study quality, patient-related and simulation-based outcomes, and magnitude of correlation. All correlations were pooled using random-effects meta-analysis. Of 11,628 potentially relevant articles, the 33 included studies enrolled 1,203 participants, including postgraduate physicians (n = 24 studies), practicing physicians (n = 8), medical students (n = 6), dentists (n = 2), and nurses (n = 1). The pooled correlation for provider behaviors was 0.51 (95% confidence interval [CI], 0.38 to 0.62; n = 27 studies); for time behaviors, 0.44 (95% CI, 0.15 to 0.66; n = 7); and for patient outcomes, 0.24 (95% CI, -0.02 to 0.47; n = 5). Most reported validity evidence was favorable, though studies often included only correlational evidence. Validity evidence of internal structure (n = 13 studies), content (n = 12), response process (n = 2), and consequences (n = 1) were reported less often. Three tools showed large pooled correlations and favorable (albeit incomplete) validity evidence. Simulation-based assessments often correlate positively with patient-related outcomes. Although these surrogates are imperfect, tools with established validity evidence may replace workplace-based assessments for evaluating select procedural skills.

Reliability and Validity of the Evidence-Based Practice Confidence (EPIC) Scale

ERIC Educational Resources Information Center

Salbach, Nancy M.; Jaglal, Susan B.; Williams, Jack I.

2013-01-01

Introduction: The reliability, minimal detectable change (MDC), and construct validity of the evidence-based practice confidence (EPIC) scale were evaluated among physical therapists (PTs) in clinical practice. Methods: A longitudinal mail survey was conducted. Internal consistency and test-retest reliability were estimated using Cronbach's alpha…

Validation of the Evidence-Based Practice Process Assessment Scale

ERIC Educational Resources Information Center

Rubin, Allen; Parrish, Danielle E.

2011-01-01

Objective: This report describes the reliability, validity, and sensitivity of a scale that assesses practitioners' perceived familiarity with, attitudes of, and implementation of the evidence-based practice (EBP) process. Method: Social work practitioners and second-year master of social works (MSW) students (N = 511) were surveyed in four sites…

20 CFR 220.14 - Weighing of evidence.

Code of Federal Regulations, 2010 CFR

2010-04-01

... capacity evaluation is based upon functional objective tests with high validity and reliability; (2) The... consists of objective findings of exams that have poor reliability or validity; (7) The evidence consists...

What Counts as Validity Evidence? Examples and Prevalence in a Systematic Review of Simulation-Based Assessment

ERIC Educational Resources Information Center

Cook, David A.; Zendejas, Benjamin; Hamstra, Stanley J.; Hatala, Rose; Brydges, Ryan

2014-01-01

Ongoing transformations in health professions education underscore the need for valid and reliable assessment. The current standard for assessment validation requires evidence from five sources: content, response process, internal structure, relations with other variables, and consequences. However, researchers remain uncertain regarding the types…

Are validated outcome measures used in distal radial fractures truly valid?

PubMed Central

Nienhuis, R. W.; Bhandari, M.; Goslings, J. C.; Poolman, R. W.; Scholtes, V. A. B.

2016-01-01

Objectives Patient-reported outcome measures (PROMs) are often used to evaluate the outcome of treatment in patients with distal radial fractures. Which PROM to select is often based on assessment of measurement properties, such as validity and reliability. Measurement properties are assessed in clinimetric studies, and results are often reviewed without considering the methodological quality of these studies. Our aim was to systematically review the methodological quality of clinimetric studies that evaluated measurement properties of PROMs used in patients with distal radial fractures, and to make recommendations for the selection of PROMs based on the level of evidence of each individual measurement property. Methods A systematic literature search was performed in PubMed, EMbase, CINAHL and PsycINFO databases to identify relevant clinimetric studies. Two reviewers independently assessed the methodological quality of the studies on measurement properties, using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. Level of evidence (strong / moderate / limited / lacking) for each measurement property per PROM was determined by combining the methodological quality and the results of the different clinimetric studies. Results In all, 19 out of 1508 identified unique studies were included, in which 12 PROMs were rated. The Patient-rated wrist evaluation (PRWE) and the Disabilities of Arm, Shoulder and Hand questionnaire (DASH) were evaluated on most measurement properties. The evidence for the PRWE is moderate that its reliability, validity (content and hypothesis testing), and responsiveness are good. The evidence is limited that its internal consistency and cross-cultural validity are good, and its measurement error is acceptable. There is no evidence for its structural and criterion validity. The evidence for the DASH is moderate that its responsiveness is good. The evidence is limited that its reliability and the validity on hypothesis testing are good. There is no evidence for the other measurement properties. Conclusion According to this systematic review, there is, at best, moderate evidence that the responsiveness of the PRWE and DASH are good, as are the reliability and validity of the PRWE. We recommend these PROMs in clinical studies in patients with distal radial fractures; however, more clinimetric studies of higher methodological quality are needed to adequately determine the other measurement properties. Cite this article: Dr Y. V. Kleinlugtenbelt. Are validated outcome measures used in distal radial fractures truly valid?: A critical assessment using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. Bone Joint Res 2016;5:153–161. DOI: 10.1302/2046-3758.54.2000462. PMID:27132246

Development and Validation of the Evidence-Based Practice Process Assessment Scale: Preliminary Findings

ERIC Educational Resources Information Center

Rubin, Allen; Parrish, Danielle E.

2010-01-01

Objective: This report describes the development and preliminary findings regarding the reliability, validity, and sensitivity of a scale that has been developed to assess practitioners' perceived familiarity with, attitudes about, and implementation of the phases of the evidence-based practice (EBP) process. Method: After a panel of national…

Analytic Validation of Immunohistochemical Assays: A Comparison of Laboratory Practices Before and After Introduction of an Evidence-Based Guideline.

PubMed

Fitzgibbons, Patrick L; Goldsmith, Jeffrey D; Souers, Rhona J; Fatheree, Lisa A; Volmar, Keith E; Stuart, Lauren N; Nowak, Jan A; Astles, J Rex; Nakhleh, Raouf E

2017-09-01

- Laboratories must demonstrate analytic validity before any test can be used clinically, but studies have shown inconsistent practices in immunohistochemical assay validation. - To assess changes in immunohistochemistry analytic validation practices after publication of an evidence-based laboratory practice guideline. - A survey on current immunohistochemistry assay validation practices and on the awareness and adoption of a recently published guideline was sent to subscribers enrolled in one of 3 relevant College of American Pathologists proficiency testing programs and to additional nonsubscribing laboratories that perform immunohistochemical testing. The results were compared with an earlier survey of validation practices. - Analysis was based on responses from 1085 laboratories that perform immunohistochemical staining. Of 1057 responses, 65.4% (691) were aware of the guideline recommendations before this survey was sent and 79.9% (550 of 688) of those have already adopted some or all of the recommendations. Compared with the 2010 survey, a significant number of laboratories now have written validation procedures for both predictive and nonpredictive marker assays and specifications for the minimum numbers of cases needed for validation. There was also significant improvement in compliance with validation requirements, with 99% (100 of 102) having validated their most recently introduced predictive marker assay, compared with 74.9% (326 of 435) in 2010. The difficulty in finding validation cases for rare antigens and resource limitations were cited as the biggest challenges in implementing the guideline. - Dissemination of the 2014 evidence-based guideline validation practices had a positive impact on laboratory performance; some or all of the recommendations have been adopted by nearly 80% of respondents.

Examining Evidence for the Validity of PISA Learning Strategy Scales Based on Student Response Processes

ERIC Educational Resources Information Center

Hopfenbeck, Therese N.; Maul, Andrew

2011-01-01

The aim of this study was to investigate response-process based evidence for the validity of the Programme for International Student Assessment's (PISA) self-report questionnaire scales as measures of specific psychological constructs, with a focus on scales meant to measure inclination toward specific learning strategies. Cognitive interviews (N…

Assessing Procedural Competence: Validity Considerations.

PubMed

Pugh, Debra M; Wood, Timothy J; Boulet, John R

2015-10-01

Simulation-based medical education (SBME) offers opportunities for trainees to learn how to perform procedures and to be assessed in a safe environment. However, SBME research studies often lack robust evidence to support the validity of the interpretation of the results obtained from tools used to assess trainees' skills. The purpose of this paper is to describe how a validity framework can be applied when reporting and interpreting the results of a simulation-based assessment of skills related to performing procedures. The authors discuss various sources of validity evidence because they relate to SBME. A case study is presented.

The Students' Perceptions of School Success Promoting Strategies Inventory (SPSI): development and validity evidence based studies.

PubMed

Moreira, Paulo A S; Oliveira, João Tiago; Dias, Paulo; Vaz, Filipa Machado; Torres-Oliveira, Isabel

2014-08-04

Students' perceptions about school success promotion strategies are of great importance for schools, as they are an indicator of how students perceive the school success promotion strategies. The objective of this study was to develop and analyze the validity evidence based of The Students' Perceptions of School Success Promoting Strategies Inventory (SPSI), which assesses both individual students' perceptions of their school success promoting strategies, and dimensions of school quality. A structure of 7 related factors was found, which showed good adjustment indices in two additional different samples, suggesting that this is a well-fitting multi-group model (p < .001). All scales presented good reliability values. Schools with good academic results registered higher values in Career development, Active learning, Proximity, Educational Technologies and Extra-curricular activities (p < .05). SPSI showed to be adequate to measure within-schools (students within schools) dimensions of school success. In addition, there is preliminary evidence for its adequacy for measuring school success promotion dimensions between schools for 4 dimensions. This study supports the validity evidence based of the SPSI (validity evidence based on test content, on internal structure, on relations to other variables and on consequences of testing). Future studies should test for within- and between-level variance in a bigger sample of schools.

Development and Validation of a Measure of Elementary Teachers' Science Content Knowledge in Two Multiyear Teacher Professional Development Intervention Projects

ERIC Educational Resources Information Center

Maerten-Rivera, Jaime Lynn; Huggins-Manley, Anne Corinne; Adamson, Karen; Lee, Okhee; Llosa, Lorena

2015-01-01

Using data collected from two multiyear teacher professional development projects employing randomized control trials, this study describes the development and validation of a paper-based test of elementary teachers' science content knowledge (SCK). Evidence of construct validity is presented, including evidence on internal structural…

Assessing mental health clinicians' intentions to adopt evidence-based treatments: reliability and validity testing of the evidence-based treatment intentions scale.

PubMed

Williams, Nathaniel J

2016-05-05

Intentions play a central role in numerous empirically supported theories of behavior and behavior change and have been identified as a potentially important antecedent to successful evidence-based treatment (EBT) implementation. Despite this, few measures of mental health clinicians' EBT intentions exist and available measures have not been subject to thorough psychometric evaluation or testing. This paper evaluates the psychometric properties of the evidence-based treatment intentions (EBTI) scale, a new measure of mental health clinicians' intentions to adopt EBTs. The study evaluates the reliability and validity of inferences made with the EBTI using multi-method, multi-informant criterion variables collected over 12 months from a sample of 197 mental health clinicians delivering services in 13 mental health agencies. Structural, predictive, and discriminant validity evidence is assessed. Findings support the EBTI's factor structure (χ (2) = 3.96, df = 5, p = .556) and internal consistency reliability (α = .80). Predictive validity evidence was provided by robust and significant associations between EBTI scores and clinicians' observer-reported attendance at a voluntary EBT workshop at a 1-month follow-up (OR = 1.92, p < .05), self-reported EBT adoption at a 12-month follow-up (R (2) = .17, p < .001), and self-reported use of EBTs with clients at a 12-month follow-up (R (2) = .25, p < .001). Discriminant validity evidence was provided by small associations with clinicians' concurrently measured psychological work climate perceptions of functionality (R (2) = .06, p < .05), engagement (R (2) = .06, p < .05), and stress (R (2) = .00, ns). The EBTI is a practical and theoretically grounded measure of mental health clinicians' EBT intentions. Scores on the EBTI provide a basis for valid inferences regarding mental health clinicians' intentions to adopt EBTs. Discussion focuses on research and practice applications.

The Utrecht questionnaire (U-CEP) measuring knowledge on clinical epidemiology proved to be valid.

PubMed

Kortekaas, Marlous F; Bartelink, Marie-Louise E L; de Groot, Esther; Korving, Helen; de Wit, Niek J; Grobbee, Diederick E; Hoes, Arno W

2017-02-01

Knowledge on clinical epidemiology is crucial to practice evidence-based medicine. We describe the development and validation of the Utrecht questionnaire on knowledge on Clinical epidemiology for Evidence-based Practice (U-CEP); an assessment tool to be used in the training of clinicians. The U-CEP was developed in two formats: two sets of 25 questions and a combined set of 50. The validation was performed among postgraduate general practice (GP) trainees, hospital trainees, GP supervisors, and experts. Internal consistency, internal reliability (item-total correlation), item discrimination index, item difficulty, content validity, construct validity, responsiveness, test-retest reliability, and feasibility were assessed. The questionnaire was externally validated. Internal consistency was good with a Cronbach alpha of 0.8. The median item-total correlation and mean item discrimination index were satisfactory. Both sets were perceived as relevant to clinical practice. Construct validity was good. Both sets were responsive but failed on test-retest reliability. One set took 24 minutes and the other 33 minutes to complete, on average. External GP trainees had comparable results. The U-CEP is a valid questionnaire to assess knowledge on clinical epidemiology, which is a prerequisite for practicing evidence-based medicine in daily clinical practice. Copyright © 2016 Elsevier Inc. All rights reserved.

Policy and Validity Prospects for Performance-Based Assessment.

ERIC Educational Resources Information Center

Baker, Eva L.; And Others

1994-01-01

This article describes performance-based assessment as expounded by its proponents, comments on these conceptions, reviews evidence regarding the technical quality of performance-based assessment, and considers its validity under various policy options. (JDD)

Validity evidence for an OSCE to assess competency in systems-based practice and practice-based learning and improvement: a preliminary investigation.

PubMed

Varkey, Prathibha; Natt, Neena; Lesnick, Timothy; Downing, Steven; Yudkowsky, Rachel

2008-08-01

To determine the psychometric properties and validity of an OSCE to assess the competencies of Practice-Based Learning and Improvement (PBLI) and Systems-Based Practice (SBP) in graduate medical education. An eight-station OSCE was piloted at the end of a three-week Quality Improvement elective for nine preventive medicine and endocrinology fellows at Mayo Clinic. The stations assessed performance in quality measurement, root cause analysis, evidence-based medicine, insurance systems, team collaboration, prescription errors, Nolan's model, and negotiation. Fellows' performance in each of the stations was assessed by three faculty experts using checklists and a five-point global competency scale. A modified Angoff procedure was used to set standards. Evidence for the OSCE's validity, feasibility, and acceptability was gathered. Evidence for content and response process validity was judged as excellent by institutional content experts. Interrater reliability of scores ranged from 0.85 to 1 for most stations. Interstation correlation coefficients ranged from -0.62 to 0.99, reflecting case specificity. Implementation cost was approximately $255 per fellow. All faculty members agreed that the OSCE was realistic and capable of providing accurate assessments. The OSCE provides an opportunity to systematically sample the different subdomains of Quality Improvement. Furthermore, the OSCE provides an opportunity for the demonstration of skills rather than the testing of knowledge alone, thus making it a potentially powerful assessment tool for SBP and PBLI. The study OSCE was well suited to assess SBP and PBLI. The evidence gathered through this study lays the foundation for future validation work.

Testing of the SEE and OEE post-hip fracture.

PubMed

Resnick, Barbara; Orwig, Denise; Zimmerman, Sheryl; Hawkes, William; Golden, Justine; Werner-Bronzert, Michelle; Magaziner, Jay

2006-08-01

The purpose of this study was to test the reliability and validity of the Self-Efficacy for Exercise (SEE) and the Outcome Expectations for Exercise (OEE) scales in a sample of 166 older women post-hip fracture. There was some evidence of validity of the SEE and OEE based on confirmatory factor analysis and Rasch model testing, criterion based and convergent validity, and evidence of internal consistency based on alpha coefficients and separation indices and reliability based on R2 estimates. Rasch model testing demonstrated that some items had high variability. Based on these findings suggestions are made for how items could be revised and the scales improved for future use.

Gathering Validity Evidence for Surgical Simulation: A Systematic Review.

PubMed

Borgersen, Nanna Jo; Naur, Therese M H; Sørensen, Stine M D; Bjerrum, Flemming; Konge, Lars; Subhi, Yousif; Thomsen, Ann Sofia S

2018-06-01

To identify current trends in the use of validity frameworks in surgical simulation, to provide an overview of the evidence behind the assessment of technical skills in all surgical specialties, and to present recommendations and guidelines for future validity studies. Validity evidence for assessment tools used in the evaluation of surgical performance is of paramount importance to ensure valid and reliable assessment of skills. We systematically reviewed the literature by searching 5 databases (PubMed, EMBASE, Web of Science, PsycINFO, and the Cochrane Library) for studies published from January 1, 2008, to July 10, 2017. We included original studies evaluating simulation-based assessments of health professionals in surgical specialties and extracted data on surgical specialty, simulator modality, participant characteristics, and the validity framework used. Data were synthesized qualitatively. We identified 498 studies with a total of 18,312 participants. Publications involving validity assessments in surgical simulation more than doubled from 2008 to 2010 (∼30 studies/year) to 2014 to 2016 (∼70 to 90 studies/year). Only 6.6% of the studies used the recommended contemporary validity framework (Messick). The majority of studies used outdated frameworks such as face validity. Significant differences were identified across surgical specialties. The evaluated assessment tools were mostly inanimate or virtual reality simulation models. An increasing number of studies have gathered validity evidence for simulation-based assessments in surgical specialties, but the use of outdated frameworks remains common. To address the current practice, this paper presents guidelines on how to use the contemporary validity framework when designing validity studies.

Automated Assessment of the Quality of Depression Websites

PubMed Central

Tang, Thanh Tin; Hawking, David; Christensen, Helen

2005-01-01

Background Since health information on the World Wide Web is of variable quality, methods are needed to assist consumers to identify health websites containing evidence-based information. Manual assessment tools may assist consumers to evaluate the quality of sites. However, these tools are poorly validated and often impractical. There is a need to develop better consumer tools, and in particular to explore the potential of automated procedures for evaluating the quality of health information on the web. Objective This study (1) describes the development of an automated quality assessment procedure (AQA) designed to automatically rank depression websites according to their evidence-based quality; (2) evaluates the validity of the AQA relative to human rated evidence-based quality scores; and (3) compares the validity of Google PageRank and the AQA as indicators of evidence-based quality. Method The AQA was developed using a quality feedback technique and a set of training websites previously rated manually according to their concordance with statements in the Oxford University Centre for Evidence-Based Mental Health’s guidelines for treating depression. The validation phase involved 30 websites compiled from the DMOZ, Yahoo! and LookSmart Depression Directories by randomly selecting six sites from each of the Google PageRank bands of 0, 1-2, 3-4, 5-6 and 7-8. Evidence-based ratings from two independent raters (based on concordance with the Oxford guidelines) were then compared with scores derived from the automated AQA and Google algorithms. There was no overlap in the websites used in the training and validation phases of the study. Results The correlation between the AQA score and the evidence-based ratings was high and significant (r=0.85, P<.001). Addition of a quadratic component improved the fit, the combined linear and quadratic model explaining 82 percent of the variance. The correlation between Google PageRank and the evidence-based score was lower than that for the AQA. When sites with zero PageRanks were included the association was weak and non-significant (r=0.23, P=.22). When sites with zero PageRanks were excluded, the correlation was moderate (r=.61, P=.002). Conclusions Depression websites of different evidence-based quality can be differentiated using an automated system. If replicable, generalizable to other health conditions and deployed in a consumer-friendly form, the automated procedure described here could represent an important advance for consumers of Internet medical information. PMID:16403723

Development and validation of the Evidence Based Medicine Questionnaire (EBMQ) to assess doctors' knowledge, practice and barriers regarding the implementation of evidence-based medicine in primary care.

PubMed

Hisham, Ranita; Ng, Chirk Jenn; Liew, Su May; Lai, Pauline Siew Mei; Chia, Yook Chin; Khoo, Ee Ming; Hanafi, Nik Sherina; Othman, Sajaratulnisah; Lee, Ping Yein; Abdullah, Khatijah Lim; Chinna, Karuthan

2018-06-23

Evidence-Based Medicine (EBM) integrates best available evidence from literature and patients' values, which then informs clinical decision making. However, there is a lack of validated instruments to assess the knowledge, practice and barriers of primary care physicians in the implementation of EBM. This study aimed to develop and validate an Evidence-Based Medicine Questionnaire (EBMQ) in Malaysia. The EBMQ was developed based on a qualitative study, literature review and an expert panel. Face and content validity was verified by the expert panel and piloted among 10 participants. Primary care physicians with or without EBM training who could understand English were recruited from December 2015 to January 2016. The EBMQ was administered at baseline and two weeks later. A higher score indicates better knowledge, better practice of EBM and less barriers towards the implementation of EBM. We hypothesized that the EBMQ would have three domains: knowledge, practice and barriers. The final version of the EBMQ consists of 80 items: 62 items were measured on a nominal scale, 22 items were measured on a 5 point Likert-scale. Flesch reading ease was 61.2. A total of 343 participants were approached; of whom 320 agreed to participate (response rate = 93.2%). Factor analysis revealed that the EBMQ had eight domains after 13 items were removed: "EBM websites", "evidence-based journals", "types of studies", "terms related to EBM", "practice", "access", "patient preferences" and "support". Cronbach alpha for the overall EBMQ was 0.909, whilst the Cronbach alpha for the individual domain ranged from 0.657-0.940. The EBMQ was able to discriminate between doctors with and without EBM training for 24 out of 42 items. At test-retest, kappa values ranged from 0.155 to 0.620. The EBMQ was found to be a valid and reliable instrument to assess the knowledge, practice and barriers towards the implementation of EBM among primary care physicians in Malaysia.

Creation of a Tool for Assessing Knowledge in Evidence-Based Decision-Making in Practicing Health Care Providers.

PubMed

Spurr, Kathy; Dechman, Gail; Lackie, Kelly; Gilbert, Robert

2016-01-01

Evidence-based decision-making (EBDM) is the process health care providers (HCPs) use to identify and appraise potential evidence. It supports the integration of best research evidence with clinical expertise and patient values into the decision-making process for patient care. Competence in this process is essential to delivery of optimal care. There is no objective tool that assesses EBDM across HCP groups. This research aimed to develop a content valid tool to assess knowledge of the principles of evidence-based medicine and the EBDM process, for use with all HCPs. A Delphi process was used in the creation of the tool. Pilot testing established its content validity with the added benefit of evaluating HCPs' knowledge of EBDM. Descriptive statistics and multivariate mixed models were used to evaluate individual survey responses in total, as well as within each EBDM component. The tool consisted of 26 multiple-choice questions. A total of 12,884 HCPs in Nova Scotia were invited to participate in the web-based validation study, yielding 818 (6.3%) participants, 471 of whom completed all questions. The mean overall score was 68%. Knowledge in one component, integration of evidence with clinical expertise and patient preferences, was identified as needing development across all HCPs surveyed. A content valid tool for assessing HCP EBDM knowledge was created and can be used to support the development of continuing education programs to enhance EBDM competency.

Update on simulation-based surgical training and assessment in ophthalmology: a systematic review.

PubMed

Thomsen, Ann Sofia S; Subhi, Yousif; Kiilgaard, Jens Folke; la Cour, Morten; Konge, Lars

2015-06-01

This study reviews the evidence behind simulation-based surgical training of ophthalmologists to determine (1) the validity of the reported models and (2) the ability to transfer skills to the operating room. Simulation-based training is established widely within ophthalmology, although it often lacks a scientific basis for implementation. We conducted a systematic review of trials involving simulation-based training or assessment of ophthalmic surgical skills among health professionals. The search included 5 databases (PubMed, EMBASE, PsycINFO, Cochrane Library, and Web of Science) and was completed on March 1, 2014. Overall, the included trials were divided into animal, cadaver, inanimate, and virtual-reality models. Risk of bias was assessed using the Cochrane Collaboration's tool. Validity evidence was evaluated using a modern validity framework (Messick's). We screened 1368 reports for eligibility and included 118 trials. The most common surgery simulated was cataract surgery. Most validity trials investigated only 1 or 2 of 5 sources of validity (87%). Only 2 trials (48 participants) investigated transfer of skills to the operating room; 4 trials (65 participants) evaluated the effect of simulation-based training on patient-related outcomes. Because of heterogeneity of the studies, it was not possible to conduct a quantitative analysis. The methodologic rigor of trials investigating simulation-based surgical training in ophthalmology is inadequate. To ensure effective implementation of training models, evidence-based knowledge of validity and efficacy is needed. We provide a useful tool for implementation and evaluation of research in simulation-based training. Copyright © 2015 American Academy of Ophthalmology. Published by Elsevier Inc. All rights reserved.

Construct Validity Evidence for Single-Response Items to Estimate Physical Activity Levels in Large Sample Studies

ERIC Educational Resources Information Center

Jackson, Allen W.; Morrow, James R., Jr.; Bowles, Heather R.; FitzGerald, Shannon J.; Blair, Steven N.

2007-01-01

Valid measurement of physical activity is important for studying the risks for morbidity and mortality. The purpose of this study was to examine evidence of construct validity of two similar single-response items assessing physical activity via self-report. Both items are based on the stages of change model. The sample was 687 participants (men =…

Testing Reading Comprehension of Theoretical Discourse with Cloze.

ERIC Educational Resources Information Center

Greene, Benjamin B., Jr.

2001-01-01

Presents evidence from a large sample of reading test scores for the validity of cloze-based assessments of reading comprehension for the discourse typically encountered in introductory college economics textbooks. Notes that results provide strong evidence that appropriately designed cloze tests permit valid assessments of reading comprehension…

Nutrition screening tools: an analysis of the evidence.

PubMed

Skipper, Annalynn; Ferguson, Maree; Thompson, Kyle; Castellanos, Victoria H; Porcari, Judy

2012-05-01

In response to questions about tools for nutrition screening, an evidence analysis project was developed to identify the most valid and reliable nutrition screening tools for use in acute care and hospital-based ambulatory care settings. An oversight group defined nutrition screening and literature search criteria. A trained analyst conducted structured searches of the literature for studies of nutrition screening tools according to predetermined criteria. Eleven nutrition screening tools designed to detect undernutrition in patients in acute care and hospital-based ambulatory care were identified. Trained analysts evaluated articles for quality using criteria specified by the American Dietetic Association's Evidence Analysis Library. Members of the oversight group assigned quality grades to the tools based on the quality of the supporting evidence, including reliability and validity data. One tool, the NRS-2002, received a grade I, and 4 tools-the Simple Two-Part Tool, the Mini-Nutritional Assessment-Short Form (MNA-SF), the Malnutrition Screening Tool (MST), and Malnutrition Universal Screening Tool (MUST)-received a grade II. The MST was the only tool shown to be both valid and reliable for identifying undernutrition in the settings studied. Thus, validated nutrition screening tools that are simple and easy to use are available for application in acute care and hospital-based ambulatory care settings.

Why the Evidence-Based Paradigm in Early Childhood Education and Care Is Anything but Evident

ERIC Educational Resources Information Center

Vandenbroeck, Michel; Roets, Griet; Roose, Rudi

2012-01-01

Praxeological research is a necessary contribution to the research field in early childhood education and care, which is currently dominated by an evidence-based paradigm that tends to consider the measurement of predefined outcomes as the most valid form of research. We analyse the history of the evidence-based paradigm in the field of medicine…

Factor analysis methods and validity evidence: A systematic review of instrument development across the continuum of medical education

NASA Astrophysics Data System (ADS)

Wetzel, Angela Payne

Previous systematic reviews indicate a lack of reporting of reliability and validity evidence in subsets of the medical education literature. Psychology and general education reviews of factor analysis also indicate gaps between current and best practices; yet, a comprehensive review of exploratory factor analysis in instrument development across the continuum of medical education had not been previously identified. Therefore, the purpose for this study was critical review of instrument development articles employing exploratory factor or principal component analysis published in medical education (2006--2010) to describe and assess the reporting of methods and validity evidence based on the Standards for Educational and Psychological Testing and factor analysis best practices. Data extraction of 64 articles measuring a variety of constructs that have been published throughout the peer-reviewed medical education literature indicate significant errors in the translation of exploratory factor analysis best practices to current practice. Further, techniques for establishing validity evidence tend to derive from a limited scope of methods including reliability statistics to support internal structure and support for test content. Instruments reviewed for this study lacked supporting evidence based on relationships with other variables and response process, and evidence based on consequences of testing was not evident. Findings suggest a need for further professional development within the medical education researcher community related to (1) appropriate factor analysis methodology and reporting and (2) the importance of pursuing multiple sources of reliability and validity evidence to construct a well-supported argument for the inferences made from the instrument. Medical education researchers and educators should be cautious in adopting instruments from the literature and carefully review available evidence. Finally, editors and reviewers are encouraged to recognize this gap in best practices and subsequently to promote instrument development research that is more consistent through the peer-review process.

Evaluating an Instrument to Measure Mental Load and Mental Effort Considering Different Sources of Validity Evidence

ERIC Educational Resources Information Center

Krell, Moritz

2017-01-01

This study evaluates a 12-item instrument for subjective measurement of mental load (ML) and mental effort (ME) by analysing different sources of validity evidence. The findings of an expert judgement (N = 8) provide "evidence based on test content" that the formulation of the items corresponds to the meaning of ML and ME. An empirical…

Hierarchy of evidence: a simple system for orthopaedic research?

PubMed

Pemberton, Julia; Kraeva, Juliana; Bhandari, Mohit

2007-01-01

To be able to make a sound recommendation for a treatment based on the best available evidence, it is necessary to follow specific steps in acquiring literature, appraising the study design and quality, and assessing the results. Evidence-based medicine is founded on the concepts of using best evidence, levels of evidence, and grades of recommendation, and aims to provide clinicians with standardized rules to help them appraise the validity of published research. A number of systems have been developed to categorize research studies into consistent levels of evidence. These systems are based primarily on consensus expert opinion, and have not been validated to any extent. The use of different systems does not allow for effective communication between users; there is a lack of accord even between users of the same system. The GRADE working group has devised a new rating system that attempts to address deficiencies seen within other systems.

Social Skills Questionnaire for Argentinean College Students (SSQ-U) Development and Validation.

PubMed

Morán, Valeria E; Olaz, Fabián O; Del Prette, Zilda A P

2015-11-27

In this paper we present a new instrument called Social Skills Questionnaire for Argentinean College Students (SSQ-U). Based on the adapted version of the Social Skills Inventory - Del Prette (SSI-Del Prette) (Olaz, Medrano, Greco, & Del Prette, 2009), we wrote new items for the scale, and carried out psychometric analysis to assess the validity and reliability of the instrument. In the first study, we collected evidence based on test content through expert judges who evaluated the quality and the relevance of the items. In the second and third studies, we provided validity evidence based on the internal structure of the instrument using exploratory (n = 1067) and confirmatory (n = 661) factor analysis. Results suggested a five-factor structure consistent with the dimensions of social skills, as proposed by Kelly (2002). The fit indexes corresponding to the obtained model were adequate, and composite reliability coefficients of each factor were excellent (above .75). Finally, in the fourth study, we provided evidence of convergent and discriminant validity. The obtained results allow us to conclude that the SSQ-U is the first valid and reliable instrument for measuring social skills in Argentinean college students.

Developing an evidence-based practice protocol: implications for midwifery practice.

PubMed

Carr, K C

2000-01-01

Evidence-based practice is defined and its importance to midwifery practice is presented. Guidelines are provided for the development of an evidence-based practice protocol. These include: identifying the clinical question, obtaining the evidence, evaluating the validity and importance of the evidence, synthesizing the evidence and applying it to the development of a protocol or clinical algorithm, and, finally, developing an evaluation plan or measurement strategy to see if the new protocol is effective.

Evidence-based medicine for every day, everyone, and every therapeutic study.

PubMed

Govindarajan, Raghav; Narayanaswami, Pushpa

2018-04-17

The rapid growth in published medical literature makes it difficult for clinicians to keep up with advances in their fields. This may result in a cursory scan of the abstract and conclusion of a study without critically evaluating study quality. The application of evidence-based medicine (EBM) is the process of converting the abstract task of reading the literature into a practical method of using the literature to inform care in a specific clinical context while simultaneously expanding one's knowledge. EBM involves 4 steps: (1) stating the clinical problem in a defined question; (2) searching the literature for the evidence; (3) critically appraising the evidence for its validity; and (4) applying the evidence in the context of the patient's situation, preferences, and values. In this review, we use the recently published trial of thymectomy in myasthenia gravis as an example and systematically go through the steps of assessing internal validity, precision, and external validity. Muscle Nerve, 2018. © 2018 Wiley Periodicals, Inc.

Predicting implementation from organizational readiness for change: a study protocol

PubMed Central

2011-01-01

Background There is widespread interest in measuring organizational readiness to implement evidence-based practices in clinical care. However, there are a number of challenges to validating organizational measures, including inferential bias arising from the halo effect and method bias - two threats to validity that, while well-documented by organizational scholars, are often ignored in health services research. We describe a protocol to comprehensively assess the psychometric properties of a previously developed survey, the Organizational Readiness to Change Assessment. Objectives Our objective is to conduct a comprehensive assessment of the psychometric properties of the Organizational Readiness to Change Assessment incorporating methods specifically to address threats from halo effect and method bias. Methods and Design We will conduct three sets of analyses using longitudinal, secondary data from four partner projects, each testing interventions to improve the implementation of an evidence-based clinical practice. Partner projects field the Organizational Readiness to Change Assessment at baseline (n = 208 respondents; 53 facilities), and prospectively assesses the degree to which the evidence-based practice is implemented. We will conduct predictive and concurrent validities using hierarchical linear modeling and multivariate regression, respectively. For predictive validity, the outcome is the change from baseline to follow-up in the use of the evidence-based practice. We will use intra-class correlations derived from hierarchical linear models to assess inter-rater reliability. Two partner projects will also field measures of job satisfaction for convergent and discriminant validity analyses, and will field Organizational Readiness to Change Assessment measures at follow-up for concurrent validity (n = 158 respondents; 33 facilities). Convergent and discriminant validities will test associations between organizational readiness and different aspects of job satisfaction: satisfaction with leadership, which should be highly correlated with readiness, versus satisfaction with salary, which should be less correlated with readiness. Content validity will be assessed using an expert panel and modified Delphi technique. Discussion We propose a comprehensive protocol for validating a survey instrument for assessing organizational readiness to change that specifically addresses key threats of bias related to halo effect, method bias and questions of construct validity that often go unexplored in research using measures of organizational constructs. PMID:21777479

Literature evidence in open targets - a target validation platform.

PubMed

Kafkas, Şenay; Dunham, Ian; McEntyre, Johanna

2017-06-06

We present the Europe PMC literature component of Open Targets - a target validation platform that integrates various evidence to aid drug target identification and validation. The component identifies target-disease associations in documents and ranks the documents based on their confidence from the Europe PMC literature database, by using rules utilising expert-provided heuristic information. The confidence score of a given document represents how valuable the document is in the scope of target validation for a given target-disease association by taking into account the credibility of the association based on the properties of the text. The component serves the platform regularly with the up-to-date data since December, 2015. Currently, there are a total number of 1168365 distinct target-disease associations text mined from >26 million PubMed abstracts and >1.2 million Open Access full text articles. Our comparative analyses on the current available evidence data in the platform revealed that 850179 of these associations are exclusively identified by literature mining. This component helps the platform's users by providing the most relevant literature hits for a given target and disease. The text mining evidence along with the other types of evidence can be explored visually through https://www.targetvalidation.org and all the evidence data is available for download in json format from https://www.targetvalidation.org/downloads/data .

Evidence flow graph methods for validation and verification of expert systems

NASA Technical Reports Server (NTRS)

Becker, Lee A.; Green, Peter G.; Bhatnagar, Jayant

1989-01-01

The results of an investigation into the use of evidence flow graph techniques for performing validation and verification of expert systems are given. A translator to convert horn-clause rule bases into evidence flow graphs, a simulation program, and methods of analysis were developed. These tools were then applied to a simple rule base which contained errors. It was found that the method was capable of identifying a variety of problems, for example that the order of presentation of input data or small changes in critical parameters could affect the output from a set of rules.

The Application of Evidence-Based Practice to Nonspeech Oral Motor Treatments

ERIC Educational Resources Information Center

Lass, Norman J.; Pannbacker, Mary

2008-01-01

Purpose: The purpose of this article is to help speech-language pathologists (SLPs) apply the principles of evidence-based practice (EBP) to nonspeech oral motor treatments (NSOMTs) in order to make valid, evidence-based decisions about NSOMTs and thus determine if they are viable treatment approaches for the management of communication disorders.…

The predictive validity of quality of evidence grades for the stability of effect estimates was low: a meta-epidemiological study.

PubMed

Gartlehner, Gerald; Dobrescu, Andreea; Evans, Tammeka Swinson; Bann, Carla; Robinson, Karen A; Reston, James; Thaler, Kylie; Skelly, Andrea; Glechner, Anna; Peterson, Kimberly; Kien, Christina; Lohr, Kathleen N

2016-02-01

To determine the predictive validity of the U.S. Evidence-based Practice Center (EPC) approach to GRADE (Grading of Recommendations Assessment, Development and Evaluation). Based on Cochrane reports with outcomes graded as high quality of evidence (QOE), we prepared 160 documents which represented different levels of QOE. Professional systematic reviewers dually graded the QOE. For each document, we determined whether estimates were concordant with high QOE estimates of the Cochrane reports. We compared the observed proportion of concordant estimates with the expected proportion from an international survey. To determine the predictive validity, we used the Hosmer-Lemeshow test to assess calibration and the C (concordance) index to assess discrimination. The predictive validity of the EPC approach to GRADE was limited. Estimates graded as high QOE were less likely, estimates graded as low or insufficient QOE more likely to remain stable than expected. The EPC approach to GRADE could not reliably predict the likelihood that individual bodies of evidence remain stable as new evidence becomes available. C-indices ranged between 0.56 (95% CI, 0.47 to 0.66) and 0.58 (95% CI, 0.50 to 0.67) indicating a low discriminatory ability. The limited predictive validity of the EPC approach to GRADE seems to reflect a mismatch between expected and observed changes in treatment effects as bodies of evidence advance from insufficient to high QOE. Copyright © 2016 Elsevier Inc. All rights reserved.

Policy Claims and Problem Frames: A Cross-Case Comparison of Evidence-Based Policy in an Australian Context

ERIC Educational Resources Information Center

van Toorn, Georgia; Dowse, Leanne

2016-01-01

This paper explores the signification of evidence-based policy as a new policy-making paradigm in Australia through a cross-case comparison of the role of evidence in two key areas: child protection and illicit drug policy. Although evidence makes certain courses of action appear valid and credible, quality evidence is not necessarily the critical…

Sicily statement on evidence-based practice

PubMed Central

Dawes, Martin; Summerskill, William; Glasziou, Paul; Cartabellotta, Antonino; Martin, Janet; Hopayian, Kevork; Porzsolt, Franz; Burls, Amanda; Osborne, James

2005-01-01

Background A variety of definitions of evidence-based practice (EBP) exist. However, definitions are in themselves insufficient to explain the underlying processes of EBP and to differentiate between an evidence-based process and evidence-based outcome. There is a need for a clear statement of what Evidence-Based Practice (EBP) means, a description of the skills required to practise in an evidence-based manner and a curriculum that outlines the minimum requirements for training health professionals in EBP. This consensus statement is based on current literature and incorporating the experience of delegates attending the 2003 Conference of Evidence-Based Health Care Teachers and Developers ("Signposting the future of EBHC"). Discussion Evidence-Based Practice has evolved in both scope and definition. Evidence-Based Practice (EBP) requires that decisions about health care are based on the best available, current, valid and relevant evidence. These decisions should be made by those receiving care, informed by the tacit and explicit knowledge of those providing care, within the context of available resources. Health care professionals must be able to gain, assess, apply and integrate new knowledge and have the ability to adapt to changing circumstances throughout their professional life. Curricula to deliver these aptitudes need to be grounded in the five-step model of EBP, and informed by ongoing research. Core assessment tools for each of the steps should continue to be developed, validated, and made freely available. Summary All health care professionals need to understand the principles of EBP, recognise EBP in action, implement evidence-based policies, and have a critical attitude to their own practice and to evidence. Without these skills, professionals and organisations will find it difficult to provide 'best practice'. PMID:15634359

[The added value of information summaries supporting clinical decisions at the point-of-care.

PubMed

Banzi, Rita; González-Lorenzo, Marien; Kwag, Koren Hyogene; Bonovas, Stefanos; Moja, Lorenzo

2016-11-01

Evidence-based healthcare requires the integration of the best research evidence with clinical expertise and patients' values. International publishers are developing evidence-based information services and resources designed to overcome the difficulties in retrieving, assessing and updating medical information as well as to facilitate a rapid access to valid clinical knowledge. Point-of-care information summaries are defined as web-based medical compendia that are specifically designed to deliver pre-digested, rapidly accessible, comprehensive, and periodically updated information to health care providers. Their validity must be assessed against marketing claims that they are evidence-based. We periodically evaluate the content development processes of several international point-of-care information summaries. The number of these products has increased along with their quality. The last analysis done in 2014 identified 26 products and found that three of them (Best Practice, Dynamed e Uptodate) scored the highest across all evaluated dimensions (volume, quality of the editorial process and evidence-based methodology). Point-of-care information summaries as stand-alone products or integrated with other systems, are gaining ground to support clinical decisions. The choice of one product over another depends both on the properties of the service and the preference of users. However, even the most innovative information system must rely on transparent and valid contents. Individuals and institutions should regularly assess the value of point-of-care summaries as their quality changes rapidly over time.

Examining the Validity of GED[R] Tests Scores with Scheduling and Setting Accommodations. GED Testing Service Research Studies, 2004-1

ERIC Educational Resources Information Center

George-Ezzelle, Carol E.; Skaggs, Gary

2004-01-01

Current testing standards call for test developers to provide evidence that testing procedures and test scores, and the inferences made based on the test scores, show evidence of validity and are comparable across subpopulations (American Educational Research Association [AERA], American Psychological Association [APA], & National Council on…

A New Method for Analyzing Content Validity Data Using Multidimensional Scaling

ERIC Educational Resources Information Center

Li, Xueming; Sireci, Stephen G.

2013-01-01

Validity evidence based on test content is of essential importance in educational testing. One source for such evidence is an alignment study, which helps evaluate the congruence between tested objectives and those specified in the curriculum. However, the results of an alignment study do not always sufficiently capture the degree to which a test…

Measurement properties of tools measuring mental health knowledge: a systematic review.

PubMed

Wei, Yifeng; McGrath, Patrick J; Hayden, Jill; Kutcher, Stan

2016-08-23

Mental health literacy has received great attention recently to improve mental health knowledge, decrease stigma and enhance help-seeking behaviors. We conducted a systematic review to critically appraise the qualities of studies evaluating the measurement properties of mental health knowledge tools and the quality of included measurement properties. We searched PubMed, PsycINFO, EMBASE, CINAHL, the Cochrane Library, and ERIC for studies addressing psychometrics of mental health knowledge tools and published in English. We applied the COSMIN checklist to assess the methodological quality of each study as "excellent", "good", "fair", or "indeterminate". We ranked the level of evidence of the overall quality of each measurement property across studies as "strong", "moderate", "limited", "conflicting", or "unknown". We identified 16 mental health knowledge tools in 17 studies, addressing reliability, validity, responsiveness or measurement errors. The methodological quality of included studies ranged from "poor" to "excellent" including 6 studies addressing the content validity, internal consistency or structural validity demonstrating "excellent" quality. We found strong evidence of the content validity or internal consistency of 6 tools; moderate evidence of the internal consistency, the content validity or the reliability of 8 tools; and limited evidence of the reliability, the structural validity, the criterion validity, or the construct validity of 12 tools. Both the methodological qualities of included studies and the overall evidence of measurement properties are mixed. Based on the current evidence, we recommend that researchers consider using tools with measurement properties of strong or moderate evidence that also reached the threshold for positive ratings according to COSMIN checklist.

Measurement properties of depression questionnaires in patients with diabetes: a systematic review.

PubMed

van Dijk, Susan E M; Adriaanse, Marcel C; van der Zwaan, Lennart; Bosmans, Judith E; van Marwijk, Harm W J; van Tulder, Maurits W; Terwee, Caroline B

2018-06-01

To conduct a systematic review on measurement properties of questionnaires measuring depressive symptoms in adult patients with type 1 or type 2 diabetes. A systematic review of the literature in MEDLINE, EMbase and PsycINFO was performed. Full text, original articles, published in any language up to October 2016 were included. Eligibility for inclusion was independently assessed by three reviewers who worked in pairs. Methodological quality of the studies was evaluated by two independent reviewers using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. Quality of the questionnaires was rated per measurement property, based on the number and quality of the included studies and the reported results. Of 6286 unique hits, 21 studies met our criteria evaluating nine different questionnaires in multiple settings and languages. The methodological quality of the included studies was variable for the different measurement properties: 9/15 studies scored 'good' or 'excellent' on internal consistency, 2/5 on reliability, 0/1 on content validity, 10/10 on structural validity, 8/11 on hypothesis testing, 1/5 on cross-cultural validity, and 4/9 on criterion validity. For the CES-D, there was strong evidence for good internal consistency, structural validity, and construct validity; moderate evidence for good criterion validity; and limited evidence for good cross-cultural validity. The PHQ-9 and WHO-5 also performed well on several measurement properties. However, the evidence for structural validity of the PHQ-9 was inconclusive. The WHO-5 was less extensively researched and originally not developed to measure depression. Currently, the CES-D is best supported for measuring depressive symptoms in diabetes patients.

Validating workplace performance assessments in health sciences students: a case study from speech pathology.

PubMed

McAllister, Sue; Lincoln, Michelle; Ferguson, Allison; McAllister, Lindy

2013-01-01

Valid assessment of health science students' ability to perform in the real world of workplace practice is critical for promoting quality learning and ultimately certifying students as fit to enter the world of professional practice. Current practice in performance assessment in the health sciences field has been hampered by multiple issues regarding assessment content and process. Evidence for the validity of scores derived from assessment tools are usually evaluated against traditional validity categories with reliability evidence privileged over validity, resulting in the paradoxical effect of compromising the assessment validity and learning processes the assessments seek to promote. Furthermore, the dominant statistical approaches used to validate scores from these assessments fall under the umbrella of classical test theory approaches. This paper reports on the successful national development and validation of measures derived from an assessment of Australian speech pathology students' performance in the workplace. Validation of these measures considered each of Messick's interrelated validity evidence categories and included using evidence generated through Rasch analyses to support score interpretation and related action. This research demonstrated that it is possible to develop an assessment of real, complex, work based performance of speech pathology students, that generates valid measures without compromising the learning processes the assessment seeks to promote. The process described provides a model for other health professional education programs to trial.

Evidence flow graph methods for validation and verification of expert systems

NASA Technical Reports Server (NTRS)

Becker, Lee A.; Green, Peter G.; Bhatnagar, Jayant

1988-01-01

This final report describes the results of an investigation into the use of evidence flow graph techniques for performing validation and verification of expert systems. This was approached by developing a translator to convert horn-clause rule bases into evidence flow graphs, a simulation program, and methods of analysis. These tools were then applied to a simple rule base which contained errors. It was found that the method was capable of identifying a variety of problems, for example that the order of presentation of input data or small changes in critical parameters could effect the output from a set of rules.

Circular instead of hierarchical: methodological principles for the evaluation of complex interventions

PubMed Central

Walach, Harald; Falkenberg, Torkel; Fønnebø, Vinjar; Lewith, George; Jonas, Wayne B

2006-01-01

Background The reasoning behind evaluating medical interventions is that a hierarchy of methods exists which successively produce improved and therefore more rigorous evidence based medicine upon which to make clinical decisions. At the foundation of this hierarchy are case studies, retrospective and prospective case series, followed by cohort studies with historical and concomitant non-randomized controls. Open-label randomized controlled studies (RCTs), and finally blinded, placebo-controlled RCTs, which offer most internal validity are considered the most reliable evidence. Rigorous RCTs remove bias. Evidence from RCTs forms the basis of meta-analyses and systematic reviews. This hierarchy, founded on a pharmacological model of therapy, is generalized to other interventions which may be complex and non-pharmacological (healing, acupuncture and surgery). Discussion The hierarchical model is valid for limited questions of efficacy, for instance for regulatory purposes and newly devised products and pharmacological preparations. It is inadequate for the evaluation of complex interventions such as physiotherapy, surgery and complementary and alternative medicine (CAM). This has to do with the essential tension between internal validity (rigor and the removal of bias) and external validity (generalizability). Summary Instead of an Evidence Hierarchy, we propose a Circular Model. This would imply a multiplicity of methods, using different designs, counterbalancing their individual strengths and weaknesses to arrive at pragmatic but equally rigorous evidence which would provide significant assistance in clinical and health systems innovation. Such evidence would better inform national health care technology assessment agencies and promote evidence based health reform. PMID:16796762

The quality of instruments to assess the process of shared decision making: A systematic review.

PubMed

Gärtner, Fania R; Bomhof-Roordink, Hanna; Smith, Ian P; Scholl, Isabelle; Stiggelbout, Anne M; Pieterse, Arwen H

2018-01-01

To inventory instruments assessing the process of shared decision making and appraise their measurement quality, taking into account the methodological quality of their validation studies. In a systematic review we searched seven databases (PubMed, Embase, Emcare, Cochrane, PsycINFO, Web of Science, Academic Search Premier) for studies investigating instruments measuring the process of shared decision making. Per identified instrument, we assessed the level of evidence separately for 10 measurement properties following a three-step procedure: 1) appraisal of the methodological quality using the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist, 2) appraisal of the psychometric quality of the measurement property using three possible quality scores, 3) best-evidence synthesis based on the number of studies, their methodological and psychometrical quality, and the direction and consistency of the results. The study protocol was registered at PROSPERO: CRD42015023397. We included 51 articles describing the development and/or evaluation of 40 shared decision-making process instruments: 16 patient questionnaires, 4 provider questionnaires, 18 coding schemes and 2 instruments measuring multiple perspectives. There is an overall lack of evidence for their measurement quality, either because validation is missing or methods are poor. The best-evidence synthesis indicated positive results for a major part of instruments for content validity (50%) and structural validity (53%) if these were evaluated, but negative results for a major part of instruments when inter-rater reliability (47%) and hypotheses testing (59%) were evaluated. Due to the lack of evidence on measurement quality, the choice for the most appropriate instrument can best be based on the instrument's content and characteristics such as the perspective that they assess. We recommend refinement and validation of existing instruments, and the use of COSMIN-guidelines to help guarantee high-quality evaluations.

Case Study: The Venous Thromboembolism Collaborative Team at the Johns Hopkins Hospital

DTIC Science & Technology

2009-05-21

the use of evidence based medicine as well as a Collaborative of medical and administrative staff, the team developed a computer based decision...audits were conducted for some of the high-risk departments to validate adherence to compliance with evidence - based medicine supporting prevention

Assessing Perceived Emotional Intelligence in Adolescents: New Validity Evidence of Trait Meta-Mood Scale-24

ERIC Educational Resources Information Center

Pedrosa, Ignacio; Suárez-Álvarez, Javier; Lozano, Luis M.; Muñiz, José; García-Cueto, Eduardo

2014-01-01

Adolescence is a critical period of life during which significant psychosocial adjustment occurs and in which emotional intelligence plays an essential role. This article provides validity evidence for the Trait Meta-Mood Scale-24 (TMMS-24) scores based on an item response theory (IRT) approach. A sample of 2,693 Spanish adolescents (M = 16.52…

A systematic review of the reliability and validity of discrete choice experiments in valuing non-market environmental goods.

PubMed

Rakotonarivo, O Sarobidy; Schaafsma, Marije; Hockley, Neal

2016-12-01

While discrete choice experiments (DCEs) are increasingly used in the field of environmental valuation, they remain controversial because of their hypothetical nature and the contested reliability and validity of their results. We systematically reviewed evidence on the validity and reliability of environmental DCEs from the past thirteen years (Jan 2003-February 2016). 107 articles met our inclusion criteria. These studies provide limited and mixed evidence of the reliability and validity of DCE. Valuation results were susceptible to small changes in survey design in 45% of outcomes reporting reliability measures. DCE results were generally consistent with those of other stated preference techniques (convergent validity), but hypothetical bias was common. Evidence supporting theoretical validity (consistency with assumptions of rational choice theory) was limited. In content validity tests, 2-90% of respondents protested against a feature of the survey, and a considerable proportion found DCEs to be incomprehensible or inconsequential (17-40% and 10-62% respectively). DCE remains useful for non-market valuation, but its results should be used with caution. Given the sparse and inconclusive evidence base, we recommend that tests of reliability and validity are more routinely integrated into DCE studies and suggest how this might be achieved. Copyright © 2016 The Authors. Published by Elsevier Ltd.. All rights reserved.

Implementing assessments of robot-assisted technical skill in urological education: a systematic review and synthesis of the validity evidence.

PubMed

Goldenberg, Mitchell G; Lee, Jason Y; Kwong, Jethro C C; Grantcharov, Teodor P; Costello, Anthony

2018-03-31

To systematically review and synthesise the validity evidence supporting intraoperative and simulation-based assessments of technical skill in urological robot-assisted surgery (RAS), and make evidence-based recommendations for the implementation of these assessments in urological training. A literature search of the Medline, PsycINFO and Embase databases was performed. Articles using technical skill and simulation-based assessments in RAS were abstracted. Only studies involving urology trainees or faculty were included in the final analysis. Multiple tools for the assessment of technical robotic skill have been published, with mixed sources of validity evidence to support their use. These evaluations have been used in both the ex vivo and in vivo settings. Performance evaluations range from global rating scales to psychometrics, and assessments are carried out through automation, expert analysts, and crowdsourcing. There have been rapid expansions in approaches to RAS technical skills assessment, both in simulated and clinical settings. Alternative approaches to assessment in RAS, such as crowdsourcing and psychometrics, remain under investigation. Evidence to support the use of these metrics in high-stakes decisions is likely insufficient at present. © 2018 The Authors BJU International © 2018 BJU International Published by John Wiley & Sons Ltd.

Evidence-based dental practice: part I. Formulating clinical questions and searching for answers.

PubMed

Adeyemo, W L; Akinwande, J A; Bamgbose, B O

2007-01-01

Evidence-based dentistry (EBD) is an approach to oral health care that requires the judicious integration of systematic assessments of clinically relevant scientific evidence, relating to the patient's oral and medical condition and history, with the dentist's clinical expertise and the patient's treatment needs and preferences. Evidence-based care is now regarded as the "gold standard" in health care delivery worldwide. EBD involves tracking down the available evidence, assessing its validity and relevance, and then using the "best" evidence to inform decisions regarding care. Although, the concept of evidence-based dentistry is not new, however, anecdotal evidence suggests that the awareness of this concept among Nigerian dental practitioners is low. This first of three articles on evidence-based dental practice discusses the historical background of evidence-based medicine/evidence-based dentistry, how to formulate clear clinical questions and how to track down (search) the available evidence in the literature databases.

Construct Validation in Counseling Psychology Research

ERIC Educational Resources Information Center

Hoyt, William T.; Warbasse, Rosalia E.; Chu, Erica Y.

2006-01-01

Counseling psychology researchers devote little attention to theory-based measurement validation, as evidenced by cursory mention of validity issues in the method and discussion sections of published research reports. Especially, many researchers appear unaware of the limitations of correlations between pairs of self-report measures as evidence of…

The Anomalous Sentences Repetition Test: Replication and Validation Study.

ERIC Educational Resources Information Center

Weeks, David J.

1986-01-01

Presents a brief clinical test, derived from earlier neuropsychological instruments, with evidence for its reliability, interscorer agreement, and validity. The latter is based upon correlations with both CAT scan measures of cortical atrophy and ventricular enlargement, as well as correlations with seven other previously validated cognitive…

"La Clave Profesional": Validation of a Vocational Guidance Instrument

ERIC Educational Resources Information Center

Mudarra, Maria J.; Lázaro Martínez, Ángel

2014-01-01

Introduction: The current study demonstrates empirical and cultural validity of "La Clave Profesional" (Spanish adaptation of Career Key, Jones's test based Holland's RIASEC model). The process of providing validity evidence also includes a reflection on personal and career development and examines the relationahsips between RIASEC…

Further Validation of the IDAS: Evidence of Convergent, Discriminant, Criterion, and Incremental Validity

ERIC Educational Resources Information Center

Watson, David; O'Hara, Michael W.; Chmielewski, Michael; McDade-Montez, Elizabeth A.; Koffel, Erin; Naragon, Kristin; Stuart, Scott

2008-01-01

The authors explicated the validity of the Inventory of Depression and Anxiety Symptoms (IDAS; D. Watson et al., 2007) in 2 samples (306 college students and 605 psychiatric patients). The IDAS scales showed strong convergent validity in relation to parallel interview-based scores on the Clinician Rating version of the IDAS; the mean convergent…

The 2002 NIMH Provisional Diagnostic Criteria for Depression of Alzheimer's Disease (PDC-dAD): Gauging their Validity over a Decade Later.

PubMed

Sepehry, Amir A; Lee, Philip E; Hsiung, Ging-Yuek R; Beattie, B Lynn; Feldman, Howard H; Jacova, Claudia

2017-01-01

Presented herein is evidence for criterion, content, and convergent/discriminant validity of the NIMH-Provisional Diagnostic Criteria for depression of Alzheimer's Disease (PDC-dAD) that were formulated to address depression in Alzheimer's disease (AD). Using meta-analytic and systematic review methods, we examined criterion validity evidence in epidemiological and clinical studies comparing the PDC-dAD to Diagnostic and Statistical Manual of Mental Disorders fourth edition (DSM-IV), and International Classification of Disease (ICD 9) depression diagnostic criteria. We estimated prevalence of depression by PDC, DSM, and ICD with an omnibus event rate effect-size. We also examined diagnostic agreement between PDC and DSM. To gauge content validity, we reviewed rates of symptom endorsement for each diagnostic approach. Finally, we examined the PDC's relationship with assessment scales (global cognition, neuropsychiatric, and depression definition) for convergent validity evidence. The aggregate evidence supports the validity of the PDC-dAD. Our findings suggest that depression in AD differs from other depressive disorders including Major Depressive Disorder (MDD) in that dAD is more prevalent, with generally a milder presentation and with unique features not captured by the DSM. Although the PDC are the current standard for diagnosis of depression in AD, we identified the need for their further optimization based on predictive validity evidence.

Validity and reliability of the Turkish version of the Optimality Index-US (OI-US) to assess maternity care outcomes.

PubMed

Yucel, Cigdem; Taskin, Lale; Low, Lisa Kane

2015-12-01

Although obstetrical interventions are used commonly in Turkey, there is no standardized evidence-based assessment tool to evaluate maternity care outcomes. The Optimality Index-US (OI-US) is an evidence-based tool that was developed for the purpose of measuring aggregate perinatal care processes and outcomes against an optimal or best possible standard. This index has been validated and used in Netherlands, USA and UK until now. The objective of this study was to adapt the OI-US to assess maternity care outcomes in Turkey. Translation and back translation were used to develop the Optimality Index-Turkey (OI-TR) version. To evaluate the content validity of the OI-TR, an expert panel group (n=10) reviewed the items and evidence-based quality of the OI-TR for application in Turkey. Following the content validity process, the OI-TR was used to assess 150 healthy and 150 high-risk pregnant women who gave birth at a high volume, urban maternity hospital in Turkey. The scores between the two groups were compared to assess the discriminant validity of the OI-TR. The percentage of agreement between two raters and the Kappa statistic were calculated to evaluate the reliability. Content validity was established for the OI-TR by an expert group. Discriminant validity was confirmed by comparing the OI scores of healthy pregnant women (mean OI score=77.65%) and those of high-risk pregnant women (mean OI score=78.60%). The percentage of agreement between the two raters was 96.19, and inter-rater agreement was provided for each item in the OI-TR. OI-TR is a valid and reliable tool that can be used to assess maternity care outcomes in Turkey. The results of this study indicate that although the risk statuses of the women differed, the type of care they received was essentially the same, as measured by the OI-TR. Care was not individualised based on risk and for a majority of items was inconsistent with evidence based practice, which is not optimal. Use of the OI-TR will help to provide a standardized way to assess maternity care process and outcomes of maternity care in Turkey which can inform future research aimed at improving maternity care outcomes. Copyright © 2015 Elsevier Ltd. All rights reserved.

20 CFR 220.14 - Weighing of evidence.

Code of Federal Regulations, 2013 CFR

2013-04-01

... capacity evaluation is based upon functional objective tests with high validity and reliability; (2) The... exam findings which is indicative of exaggerated or potential malingering response; (6) The evidence...

20 CFR 220.14 - Weighing of evidence.

Code of Federal Regulations, 2011 CFR

2011-04-01

... capacity evaluation is based upon functional objective tests with high validity and reliability; (2) The... exam findings which is indicative of exaggerated or potential malingering response; (6) The evidence...

20 CFR 220.14 - Weighing of evidence.

Code of Federal Regulations, 2014 CFR

2014-04-01

... capacity evaluation is based upon functional objective tests with high validity and reliability; (2) The... exam findings which is indicative of exaggerated or potential malingering response; (6) The evidence...

20 CFR 220.14 - Weighing of evidence.

Code of Federal Regulations, 2012 CFR

2012-04-01

... capacity evaluation is based upon functional objective tests with high validity and reliability; (2) The... exam findings which is indicative of exaggerated or potential malingering response; (6) The evidence...

Construct Definition Using Cognitively Based Evidence: A Framework for Practice

ERIC Educational Resources Information Center

Ketterlin-Geller, Leanne R.; Yovanoff, Paul; Jung, EunJu; Liu, Kimy; Geller, Josh

2013-01-01

In this article, we highlight the need for a precisely defined construct in score-based validation and discuss the contribution of cognitive theories to accurately and comprehensively defining the construct. We propose a framework for integrating cognitively based theoretical and empirical evidence to specify and evaluate the construct. We apply…

Validation of Scales from the Deployment Risk and Resilience Inventory in a Sample of Operation Iraqi Freedom Veterans

ERIC Educational Resources Information Center

Vogt, Dawne S.; Proctor, Susan P.; King, Daniel W.; King, Lynda A.; Vasterling, Jennifer J.

2008-01-01

The Deployment Risk and Resilience Inventory (DRRI) is a suite of scales that can be used to assess deployment-related factors implicated in the health and well-being of military veterans. Although initial evidence for the reliability and validity of DRRI scales based on Gulf War veteran samples is encouraging, evidence with respect to a more…

Validating the Proposed Structure of the Relationships among Test Anxiety and Its Predictors Based on Control-Value Theory: Evidence for Gender-Specific Patterns

ERIC Educational Resources Information Center

Ringeisen, Tobias; Raufelder, Diana; Schnell, Kerstin; Rohrmann, Sonja

2016-01-01

Control-value theory (CVT) proposes a framework for the structure of the relationships between the various predictors of achievement-related emotions, particularly anxiety. Despite existing evidence for the role of anxiety predictors, research has not yet justified their proposed structure. Hence, the current study validated the structure of test…

Reliability and validity of the Outcome Expectations for Exercise Scale-2.

PubMed

Resnick, Barbara

2005-10-01

Development of a reliable and valid measure of outcome expectations for exercise for older adults will help establish the relationship between outcome expectations and exercise and facilitate the development of interventions to increase physical activity in older adults. The purpose of this study was to test the reliability and validity of the Outcome Expectations for Exercise-2 Scale (OEE-2), a 13-item measure with two subscales: positive OEE (POEE) and negative OEE (NOEE). The OEE-2 scale was given to 161 residents in a continuing-care retirement community. There was some evidence of validity based on confirmatory factor analysis, Rasch-analysis INFIT and OUTFIT statistics, and convergent validity and test criterion relationships. There was some evidence for reliability of the OEE-2 based on alpha coefficients, person- and item-separation reliability indexes, and R(2)values. Based on analyses, suggested revisions are provided for future use of the OEE-2. Although ongoing reliability and validity testing are needed, the OEE-2 scale can be used to identify older adults with low outcome expectations for exercise, and interventions can then be implemented to strengthen these expectations and improve exercise behavior.

Diagnostic validity of physical examination tests for common knee disorders: An overview of systematic reviews and meta-analysis.

PubMed

Décary, Simon; Ouellet, Philippe; Vendittoli, Pascal-André; Roy, Jean-Sébastien; Desmeules, François

2017-01-01

More evidence on diagnostic validity of physical examination tests for knee disorders is needed to lower frequently used and costly imaging tests. To conduct a systematic review of systematic reviews (SR) and meta-analyses (MA) evaluating the diagnostic validity of physical examination tests for knee disorders. A structured literature search was conducted in five databases until January 2016. Methodological quality was assessed using the AMSTAR. Seventeen reviews were included with mean AMSTAR score of 5.5 ± 2.3. Based on six SR, only the Lachman test for ACL injuries is diagnostically valid when individually performed (Likelihood ratio (LR+):10.2, LR-:0.2). Based on two SR, the Ottawa Knee Rule is a valid screening tool for knee fractures (LR-:0.05). Based on one SR, the EULAR criteria had a post-test probability of 99% for the diagnosis of knee osteoarthritis. Based on two SR, a complete physical examination performed by a trained health provider was found to be diagnostically valid for ACL, PCL and meniscal injuries as well as for cartilage lesions. When individually performed, common physical tests are rarely able to rule in or rule out a specific knee disorder, except the Lachman for ACL injuries. There is low-quality evidence concerning the validity of combining history elements and physical tests. Copyright © 2016 Elsevier Ltd. All rights reserved.

Developing evidence-based dentistry skills: how to interpret randomized clinical trials and systematic reviews.

PubMed

Kiriakou, Juliana; Pandis, Nikolaos; Madianos, Phoebus; Polychronopoulou, Argy

2014-10-30

Decision-making based on reliable evidence is more likely to lead to effective and efficient treatments. Evidence-based dentistry was developed, similarly to evidence-based medicine, to help clinicians apply current and valid research findings into their own clinical practice. Interpreting and appraising the literature is fundamental and involves the development of evidence-based dentistry (EBD) skills. Systematic reviews (SRs) of randomized controlled trials (RCTs) are considered to be evidence of the highest level in evaluating the effectiveness of interventions. Furthermore, the assessment of the report of a RCT, as well as a SR, can lead to an estimation of how the study was designed and conducted.

Evidence of the Impact of Scholarship of Teaching and Learning Purposes

ERIC Educational Resources Information Center

Trigwell, Keith

2013-01-01

This paper identifies a need for empirical studies to validate the purposes of the Scholarship of Teaching and Learning (SoTL) and reports the results of an investigation into one purpose based on one definition. The SoTL movement needs to be seen to be scholarly and to be engaging in evidence-based practice. More evidence is needed on whether…

Validating a Theory-Based Survey to Evaluate Teaching Effectiveness in Higher Education

ERIC Educational Resources Information Center

Amrein-Beardsley, A.; Haladyna, T.

2012-01-01

Surveys to evaluate instructor effectiveness are commonly used in higher education. Yet the survey items included are often drawn from other surveys without reference to a theory of adult learning. The authors present the results from a validation study of such a theory-based survey. They evidence that an evaluation survey based on a theory that…

Development and validation of the Body and Appearance Self-Conscious Emotions Scale (BASES).

PubMed

Castonguay, Andrée L; Sabiston, Catherine M; Crocker, Peter R E; Mack, Diane E

2014-03-01

The purpose of these studies was to develop a psychometrically sound measure of shame, guilt, authentic pride, and hubristic pride for use in body and appearance contexts. In Study 1, 41 potential items were developed and assessed for item quality and comprehension. In Study 2, a panel of experts (N=8; M=11, SD=6.5 years of experience) reviewed the scale and items for evidence of content validity. Participants in Study 3 (n=135 males, n=300 females) completed the BASES and various body image, personality, and emotion scales. A separate sample (n=155; 35.5% male) in Study 3 completed the BASES twice using a two-week time interval. The BASES subscale scores demonstrated evidence for internal consistency, item-total correlations, concurrent, convergent, incremental, and discriminant validity, and 2-week test-retest reliability. The 4-factor solution was a good fit in confirmatory factor analysis, reflecting body-related shame, guilt, authentic and hubristic pride subscales of the BASES. The development and validation of the BASES may help advance body image and self-conscious emotion research by providing a foundation to examine the unique antecedents and outcomes of these specific emotional experiences. Copyright © 2014 Elsevier Ltd. All rights reserved.

Comparison of consumer derived evidence with an omaha system evidence-based practice guideline for community dwelling older adults.

PubMed

Pruinelli, Lisiane; Fu, Helen; Monsen, Karen A; Westra, Bonnie L

2014-01-01

Consumer involvement in healthcare is critical to support continuity of care for consumers to manage their health while transitioning from one care setting to another. Validation of evidence-based practice (EBP) guideline by consumers is essential to achieving consumer health goals over time that is consistent with their needs and preferences. The purpose of this study was to compare an Omaha System EBP guideline for community dwelling older adults with consumer-derived evidence of their ongoing needs, resources, and strategies after home care discharge. All identified problems were relevant for all patients except for Neglect and Substance use. Ten additional problems were identified from the interviews, five of which affected at least 10% of the participants. Consumer derived evidence both validated and expanded EBP guidelines; thus further emphasizing the importance of consumer involvement in the delivery of home healthcare.

Psychometric instrumentation: reliability and validity of instruments used for clinical practice, evidence-based practice projects and research studies.

PubMed

Mayo, Ann M

2015-01-01

It is important for CNSs and other APNs to consider the reliability and validity of instruments chosen for clinical practice, evidence-based practice projects, or research studies. Psychometric testing uses specific research methods to evaluate the amount of error associated with any particular instrument. Reliability estimates explain more about how well the instrument is designed, whereas validity estimates explain more about scores that are produced by the instrument. An instrument may be architecturally sound overall (reliable), but the same instrument may not be valid. For example, if a specific group does not understand certain well-constructed items, then the instrument does not produce valid scores when used with that group. Many instrument developers may conduct reliability testing only once, yet continue validity testing in different populations over many years. All CNSs should be advocating for the use of reliable instruments that produce valid results. Clinical nurse specialists may find themselves in situations where reliability and validity estimates for some instruments that are being utilized are unknown. In such cases, CNSs should engage key stakeholders to sponsor nursing researchers to pursue this most important work.

Evidence-Based Assessment of Depression in Adults

ERIC Educational Resources Information Center

Joiner, Thomas E.; Walker, Rheeda L.; Pettit, Jeremy W.; Perez, Marisol; Cukrowicz, Kelly C.

2005-01-01

From diverse perspectives, there is little doubt that depressive symptoms cohere to form a valid and distinct syndrome. Research indicates that an evidence-based assessment of depression would include (a) measures with adequate psychometric properties; (b) adequate coverage of symptoms; (c) adequate coverage of depressed mood, anhedonia, and…

Adaptation of the ESPA29 Parental Socialization Styles Scale to the Basque language: evidence of validity.

PubMed

López-Jáuregui, Alicia; Oliden, Paula Elosua

2009-11-01

The aim of this study is to adapt the ESPA29 scale of parental socialization styles in adolescence to the Basque language. The study of its psychometric properties is based on the search for evidence of internal and external validity. The first focuses on the assessment of the dimensionality of the scale by means of exploratory factor analysis. The relationship between the dimensions of parental socialization styles and gender and age guarantee the external validity of the scale. The study of the equivalence of the adapted and original versions is based on the comparisons of the reliability coefficients and on factor congruence. The results allow us to conclude the equivalence of the two scales.

The Importance of Replication in Measurement Research: Using Curriculum-Based Measures with Postsecondary Students with Developmental Disabilities

ERIC Educational Resources Information Center

Hosp, John L.; Ford, Jeremy W.; Huddle, Sally M.; Hensley, Kiersten K.

2018-01-01

Replication is a foundation of the development of a knowledge base in an evidence-based field such as education. This study includes two direct replications of Hosp, Hensley, Huddle, and Ford which found evidence of criterion-related validity of curriculum-based measurement (CBM) for reading and mathematics with postsecondary students with…

Validity evidence for procedural competency in virtual reality robotic simulation, establishing a credible pass/fail standard for the vaginal cuff closure procedure.

PubMed

Hovgaard, Lisette Hvid; Andersen, Steven Arild Wuyts; Konge, Lars; Dalsgaard, Torur; Larsen, Christian Rifbjerg

2018-03-30

The use of robotic surgery for minimally invasive procedures has increased considerably over the last decade. Robotic surgery has potential advantages compared to laparoscopic surgery but also requires new skills. Using virtual reality (VR) simulation to facilitate the acquisition of these new skills could potentially benefit training of robotic surgical skills and also be a crucial step in developing a robotic surgical training curriculum. The study's objective was to establish validity evidence for a simulation-based test for procedural competency for the vaginal cuff closure procedure that can be used in a future simulation-based, mastery learning training curriculum. Eleven novice gynaecological surgeons without prior robotic experience and 11 experienced gynaecological robotic surgeons (> 30 robotic procedures) were recruited. After familiarization with the VR simulator, participants completed the module 'Guided Vaginal Cuff Closure' six times. Validity evidence was investigated for 18 preselected simulator metrics. The internal consistency was assessed using Cronbach's alpha and a composite score was calculated based on metrics with significant discriminative ability between the two groups. Finally, a pass/fail standard was established using the contrasting groups' method. The experienced surgeons significantly outperformed the novice surgeons on 6 of the 18 metrics. The internal consistency was 0.58 (Cronbach's alpha). The experienced surgeons' mean composite score for all six repetitions were significantly better than the novice surgeons' (76.1 vs. 63.0, respectively, p < 0.001). A pass/fail standard of 75/100 was established. Four novice surgeons passed this standard (false positives) and three experienced surgeons failed (false negatives). Our study has gathered validity evidence for a simulation-based test for procedural robotic surgical competency in the vaginal cuff closure procedure and established a credible pass/fail standard for future proficiency-based training.

The development and validation of the core competencies scale (CCS) for the college and university students.

PubMed

Ruan, Bin; Mok, Magdalena Mo Ching; Edginton, Christopher R; Chin, Ming Kai

2012-01-01

This article describes the development and validation of the Core Competencies Scale (CCS) using Bok's (2006) competency framework for undergraduate education. The framework included: communication, critical thinking, character development, citizenship, diversity, global understanding, widening of interest, and career and vocational development. The sample comprised 70 college and university students. Results of analysis using Rasch rating scale modelling showed that there was strong empirical evidence on the validity of the measures in contents, structure, interpretation, generalizability, and response options of the CCS scale. The implication of having developed Rasch-based valid and dependable measures in this study for gauging the value added of college and university education to their students is that the feedback generated from CCS will enable evidence-based decision and policy making to be implemented and strategized. Further, program effectiveness can be measured and thus accountability on the achievement of the program objectives.

Competency-Based Training and Simulation: Making a "Valid" Argument.

PubMed

Noureldin, Yasser A; Lee, Jason Y; McDougall, Elspeth M; Sweet, Robert M

2018-02-01

The use of simulation as an assessment tool is much more controversial than is its utility as an educational tool. However, without valid simulation-based assessment tools, the ability to objectively assess technical skill competencies in a competency-based medical education framework will remain challenging. The current literature in urologic simulation-based training and assessment uses a definition and framework of validity that is now outdated. This is probably due to the absence of awareness rather than an absence of comprehension. The following review article provides the urologic community an updated taxonomy on validity theory as it relates to simulation-based training and assessments and translates our simulation literature to date into this framework. While the old taxonomy considered validity as distinct subcategories and focused on the simulator itself, the modern taxonomy, for which we translate the literature evidence, considers validity as a unitary construct with a focus on interpretation of simulator data/scores.

Assessing Student Understanding of the "New Biology": Development and Evaluation of a Criterion-Referenced Genomics and Bioinformatics Assessment

NASA Astrophysics Data System (ADS)

Campbell, Chad Edward

Over the past decade, hundreds of studies have introduced genomics and bioinformatics (GB) curricula and laboratory activities at the undergraduate level. While these publications have facilitated the teaching and learning of cutting-edge content, there has yet to be an evaluation of these assessment tools to determine if they are meeting the quality control benchmarks set forth by the educational research community. An analysis of these assessment tools indicated that <10% referenced any quality control criteria and that none of the assessments met more than one of the quality control benchmarks. In the absence of evidence that these benchmarks had been met, it is unclear whether these assessment tools are capable of generating valid and reliable inferences about student learning. To remedy this situation the development of a robust GB assessment aligned with the quality control benchmarks was undertaken in order to ensure evidence-based evaluation of student learning outcomes. Content validity is a central piece of construct validity, and it must be used to guide instrument and item development. This study reports on: (1) the correspondence of content validity evidence gathered from independent sources; (2) the process of item development using this evidence; (3) the results from a pilot administration of the assessment; (4) the subsequent modification of the assessment based on the pilot administration results and; (5) the results from the second administration of the assessment. Twenty-nine different subtopics within GB (Appendix B: Genomics and Bioinformatics Expert Survey) were developed based on preliminary GB textbook analyses. These subtopics were analyzed using two methods designed to gather content validity evidence: (1) a survey of GB experts (n=61) and (2) a detailed content analyses of GB textbooks (n=6). By including only the subtopics that were shown to have robust support across these sources, 22 GB subtopics were established for inclusion in the assessment. An expert panel subsequently developed, evaluated, and revised two multiple-choice items to align with each of the 22 subtopics, producing a final item pool of 44 items. These items were piloted with student samples of varying content exposure levels. Both Classical Test Theory (CTT) and Item Response Theory (IRT) methodologies were used to evaluate the assessment's validity, reliability and ability inferences, and its ability to differentiate students with different magnitudes of content exposure. A total of 18 items were subsequently modified and reevaluated by an expert panel. The 26 original and 18 modified items were once again piloted with student samples of varying content exposure levels. Both CTT and IRT methodologies were once again used to evaluate student responses in order to evaluate the assessment's validity and reliability inferences as well as its ability to differentiate students with different magnitudes of content exposure. Interviews with students from different content exposure levels were also performed in order to gather convergent validity evidence (external validity evidence) as well as substantive validity evidence. Also included are the limitations of the assessment and a set of guidelines on how the assessment can best be used.

The Validity and Responsiveness of Isometric Lower Body Multi-Joint Tests of Muscular Strength: a Systematic Review.

PubMed

Drake, David; Kennedy, Rodney; Wallace, Eric

2017-12-01

Researchers and practitioners working in sports medicine and science require valid tests to determine the effectiveness of interventions and enhance understanding of mechanisms underpinning adaptation. Such decision making is influenced by the supportive evidence describing the validity of tests within current research. The objective of this study is to review the validity of lower body isometric multi-joint tests ability to assess muscular strength and determine the current level of supporting evidence. Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines were followed in a systematic fashion to search, assess and synthesize existing literature on this topic. Electronic databases such as Web of Science, CINAHL and PubMed were searched up to 18 March 2015. Potential inclusions were screened against eligibility criteria relating to types of test, measurement instrument, properties of validity assessed and population group and were required to be published in English. The Consensus-based Standards for the Selection of health Measurement Instruments (COSMIN) checklist was used to assess methodological quality and measurement property rating of included studies. Studies rated as fair or better in methodological quality were included in the best evidence synthesis. Fifty-nine studies met the eligibility criteria for quality appraisal. The ten studies that rated fair or better in methodological quality were included in the best evidence synthesis. The most frequently investigated lower body isometric multi-joint tests for validity were the isometric mid-thigh pull and isometric squat. The validity of each of these tests was strong in terms of reliability and construct validity. The evidence for responsiveness of tests was found to be moderate for the isometric squat test and unknown for the isometric mid-thigh pull. No tests using the isometric leg press met the criteria for inclusion in the best evidence synthesis. Researchers and practitioners can use the isometric squat and isometric mid-thigh pull with confidence in terms of reliability and construct validity. Further work to investigate other validity components such as criterion validity, smallest detectable change and responsiveness to resistance exercise interventions may be beneficial to the current level of evidence.

The Construct of the Learning Organization: Dimensions, Measurement, and Validation

ERIC Educational Resources Information Center

Yang, Baiyin; Watkins, Karen E.; Marsick, Victoria J.

2004-01-01

This research describes efforts to develop and validate a multidimensional measure of the learning organization. An instrument was developed based on a critical review of both the conceptualization and practice of this construct. Supporting validity evidence for the instrument was obtained from several sources, including best model-data fit among…

Evaluating the Content Validity of Multistage-Adaptive Tests

ERIC Educational Resources Information Center

Crotts, Katrina; Sireci, Stephen G.; Zenisky, April

2012-01-01

Validity evidence based on test content is important for educational tests to demonstrate the degree to which they fulfill their purposes. Most content validity studies involve subject matter experts (SMEs) who rate items that comprise a test form. In computerized-adaptive testing, examinees take different sets of items and test "forms"…

Validity of premature ejaculation diagnostic tool and its association with International Index of Erectile Function-15 in Chinese men with evidence-based-defined premature ejaculation.

PubMed

Tang, Dong-Dong; Li, Chao; Peng, Dang-Wei; Zhang, Xian-Sheng

2018-01-01

The premature ejaculation diagnostic tool (PEDT) is a brief diagnostic measure to assess premature ejaculation (PE). However, there is insufficient evidence regarding its validity in the new evidence-based-defined PE. This study was performed to evaluate the validity of PEDT and its association with IIEF-15 in different types of evidence-based-defined PE. From June 2015 to January 2016, a total of 260 men complaining of PE and defined as lifelong PE (LPE)/acquired PE (APE) according to the evidence-based definition from Andrology Clinic of the First Affiliated Hospital of Anhui Medical University, along with 104 male healthy controls without PE from a medical examination center, were enrolled in this study. All individuals completed questionnaires including demographics, medical and sexual history, as well as PEDT and IIEF-15. After statistical analysis, it was found that men with PE reported higher PEDT scores (14.28 ± 3.05) and lower IIEF-15 (41.26 ± 8.20) than men without PE (PEDT: 5.32 ± 3.42, IIEF-15: 52.66 ± 6.86, P < 0.001 for both). It was suggested that a score of ≥9 indicated PE in both LPE and APE by sensitivity and specificity analyses (sensitivity: 0.875, 0.913; specificity: 0.865, 0.865, respectively). In addition, IIEF-15 were higher in men with LPE (42.64 ± 8.11) than APE (39.43 ± 7.84, P < 0.001). After adjusting for age, IIEF-15 was negatively related to PEDT in men with LPE (adjust r = -0.225, P < 0.001) and APE (adjust r = -0.378, P < 0.001). In this study, we concluded that PEDT was valid in the diagnosis of evidenced-based-defined PE. Furthermore, IIEF-15 was negatively related to PEDT in men with different types of PE.

Health risk behaviors in adolescents and emerging adults with congenital heart disease: psychometric properties of the Health Behavior Scale-Congenital Heart Disease.

PubMed

Goossens, Eva; Luyckx, Koen; Mommen, Nele; Gewillig, Marc; Budts, Werner; Zupancic, Nele; Moons, Philip

2013-12-01

To optimize long-term outcomes, patients with congenital heart disease (CHD) should adopt health-promoting behaviors. Studies on health behavior in afflicted patients are scarce and comparability of study results is limited. To enlarge the body of evidence, we have developed the Health Behavior Scale-Congenital Heart Disease (HBS-CHD). We examined the psychometric properties of the HBS-CHD by providing evidence for (a) the content validity; (b) validity based on the relationships with other variables; (c) reliability in terms of stability; and (d) responsiveness. Ten experts rated the relevance of the HBS-CHD items. The item content validity index (I-CVI) and the averaged scale content validity index (S-CVI/Ave); the modified multi-rater Kappa and proportion of missing values for each question were calculated. Relationships with other variables were evaluated using six hypotheses that were tested in 429 adolescents with CHD. Stability of the instrument was assessed using Heise's method; and responsiveness was tested by calculating the Guyatt's Responsiveness Index (GRI). Overall, 86.3% of the items had a good to excellent content validity; the S-CVI/Ave (0.81) and multi-rater Kappa (0.78) were adequate. The average proportion of missing values was low (1.2%). Because five out of six hypotheses were confirmed, evidence for the validity of the HBS-CHD based on relationships with other variables was provided. The stability of the instrument could not be confirmed based on our data. The GRI showed good to excellent capacity of the HBS-CHD to detect clinical changes in the health behavior over time. We found that the HBS-CHD is a valid and responsive questionnaire to assess health behaviors in patients with CHD.

Patient-reported outcome instruments that evaluate adherence behaviours in adults with asthma: A systematic review of measurement properties.

PubMed

Gagné, Myriam; Boulet, Louis-Philippe; Pérez, Norma; Moisan, Jocelyne

2018-04-30

To systematically identify the measurement properties of patient-reported outcome instruments (PROs) that evaluate adherence to inhaled maintenance medication in adults with asthma. We conducted a systematic review of six databases. Two reviewers independently included studies on the measurement properties of PROs that evaluated adherence in asthmatic participants aged ≥18 years. Based on the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN), the reviewers (1) extracted data on internal consistency, reliability, measurement error, content validity, structural validity, hypotheses testing, cross-cultural validity, criterion validity, and responsiveness; (2) assessed the methodological quality of the included studies; (3) assessed the quality of the measurement properties (positive or negative); and (4) summarised the level of evidence (limited, moderate, or strong). We screened 6,068 records and included 15 studies (14 PROs). No studies evaluated measurement error or responsiveness. Based on methodological and measurement property quality assessments, we found limited positive evidence of: (a) internal consistency of the Adherence Questionnaire, Refined Medication Adherence Reason Scale (MAR-Scale), Medication Adherence Report Scale for Asthma (MARS-A), and Test of the Adherence to Inhalers (TAI); (b) reliability of the TAI; and (c) structural validity of the Adherence Questionnaire, MAR-Scale, MARS-A, and TAI. We also found limited negative evidence of: (d) hypotheses testing of Adherence Questionnaire; (e) reliability of the MARS-A; and (f) criterion validity of the MARS-A and TAI. Our results highlighted the need to conduct further high-quality studies that will positively evaluate the reliability, validity, and responsiveness of the available PROs. This article is protected by copyright. All rights reserved.

Investigating the validity and usability of an interactive computer programme for assessing competence in telephone-based mental health triage.

PubMed

Sands, Natisha; Elsom, Stephen; Keppich-Arnold, Sandra; Henderson, Kathryn; King, Peter; Bourke-Finn, Karen; Brunning, Debra

2016-02-01

Telephone-based mental health triage services are frontline health-care providers that operate 24/7 to facilitate access to psychiatric assessment and intervention for people requiring assistance with a mental health problem. The mental health triage clinical role is complex, and the populations triage serves are typically high risk; yet to date, no evidence-based methods have been available to assess clinician competence to practice telephone-based mental health triage. The present study reports the findings of a study that investigated the validity and usability of the Mental Health Triage Competency Assessment Tool, an evidence-based, interactive computer programme designed to assist clinicians in developing and assessing competence to practice telephone-based mental health triage. © 2015 Australian College of Mental Health Nurses Inc.

Montreal Accord on Patient-Reported Outcomes (PROs) use series-Paper 7: modern perspectives of measurement validation emphasize justification of inferences based on patient reported outcome scores.

PubMed

Sawatzky, Richard; Chan, Eric K H; Zumbo, Bruno D; Ahmed, Sara; Bartlett, Susan J; Bingham, Clifton O; Gardner, William; Jutai, Jeffrey; Kuspinar, Ayse; Sajobi, Tolulope; Lix, Lisa M

2017-09-01

Obtaining the patient's view about the outcome of care is an essential component of patient-centered care. Many patient-reported outcome (PRO) instruments for different purposes have been developed since the 1960s. Measurement validation is fundamental in the development, evaluation, and use of PRO instruments. This paper provides a review of modern perspectives of measurement validation in relation to the followings three questions as applied to PROs: (1) What evidence is needed to warrant comparisons between groups and individuals? (2) What evidence is needed to warrant comparisons over time? and (3) What are the value implications, including personal and societal consequences, of using PRO scores? Measurement validation is an ongoing process that involves the accumulation of evidence regarding the justification of inferences, actions, and decisions based on measurement scores. These include inferences pertaining to comparisons between groups and comparisons over time as well as consideration of value implications of using PRO scores. Personal and societal consequences must be examined as part of a comprehensive approach to measurement validation. The answers to these three questions are fundamental to the the validity of different types of inferences, actions, and decisions made on PRO scores in health research, health care administration, and clinical practice. Copyright © 2016 Elsevier Inc. All rights reserved.

Model testing for reliability and validity of the Outcome Expectations for Exercise Scale.

PubMed

Resnick, B; Zimmerman, S; Orwig, D; Furstenberg, A L; Magaziner, J

2001-01-01

Development of a reliable and valid measure of outcome expectations for exercise appropriate for older adults will help establish the relationship between outcome expectations and exercise. Once established, this measure can be used to facilitate the development of interventions to strengthen outcome expectations and improve adherence to regular exercise in older adults. Building on initial psychometrics of the Outcome Expectation for Exercise (OEE) Scale, the purpose of the current study was to use structural equation modeling to provide additional support for the reliability and validity of this measure. The OEE scale is a 9-item measure specifically focusing on the perceived consequences of exercise for older adults. The OEE scale was given to 191 residents in a continuing care retirement community. The mean age of the participants was 85 +/- 6.1 and the majority were female (76%), White (99%), and unmarried (76%). Using structural equation modeling, reliability was based on R2 values, and validity was based on a confirmatory factor analysis and path coefficients. There was continued evidence for reliability of the OEE based on R2 values ranging from .42 to .77, and validity with path coefficients ranging from .69 to .87, and evidence of model fit (X2 of 69, df = 27, p < .05, NFI = .98, RMSEA = .07). The evidence of reliability and validity of this measure has important implications for clinical work and research. The OEE scale can be used to identify older adults who have low outcome expectations for exercise, and interventions can then be implemented to strengthen these expectations and thereby improve exercise behavior.

A Rasch-Based Validation of the Vocabulary Size Test

ERIC Educational Resources Information Center

Beglar, David

2010-01-01

The primary purpose of this study was to provide preliminary validity evidence for a 140-item form of the Vocabulary Size Test, which is designed to measure written receptive knowledge of the first 14,000 words of English. Nineteen native speakers of English and 178 native speakers of Japanese participated in the study. Analyses based on the Rasch…

Current Status of Simulation-based Training Tools in Orthopedic Surgery: A Systematic Review.

PubMed

Morgan, Michael; Aydin, Abdullatif; Salih, Alan; Robati, Shibby; Ahmed, Kamran

To conduct a systematic review of orthopedic training and assessment simulators with reference to their level of evidence (LoE) and level of recommendation. Medline and EMBASE library databases were searched for English language articles published between 1980 and 2016, describing orthopedic simulators or validation studies of these models. All studies were assessed for LoE, and each model was subsequently awarded a level of recommendation using a modified Oxford Centre for Evidence-Based Medicine classification, adapted for education. A total of 76 articles describing orthopedic simulators met the inclusion criteria, 47 of which described at least 1 validation study. The most commonly identified models (n = 34) and validation studies (n = 26) were for knee arthroscopy. Construct validation was the most frequent validation study attempted by authors. In all, 62% (47 of 76) of the simulator studies described arthroscopy simulators, which also contained validation studies with the highest LoE. Orthopedic simulators are increasingly being subjected to validation studies, although the LoE of such studies generally remain low. There remains a lack of focus on nontechnical skills and on cost analyses of orthopedic simulators. Copyright © 2017 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Exploring the validity and reliability of a questionnaire for evaluating veterinary clinical teachers' supervisory skills during clinical rotations.

PubMed

Boerboom, T B B; Dolmans, D H J M; Jaarsma, A D C; Muijtjens, A M M; Van Beukelen, P; Scherpbier, A J J A

2011-01-01

Feedback to aid teachers in improving their teaching requires validated evaluation instruments. When implementing an evaluation instrument in a different context, it is important to collect validity evidence from multiple sources. We examined the validity and reliability of the Maastricht Clinical Teaching Questionnaire (MCTQ) as an instrument to evaluate individual clinical teachers during short clinical rotations in veterinary education. We examined four sources of validity evidence: (1) Content was examined based on theory of effective learning. (2) Response process was explored in a pilot study. (3) Internal structure was assessed by confirmatory factor analysis using 1086 student evaluations and reliability was examined utilizing generalizability analysis. (4) Relations with other relevant variables were examined by comparing factor scores with other outcomes. Content validity was supported by theory underlying the cognitive apprenticeship model on which the instrument is based. The pilot study resulted in an additional question about supervision time. A five-factor model showed a good fit with the data. Acceptable reliability was achievable with 10-12 questionnaires per teacher. Correlations between the factors and overall teacher judgement were strong. The MCTQ appears to be a valid and reliable instrument to evaluate clinical teachers' performance during short rotations.

Instruments assessing attitudes toward or capability regarding self-management of osteoarthritis: a systematic review of measurement properties.

PubMed

Eyles, J P; Hunter, D J; Meneses, S R F; Collins, N J; Dobson, F; Lucas, B R; Mills, K

2017-08-01

To make a recommendation on the "best" instrument to assess attitudes toward and/or capabilities regarding self-management of osteoarthritis (OA) based on available measurement property evidence. Electronic searches were performed in MEDLINE, EMBASE, CINAHL and PsychINFO (inception to 27 December 2016). Two reviewers independently rated measurement properties using the Consensus-based Standards for the selection of Health Measurement Instruments (COSMIN) 4-point scale. Best evidence synthesis was determined by considering COSMIN ratings for measurement property results and the level of evidence available for each measurement property of each instrument. Eight studies out of 5653 publications met the inclusion criteria, with eight instruments identified for evaluation: Multidimensional Health Locus of Control (MHLC), Perceived Behavioural Control (PBC), Patient Activation Measure (PAM), Educational Needs Assessment (ENAT), Stages of Change Questionnaire in Osteoarthritis (SCQOA), Effective Consumer Scale (EC-17) and Perceived Efficacy in Patient-Physician Interactions five item (PEPPI-5) and ten item scales. Measurement properties assessed for these instruments included internal consistency (k = 8), structural validity (k = 8), test-retest reliability (k = 2), measurement error (k = 1), hypothesis testing (k = 3) and cross-cultural validity (k = 3). No information was available for content validity, responsiveness or minimal important change (MIC)/minimal important difference (MID). The Dutch PEPPI-5 demonstrated the best measurement property evidence; strong evidence for internal consistency and structural validity but limited evidence for reliability and construct validity. Although PEPPI-5 was identified as having the best measurement properties, overall there is a poor level of evidence currently available concerning measurement properties of instruments to assess attitudes toward and/or capabilities regarding osteoarthritis self-management. Further well-designed studies investigating measurement properties of existing instruments are required. Copyright © 2017 Osteoarthritis Research Society International. All rights reserved.

The quality of instruments to assess the process of shared decision making: A systematic review

PubMed Central

Bomhof-Roordink, Hanna; Smith, Ian P.; Scholl, Isabelle; Stiggelbout, Anne M.; Pieterse, Arwen H.

2018-01-01

Objective To inventory instruments assessing the process of shared decision making and appraise their measurement quality, taking into account the methodological quality of their validation studies. Methods In a systematic review we searched seven databases (PubMed, Embase, Emcare, Cochrane, PsycINFO, Web of Science, Academic Search Premier) for studies investigating instruments measuring the process of shared decision making. Per identified instrument, we assessed the level of evidence separately for 10 measurement properties following a three-step procedure: 1) appraisal of the methodological quality using the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist, 2) appraisal of the psychometric quality of the measurement property using three possible quality scores, 3) best-evidence synthesis based on the number of studies, their methodological and psychometrical quality, and the direction and consistency of the results. The study protocol was registered at PROSPERO: CRD42015023397. Results We included 51 articles describing the development and/or evaluation of 40 shared decision-making process instruments: 16 patient questionnaires, 4 provider questionnaires, 18 coding schemes and 2 instruments measuring multiple perspectives. There is an overall lack of evidence for their measurement quality, either because validation is missing or methods are poor. The best-evidence synthesis indicated positive results for a major part of instruments for content validity (50%) and structural validity (53%) if these were evaluated, but negative results for a major part of instruments when inter-rater reliability (47%) and hypotheses testing (59%) were evaluated. Conclusions Due to the lack of evidence on measurement quality, the choice for the most appropriate instrument can best be based on the instrument’s content and characteristics such as the perspective that they assess. We recommend refinement and validation of existing instruments, and the use of COSMIN-guidelines to help guarantee high-quality evaluations. PMID:29447193

Rationality versus reality: the challenges of evidence-based decision making for health policy makers

PubMed Central

2010-01-01

Background Current healthcare systems have extended the evidence-based medicine (EBM) approach to health policy and delivery decisions, such as access-to-care, healthcare funding and health program continuance, through attempts to integrate valid and reliable evidence into the decision making process. These policy decisions have major impacts on society and have high personal and financial costs associated with those decisions. Decision models such as these function under a shared assumption of rational choice and utility maximization in the decision-making process. Discussion We contend that health policy decision makers are generally unable to attain the basic goals of evidence-based decision making (EBDM) and evidence-based policy making (EBPM) because humans make decisions with their naturally limited, faulty, and biased decision-making processes. A cognitive information processing framework is presented to support this argument, and subtle cognitive processing mechanisms are introduced to support the focal thesis: health policy makers' decisions are influenced by the subjective manner in which they individually process decision-relevant information rather than on the objective merits of the evidence alone. As such, subsequent health policy decisions do not necessarily achieve the goals of evidence-based policy making, such as maximizing health outcomes for society based on valid and reliable research evidence. Summary In this era of increasing adoption of evidence-based healthcare models, the rational choice, utility maximizing assumptions in EBDM and EBPM, must be critically evaluated to ensure effective and high-quality health policy decisions. The cognitive information processing framework presented here will aid health policy decision makers by identifying how their decisions might be subtly influenced by non-rational factors. In this paper, we identify some of the biases and potential intervention points and provide some initial suggestions about how the EBDM/EBPM process can be improved. PMID:20504357

Rationality versus reality: the challenges of evidence-based decision making for health policy makers.

PubMed

McCaughey, Deirdre; Bruning, Nealia S

2010-05-26

Current healthcare systems have extended the evidence-based medicine (EBM) approach to health policy and delivery decisions, such as access-to-care, healthcare funding and health program continuance, through attempts to integrate valid and reliable evidence into the decision making process. These policy decisions have major impacts on society and have high personal and financial costs associated with those decisions. Decision models such as these function under a shared assumption of rational choice and utility maximization in the decision-making process. We contend that health policy decision makers are generally unable to attain the basic goals of evidence-based decision making (EBDM) and evidence-based policy making (EBPM) because humans make decisions with their naturally limited, faulty, and biased decision-making processes. A cognitive information processing framework is presented to support this argument, and subtle cognitive processing mechanisms are introduced to support the focal thesis: health policy makers' decisions are influenced by the subjective manner in which they individually process decision-relevant information rather than on the objective merits of the evidence alone. As such, subsequent health policy decisions do not necessarily achieve the goals of evidence-based policy making, such as maximizing health outcomes for society based on valid and reliable research evidence. In this era of increasing adoption of evidence-based healthcare models, the rational choice, utility maximizing assumptions in EBDM and EBPM, must be critically evaluated to ensure effective and high-quality health policy decisions. The cognitive information processing framework presented here will aid health policy decision makers by identifying how their decisions might be subtly influenced by non-rational factors. In this paper, we identify some of the biases and potential intervention points and provide some initial suggestions about how the EBDM/EBPM process can be improved.

Predictive Validity of a Student Self-Report Screener of Behavioral and Emotional Risk in an Urban High School

ERIC Educational Resources Information Center

Dowdy, Erin; Harrell-Williams, Leigh; Dever, Bridget V.; Furlong, Michael J.; Moore, Stephanie; Raines, Tara; Kamphaus, Randy W.

2016-01-01

Increasingly, schools are implementing school-based screening for risk of behavioral and emotional problems; hence, foundational evidence supporting the predictive validity of screening instruments is important to assess. This study examined the predictive validity of the Behavior Assessment System for Children-2 Behavioral and Emotional Screening…

KIPS: An Evidence-Based Tool for Assessing Parenting Strengths and Needs in Diverse Families

ERIC Educational Resources Information Center

Comfort, Marilee; Gordon, Philip R.; Naples, Denise

2011-01-01

The movement toward evidence-based practices has stimulated greater interest in assessing parenting outcomes. The purpose of these studies was to further validate the Keys to Interactive Parenting Scale (KIPS), a structured observational assessment of parenting quality, with 397 diverse families. Factor analysis demonstrated that the 12 KIPS items…

Assessing Sensitivity of Early Head Start Study Findings to Manipulated Randomization Threats

ERIC Educational Resources Information Center

Green, Sheridan

2013-01-01

Increasing demands for design rigor and an emphasis on evidence-based practice on a national level indicated a need for further guidance related to successful implementation of randomized studies in education. Rigorous and meaningful experimental research and its conclusions help establish a valid theoretical and evidence base for educational…

Moderators and Mediators in Social Work Research: Toward a More Ecologically Valid Evidence Base for Practice

PubMed Central

Magill, Molly

2012-01-01

Summary Evidence-based practice involves the consistent and critical consumption of the social work research literature. As methodologies advance, primers to guide such efforts are often needed. In the present work, common statistical methods for testing moderation and mediation are identified, summarized, and corresponding examples, drawn from the substance abuse, domestic violence, and mental health literature, are provided. Findings While methodologically complex, analyses of these third variable effects can provide an optimal fit for the complexity involved in the provision of evidence-based social work services. While a moderator may identify the trait or state requirement for a causal relationship to occur, a mediator is concerned with the transmission of that relationship. In social work practice, these are questions of “under what conditions and for whom?” and of the “how?” of behavior change. Implications Implications include a need for greater attention to these methods among practitioners and evaluation researchers. With knowledge gained through the present review, social workers can benefit from a more ecologically valid evidence base for practice. PMID:22833701

Consistency between direct and indirect trial evidence: is direct evidence always more reliable?

PubMed

Madan, Jason; Stevenson, Matt D; Cooper, Katy L; Ades, A E; Whyte, Sophie; Akehurst, Ron

2011-01-01

To present a case study involving the reduction in incidence of febrile neutropenia (FN) after chemotherapy with granulocyte colony-stimulating factors (G-CSFs), illustrating difficulties that may arise when following the common preference for direct evidence over indirect evidence. Evidence of the efficacy of treatments was identified from two previous systematic reviews. We used Bayesian evidence synthesis to estimate relative treatment effects based on direct evidence, indirect evidence, and both pooled together. We checked for inconsistency between direct and indirect evidence and explored the role of one specific trial using cross-validation. A subsequent review identified further studies not available at the time of the original analysis. We repeated the analyses on the enlarged evidence base. We found substantial inconsistency in the original evidence base. The median odds ratio of FN for primary pegfilgrastim versus no primary G-CSF was 0.06 (95% credible interval: 0.02-0.19) based on direct evidence, but 0.27 (95% credible interval: 0.13-0.53) based on indirect evidence (P value for consistency hypothesis 0.027). The additional trials were consistent with the earlier indirect, rather than the direct, evidence, and there was no inconsistency between direct and indirect estimates in the updated evidence. The earlier inconsistency was due to one trial comparing primary pegfilgrastim with no primary G-CSF. Predictive cross-validation showed that this study was inconsistent with the evidence as a whole and with other trials making this comparison. Both the Cochrane Handbook and the NICE Methods Guide express a preference for direct evidence. A more robust strategy, which is in line with the accepted principles of evidence synthesis, would be to combine all relevant and appropriate information, whether direct or indirect. Copyright © 2011 International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. All rights reserved.

Content validation of terms and definitions in a wound glossary.

PubMed

Milne, Catherine T; Paine, Tim; Sullivan, Valerie; Sawyer, Allen

2011-12-01

A common language and lexicon provide the easiest means of mutual understanding. Inconsistency in terminology makes effective information exchange difficult. Previous studies identified the need to determine standard, accepted definitions for the vocabulary frequently used in wound care. The objective of this study was to establish content validation for these terms and develop an evidence-based glossary for this specialty. Members of the Association for the Advancement of Wound Care Quality of Care Task Force reviewed literature to determine glossary content generation and the associated literature-based definitions. Thirty-nine wound care professionals from wound care stakeholder professional organizations in the United States and Canada participated in the content validation process. Participants were asked to quantify the degree of validity using a 367-item, 4-point Likert-type scale. On a scale of 1 to 4, the mean score of the entire instrument was 3.84. The instrument's overall scale content validity index was 0.96. Terms with an item content validity index of less than 0.70 were removed from the glossary, leaving 365 items with established content validity. Qualitative data analysis revealed themes suggesting that enhanced communication between providers improves patient outcomes. The need for ongoing updates of the glossary was also identified. The wound care glossary in its finalized form proved valid. An evidence-based glossary bridges the chasm of miscommunication and nonstandardization so that wound care, as an emerging specialized medical science field, can move forward to optimize both process and clinical outcomes.

Construction and Validation of the Clinical Judgment Skill Inventory: Clinical Judgment Skill Competencies That Measure Counselor Debiasing Techniques

ERIC Educational Resources Information Center

Austin, Bryan S.; Leahy, Michael J.

2015-01-01

Purpose: To construct and validate a new self-report instrument, the Clinical Judgment Skill Inventory (CJSI), inclusive of clinical judgment skill competencies that address counselor biases and evidence-based strategies. Method: An Internet-based survey design was used and an exploratory factor analysis was performed on a sample of rehabilitation…

Six Years of Comprehensive, Clinical, Performance-Based Assessment Using Standardized Patients at the Southern Illinois University School of Medicine.

ERIC Educational Resources Information Center

Vu, Nu Viet; And Others

1992-01-01

The use of a performance-based assessment of senior medical students' clinical skills utilizing standardized patients was evaluated, with 6,804 student-patient encounters involving 405 students over 6 years. Results provide evidence for test security, content validity, construct validity, reliability, and test ability to discriminate a wide range…

Measuring activities and participation in persons with haemophilia: A systematic review of commonly used instruments.

PubMed

Timmer, M A; Gouw, S C; Feldman, B M; Zwagemaker, A; de Kleijn, P; Pisters, M F; Schutgens, R E G; Blanchette, V; Srivastava, A; David, J A; Fischer, K; van der Net, J

2018-03-01

Monitoring clinical outcome in persons with haemophilia (PWH) is essential in order to provide optimal treatment for individual patients and compare effectiveness of treatment strategies. Experience with measurement of activities and participation in haemophilia is limited and consensus on preferred tools is lacking. The aim of this study was to give a comprehensive overview of the measurement properties of a selection of commonly used tools developed to assess activities and participation in PWH. Electronic databases were searched for articles that reported on reliability, validity or responsiveness of predetermined measurement tools (5 self-reported and 4 performance based measurement tools). Methodological quality of the studies was assessed according to the COSMIN checklist. Best evidence synthesis was used to summarize evidence on the measurement properties. The search resulted in 3453 unique hits. Forty-two articles were included. The self-reported Haemophilia Acitivity List (HAL), Pediatric HAL (PedHAL) and the performance based Functional Independence Score in Haemophilia (FISH) were studied most extensively. Methodological quality of the studies was limited. Measurement error, cross-cultural validity and responsiveness have been insufficiently evaluated. Albeit based on limited evidence, the measurement properties of the PedHAL, HAL and FISH are currently considered most satisfactory. Further research needs to focus on measurement error, responsiveness, interpretability and cross-cultural validity of the self-reported tools and validity of performance based tools which are able to assess limitations in sports and leisure activities. © 2018 The Authors. Haemophilia Published by John Wiley & Sons Ltd.

Management of lumbar zygapophysial (facet) joint pain

PubMed Central

Manchikanti, Laxmaiah; Hirsch, Joshua A; Falco, Frank JE; Boswell, Mark V

2016-01-01

AIM: To investigate the diagnostic validity and therapeutic value of lumbar facet joint interventions in managing chronic low back pain. METHODS: The review process applied systematic evidence-based assessment methodology of controlled trials of diagnostic validity and randomized controlled trials of therapeutic efficacy. Inclusion criteria encompassed all facet joint interventions performed in a controlled fashion. The pain relief of greater than 50% was the outcome measure for diagnostic accuracy assessment of the controlled studies with ability to perform previously painful movements, whereas, for randomized controlled therapeutic efficacy studies, the primary outcome was significant pain relief and the secondary outcome was a positive change in functional status. For the inclusion of the diagnostic controlled studies, all studies must have utilized either placebo controlled facet joint blocks or comparative local anesthetic blocks. In assessing therapeutic interventions, short-term and long-term reliefs were defined as either up to 6 mo or greater than 6 mo of relief. The literature search was extensive utilizing various types of electronic search media including PubMed from 1966 onwards, Cochrane library, National Guideline Clearinghouse, clinicaltrials.gov, along with other sources including previous systematic reviews, non-indexed journals, and abstracts until March 2015. Each manuscript included in the assessment was assessed for methodologic quality or risk of bias assessment utilizing the Quality Appraisal of Reliability Studies checklist for diagnostic interventions, and Cochrane review criteria and the Interventional Pain Management Techniques - Quality Appraisal of Reliability and Risk of Bias Assessment tool for therapeutic interventions. Evidence based on the review of the systematic assessment of controlled studies was graded utilizing a modified schema of qualitative evidence with best evidence synthesis, variable from level I to level V. RESULTS: Across all databases, 16 high quality diagnostic accuracy studies were identified. In addition, multiple studies assessed the influence of multiple factors on diagnostic validity. In contrast to diagnostic validity studies, therapeutic efficacy trials were limited to a total of 14 randomized controlled trials, assessing the efficacy of intraarticular injections, facet or zygapophysial joint nerve blocks, and radiofrequency neurotomy of the innervation of the facet joints. The evidence for the diagnostic validity of lumbar facet joint nerve blocks with at least 75% pain relief with ability to perform previously painful movements was level I, based on a range of level I to V derived from a best evidence synthesis. For therapeutic interventions, the evidence was variable from level II to III, with level II evidence for lumbar facet joint nerve blocks and radiofrequency neurotomy for long-term improvement (greater than 6 mo), and level III evidence for lumbosacral zygapophysial joint injections for short-term improvement only. CONCLUSION: This review provides significant evidence for the diagnostic validity of facet joint nerve blocks, and moderate evidence for therapeutic radiofrequency neurotomy and therapeutic facet joint nerve blocks in managing chronic low back pain. PMID:27190760

Diagnostic flexible pharyngo-laryngoscopy: development of a procedure specific assessment tool using a Delphi methodology.

PubMed

Melchiors, Jacob; Henriksen, Mikael Johannes Vuokko; Dikkers, Frederik G; Gavilán, Javier; Noordzij, J Pieter; Fried, Marvin P; Novakovic, Daniel; Fagan, Johannes; Charabi, Birgitte W; Konge, Lars; von Buchwald, Christian

2018-05-01

Proper training and assessment of skill in flexible pharyngo-laryngoscopy are central in the education of otorhinolaryngologists. To facilitate an evidence-based approach to curriculum development in this field, a structured analysis of what constitutes flexible pharyngo-laryngoscopy is necessary. Our aim was to develop an assessment tool based on this analysis. We conducted an international Delphi study involving experts from twelve countries in five continents. Utilizing reiterative assessment, the panel defined the procedure and reached consensus (defined as 80% agreement) on the phrasing of an assessment tool. FIFTY PANELISTS COMPLETED THE DELPHI PROCESS. THE MEDIAN AGE OF THE PANELISTS WAS 44 YEARS (RANGE 33-64 YEARS). MEDIAN EXPERIENCE IN OTORHINOLARYNGOLOGY WAS 15 YEARS (RANGE 6-35 YEARS). TWENTY-FIVE WERE SPECIALIZED IN LARYNGOLOGY, 16 WERE HEAD AND NECK SURGEONS, AND NINE WERE GENERAL OTORHINOLARYNGOLOGISTS. AN ASSESSMENT TOOL WAS CREATED CONSISTING OF TWELVE DISTINCT ITEMS.: Conclusion The gathering of validity evidence for assessment of core procedural skills within Otorhinolaryngology is central to the development of a competence-based education. The use of an international Delphi panel allows for the creation of an assessment tool which is widely applicable and valid. This work allows for an informed approach to technical skills training for flexible pharyngo-laryngoscopy and as further validity evidence is gathered allows for a valid assessment of clinical performance within this important skillset.

Initial construct validity evidence of a virtual human application for competency assessment in breaking bad news to a cancer patient.

PubMed

Guetterman, Timothy C; Kron, Frederick W; Campbell, Toby C; Scerbo, Mark W; Zelenski, Amy B; Cleary, James F; Fetters, Michael D

2017-01-01

Despite interest in using virtual humans (VHs) for assessing health care communication, evidence of validity is limited. We evaluated the validity of a VH application, MPathic-VR, for assessing performance-based competence in breaking bad news (BBN) to a VH patient. We used a two-group quasi-experimental design, with residents participating in a 3-hour seminar on BBN. Group A (n=15) completed the VH simulation before and after the seminar, and Group B (n=12) completed the VH simulation only after the BBN seminar to avoid the possibility that testing alone affected performance. Pre- and postseminar differences for Group A were analyzed with a paired t -test, and comparisons between Groups A and B were analyzed with an independent t -test. Compared to the preseminar result, Group A's postseminar scores improved significantly, indicating that the VH program was sensitive to differences in assessing performance-based competence in BBN. Postseminar scores of Group A and Group B were not significantly different, indicating that both groups performed similarly on the VH program. Improved pre-post scores demonstrate acquisition of skills in BBN to a VH patient. Pretest sensitization did not appear to influence posttest assessment. These results provide initial construct validity evidence that the VH program is effective for assessing BBN performance-based communication competence.

Initial construct validity evidence of a virtual human application for competency assessment in breaking bad news to a cancer patient

PubMed Central

Guetterman, Timothy C; Kron, Frederick W; Campbell, Toby C; Scerbo, Mark W; Zelenski, Amy B; Cleary, James F; Fetters, Michael D

2017-01-01

Background Despite interest in using virtual humans (VHs) for assessing health care communication, evidence of validity is limited. We evaluated the validity of a VH application, MPathic-VR, for assessing performance-based competence in breaking bad news (BBN) to a VH patient. Methods We used a two-group quasi-experimental design, with residents participating in a 3-hour seminar on BBN. Group A (n=15) completed the VH simulation before and after the seminar, and Group B (n=12) completed the VH simulation only after the BBN seminar to avoid the possibility that testing alone affected performance. Pre- and postseminar differences for Group A were analyzed with a paired t-test, and comparisons between Groups A and B were analyzed with an independent t-test. Results Compared to the preseminar result, Group A’s postseminar scores improved significantly, indicating that the VH program was sensitive to differences in assessing performance-based competence in BBN. Postseminar scores of Group A and Group B were not significantly different, indicating that both groups performed similarly on the VH program. Conclusion Improved pre–post scores demonstrate acquisition of skills in BBN to a VH patient. Pretest sensitization did not appear to influence posttest assessment. These results provide initial construct validity evidence that the VH program is effective for assessing BBN performance-based communication competence. PMID:28794664

Adaptive Practice: Next Generation Evidence-Based Practice in Digital Environments.

PubMed

Kennedy, Margaret Ann

2016-01-01

Evidence-based practice in nursing is considered foundational to safe, competent care. To date, rigid traditional perceptions of what constitutes 'evidence' have constrained the recognition and use of practice-based evidence and the exploitation of novel forms of evidence from data rich environments. Advancements such as the conceptualization of clinical intelligence, the prevalence of increasingly sophisticated digital health information systems, and the advancement of the Big Data phenomenon have converged to generate a new contemporary context. In today's dynamic data-rich environments, clinicians have new sources of valid evidence, and need a new paradigm supporting clinical practice that is adaptive to information generated by diverse electronic sources. This opinion paper presents adaptive practice as the next generation of evidence-based practice in contemporary evidence-rich environments and provides recommendations for the next phase of evolution.

Constructing a Validity Argument for the Objective Structured Assessment of Technical Skills (OSATS): A Systematic Review of Validity Evidence

ERIC Educational Resources Information Center

Hatala, Rose; Cook, David A.; Brydges, Ryan; Hawkins, Richard

2015-01-01

In order to construct and evaluate the validity argument for the Objective Structured Assessment of Technical Skills (OSATS), based on Kane's framework, we conducted a systematic review. We searched MEDLINE, EMBASE, CINAHL, PsycINFO, ERIC, Web of Science, Scopus, and selected reference lists through February 2013. Working in duplicate, we selected…

A Field-Based Testing Protocol for Assessing Gross Motor Skills in Preschool Children: The Children's Activity and Movement in Preschool Study Motor Skills Protocol

ERIC Educational Resources Information Center

Williams, Harriet G.; Pfeiffer, Karin A.; Dowda, Marsha; Jeter, Chevy; Jones, Shaverra; Pate, Russell R.

2009-01-01

The purpose of this study was to develop a valid and reliable tool for use in assessing motor skills in preschool children in field-based settings. The development of the Children's Activity and Movement in Preschool Study Motor Skills Protocol included evidence of its reliability and validity for use in field-based environments as part of large…

Systematic review of measurement properties of questionnaires measuring somatization in primary care patients.

PubMed

Sitnikova, Kate; Dijkstra-Kersten, Sandra M A; Mokkink, Lidwine B; Terluin, Berend; van Marwijk, Harm W J; Leone, Stephanie S; van der Horst, Henriëtte E; van der Wouden, Johannes C

2017-12-01

The aim of this review is to critically appraise the evidence on measurement properties of self-report questionnaires measuring somatization in adult primary care patients and to provide recommendations about which questionnaires are most useful for this purpose. We assessed the methodological quality of included studies using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. To draw overall conclusions about the quality of the questionnaires, we conducted an evidence synthesis using predefined criteria for judging the measurement properties. We found 24 articles on 9 questionnaires. Studies on the Patient Health Questionnaire-15 (PHQ-15) and the Four-Dimensional Symptom Questionnaire (4DSQ) somatization subscale prevailed and covered the broadest range of measurement properties. These questionnaires had the best internal consistency, test-retest reliability, structural validity, and construct validity. The PHQ-15 also had good criterion validity, whereas the 4DSQ somatization subscale was validated in several languages. The Bodily Distress Syndrome (BDS) checklist had good internal consistency and structural validity. Some evidence was found for good construct validity and criterion validity of the Physical Symptom Checklist (PSC-51) and good construct validity of the Symptom Check-List (SCL-90-R) somatization subscale. However, these three questionnaires were only studied in a small number of primary care studies. Based on our findings, we recommend the use of either the PHQ-15 or 4DSQ somatization subscale for somatization in primary care. Other questionnaires, such as the BDS checklist, PSC-51 and the SCL-90-R somatization subscale show promising results but have not been studied extensively in primary care. Copyright © 2017 Elsevier Inc. All rights reserved.

Development and psychometric testing of a trans-professional evidence-based practice profile questionnaire.

PubMed

McEvoy, Maureen Patricia; Williams, Marie T; Olds, Timothy Stephen

2010-01-01

Previous survey tools operationalising knowledge, attitudes or beliefs about evidence-based practice (EBP) have shortcomings in content, psychometric properties and target audience. This study developed and psychometrically assessed a self-report trans-professional questionnaire to describe an EBP profile. Sixty-six items were collated from existing EBP questionnaires and administered to 526 academics and students from health and non-health backgrounds. Principal component factor analysis revealed the presence of five factors (Relevance, Terminology, Confidence, Practice and Sympathy). Following expert panel review and pilot testing, the 58-item final questionnaire was disseminated to 105 subjects on two occasions. Test-retest and internal reliability were quantified using intra-class correlation coefficients (ICCs) and Cronbach's alpha, convergent validity against a commonly used EBP questionnaire by Pearson's correlation coefficient and discriminative validity via analysis of variance (ANOVA) based on exposure to EBP training. The final questionnaire demonstrated acceptable internal consistency (Cronbach's alpha 0.96), test-retest reliability (ICCs range 0.77-0.94) and convergent validity (Practice 0.66, Confidence 0.80 and Sympathy 0.54). Three factors (Relevance, Terminology and Confidence) distinguished EBP exposure groups (ANOVA p < 0.001-0.004). The evidence-based practice profile (EBP(2)) questionnaire is a reliable instrument with the ability to discriminate for three factors, between respondents with differing EBP exposures.

What Does It Take to Scale Up and Sustain Evidence-Based Practices?

ERIC Educational Resources Information Center

Klingner, Janette K.; Boardman, Alison G.; Mcmaster, Kristen L.

2013-01-01

This article discusses the strategic scaling up of evidence-based practices. The authors draw from the scholarly work of fellow special education researchers and from the field of learning sciences. The article defines scaling up as the process by which researchers or educators initially implement interventions on a small scale, validate them, and…

Comparison of Evidence-Based Practice between Physicians and Nurses: A National Survey of Regional Hospitals in Taiwan

ERIC Educational Resources Information Center

Chiu, Ya-Wen; Weng, Yi-Hao; Lo, Heng-Lien; Hsu, Chih-Cheng; Shih, Ya-Hui; Kuo, Ken N.

2010-01-01

Introduction: Although evidence-based practice (EBP) has been widely investigated, few studies compare physicians and nurses on performance. Methods: A structured questionnaire survey was used to investigate EBP among physicians and nurses in 61 regional hospitals of Taiwan. Valid postal questionnaires were collected from 605 physicians and 551…

Validation of a short measure of effort-reward imbalance in the workplace: evidence from China.

PubMed

Li, Jian; Loerbroks, Adrian; Shang, Li; Wege, Natalia; Wahrendorf, Morten; Siegrist, Johannes

2012-01-01

Work stress is an emergent risk in occupational health in China, and its measurement is still a critical issue. The aim of this study was to examine the reliability and validity of a short version of the effort-reward imbalance (ERI) questionnaire in a sample of Chinese workers. A community-based survey was conducted in 1,916 subjects aged 30-65 years with paid employment (971 men and 945 women). Acceptable internal consistencies of the three scales, effort, reward and overcommitment, were obtained. Confirmatory factor analysis showed a good model fit of the data with the theoretical structure (goodness-of-fit index = 0.95). Evidence of criterion validity was demonstrated, as all three scales were independently associated with elevated odds ratios of both poor physical and mental health. Based on the findings of our study, this short version of the ERI questionnaire is considered to be a reliable and valid tool for measuring psychosocial work environment in Chinese working populations.

Motivation and personality: relationships between putative motive dimensions and the five factor model of personality.

PubMed

Bernard, Larry C

2010-04-01

There are few multidimensional measures of individual differences in motivation available. The Assessment of Individual Motives-Questionnaire assesses 15 putative dimensions of motivation. The dimensions are based on evolutionary theory and preliminary evidence suggests the motive scales have good psychometric properties. The scales are reliable and there is evidence of their consensual validity (convergence of self-other ratings) and behavioral validity (relationships with self-other reported behaviors of social importance). Additional validity research is necessary, however, especially with respect to current models of personality. The present study tested two general and 24 specific hypotheses based on proposed evolutionary advantages/disadvantages and fitness benefits/costs of the five-factor model of personality together with the new motive scales in a sample of 424 participants (M age=28.8 yr., SD=14.6). Results were largely supportive of the hypotheses. These results support the validity of new motive dimensions and increase understanding of the five-factor model of personality.

20 CFR 404.727 - Evidence of a deemed valid marriage.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 20 Employees' Benefits 2 2010-04-01 2010-04-01 false Evidence of a deemed valid marriage. 404.727... DISABILITY INSURANCE (1950- ) Evidence Evidence of Age, Marriage, and Death § 404.727 Evidence of a deemed valid marriage. (a) General. A deemed valid marriage is a ceremonial marriage we consider valid even...

Building Evidence in Early Childhood Special Education: A Systematic Review of Replication Intervention Studies

ERIC Educational Resources Information Center

Banerjee, Rashida; Movahedazarhouligh, Sara; Millen, Kaitlyn; Luckner, John L.

2018-01-01

Valid and evidence-informed practices are critical to help young children with disabilities and their families with highly effective interventions and instruction to reach their potentials. Replication research is critical for appraising research and identifying evidence-based practices. The purpose of this study was to replicate the methods used…

Identifying and Evaluating External Validity Evidence for Passing Scores

ERIC Educational Resources Information Center

Davis-Becker, Susan L.; Buckendahl, Chad W.

2013-01-01

A critical component of the standard setting process is collecting evidence to evaluate the recommended cut scores and their use for making decisions and classifying students based on test performance. Kane (1994, 2001) proposed a framework by which practitioners can identify and evaluate evidence of the results of the standard setting from (1)…

Methodologic limitations of prescription opioid safety research and recommendations for improving the evidence base.

PubMed

Ranapurwala, Shabbar I; Naumann, Rebecca B; Austin, Anna E; Dasgupta, Nabarun; Marshall, Stephen W

2018-06-03

The ongoing opioid epidemic has claimed more than a quarter million Americans' lives over the past 15 years. The epidemic began with an escalation of prescription opioid deaths and has now evolved to include secondary waves of illicit heroin and fentanyl deaths, while the deaths due to prescription opioid overdoses are still increasing. In response, the Centers for Disease Control and Prevention (CDC) moved to limit opioid prescribing with the release of opioid prescribing guidelines for chronic noncancer pain in March 2016. The guidelines represent a logical and timely federal response to this growing crisis. However, CDC acknowledged that the evidence base linking opioid prescribing to opioid use disorders and overdose was grades 3 and 4. Motivated by the need to strengthen the evidence base, this review details limitations of the opioid safety studies cited in the CDC guidelines with a focus on methodological limitations related to internal and external validity. Internal validity concerns were related to poor confounding control, variable misclassification, selection bias, competing risks, and potential competing interventions. External validity concerns arose from the use of limited source populations, historical data (in a fast-changing epidemic), and issues with handling of cancer and acute pain patients' data. We provide a nonexhaustive list of 7 recommendations to address these limitations in future opioid safety studies. Strengthening the opioid safety evidence base will aid any future revisions of the CDC guidelines and enhance their prevention impact. Copyright © 2018 John Wiley & Sons, Ltd.

Development of a proficiency-based virtual reality simulation training curriculum for laparoscopic appendicectomy.

PubMed

Sirimanna, Pramudith; Gladman, Marc A

2017-10-01

Proficiency-based virtual reality (VR) training curricula improve intraoperative performance, but have not been developed for laparoscopic appendicectomy (LA). This study aimed to develop an evidence-based training curriculum for LA. A total of 10 experienced (>50 LAs), eight intermediate (10-30 LAs) and 20 inexperienced (<10 LAs) operators performed guided and unguided LA tasks on a high-fidelity VR simulator using internationally relevant techniques. The ability to differentiate levels of experience (construct validity) was measured using simulator-derived metrics. Learning curves were analysed. Proficiency benchmarks were defined by the performance of the experienced group. Intermediate and experienced participants completed a questionnaire to evaluate the realism (face validity) and relevance (content validity). Of 18 surgeons, 16 (89%) considered the VR model to be visually realistic and 17 (95%) believed that it was representative of actual practice. All 'guided' modules demonstrated construct validity (P < 0.05), with learning curves that plateaued between sessions 6 and 9 (P < 0.01). When comparing inexperienced to intermediates to experienced, the 'unguided' LA module demonstrated construct validity for economy of motion (5.00 versus 7.17 versus 7.84, respectively; P < 0.01) and task time (864.5 s versus 477.2 s versus 352.1 s, respectively, P < 0.01). Construct validity was also confirmed for number of movements, path length and idle time. Validated modules were used for curriculum construction, with proficiency benchmarks used as performance goals. A VR LA model was realistic and representative of actual practice and was validated as a training and assessment tool. Consequently, the first evidence-based internationally applicable training curriculum for LA was constructed, which facilitates skill acquisition to proficiency. © 2017 Royal Australasian College of Surgeons.

Validity Evidence for the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT).

PubMed

Breimer, Gerben E; Haji, Faizal A; Cinalli, Giuseppe; Hoving, Eelco W; Drake, James M

2017-02-01

Growing demand for transparent and standardized methods for evaluating surgical competence prompted the construction of the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT). To provide validity evidence of the NEVAT by reporting on the tool's internal structure and its relationship with surgical expertise during simulation-based training. The NEVAT was used to assess performance of trainees and faculty at an international neuroendoscopy workshop. All participants performed an endoscopic third ventriculostomy (ETV) on a synthetic simulator. Participants were simultaneously scored by 2 raters using the NEVAT procedural checklist and global rating scale (GRS). Evidence of internal structure was collected by calculating interrater reliability and internal consistency of raters' scores. Evidence of relationships with other variables was collected by comparing the ETV performance of experts, experienced trainees, and novices using Jonckheere's test (evidence of construct validity). Thirteen experts, 11 experienced trainees, and 10 novices participated. The interrater reliability by the intraclass correlation coefficient for the checklist and GRS was 0.82 and 0.94, respectively. Internal consistency (Cronbach's α) for the checklist and the GRS was 0.74 and 0.97, respectively. Median scores with interquartile range on the checklist and GRS for novices, experienced trainees, and experts were 0.69 (0.58-0.86), 0.85 (0.63-0.89), and 0.85 (0.81-0.91) and 3.1 (2.5-3.8), 3.7 (2.2-4.3) and 4.6 (4.4-4.9), respectively. Jonckheere's test showed that the median checklist and GRS score increased with performer expertise ( P = .04 and .002, respectively). This study provides validity evidence for the NEVAT to support its use as a standardized method of evaluating neuroendoscopic competence during simulation-based training. Copyright © 2016 by the Congress of Neurological Surgeons

In 'big bang' major incidents do triage tools accurately predict clinical priority?: a systematic review of the literature.

PubMed

Kilner, T M; Brace, S J; Cooke, M W; Stallard, N; Bleetman, A; Perkins, G D

2011-05-01

The term "big bang" major incidents is used to describe sudden, usually traumatic,catastrophic events, involving relatively large numbers of injured individuals, where demands on clinical services rapidly outstrip the available resources. Triage tools support the pre-hospital provider to prioritise which patients to treat and/or transport first based upon clinical need. The aim of this review is to identify existing triage tools and to determine the extent to which their reliability and validity have been assessed. A systematic review of the literature was conducted to identify and evaluate published data validating the efficacy of the triage tools. Studies using data from trauma patients that report on the derivation, validation and/or reliability of the specific pre-hospital triage tools were eligible for inclusion.Purely descriptive studies, reviews, exercises or reports (without supporting data) were excluded. The search yielded 1982 papers. After initial scrutiny of title and abstract, 181 papers were deemed potentially applicable and from these 11 were identified as relevant to this review (in first figure). There were two level of evidence one studies, three level of evidence two studies and six level of evidence three studies. The two level of evidence one studies were prospective validations of Clinical Decision Rules (CDR's) in children in South Africa, all the other studies were retrospective CDR derivation, validation or cohort studies. The quality of the papers was rated as good (n=3), fair (n=7), poor (n=1). There is limited evidence for the validity of existing triage tools in big bang major incidents.Where evidence does exist it focuses on sensitivity and specificity in relation to prediction of trauma death or severity of injury based on data from single or small number patient incidents. The Sacco system is unique in combining survivability modelling with the degree by which the system is overwhelmed in the triage decision system. The practicalities, training implications, performance characteristics and reliance on computer technology during a mass casualty incident require further evaluation. 2010 Elsevier Ltd. All rights reserved.

Reliability and validity of advanced theory-of-mind measures in middle childhood and adolescence.

PubMed

Hayward, Elizabeth O; Homer, Bruce D

2017-09-01

Although theory-of-mind (ToM) development is well documented for early childhood, there is increasing research investigating changes in ToM reasoning in middle childhood and adolescence. However, the psychometric properties of most advanced ToM measures for use with older children and adolescents have not been firmly established. We report on the reliability and validity of widely used, conventional measures of advanced ToM with this age group. Notable issues with both reliability and validity of several of the measures were evident in the findings. With regard to construct validity, results do not reveal a clear empirical commonality between tasks, and, after accounting for comprehension, developmental trends were evident in only one of the tasks investigated. Statement of contribution What is already known on this subject? Second-order false belief tasks have acceptable internal consistency. The Eyes Test has poor internal consistency. Validity of advanced theory-of-mind tasks is often based on the ability to distinguish clinical from typical groups. What does this study add? This study examines internal consistency across six widely used advanced theory-of-mind tasks. It investigates validity of tasks based on comprehension of items by typically developing individuals. It further assesses construct validity, or commonality between tasks. © 2017 The British Psychological Society.

Toward a New Process-Based Indicator for Measuring Writing Fluency: Evidence from L2 Writers' Think-Aloud Protocols

ERIC Educational Resources Information Center

Abdel Latif, Muhammad M.

2009-01-01

This article reports on a study aimed at testing the hypothesis that, because of strategic and temporal variables, composing rate and text quantity may not be valid measures of writing fluency. A second objective was to validate the mean length of writers' translating episodes as a process-based indicator that mirrors their fluent written…

Reliability and validity evidence of the Assessment of Language Use in Social Contexts for Adults (ALUSCA).

PubMed

Valente, Ana Rita S; Hall, Andreia; Alvelos, Helena; Leahy, Margaret; Jesus, Luis M T

2018-04-12

The appropriate use of language in context depends on the speaker's pragmatic language competencies. A coding system was used to develop a specific and adult-focused self-administered questionnaire to adults who stutter and adults who do not stutter, The Assessment of Language Use in Social Contexts for Adults, with three categories: precursors, basic exchanges, and extended literal/non-literal discourse. This paper presents the content validity, item analysis, reliability coefficients and evidences of construct validity of the instrument. Content validity analysis was based on a two-stage process: first, 11 pragmatic questionnaires were assessed to identify items that probe each pragmatic competency and to create the first version of the instrument; second, items were assessed qualitatively by an expert panel composed by adults who stutter and controls, and quantitatively and qualitatively by an expert panel composed by clinicians. A pilot study was conducted with five adults who stutter and five controls to analyse items and calculate reliability. Construct validity evidences were obtained using the hypothesized relationships method and factor analysis with 28 adults who stutter and 28 controls. Concerning content validity, the questionnaires assessed up to 13 pragmatic competencies. Qualitative and quantitative analysis revealed ambiguities in items construction. Disagreement between experts was solved through item modification. The pilot study showed that the instrument presented internal consistency and temporal stability. Significant differences between adults who stutter and controls and different response profiles revealed the instrument's underlying construct. The instrument is reliable and presented evidences of construct validity.

The Illusory Beliefs Inventory: a new measure of magical thinking and its relationship with obsessive compulsive disorder.

PubMed

Kingdon, Bianca L; Egan, Sarah J; Rees, Clare S

2012-01-01

Magical thinking has been proposed to have an aetiological role in obsessive compulsive disorder (OCD). To address the limitations of existing measures of magical thinking we developed and validated a new 24-item measure of magical thinking, the Illusory Beliefs Inventory (IBI). The validation sample comprised a total of 1194 individuals across two samples recruited via an Internet based survey. Factor analysis identified three subscales representing domains relevant to the construct of magical thinking: Magical Beliefs, Spirituality, and Internal State and Thought Action Fusion. The scale had excellent internal consistency and evidence of convergent and discriminant validity. Evidence of criterion-related concurrent validity confirmed that magical thinking is a cognitive domain associated with OCD and is largely relevant to neutralizing, obsessing and hoarding symptoms. It is important for future studies to extend the evidence of the psychometric properties of the IBI in new populations and to conduct longitudinal studies to examine the aetiological role of magical thinking.

Assessment, Selection, Use, and Evaluation of Body-Worn Absorbent Products for Adults With Incontinence: A WOCN Society Consensus Conference.

PubMed

Gray, Mikel; Kent, Dea; Ermer-Seltun, JoAnn; McNichol, Laurie

The Wound, Ostomy and Continence Nurses (WOCN) Society charged a task force with creating recommendations for assessment, selection, use, and evaluation of body-worn absorbent products. The 3-member task force, assisted by a moderator with knowledge of this area of care, completed a scoping literature review to identify recommendations supported by adequate research to qualify as evidence-based, and area of care where evidence needed to guide care was missing. Based on findings of this scoping review, the Society then convened a panel of experts to develop consensus statements guiding assessment, use, and evaluation of the effect of body-worn absorbent products for adults with urinary and/or fecal incontinence. These consensus-based statements underwent a second round of content validation using a modified Delphi technique using a different panel of clinicians with expertise in this area of care. This article reports on the scoping review and subsequent evidence-based statements, along with generation and validation of consensus-based statements that will be used to create an algorithm to aid clinical decision making.

Comprehension of Written Grammar Test: Reliability and Known-Groups Validity Study With Hearing and Deaf and Hard-of-Hearing Students.

PubMed

Cannon, Joanna E; Hubley, Anita M; Millhoff, Courtney; Mazlouman, Shahla

2016-01-01

The aim of the current study was to gather validation evidence for the Comprehension of Written Grammar (CWG; Easterbrooks, 2010) receptive test of 26 grammatical structures of English print for use with children who are deaf and hard of hearing (DHH). Reliability and validity data were collected for 98 participants (49 DHH and 49 hearing) in Grades 2-6. The objectives were to: (a) examine 4-week test-retest reliability data; and (b) provide evidence of known-groups validity by examining expected differences between the groups on the CWG vocabulary pretest and main test, as well as selected structures. Results indicated excellent test-retest reliability estimates for CWG test scores. DHH participants performed statistically significantly lower on the CWG vocabulary pretest and main test than the hearing participants. Significantly lower performance by DHH participants on most expected grammatical structures (e.g., basic sentence patterns, auxiliary "be" singular/plural forms, tense, comparatives, and complementation) also provided known groups evidence. Overall, the findings of this study showed strong evidence of the reliability of scores and known group-based validity of inferences made from the CWG. © The Author 2015. Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions@oup.com.

Applying ergonomics to systems: some documented "lessons learned".

PubMed

Hendrick, Hal W

2008-07-01

Based on evidence accumulated during the author's 45 years of professional experience, the author presents 23 important "lessons learned" regarding applying ergonomics to systems. Documented results from reported cases or other evidence are presented to validate each of these practical learning points.

Training of Lay Health Educators to Implement an Evidence-Based Behavioral Weight Loss Intervention in Rural Senior Centers

ERIC Educational Resources Information Center

Krukowski, Rebecca A.; Lensing, Shelly; Love, ShaRhonda; Prewitt, T. Elaine; Adams, Becky; Cornell, Carol E.; Felix, Holly C.; West, Delia

2013-01-01

Purpose of the Study: Lay health educators (LHEs) offer great promise for facilitating the translation of evidence-based health promotion programs to underserved areas; yet, there is little guidance on how to train LHEs to implement these programs, particularly in the crucial area of empirically validated obesity interventions. Design and Methods:…

Development and Validation of a Measurement Scale to Analyze the Environment for Evidence-Based Medicine Learning and Practice by Medical Residents

ERIC Educational Resources Information Center

Mi, Fangqiong

2010-01-01

A growing number of residency programs are instituting curricula to include the component of evidence-based medicine (EBM) principles and process. However, these curricula may not be able to achieve the optimal learning outcomes, perhaps because various contextual factors are often overlooked when EBM training is being designed, developed, and…

Evaluating the Validity of Systematic Reviews to Identify Empirically Supported Treatments

ERIC Educational Resources Information Center

Slocum, Timothy A.; Detrich, Ronnie; Spencer, Trina D.

2012-01-01

The "best available evidence" is one of the three basic inputs into evidence-based practice. This paper sets out a framework for evaluating the quality of systematic reviews that are intended to identify empirically supported interventions as a way of summarizing the best available evidence. The premise of this paper is that the process of…

Validating an Observation Protocol to Measure Special Education Teacher Effectiveness

ERIC Educational Resources Information Center

Johnson, Evelyn S.; Semmelroth, Carrie L.

2015-01-01

This study used Kane's (2013) Interpretation/Use Argument (IUA) to measure validity on the Recognizing Effective Special Education Teachers (RESET) observation tool. The RESET observation tool is designed to evaluate special education teacher effectiveness using evidence-based instructional practices as the basis for evaluation. In alignment with…

Does Linguistic Analysis Confirm the Validity of Facilitated Communication?

ERIC Educational Resources Information Center

Saloviita, Timo

2018-01-01

Facilitated communication (FC) has been interpreted as an ideomotor phenomenon, in which one person physically supports another person's hand and unconsciously affects the content of the writing. Despite the strong experimental evidence against the authenticity of FC output, several studies claim to support its validity based on idiosyncrasies…

Validation methodology in publications describing epidemiological registration methods of dental caries: a systematic review.

PubMed

Sjögren, P; Ordell, S; Halling, A

2003-12-01

The aim was to describe and systematically review the methodology and reporting of validation in publications describing epidemiological registration methods for dental caries. BASIC RESEARCH METHODOLOGY: Literature searches were conducted in six scientific databases. All publications fulfilling the predetermined inclusion criteria were assessed for methodology and reporting of validation using a checklist including items described previously as well as new items. The frequency of endorsement of the assessed items was analysed. Moreover, the type and strength of evidence, was evaluated. Reporting of predetermined items relating to methodology of validation and the frequency of endorsement of the assessed items were of primary interest. Initially 588 publications were located. 74 eligible publications were obtained, 23 of which fulfilled the inclusion criteria and remained throughout the analyses. A majority of the studies reported the methodology of validation. The reported methodology of validation was generally inadequate, according to the recommendations of evidence-based medicine. The frequencies of reporting the assessed items (frequencies of endorsement) ranged from four to 84 per cent. A majority of the publications contributed to a low strength of evidence. There seems to be a need to improve the methodology and the reporting of validation in publications describing professionally registered caries epidemiology. Four of the items assessed in this study are potentially discriminative for quality assessments of reported validation.

Diffusion of robotics into clinical practice in the United States: process, patient safety, learning curves, and the public health.

PubMed

Mirheydar, Hossein S; Parsons, J Kellogg

2013-06-01

Robotic technology disseminated into urological practice without robust comparative effectiveness data. To review the diffusion of robotic surgery into urological practice. We performed a comprehensive literature review focusing on diffusion patterns, patient safety, learning curves, and comparative costs for robotic radical prostatectomy, partial nephrectomy, and radical cystectomy. Robotic urologic surgery diffused in patterns typical of novel technology spreading among practicing surgeons. Robust evidence-based data comparing outcomes of robotic to open surgery were sparse. Although initial Level 3 evidence for robotic prostatectomy observed complication outcomes similar to open prostatectomy, subsequent population-based Level 2 evidence noted an increased prevalence of adverse patient safety events and genitourinary complications among robotic patients during the early years of diffusion. Level 2 evidence indicated comparable to improved patient safety outcomes for robotic compared to open partial nephrectomy and cystectomy. Learning curve recommendations for robotic urologic surgery have drawn exclusively on Level 4 evidence and subjective, non-validated metrics. The minimum number of cases required to achieve competency for robotic prostatectomy has increased to unrealistically high levels. Most comparative cost-analyses have demonstrated that robotic surgery is significantly more expensive than open or laparoscopic surgery. Evidence-based data are limited but suggest an increased prevalence of adverse patient safety events for robotic prostatectomy early in the national diffusion period. Learning curves for robotic urologic surgery are subjective and based on non-validated metrics. The urological community should develop rigorous, evidence-based processes by which future technological innovations may diffuse in an organized and safe manner.

Simulation-Based Abdominal Ultrasound Training - A Systematic Review.

PubMed

Østergaard, M L; Ewertsen, C; Konge, L; Albrecht-Beste, E; Bachmann Nielsen, M

2016-06-01

The aim is to provide a complete overview of the different simulation-based training options for abdominal ultrasound and to explore the evidence of their effect. This systematic review was performed according to the PRISMA guidelines and Medline, Embase, Web of Science, and the Cochrane Library was searched. Articles were divided into three categories based on study design (randomized controlled trials, before-and-after studies and descriptive studies) and assessed for level of evidence using the Oxford Centre for Evidence Based Medicine (OCEBM) system and for bias using the Cochrane Collaboration risk of bias assessment tool. Seventeen studies were included in the analysis: four randomized controlled trials, eight before-and-after studies with pre- and post-test evaluations, and five descriptive studies. No studies scored the highest level of evidence, and 14 had the lowest level. Bias was high for 11 studies, low for four, and unclear for two. No studies used a test with established evidence of validity or examined the correlation between obtained skills on the simulators and real-life clinical skills. Only one study used blinded assessors. The included studies were heterogeneous in the choice of simulator, study design, participants, and outcome measures, and the level of evidence for effect was inadequate. In all studies simulation training was equally or more beneficial than other instructions or no instructions. Study designs had significant built-in bias and confounding issues; therefore, further research should be based on randomized controlled trials using tests with validity evidence and blinded assessors. © Georg Thieme Verlag KG Stuttgart · New York.

An Adaptation of the Original Fresno Test to Measure Evidence-Based Practice Competence in Pediatric Bedside Nurses.

PubMed

Laibhen-Parkes, Natasha; Kimble, Laura P; Melnyk, Bernadette Mazurek; Sudia, Tanya; Codone, Susan

2018-06-01

Instruments used to assess evidence-based practice (EBP) competence in nurses have been subjective, unreliable, or invalid. The Fresno test was identified as the only instrument to measure all the steps of EBP with supportive reliability and validity data. However, the items and psychometric properties of the original Fresno test are only relevant to measure EBP with medical residents. Therefore, the purpose of this paper is to describe the development of the adapted Fresno test for pediatric nurses, and provide preliminary validity and reliability data for its use with Bachelor of Science in Nursing-prepared pediatric bedside nurses. General adaptations were made to the original instrument's case studies, item content, wording, and format to meet the needs of a pediatric nursing sample. The scoring rubric was also modified to complement changes made to the instrument. Content and face validity, and intrarater reliability of the adapted Fresno test were assessed during a mixed-methods pilot study conducted from October to December 2013 with 29 Bachelor of Science in Nursing-prepared pediatric nurses. Validity data provided evidence for good content and face validity. Intrarater reliability estimates were high. The adapted Fresno test presented here appears to be a valid and reliable assessment of EBP competence in Bachelor of Science in Nursing-prepared pediatric nurses. However, further testing of this instrument is warranted using a larger sample of pediatric nurses in diverse settings. This instrument can be a starting point for evaluating the impact of EBP competence on patient outcomes. © 2018 Sigma Theta Tau International.

TENI: A comprehensive battery for cognitive assessment based on games and technology.

PubMed

Delgado, Marcela Tenorio; Uribe, Paulina Arango; Alonso, Andrés Aparicio; Díaz, Ricardo Rosas

2016-01-01

TENI (Test de Evaluación Neuropsicológica Infantil) is an instrument developed to assess cognitive abilities in children between 3 and 9 years of age. It is based on a model that incorporates games and technology as tools to improve the assessment of children's capacities. The test was standardized with two Chilean samples of 524 and 82 children living in urban zones. Evidence of reliability and validity based on current standards is presented. Data show good levels of reliability for all subtests. Some evidence of validity in terms of content, test structure, and association with other variables is presented. This instrument represents a novel approach and a new frontier in cognitive assessment. Further studies with clinical, rural, and cross-cultural populations are required.

20 CFR 404.725 - Evidence of a valid ceremonial marriage.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 20 Employees' Benefits 2 2010-04-01 2010-04-01 false Evidence of a valid ceremonial marriage. 404... DISABILITY INSURANCE (1950- ) Evidence Evidence of Age, Marriage, and Death § 404.725 Evidence of a valid ceremonial marriage. (a) General. A valid ceremonial marriage is one that follows procedures set by law in...

Therapist self-report of evidence-based practices in usual care for adolescent behavior problems: factor and construct validity.

PubMed

Hogue, Aaron; Dauber, Sarah; Henderson, Craig E

2014-01-01

This study introduces a therapist-report measure of evidence-based practices for adolescent conduct and substance use problems. The Inventory of Therapy Techniques-Adolescent Behavior Problems (ITT-ABP) is a post-session measure of 27 techniques representing four approaches: cognitive-behavioral therapy (CBT), family therapy (FT), motivational interviewing (MI), and drug counseling (DC). A total of 822 protocols were collected from 32 therapists treating 71 adolescents in six usual care sites. Factor analyses identified three clinically coherent scales with strong internal consistency across the full sample: FT (8 items; α = .79), MI/CBT (8 items; α = .87), and DC (9 items, α = .90). The scales discriminated between therapists working in a family-oriented site versus other sites and showed moderate convergent validity with therapist reports of allegiance and skill in each approach. The ITT-ABP holds promise as a cost-efficient quality assurance tool for supporting high-fidelity delivery of evidence-based practices in usual care.

[Traceability of Wine Varieties Using Near Infrared Spectroscopy Combined with Cyclic Voltammetry].

PubMed

Li, Meng-hua; Li, Jing-ming; Li, Jun-hui; Zhang, Lu-da; Zhao, Long-lian

2015-06-01

To achieve the traceability of wine varieties, a method was proposed to fuse Near-infrared (NIR) spectra and cyclic voltammograms (CV) which contain different information using D-S evidence theory. NIR spectra and CV curves of three different varieties of wines (cabernet sauvignon, merlot, cabernet gernischt) which come from seven different geographical origins were collected separately. The discriminant models were built using PLS-DA method. Based on this, D-S evidence theory was then applied to achieve the integration of the two kinds of discrimination results. After integrated by D-S evidence theory, the accuracy rate of cross-validation is 95.69% and validation set is 94.12% for wine variety identification. When only considering the wine that come from Yantai, the accuracy rate of cross-validation is 99.46% and validation set is 100%. All the traceability models after fusion achieved better results on classification than individual method. These results suggest that the proposed method combining electrochemical information with spectral information using the D-S evidence combination formula is benefit to the improvement of model discrimination effect, and is a promising tool for discriminating different kinds of wines.

Assessing the validity of surrogate endpoints in the context of a controversy about the measurement of effectiveness of hepatitis C virus treatment.

PubMed

Dobler, Claudia C; Morgan, Rebecca L; Falck-Ytter, Yngve; Montori, Victor M; Murad, M Hassan

2018-04-01

Surrogate endpoints are often used in clinical trials, as they allow for indirect measures of outcomes (eg, shorter trials with less participants). Improvements in surrogate endpoints (eg, reduction in low density lipoprotein cholesterol, normalisation of glycated haemoglobin) achieved with an intervention are, however, not always associated with improvements in patient-important outcomes. The common tendency in evidence-based medicine is to view results based on surrogate endpoints as less certain than results based on long term, final patient-important outcomes and rate them as 'lower quality evidence'. However, careful appraisal of the validity of a surrogate endpoint as a measure of the final, patient-important outcome is more useful than an automatic judgement. In this guide, we use a contemporary and currently highly debated example of the surrogate endpoint 'sustained viral response' (ie, viral eradication considered to represent successful treatment) in patients treated for chronic hepatitis C virus. We demonstrate how the validity of a surrogate endpoint can be critically appraised to assess the quality of the evidence (ie, the certainty in estimates) and the implications for decision-making. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

The Predictive Validity of Measures of Teacher Candidate Programs and Performance: Toward an Evidence-Based Approach to Teacher Preparation

ERIC Educational Resources Information Center

Henry, Gary T.; Campbell, Shanyce L.; Thompson, Charles L.; Patriarca, Linda A.; Luterbach, Kenneth J.; Lys, Diana B.; Covington, Vivian Martin

2013-01-01

Calls for evidence-based reform of teacher preparation programs (TPPs) suggest the question: Do the current indicators of progress and performance used by TPPs predict effectiveness of their graduates when they become teachers? In this study, the indicators of progress and performance used by one program are examined for their ability to predict…

A systematic review of the measurement properties of the European Organisation for Research and Treatment of Cancer In-patient Satisfaction with Care Questionnaire, the EORTC IN-PATSAT32.

PubMed

Neijenhuijs, Koen I; Jansen, Femke; Aaronson, Neil K; Brédart, Anne; Groenvold, Mogens; Holzner, Bernhard; Terwee, Caroline B; Cuijpers, Pim; Verdonck-de Leeuw, Irma M

2018-05-07

The EORTC IN-PATSAT32 is a patient-reported outcome measure (PROM) to assess cancer patients' satisfaction with in-patient health care. The aim of this study was to investigate whether the initial good measurement properties of the IN-PATSAT32 are confirmed in new studies. Within the scope of a larger systematic review study (Prospero ID 42017057237), a systematic search was performed of Embase, Medline, PsycINFO, and Web of Science for studies that investigated measurement properties of the IN-PATSAT32 up to July 2017. Study quality was assessed, data were extracted, and synthesized according to the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology. Nine studies were included in this review. The evidence on reliability and construct validity were rated as sufficient and of the quality of the evidence as moderate. The evidence on structural validity was rated as insufficient and of low quality. The evidence on internal consistency was indeterminate. Measurement error, responsiveness, criterion validity, and cross-cultural validity were not reported in the included studies. Measurement error could be calculated for two studies and was judged indeterminate. In summary, the IN-PATSAT32 performs as expected with respect to reliability and construct validity. No firm conclusions can be made yet whether the IN-PATSAT32 also performs as well with respect to structural validity and internal consistency. Further research on these measurement properties of the PROM is therefore needed as well as on measurement error, responsiveness, criterion validity, and cross-cultural validity. For future studies, it is recommended to take the COSMIN methodology into account.

The Development and Validation of the Age-Based Rejection Sensitivity Questionnaire

ERIC Educational Resources Information Center

Kang, Sonia K.; Chasteen, Alison L.

2009-01-01

Purpose: There is much evidence suggesting that older adults are often negatively affected by aging stereotypes; however, no method to identify individual differences in vulnerability to these effects has yet been developed. The purpose of this study was to develop a reliable and valid questionnaire to measure individual differences in the…

How to Measure Critical Health Competences: Development and Validation of the Critical Health Competence Test (CHC Test)

ERIC Educational Resources Information Center

Steckelberg, Anke; Hulfenhaus, Christian; Kasper, Jurgen; Rost, Jurgen; Muhlhauser, Ingrid

2009-01-01

Consumers' autonomy regarding health increasingly requires competences to critically appraise health information. Critical health literacy refers to the concept of evidence-based medicine. Instruments to measure these competences in curriculum evaluation and surveys are lacking. We aimed to develop and validate an instrument to measure critical…

Validity of Adult Retrospective Reports of Adverse Childhood Experiences: Review of the Evidence

ERIC Educational Resources Information Center

Hardt, Jochen; Rutter, Michael

2004-01-01

Background: Influential studies have cast doubt on the validity of retrospective reports by adults of their own adverse experiences in childhood. Accordingly, many researchers view retrospective reports with scepticism. Method: A computer-based search, supplemented by hand searches, was used to identify studies reported between 1980 and 2001 in…

The development and validation of the Incivility from Customers Scale.

PubMed

Wilson, Nicole L; Holmvall, Camilla M

2013-07-01

Scant research has examined customers as sources of workplace incivility, despite evidence suggesting that mistreatment is more common from organizational outsiders, including customers, than from organizational members (Grandey, Kern, & Frone, 2007; Schat & Kelloway, 2005). As an important step in extending the literature on customer incivility, we conducted two studies to develop and validate a measure of this construct. Study 1 used focus groups of retail and restaurant employees (n = 30) to elicit a list of uncivil customer behaviors, based on which we wrote initial scale items. Study 2 used a correlational survey design (n = 439) to pare down the number of scale items to 10 and to garner reliability and validity evidence for the scale. Exploratory and confirmatory factor analyses show that the scale is unidimensional and distinguishable from measures of the related, but distinct, constructs of interpersonal justice and psychological aggression from customers. Reliability analyses show that the scale is internally consistent. Significant correlations between the scale and individuals' job satisfaction, turnover intentions, and general and job-specific psychological strain provide evidence of criterion-related validity. Hierarchical regression analyses show that the scale significantly predicts three of four organizational and personal strain outcomes over and above a workplace incivility measure adapted for customer incivility, providing some evidence of incremental validity. Limitations and future research directions are discussed. PsycINFO Database Record (c) 2013 APA, all rights reserved.

Validation of surrogate endpoints in advanced solid tumors: systematic review of statistical methods, results, and implications for policy makers.

PubMed

Ciani, Oriana; Davis, Sarah; Tappenden, Paul; Garside, Ruth; Stein, Ken; Cantrell, Anna; Saad, Everardo D; Buyse, Marc; Taylor, Rod S

2014-07-01

Licensing of, and coverage decisions on, new therapies should rely on evidence from patient-relevant endpoints such as overall survival (OS). Nevertheless, evidence from surrogate endpoints may also be useful, as it may not only expedite the regulatory approval of new therapies but also inform coverage decisions. It is, therefore, essential that candidate surrogate endpoints be properly validated. However, there is no consensus on statistical methods for such validation and on how the evidence thus derived should be applied by policy makers. We review current statistical approaches to surrogate-endpoint validation based on meta-analysis in various advanced-tumor settings. We assessed the suitability of two surrogates (progression-free survival [PFS] and time-to-progression [TTP]) using three current validation frameworks: Elston and Taylor's framework, the German Institute of Quality and Efficiency in Health Care's (IQWiG) framework and the Biomarker-Surrogacy Evaluation Schema (BSES3). A wide variety of statistical methods have been used to assess surrogacy. The strength of the association between the two surrogates and OS was generally low. The level of evidence (observation-level versus treatment-level) available varied considerably by cancer type, by evaluation tools and was not always consistent even within one specific cancer type. Not in all solid tumors the treatment-level association between PFS or TTP and OS has been investigated. According to IQWiG's framework, only PFS achieved acceptable evidence of surrogacy in metastatic colorectal and ovarian cancer treated with cytotoxic agents. Our study emphasizes the challenges of surrogate-endpoint validation and the importance of building consensus on the development of evaluation frameworks.

Self-esteem among nursing assistants: reliability and validity of the Rosenberg Self-Esteem Scale.

PubMed

McMullen, Tara; Resnick, Barbara

2013-01-01

To establish the reliability and validity of the Rosenberg Self-Esteem Scale (RSES) when used with nursing assistants (NAs). Testing the RSES used baseline data from a randomized controlled trial testing the Res-Care Intervention. Female NAs were recruited from nursing homes (n = 508). Validity testing for the positive and negative subscales of the RSES was based on confirmatory factor analysis (CFA) using structural equation modeling and Rasch analysis. Estimates of reliability were based on Rasch analysis and the person separation index. Evidence supports the reliability and validity of the RSES in NAs although we recommend minor revisions to the measure for subsequent use. Establishing reliable and valid measures of self-esteem in NAs will facilitate testing of interventions to strengthen workplace self-esteem, job satisfaction, and retention.

Validity in work-based assessment: expanding our horizons.

PubMed

Govaerts, Marjan; van der Vleuten, Cees P M

2013-12-01

Although work-based assessments (WBA) may come closest to assessing habitual performance, their use for summative purposes is not undisputed. Most criticism of WBA stems from approaches to validity consistent with the quantitative psychometric framework. However, there is increasing research evidence that indicates that the assumptions underlying the predictive, deterministic framework of psychometrics may no longer hold. In this discussion paper we argue that meaningfulness and appropriateness of current validity evidence can be called into question and that we need alternative strategies to assessment and validity inquiry that build on current theories of learning and performance in complex and dynamic workplace settings. Drawing from research in various professional fields we outline key issues within the mechanisms of learning, competence and performance in the context of complex social environments and illustrate their relevance to WBA. In reviewing recent socio-cultural learning theory and research on performance and performance interpretations in work settings, we demonstrate that learning, competence (as inferred from performance) as well as performance interpretations are to be seen as inherently contextualised, and can only be under-stood 'in situ'. Assessment in the context of work settings may, therefore, be more usefully viewed as a socially situated interpretive act. We propose constructivist-interpretivist approaches towards WBA in order to capture and understand contextualised learning and performance in work settings. Theoretical assumptions underlying interpretivist assessment approaches call for a validity theory that provides the theoretical framework and conceptual tools to guide the validation process in the qualitative assessment inquiry. Basic principles of rigour specific to qualitative research have been established, and they can and should be used to determine validity in interpretivist assessment approaches. If used properly, these strategies generate trustworthy evidence that is needed to develop the validity argument in WBA, allowing for in-depth and meaningful information about professional competence. © 2013 John Wiley & Sons Ltd.

Study design and "evidence" in patient-oriented research.

PubMed

Concato, John

2013-06-01

Individual studies in patient-oriented research, whether described as "comparative effectiveness" or using other terms, are based on underlying methodological designs. A simple taxonomy of study designs includes randomized controlled trials on the one hand, and observational studies (such as case series, cohort studies, and case-control studies) on the other. A rigid hierarchy of these design types is a fairly recent phenomenon, promoted as a tenet of "evidence-based medicine," with randomized controlled trials receiving gold-standard status in terms of producing valid results. Although randomized trials have many strengths, and contribute substantially to the evidence base in clinical care, making presumptions about the quality of a study based solely on category of research design is unscientific. Both the limitations of randomized trials as well as the strengths of observational studies tend to be overlooked when a priori assumptions are made. This essay presents an argument in support of a more balanced approach to evaluating evidence, and discusses representative examples from the general medical as well as pulmonary and critical care literature. The simultaneous consideration of validity (whether results are correct "internally") and generalizability (how well results apply to "external" populations) is warranted in assessing whether a study's results are accurate for patients likely to receive the intervention-examining the intersection of clinical and methodological issues in what can be called a medicine-based evidence approach. Examination of cause-effect associations in patient-oriented research should recognize both the strengths and limitations of randomized trials as well as observational studies.

Evidence-based algorithm for heparin dosing before cardiopulmonary bypass. Part 1: Development of the algorithm.

PubMed

McKinney, Mark C; Riley, Jeffrey B

2007-12-01

The incidence of heparin resistance during adult cardiac surgery with cardiopulmonary bypass has been reported at 15%-20%. The consistent use of a clinical decision-making algorithm may increase the consistency of patient care and likely reduce the total required heparin dose and other problems associated with heparin dosing. After a directed survey of practicing perfusionists regarding treatment of heparin resistance and a literature search for high-level evidence regarding the diagnosis and treatment of heparin resistance, an evidence-based decision-making algorithm was constructed. The face validity of the algorithm decisive steps and logic was confirmed by a second survey of practicing perfusionists. The algorithm begins with review of the patient history to identify predictors for heparin resistance. The definition for heparin resistance contained in the algorithm is an activated clotting time < 450 seconds with > 450 IU/kg heparin loading dose. Based on the literature, the treatment for heparin resistance used in the algorithm is anti-thrombin III supplement. The algorithm seems to be valid and is supported by high-level evidence and clinician opinion. The next step is a human randomized clinical trial to test the clinical procedure guideline algorithm vs. current standard clinical practice.

Diet Assessment Methods in the Nurses' Health Studies and Contribution to Evidence-Based Nutritional Policies and Guidelines

PubMed Central

Satija, Ambika; Rimm, Eric B.; Spiegelman, Donna; Sampson, Laura; Rosner, Bernard; Camargo, Carlos A.; Stampfer, Meir; Willett, Walter C.

2016-01-01

Objectives. To review the contribution of the Nurses’ Health Studies (NHSs) to diet assessment methods and evidence-based nutritional policies and guidelines. Methods. We performed a narrative review of the publications of the NHS and NHS II between 1976 and 2016. Results. Through periodic assessment of diet by validated dietary questionnaires over 40 years, the NHSs have identified dietary determinants of diseases such as breast and other cancers; obesity; type 2 diabetes; cardiovascular, respiratory, and eye diseases; and neurodegenerative and mental health disorders. Nutritional biomarkers were assessed using blood, urine, and toenail samples. Robust findings, from the NHSs, together with evidence from other large cohorts and randomized dietary intervention trials, have contributed to the evidence base for developing dietary guidelines and nutritional policies to reduce intakes of trans fat, saturated fat, sugar-sweetened beverages, red and processed meats, and refined carbohydrates while promoting higher intake of healthy fats and carbohydrates and overall healthful dietary patterns. Conclusions. The long-term, periodically collected dietary data in the NHSs, with documented reliability and validity, have contributed extensively to our understanding of the dietary determinants of various diseases, informing dietary guidelines and shaping nutritional policy. PMID:27459459

Nutrition Recommendations in Elderly and Aging.

PubMed

Barkoukis, Hope

2016-11-01

Maintaining optimal health and well-being in the older adult requires understanding of how physiologic changes influence nutritional status, familiarity with the available validated tools to assess status, identification of factors predisposing older adults to malnutrition, and evidence-based practice regarding the nutritional needs of this age group. Evidence-based guidance on these core practice components is provided to the clinician in this article. Copyright © 2016 Elsevier Inc. All rights reserved.

Fighting Crime by Fighting Misconceptions and Blind Spots in Policy Theories: An Evidence-Based Evaluation of Interventions and Assumed Causal Mechanisms

ERIC Educational Resources Information Center

van Noije, Lonneke; Wittebrood, Karin

2010-01-01

How effective are policy interventions to fight crime and how valid is the policy theory that underlies them? This is the twofold research question addressed in this article, which presents an evidence-based evaluation of Dutch social safety policy. By bridging the gap between actual effects and assumed effects, this study seeks to make fuller use…

Ethics and Evidence-Based Medicine: Is There a Conflict?

PubMed Central

Loewy, Erich H.

2007-01-01

This article addresses the advantages, disadvantages, and traps to which evidence-based medicine (EBM) may lead and suggests that, to be ethically valid, EBM must be aimed at the patient's best interests and not at the financial interests of others. While financial considerations are by no means trivial, it is hypocritical – if not dangerous – to hide them behind words like “evidence” or “quality.” PMID:18092036

An evidence-based systematic review of elderberry and elderflower (Sambucus nigra) by the Natural Standard Research Collaboration.

PubMed

Ulbricht, Catherine; Basch, Ethan; Cheung, Lisa; Goldberg, Harley; Hammerness, Paul; Isaac, Richard; Khalsa, Karta Purkh Singh; Romm, Aviva; Rychlik, Idalia; Varghese, Minney; Weissner, Wendy; Windsor, Regina C; Wortley, Jayme

2014-03-01

An evidence-based systematic review of elderberry and elderflower (Sambucus nigra) by the Natural Standard Research Collaboration consolidates the safety and efficacy data available in the scientific literature using a validated, reproducible grading rationale. This article includes written and statistical analysis of clinical trials, plus a compilation of expert opinion, folkloric precedent, history, pharmacology, kinetics/dynamics, interactions, adverse effects, toxicology, and dosing.

Validity and reliability of instruments aimed at measuring Evidence-Based Practice in Physical Therapy: a systematic review of the literature.

PubMed

Fernández-Domínguez, Juan Carlos; Sesé-Abad, Albert; Morales-Asencio, Jose Miguel; Oliva-Pascual-Vaca, Angel; Salinas-Bueno, Iosune; de Pedro-Gómez, Joan Ernest

2014-12-01

Our goal is to compile and analyse the characteristics - especially validity and reliability - of all the existing international tools that have been used to measure evidence-based clinical practice in physiotherapy. A systematic review conducted with data from exclusively quantitative-type studies synthesized in narrative format. An in-depth search of the literature was conducted in two phases: initial, structured, electronic search of databases and also journals with summarized evidence; followed by a residual-directed search in the bibliographical references of the main articles found in the primary search procedure. The studies included were assigned to members of the research team who acted as peer reviewers. Relevant information was extracted from each of the selected articles using a template that included the general characteristics of the instrument as well as an analysis of the quality of the validation processes carried out, by following the criteria of Terwee. Twenty-four instruments were found to comply with the review screening criteria; however, in all cases, they were found to be limited as regards the 'constructs' included. Besides, they can all be seen to be lacking as regards comprehensiveness associated to the validation process of the psychometric tests used. It seems that what constitutes a rigorously developed assessment instrument for EBP in physical therapy continues to be a challenge. © 2014 John Wiley & Sons, Ltd.

[Is evidence-based assessment fact or fiction? A bibliometric analysis of three German journals].

PubMed

Petermann, Franz; Schüssler, Gerhard; Glaesmer, Heide

2008-01-01

Despite the ongoing process for the development and dissemination of empirically supported treatments, little attention has been paid to the development of evidence-based diagnostics. The article aims at evaluating diagnostic procedures and instruments in current clinical research in terms of evidence-based assessment. Volumes 2006 and 2007 of three German psychological journals "Psychotherapeut," "Psychotherapie, Psychosomatik und Medizinische Psychologie," and "Zeitschrift für Psychiatrie, Psychologie und Psychotherapie" were screened for empirical reports and articles dealing with diagnostic issues. 93 articles were identified and evaluated. Most studies used psychometrically valid and established instruments for assessment. However, diagnostic interviews were relatively scarce, as were multimodal assessments. Measures used for outcome evaluation often lacked evidence of sensitivity to change. Clinical assessment to date does not meet criteria for evidence-based diagnostics. Implications for research and guideline development are discussed.

Measurement properties of patient-reported outcome measures (PROMs) used in adult patients with chronic kidney disease: A systematic review

PubMed Central

Kyte, Derek; Cockwell, Paul; Marshall, Tom; Gheorghe, Adrian; Keeley, Thomas; Slade, Anita; Calvert, Melanie

2017-01-01

Background Patient-reported outcome measures (PROMs) can provide valuable information which may assist with the care of patients with chronic kidney disease (CKD). However, given the large number of measures available, it is unclear which PROMs are suitable for use in research or clinical practice. To address this we comprehensively evaluated studies that assessed the measurement properties of PROMs in adults with CKD. Methods Four databases were searched; reference list and citation searching of included studies was also conducted. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist was used to appraise the methodological quality of the included studies and to inform a best evidence synthesis for each PROM. Results The search strategy retrieved 3,702 titles/abstracts. After 288 duplicates were removed, 3,414 abstracts were screened and 71 full-text articles were retrieved for further review. Of these, 24 full-text articles were excluded as they did not meet the eligibility criteria. Following reference list and citation searching, 19 articles were retrieved bringing the total number of papers included in the final analysis to 66. There was strong evidence supporting internal consistency and moderate evidence supporting construct validity for the Kidney Disease Quality of Life-36 (KDQOL-36) in pre-dialysis patients. In the dialysis population, the KDQOL-Short Form (KDQOL-SF) had strong evidence for internal consistency and structural validity and moderate evidence for test-retest reliability and construct validity while the KDQOL-36 had moderate evidence of internal consistency, test-retest reliability and construct validity. The End Stage Renal Disease-Symptom Checklist Transplantation Module (ESRD-SCLTM) demonstrated strong evidence for internal consistency and moderate evidence for test-retest reliability, structural and construct validity in renal transplant recipients. Conclusions We suggest considering the KDQOL-36 for use in pre-dialysis patients; the KDQOL-SF or KDQOL-36 for dialysis patients and the ESRD-SCLTM for use in transplant recipients. However, further research is required to evaluate the measurement error, structural validity, responsiveness and patient acceptability of PROMs used in CKD. PMID:28636678

Mental Health Smartphone Apps: Review and Evidence-Based Recommendations for Future Developments.

PubMed

Bakker, David; Kazantzis, Nikolaos; Rickwood, Debra; Rickard, Nikki

2016-03-01

The number of mental health apps (MHapps) developed and now available to smartphone users has increased in recent years. MHapps and other technology-based solutions have the potential to play an important part in the future of mental health care; however, there is no single guide for the development of evidence-based MHapps. Many currently available MHapps lack features that would greatly improve their functionality, or include features that are not optimized. Furthermore, MHapp developers rarely conduct or publish trial-based experimental validation of their apps. Indeed, a previous systematic review revealed a complete lack of trial-based evidence for many of the hundreds of MHapps available. To guide future MHapp development, a set of clear, practical, evidence-based recommendations is presented for MHapp developers to create better, more rigorous apps. A literature review was conducted, scrutinizing research across diverse fields, including mental health interventions, preventative health, mobile health, and mobile app design. Sixteen recommendations were formulated. Evidence for each recommendation is discussed, and guidance on how these recommendations might be integrated into the overall design of an MHapp is offered. Each recommendation is rated on the basis of the strength of associated evidence. It is important to design an MHapp using a behavioral plan and interactive framework that encourages the user to engage with the app; thus, it may not be possible to incorporate all 16 recommendations into a single MHapp. Randomized controlled trials are required to validate future MHapps and the principles upon which they are designed, and to further investigate the recommendations presented in this review. Effective MHapps are required to help prevent mental health problems and to ease the burden on health systems.

Mental Health Smartphone Apps: Review and Evidence-Based Recommendations for Future Developments

PubMed Central

Kazantzis, Nikolaos; Rickwood, Debra; Rickard, Nikki

2016-01-01

Background The number of mental health apps (MHapps) developed and now available to smartphone users has increased in recent years. MHapps and other technology-based solutions have the potential to play an important part in the future of mental health care; however, there is no single guide for the development of evidence-based MHapps. Many currently available MHapps lack features that would greatly improve their functionality, or include features that are not optimized. Furthermore, MHapp developers rarely conduct or publish trial-based experimental validation of their apps. Indeed, a previous systematic review revealed a complete lack of trial-based evidence for many of the hundreds of MHapps available. Objective To guide future MHapp development, a set of clear, practical, evidence-based recommendations is presented for MHapp developers to create better, more rigorous apps. Methods A literature review was conducted, scrutinizing research across diverse fields, including mental health interventions, preventative health, mobile health, and mobile app design. Results Sixteen recommendations were formulated. Evidence for each recommendation is discussed, and guidance on how these recommendations might be integrated into the overall design of an MHapp is offered. Each recommendation is rated on the basis of the strength of associated evidence. It is important to design an MHapp using a behavioral plan and interactive framework that encourages the user to engage with the app; thus, it may not be possible to incorporate all 16 recommendations into a single MHapp. Conclusions Randomized controlled trials are required to validate future MHapps and the principles upon which they are designed, and to further investigate the recommendations presented in this review. Effective MHapps are required to help prevent mental health problems and to ease the burden on health systems. PMID:26932350

External validity of a generic safety climate scale for lone workers across different industries and companies.

PubMed

Lee, Jin; Huang, Yueng-hsiang; Robertson, Michelle M; Murphy, Lauren A; Garabet, Angela; Chang, Wen-Ruey

2014-02-01

The goal of this study was to examine the external validity of a 12-item generic safety climate scale for lone workers in order to evaluate the appropriateness of generalized use of the scale in the measurement of safety climate across various lone work settings. External validity evidence was established by investigating the measurement equivalence (ME) across different industries and companies. Confirmatory factor analysis (CFA)-based and item response theory (IRT)-based perspectives were adopted to examine the ME of the generic safety climate scale for lone workers across 11 companies from the trucking, electrical utility, and cable television industries. Fairly strong evidence of ME was observed for both organization- and group-level generic safety climate sub-scales. Although significant invariance was observed in the item intercepts across the different lone work settings, absolute model fit indices remained satisfactory in the most robust step of CFA-based ME testing. IRT-based ME testing identified only one differentially functioning item from the organization-level generic safety climate sub-scale, but its impact was minimal and strong ME was supported. The generic safety climate scale for lone workers reported good external validity and supported the presence of a common feature of safety climate among lone workers. The scale can be used as an effective safety evaluation tool in various lone work situations. Copyright © 2013 Elsevier Ltd. All rights reserved.

Development and Validation of the Personal Strengths Inventory Using Exploratory and Confirmatory Factor Analyses

ERIC Educational Resources Information Center

Liau, Albert Kienfie; Chow, Daryl; Tan, Teck Kiang; Senf, Konrad

2011-01-01

The purpose of this study was to establish the reliability and validity of the scores on a brief strengths-based assessment, the 22-item Personal Strengths Inventory (PSI). In Study 1, findings from exploratory factor analysis of 410 adolescents provided evidence for a five-factor solution--social competence (four items), emotional awareness (five…

Investigating the Validity and Reliability of the Vanderbilt Assessment of Leadership in Education

ERIC Educational Resources Information Center

Porter, Andrew C.; Polikoff, Morgan S.; Goldring, Ellen B.; Murphy, Joseph; Elliott, Stephen N.; May, Henry

2010-01-01

The Vanderbilt Assessment of Leadership in Education (VAL-ED) is a multirater assessment of principals' learning-centered leadership. The instrument was developed based on the Standards for Educational and Psychological Testing. In this article, we report on the validity and reliability evidence for the VAL-ED accumulated in a national field…

A Framework for Evidence-Based Licensure of Adaptive Autonomous Systems

DTIC Science & Technology

2016-03-01

insights gleaned to DoD. The autonomy community has identified significant challenges associated with test, evaluation verification and validation of...licensure as a test, evaluation, verification , and validation (TEVV) framework that can address these challenges. IDA found that traditional...language requirements to testable (preferably machine testable) specifications • Design of architectures that treat development and verification of

Evidences of Validity of the Brazilian Scale of Learner's Attitude towards Distance Education Programs

ERIC Educational Resources Information Center

Coelho, Francisco Antonio, Jr.; Cortat, Mariane; Flores, Clarissa Leite; Santos, Flávio Augusto Mendes; Alves, Gleidilson Costa; Faiad, Cristiane; Ramos, Wilsa Maria; Rodrigues da Silva, Alan

2018-01-01

Online learning is one of the fastest growing trends in educational uses of technology. In this study, an instrument to measure the social attitudes of the Brazilian students based on distance education was developed and validated. The study population consisted of public administration undergraduate students that has been providing by distance…

Interpreting Variance Components as Evidence for Reliability and Validity.

ERIC Educational Resources Information Center

Kane, Michael T.

The reliability and validity of measurement is analyzed by a sampling model based on generalizability theory. A model for the relationship between a measurement procedure and an attribute is developed from an analysis of how measurements are used and interpreted in science. The model provides a basis for analyzing the concept of an error of…

Exploring Person Fit with an Approach Based on Multilevel Logistic Regression

ERIC Educational Resources Information Center

Walker, A. Adrienne; Engelhard, George, Jr.

2015-01-01

The idea that test scores may not be valid representations of what students know, can do, and should learn next is well known. Person fit provides an important aspect of validity evidence. Person fit analyses at the individual student level are not typically conducted and person fit information is not communicated to educational stakeholders. In…

Screening Systems and Decision Making at the Preschool Level: Application of a Comprehensive Validity Framework

ERIC Educational Resources Information Center

Kettler, Ryan J.; Feeney-Kettler, Kelly A.

2011-01-01

Universal screening is designed to be an efficient method for identifying preschool students with mental health problems, but prior to use, screening systems must be evaluated to determine their appropriateness within a specific setting. In this article, an evidence-based validity framework is applied to four screening systems for identifying…

Improving the Residency Admissions Process by Integrating a Professionalism Assessment: A Validity and Feasibility Study

ERIC Educational Resources Information Center

Bajwa, Nadia M.; Yudkowsky, Rachel; Belli, Dominique; Vu, Nu Viet; Park, Yoon Soo

2017-01-01

The purpose of this study was to provide validity and feasibility evidence in measuring professionalism using the Professionalism Mini-Evaluation Exercise (P-MEX) scores as part of a residency admissions process. In 2012 and 2013, three standardized-patient-based P-MEX encounters were administered to applicants invited for an interview at the…

Teaching and physics education research: bridging the gap.

PubMed

Fraser, James M; Timan, Anneke L; Miller, Kelly; Dowd, Jason E; Tucker, Laura; Mazur, Eric

2014-03-01

Physics faculty, experts in evidence-based research, often rely on anecdotal experience to guide their teaching practices. Adoption of research-based instructional strategies is surprisingly low, despite the large body of physics education research (PER) and strong dissemination effort of PER researchers and innovators. Evidence-based PER has validated specific non-traditional teaching practices, but many faculty raise valuable concerns toward their applicability. We address these concerns and identify future studies required to overcome the gap between research and practice.

Systematic Review of Measurement Property Evidence for 8 Financial Management Instruments in Populations With Acquired Cognitive Impairment.

PubMed

Engel, Lisa; Chui, Adora; Beaton, Dorcas E; Green, Robin E; Dawson, Deirdre R

2018-03-07

To critically appraise the measurement property evidence (ie, psychometric) for 8 observation-based financial management assessment instruments. Seven databases were searched in May 2015. Two reviewers used an independent decision-agreement process to select studies of measurement property evidence relevant to populations with adulthood acquired cognitive impairment, appraise the quality of the evidence, and extract data. Twenty-one articles were selected. This review used the COnsensus-based Standards for the selection of health Measurement Instruments review guidelines and 4-point tool to appraise evidence. After appraising the methodologic quality, the adequacy of results and volume of evidence per instrument were synthesized. Measurement property evidence with high risk of bias was excluded from the synthesis. The volume of measurement property evidence per instrument is low; most instruments had 1 to 3 included studies. Many included studies had poor methodologic quality per measurement property evidence area examined. Six of the 8 instruments reviewed had supporting construct validity/hypothesis-testing evidence of fair methodologic quality. There is a dearth of acceptable quality content validity, reliability, and responsiveness evidence for all 8 instruments. Rehabilitation practitioners assess financial management functions in adults with acquired cognitive impairments. However, there is limited published evidence to support using any of the reviewed instruments. Practitioners should exercise caution when interpreting the results of these instruments. This review highlights the importance of appraising the quality of measurement property evidence before examining the adequacy of the results and synthesizing the evidence. Copyright © 2018 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.

Reliability and Validity of the Alberta Context Tool (ACT) with Professional Nurses: Findings from a Multi-Study Analysis

PubMed Central

Squires, Janet E.; Hayduk, Leslie; Hutchinson, Alison M.; Mallick, Ranjeeta; Norton, Peter G.; Cummings, Greta G.; Estabrooks, Carole A.

2015-01-01

Although organizational context is central to evidence-based practice, underdeveloped measurement hindersitsassessment. The Alberta Context Tool, comprised of 59 items that tap10 modifiable contextual concepts, was developed to address this gap. The purpose of this study to examine the reliability and validity of scores obtained when the Alberta Context Tool is completed by professional nurses across different healthcare settings. Five separate studies (N = 2361 nurses across different care settings) comprised the study sample. Reliability and validity were assessed. Cronbach’s alpha exceeded 0.70 for9/10 Alberta Context Tool concepts. Item-total correlations exceeded acceptable standards for 56/59items. Confirmatory Factor Analysescoordinated acceptably with the Alberta Context Tool’s proposed latent structure. The mean values for each Alberta Context Tool concept increased from low to high levels of research utilization(as hypothesized) further supporting its validity. This study provides robust evidence forreliability and validity of scores obtained with the Alberta Context Tool when administered to professional nurses. PMID:26098857

Evidence based vaccinology.

PubMed

Nalin, David R

2002-02-22

Evidence based vaccinology (EBV) is the identification and use of the best evidence in making and implementing decisions during all of the stages of the life of a vaccine, including pre-licensure vaccine development and post-licensure manufacture and research, and utilization of the vaccine for disease control. Vaccines, unlike most pharmaceuticals, are in a continuous process of development both before and after licensure. Changes in biologics manufacturing technology and changes that vaccines induce in population and disease biology lead to periodic review of regimens (and sometimes dosage) based on changing immunologic data or public perceptions relevant to vaccine safety and effectiveness. EBV includes the use of evidence based medicine (EBM) both in clinical trials and in national disease containment programs. The rationale for EBV is that the highest evidentiary standards are required to maintain a rigorous scientific basis of vaccine quality control in manufacture and to ensure valid determination of vaccine efficacy, field effectiveness and safety profiles (including post-licensure safety monitoring), cost-benefit analyses, and risk:benefit ratios. EBV is increasingly based on statistically validated, clearly defined laboratory, manufacturing, clinical and epidemiological research methods and procedures, codified as good laboratory practices (GLP), good manufacturing practices (GMP), good clinical research practices (GCRP) and in clinical and public health practice (good vaccination practices, GVP). Implementation demands many data-driven decisions made by a spectrum of specialists pre- and post-licensure, and is essential to maintaining public confidence in vaccines.

Educational testing validity and reliability in pharmacy and medical education literature.

PubMed

Hoover, Matthew J; Jung, Rose; Jacobs, David M; Peeters, Michael J

2013-12-16

To evaluate and compare the reliability and validity of educational testing reported in pharmacy education journals to medical education literature. Descriptions of validity evidence sources (content, construct, criterion, and reliability) were extracted from articles that reported educational testing of learners' knowledge, skills, and/or abilities. Using educational testing, the findings of 108 pharmacy education articles were compared to the findings of 198 medical education articles. For pharmacy educational testing, 14 articles (13%) reported more than 1 validity evidence source while 83 articles (77%) reported 1 validity evidence source and 11 articles (10%) did not have evidence. Among validity evidence sources, content validity was reported most frequently. Compared with pharmacy education literature, more medical education articles reported both validity and reliability (59%; p<0.001). While there were more scholarship of teaching and learning (SoTL) articles in pharmacy education compared to medical education, validity, and reliability reporting were limited in the pharmacy education literature.

Validation of the Narrowing Beam Walking Test in Lower Limb Prosthesis Users.

PubMed

Sawers, Andrew; Hafner, Brian

2018-04-11

To evaluate the content, construct, and discriminant validity of the Narrowing Beam Walking Test (NBWT), a performance-based balance test for lower limb prosthesis users. Cross-sectional study. Research laboratory and prosthetics clinic. Unilateral transtibial and transfemoral prosthesis users (N=40). Not applicable. Content validity was examined by quantifying the percentage of participants receiving maximum or minimum scores (ie, ceiling and floor effects). Convergent construct validity was examined using correlations between participants' NBWT scores and scores or times on existing clinical balance tests regularly administered to lower limb prosthesis users. Known-groups construct validity was examined by comparing NBWT scores between groups of participants with different fall histories, amputation levels, amputation etiologies, and functional levels. Discriminant validity was evaluated by analyzing the area under each test's receiver operating characteristic (ROC) curve. No minimum or maximum scores were recorded on the NBWT. NBWT scores demonstrated strong correlations (ρ=.70‒.85) with scores/times on performance-based balance tests (timed Up and Go test, Four Square Step Test, and Berg Balance Scale) and a moderate correlation (ρ=.49) with the self-report Activities-specific Balance Confidence scale. NBWT performance was significantly lower among participants with a history of falls (P=.003), transfemoral amputation (P=.011), and a lower mobility level (P<.001). The NBWT also had the largest area under the ROC curve (.81) and was the only test to exhibit an area that was statistically significantly >.50 (ie, chance). The results provide strong evidence of content, construct, and discriminant validity for the NBWT as a performance-based test of balance ability. The evidence supports its use to assess balance impairments and fall risk in unilateral transtibial and transfemoral prosthesis users. Copyright © 2018 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.

Hypersensitivity to sound in tinnitus patients: an analysis of a construct based on questionnaire and audiological data.

PubMed

Bläsing, Lena; Goebel, Gerhard; Flötzinger, Uta; Berthold, Anke; Kröner-Herwig, Birgit

2010-07-01

The purpose of this study was to analyse the Questionnaire on Hypersensitivity to Sound (GUF; Nelting & Finlayson, 2004 ) and to improve its validity based on the analysis of intercorrelations (single item level) with other methods of assessing hyperacusis (uncomfortable loudness level, individual loudness function, self-rated severity of hyperacusis). Subjects consisted of 91 inpatients with tinnitus and hyperacusis. The GUF showed a good reliability (alpha = .92). The factorial structure of the questionnaire reported by Nelting et al (2002) was not completely supported by the evidence in this study. The total score and the single items showed small to moderate correlations with the other modes of measuring hyperacusis. Evidence for convergent and discriminant validity were found, but overall the results corroborate the conceptual heterogeneity of the construct hyperacusis and its dependency on the assessment method. Four items of the GUF with particularly low correlations were excluded from the questionnaire. The revised GUF total score showed slightly but not statistically significant higher convergent and discriminant validity.

Validation of the Mobile Information Software Evaluation Tool (MISET) With Nursing Students.

PubMed

Secco, M Loretta; Furlong, Karen E; Doyle, Glynda; Bailey, Judy

2016-07-01

This study evaluated the Mobile Information Software Evaluation Tool (MISET) with a sample of Canadian undergraduate nursing students (N = 240). Psychometric analyses determined how well the MISET assessed the extent that nursing students find mobile device-based information resources useful and supportive of learning in the clinical and classroom settings. The MISET has a valid three-factor structure with high explained variance (74.7%). Internal consistency reliabilities were high for the MISET total (.90) and three subscales: Usefulness/Helpfulness, Information Literacy Support, and Use of Evidence-Based Sources (.87 to .94). Construct validity evidence included significantly higher mean total MISET, Helpfulness/Usefulness, and Information Literacy Support scores for senior students and those with higher computer competence. The MISET is a promising tool to evaluate mobile information technologies and information literacy support; however, longitudinal assessment of changes in scores over time would determine scale sensitivity and responsiveness. [J Nurs Educ. 2016;55(7):385-390.]. Copyright 2016, SLACK Incorporated.

Validity and reliability of the Diagnostic Adaptive Behaviour Scale.

PubMed

Tassé, M J; Schalock, R L; Balboni, G; Spreat, S; Navas, P

2016-01-01

The Diagnostic Adaptive Behaviour Scale (DABS) is a new standardised adaptive behaviour measure that provides information for evaluating limitations in adaptive behaviour for the purpose of determining a diagnosis of intellectual disability. This article presents validity evidence and reliability data for the DABS. Validity evidence was based on comparing DABS scores with scores obtained on the Vineland Adaptive Behaviour Scale, second edition. The stability of the test scores was measured using a test and retest, and inter-rater reliability was assessed by computing the inter-respondent concordance. The DABS convergent validity coefficients ranged from 0.70 to 0.84, while the test-retest reliability coefficients ranged from 0.78 to 0.95, and the inter-rater concordance as measured by intraclass correlation coefficients ranged from 0.61 to 0.87. All obtained validity and reliability indicators were strong and comparable with the validity and reliability coefficients of the most commonly used adaptive behaviour instruments. These results and the advantages of the DABS for clinician and researcher use are discussed. © 2015 MENCAP and International Association of the Scientific Study of Intellectual and Developmental Disabilities and John Wiley & Sons Ltd.

Developing, Testing, and Using Theoretical Models for Promoting Quality in Education

ERIC Educational Resources Information Center

Creemers, Bert; Kyriakides, Leonidas

2015-01-01

This paper argues that the dynamic model of educational effectiveness can be used to establish stronger links between educational effectiveness research (EER) and school improvement. It provides research evidence to support the validity of the model. Thus, the importance of using the dynamic model to establish an evidence-based and theory-driven…

A Preliminary Investigation of the Empirical Validity of Study Quality Appraisal

ERIC Educational Resources Information Center

Cook, Bryan G.; Dupuis, Danielle N.; Jitendra, Asha K.

2017-01-01

When classifying the evidence base of practices, special education scholars typically appraise study quality to identify and exclude from consideration in their reviews unacceptable-quality studies that are likely biased and might bias review findings if included. However, study quality appraisals used in the process of identifying evidence-based…

Reviving Cochrane's contribution to evidence-based medicine: bridging the gap between evidence of efficacy and evidence of effectiveness and cost-effectiveness.

PubMed

James, Jack E

2017-09-01

Throughout the quarter century since the advent of evidence-based medicine (EBM), medical research has prioritized 'efficacy' (i.e. internal validity) using randomized controlled trials. EBM has consistently neglected 'effectiveness' and 'cost-effectiveness', identified in the pioneering work of Archie Cochrane as essential for establishing the external (i.e. clinical) validity of health care interventions. Neither Cochrane nor other early pioneers appear to have foreseen the extent to which EBM would be appropriated by the pharmaceutical and medical devices industries, which are responsible for extensive biases in clinical research due to selective reporting, exaggeration of benefits, minimization of risks, and misrepresentation of data. The promise of EBM to effect transformational change in health care will remain unfulfilled until (i) studies of effectiveness and cost-effectiveness are pursued with some of the same fervour that previously succeeded in elevating the status of the randomized controlled trial, and (ii) ways are found to defeat threats to scientific integrity posed by commercial conflicts of interest. © 2017 Stichting European Society for Clinical Investigation Journal Foundation.

Crafting practice guidelines in the world of evidence-based medicine.

PubMed

Chung, Kevin C; Shauver, Melissa J

2009-10-01

In the era of exponential increase in the medical literature, physicians and health policy-makers are relying on well-constructed, evidence-based practice guidelines to help ensure that the care given to patients is based on valid, scientific data. The construction of practice guidelines, however, may not always adhere to accepted research protocol. In this article, the authors detail the steps required to produce effective, evidence-based practice guidelines. The seven essential steps in crafting a practice guideline are presented: (1) defining a topic, (2) selecting a work group, (3) performing a literature review, (4) writing the guideline, (5) peer review, (6) making plans for review and revision, and (7) dissemination. Given the importance of practice guidelines in supporting everyday practice, this article strives to provide a practical guide in the development of this key component of evidence-based medicine.

Validation of persuasive messages for the promotion of physical activity among people with coronary heart disease.

PubMed

Mendez, Roberto Della Rosa; Rodrigues, Roberta Cunha Matheus; Spana, Thaís Moreira; Cornélio, Marília Estevam; Gallani, Maria Cecília Bueno Jayme; Pérez-Nebra, Amalia Raquel

2012-01-01

to validate the content of persuasive messages for promoting walking among patients with coronary heart disease (CHD). The messages were constructed to strengthen or change patients' attitudes to walking. the selection of persuasive arguments was based on behavioral beliefs (determinants of attitude) related to walking. The messages were constructed based in the Elaboration Likelihood Model and were submitted to content validation. the data was analyzed with the content validity index and by the importance which the patients attributed to the messages' persuasive arguments. Positive behavioral beliefs (i.e. positive and negative reinforcement) and self-efficacy were the appeals which the patients considered important. The messages with validation evidence will be tested in an intervention study for the promotion of the practice of physical activity among patients with CHD.

Nutrigenetics and personalized nutrition: are we ready for DNA-based dietary advice?

PubMed

Grimaldi, Keith A

2014-05-01

Common genetic variation affects individual nutrient requirements and the use of DNA-based dietary advice, derived from nutrigenetics, has been growing. The growth is about to accelerate as the cost of genotyping continues to fall and research results from major nutrigenetics projects are published. There is still some skepticism; some barriers remain including some commercial tests, which make exaggerated, incorrect claims. There is a need for more public resources dedicated to unbiased, objective review and dissemination of nutrigenetics information; however, nutrigenetics evidence should be assessed in the context of standard nutritional evidence and should not require higher standards. This article argues that we are ready for some DNA-based dietary advice in general nutrition and it can be beneficial. Examples of the scientific validity and health utility of gene-diet interactions will be given and the development of guidelines for assessment and validation of benefits will be discussed.

Gene expression profiling and expanded immunohistochemistry tests to guide the use of adjuvant chemotherapy in breast cancer management: a systematic review and cost-effectiveness analysis.

PubMed

Ward, S; Scope, A; Rafia, R; Pandor, A; Harnan, S; Evans, P; Wyld, L

2013-10-01

Gene expression profiling (GEP) and expanded immunohistochemistry (IHC) tests aim to improve decision-making relating to adjuvant chemotherapy for women with early breast cancer. The aim of this report is to assess the clinical effectiveness and cost-effectiveness of nine GEP and expanded IHC tests compared with current prognostic tools in guiding the use of adjuvant chemotherapy in patients with early breast cancer in England and Wales. The nine tests are BluePrint, Breast Cancer Index (BCI), IHC4, MammaPrint, Mammostrat, NPI plus (NPI+), OncotypeDX, PAM50 and Randox Breast Cancer Array. Databases searched included MEDLINE, MEDLINE In-Process & Other Non-Indexed Citations, EMBASE and The Cochrane Library. Databases were searched from January 2009 to May 2011 for the OncotypeDX and MammaPrint tests and from January 2002 to May 2011 for the other tests. A systematic review of the evidence on clinical effectiveness (analytical validity, clinical validity and clinical utility) and cost-effectiveness was conducted. An economic model was developed to evaluate the cost-effectiveness of adjuvant chemotherapy treatment guided by four of the nine test (OncotypeDX, IHC4, MammaPrint and Mammostrat) compared with current clinical practice in England and Wales, using clinicopathological parameters, in women with oestrogen receptor-positive (ER+), lymph node-negative (LN-), human epidermal growth factor receptor type 2-negative (HER2-) early breast cancer. The literature searches for clinical effectiveness identified 5993 citations, of which 32 full-text papers or abstracts (30 studies) satisfied the criteria for the effectiveness review. A narrative synthesis was performed. Evidence for OncotypeDX supported the prognostic capability of the test. There was some evidence on the impact of the test on decision-making and to support the case that OncotypeDX predicts chemotherapy benefit; however, few studies were UK based and limitations in relation to study design were identified. Evidence for MammaPrint demonstrated that the test score was a strong independent prognostic factor, but the evidence is non-UK based and is based on small sample sizes. Evidence on the Mammostrat test showed that the test was an independent prognostic tool for women with ER+, tamoxifen-treated breast cancer. The three studies appeared to be of reasonable quality and provided data from a UK setting (one study). One large study reported on clinical validity of the IHC4 test, with IHC4 score a highly significant predictor of distant recurrence. This study included data from a UK setting and appeared to be of reasonable quality. Evidence for the remaining five tests (PAM50, NPI+, BCI, BluePrint and Randox) was limited. The economic analysis suggests that treatment guided using IHC4 has the greatest potential to be cost-effective at a £20,000 threshold, given the low cost of the test; however, further research is needed on the analytical validity and clinical utility of IHC4, and the exact cost of the test needs to be confirmed. Current limitations in the evidence base produce significant uncertainty in the results. OncotypeDX has a more robust evidence base, but further evidence on its impact on decision-making in the UK and the predictive ability of the test in an ER+, LN-, HER- population receiving current drug regimens is needed. For MammaPrint and Mammostrat there were significant gaps in the available evidence and the estimates of cost-effectiveness produced were not considered to be robust by the External Assessment Group. Methodological weaknesses in the clinical evidence base relate to heterogeneity of patient cohorts and issues arising from the retrospective nature of the evidence. Further evidence is required on the clinical utility of all of the tests and on UK-based populations. A key area of uncertainty relates to whether the tests provide prognostic or predictive ability. The clinical evidence base for OncotypeDX is considered to be the most robust. The economic analysis suggested that treatment guided using IHC4 has the most potential to be cost-effective at a threshold of £20,000; however, the evidence base to support IHC4 needs significant further research. PROSPERO 2011:CRD42011001361, available from www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42011001361.

Development and Validation of a Multimedia-based Assessment of Scientific Inquiry Abilities

NASA Astrophysics Data System (ADS)

Kuo, Che-Yu; Wu, Hsin-Kai; Jen, Tsung-Hau; Hsu, Ying-Shao

2015-09-01

The potential of computer-based assessments for capturing complex learning outcomes has been discussed; however, relatively little is understood about how to leverage such potential for summative and accountability purposes. The aim of this study is to develop and validate a multimedia-based assessment of scientific inquiry abilities (MASIA) to cover a more comprehensive construct of inquiry abilities and target secondary school students in different grades while this potential is leveraged. We implemented five steps derived from the construct modeling approach to design MASIA. During the implementation, multiple sources of evidence were collected in the steps of pilot testing and Rasch modeling to support the validity of MASIA. Particularly, through the participation of 1,066 8th and 11th graders, MASIA showed satisfactory psychometric properties to discriminate students with different levels of inquiry abilities in 101 items in 29 tasks when Rasch models were applied. Additionally, the Wright map indicated that MASIA offered accurate information about students' inquiry abilities because of the comparability of the distributions of student abilities and item difficulties. The analysis results also suggested that MASIA offered precise measures of inquiry abilities when the components (questioning, experimenting, analyzing, and explaining) were regarded as a coherent construct. Finally, the increased mean difficulty thresholds of item responses along with three performance levels across all sub-abilities supported the alignment between our scoring rubrics and our inquiry framework. Together with other sources of validity in the pilot testing, the results offered evidence to support the validity of MASIA.

Single-Case Design Research: Building the Evidence-Base in the Field of Education of Deaf and Hard of Hearing Students.

PubMed

Cannon, Joanna E; Guardino, Caroline; Antia, Shirin D; Luckner, John L

2016-01-01

The field of education of deaf and hard of hearing (DHH) students has a paucity of evidence-based practices (EBPs) to guide instruction. The authors discussed how the research methodology of single-case design (SCD) can be used to build EBPs through direct and systematic replication of studies. An overview of SCD research methods is presented, including an explanation of how internal and external validity issues are addressed, and why SCD is appropriate for intervention research with DHH children. The authors then examine the SCD research in the field according to quality indicators (QIs; at the individual level and as a body of evidence) to determine the existing evidence base. Finally, future replication areas are recommended to fill the gaps in SCD research with students who are DHH in order to add to the evidence base in the field.

Validity evidence for the Fundamentals of Laparoscopic Surgery (FLS) program as an assessment tool: a systematic review.

PubMed

Zendejas, Benjamin; Ruparel, Raaj K; Cook, David A

2016-02-01

The Fundamentals of Laparoscopic Surgery (FLS) program uses five simulation stations (peg transfer, precision cutting, loop ligation, and suturing with extracorporeal and intracorporeal knot tying) to teach and assess laparoscopic surgery skills. We sought to summarize evidence regarding the validity of scores from the FLS assessment. We systematically searched for studies evaluating the FLS as an assessment tool (last search update February 26, 2013). We classified validity evidence using the currently standard validity framework (content, response process, internal structure, relations with other variables, and consequences). From a pool of 11,628 studies, we identified 23 studies reporting validity evidence for FLS scores. Studies involved residents (n = 19), practicing physicians (n = 17), and medical students (n = 8), in specialties of general (n = 17), gynecologic (n = 4), urologic (n = 1), and veterinary (n = 1) surgery. Evidence was most common in the form of relations with other variables (n = 22, most often expert-novice differences). Only three studies reported internal structure evidence (inter-rater or inter-station reliability), two studies reported content evidence (i.e., derivation of assessment elements), and three studies reported consequences evidence (definition of pass/fail thresholds). Evidence nearly always supported the validity of FLS total scores. However, the loop ligation task lacks discriminatory ability. Validity evidence confirms expected relations with other variables and acceptable inter-rater reliability, but other validity evidence is sparse. Given the high-stakes use of this assessment (required for board eligibility), we suggest that more validity evidence is required, especially to support its content (selection of tasks and scoring rubric) and the consequences (favorable and unfavorable impact) of assessment.

Evidence-based medicine and big genomic data.

PubMed

Ioannidis, John P A; Khoury, Muin J

2018-05-01

Genomic and other related big data (Big Genomic Data, BGD for short) are ushering a new era of precision medicine. This overview discusses whether principles of evidence-based medicine hold true for BGD and how they should be operationalized in the current era. Major evidence-based medicine principles include the systematic identification, description and analysis of the validity and utility of BGD, the combination of individual clinical expertise with individual patient needs and preferences, and the focus on obtaining experimental evidence, whenever possible. BGD emphasize information of single patients with an overemphasis on N-of-1 trials to personalize treatment. However, large-scale comparative population data remain indispensable for meaningful translation of BGD personalized information. The impact of BGD on population health depends on its ability to affect large segments of the population. While several frameworks have been proposed to facilitate and standardize decision making for use of genomic tests, there are new caveats that arise from BGD that extend beyond the limitations that were applicable for more simple genetic tests. Non-evidence-based use of BGD may be harmful and result in major waste of healthcare resources. Randomized controlled trials will continue to be the strongest arbitrator for the clinical utility of genomic technologies, including BGD. Research on BGD needs to focus not only on finding robust predictive associations (clinical validity) but also more importantly on evaluating the balance of health benefits and potential harms (clinical utility), as well as implementation challenges. Appropriate features of such useful research on BGD are discussed.

Construct Validation of Physical Activity Surveys in Culturally Diverse Older Adults: A Comparison of Four Commonly Used Questionnaires

ERIC Educational Resources Information Center

Moore, Delilah S.; Ellis, Rebecca; Allen, Priscilla D.; Cherry, Katie E.; Monroe, Pamela A.; O'Neil, Carol E.; Wood, Robert H.

2008-01-01

The purpose of this study was to establish validity evidence of four physical activity (PA) questionnaires in culturally diverse older adults by comparing self-report PA with performance-based physical function. Participants were 54 older adults who completed the Continuous Scale Physical Functional Performance 10-item Test (CS-PFP10), Physical…

Initial Evidence for the Reliability and Validity of the Student Risk Screening Scale for Internalizing and Externalizing Behaviors at the Elementary Level

ERIC Educational Resources Information Center

Lane, Kathleen Lynne; Oakes, Wendy P.; Harris, Pamela J.; Menzies, Holly Mariah; Cox, Meredith; Lambert, Warren

2012-01-01

We report findings of an exploratory validation study of a revised instrument: the Student Risk Screening Scale-Internalizing and Externalizing (SRSS-IE). The SRSS-IE was modified to include seven additional items reflecting characteristics of internalizing behaviors, with proposed items generated from the current literature base, review of…

Validity Evidence for the Chinese Version Classroom Appraisal of Resources and Demands (CARD)

ERIC Educational Resources Information Center

Zhang, Juan; Wang, Chuang; Lambert, Richard; Wu, Chenggang; Wen, Hongbo

2017-01-01

The Classroom Appraisal of Resources and Demands (CARD) was designed to evaluate teacher stress based on subjective evaluations of classroom demands and resources. However, the CARD has been mostly utilized in western countries. The aim of the current study was to provide aspects of the validity of responses to a Chinese version of the CARD that…

Using Nonsense Word Fluency to Predict Reading Proficiency in Kindergarten through Second Grade for English Learners and Native English Speakers

ERIC Educational Resources Information Center

Fien, Hank; Baker, Scott K.; Smolkowski, Keith; Smith, Jean L. Mercier; Kame'enui, Edward J.; Beck, Carrie Thomas

2008-01-01

This study examined the validity of Nonsense Word Fluency as an index of beginning reading proficiency for students in kindergarten through second grade. Validity evidence for Nonsense Word Fluency is addressed in the context of research-based instructional practices implemented on a large scale. Technical adequacy data are presented for all…

Validating a Computer-Assisted Language Learning Attitude Instrument Used in Iranian EFL Context: An Evidence-Based Approach

ERIC Educational Resources Information Center

Aryadoust, Vahid; Mehran, Parisa; Alizadeh, Mehrasa

2016-01-01

A few computer-assisted language learning (CALL) instruments have been developed in Iran to measure EFL (English as a foreign language) learners' attitude toward CALL. However, these instruments have no solid validity argument and accordingly would be unable to provide a reliable measurement of attitude. The present study aimed to develop a CALL…

Using Corpus Linguistics to Examine the Extrapolation Inference in the Validity Argument for a High-Stakes Speaking Assessment

ERIC Educational Resources Information Center

LaFlair, Geoffrey T.; Staples, Shelley

2017-01-01

Investigations of the validity of a number of high-stakes language assessments are conducted using an argument-based approach, which requires evidence for inferences that are critical to score interpretation (Chapelle, Enright, & Jamieson, 2008b; Kane, 2013). The current study investigates the extrapolation inference for a high-stakes test of…

The validation of forensic DNA extraction systems to utilize soil contaminated biological evidence.

PubMed

Kasu, Mohaimin; Shires, Karen

2015-07-01

The production of full DNA profiles from biological evidence found in soil has a high failure rate due largely to the inhibitory substance humic acid (HA). Abundant in various natural soils, HA co-extracts with DNA during extraction and inhibits DNA profiling by binding to the molecular components of the genotyping assay. To successfully utilize traces of soil contaminated evidence, such as that found at many murder and rape crime scenes in South Africa, a reliable HA removal extraction system would often be selected based on previous validation studies. However, for many standard forensic DNA extraction systems, peer-reviewed publications detailing the efficacy on soil evidence is either lacking or is incomplete. Consequently, these sample types are often not collected or fail to yield suitable DNA material due to the use of unsuitable methodology. The aim of this study was to validate the common forensic DNA collection and extraction systems used in South Africa, namely DNA IQ, FTA elute and Nucleosave for processing blood and saliva contaminated with HA. A forensic appropriate volume of biological evidence was spiked with HA (0, 0.5, 1.5 and 2.5 mg/ml) and processed through each extraction protocol for the evaluation of HA removal using QPCR and STR-genotyping. The DNA IQ magnetic bead system effectively removed HA from highly contaminated blood and saliva, and generated consistently acceptable STR profiles from both artificially spiked samples and crude soil samples. This system is highly recommended for use on soil-contaminated evidence over the cellulose card-based systems currently being preferentially used for DNA sample collection. Copyright © 2015 Elsevier Ireland Ltd. All rights reserved.

Development of an evidence-based training program for laparoscopic hysterectomy on a virtual reality simulator.

PubMed

Crochet, Patrice; Aggarwal, Rajesh; Knight, Sophie; Berdah, Stéphane; Boubli, Léon; Agostini, Aubert

2017-06-01

Substantial evidence in the scientific literature supports the use of simulation for surgical education. However, curricula lack for complex laparoscopic procedures in gynecology. The objective was to evaluate the validity of a program that reproduces key specific components of a laparoscopic hysterectomy (LH) procedure until colpotomy on a virtual reality (VR) simulator and to develop an evidence-based and stepwise training curriculum. This prospective cohort study was conducted in a Marseille teaching hospital. Forty participants were enrolled and were divided into experienced (senior surgeons who had performed more than 100 LH; n = 8), intermediate (surgical trainees who had performed 2-10 LH; n = 8) and inexperienced (n = 24) groups. Baselines were assessed on a validated basic task. Participants were tested for the LH procedure on a high-fidelity VR simulator. Validity evidence was proposed as the ability to differentiate between the three levels of experience. Inexperienced subjects performed ten repetitions for learning curve analysis. Proficiency measures were based on experienced surgeons' performances. Outcome measures were simulator-derived metrics and Objective Structured Assessment of Technical Skills (OSATS) scores. Quantitative analysis found significant inter-group differences between experienced intermediate and inexperienced groups for time (1369, 2385 and 3370 s; p < 0.001), number of movements (2033, 3195 and 4056; p = 0.001), path length (3390, 4526 and 5749 cm; p = 0.002), idle time (357, 654 and 747 s; p = 0.001), respect for tissue (24, 40 and 84; p = 0.01) and number of bladder injuries (0.13, 0 and 4.27; p < 0.001). Learning curves plateaued at the 2nd to 6th repetition. Further qualitative analysis found significant inter-group OSATS score differences at first repetition (22, 15 and 8, respectively; p < 0.001) and second repetition (25.5, 19.5 and 14; p < 0.001). The VR program for LH accrued validity evidence and allowed the development of a training curriculum using a structured scientific methodology.

Contemporary Test Validity in Theory and Practice: A Primer for Discipline-Based Education Researchers

PubMed Central

Reeves, Todd D.; Marbach-Ad, Gili

2016-01-01

Most discipline-based education researchers (DBERs) were formally trained in the methods of scientific disciplines such as biology, chemistry, and physics, rather than social science disciplines such as psychology and education. As a result, DBERs may have never taken specific courses in the social science research methodology—either quantitative or qualitative—on which their scholarship often relies so heavily. One particular aspect of (quantitative) social science research that differs markedly from disciplines such as biology and chemistry is the instrumentation used to quantify phenomena. In response, this Research Methods essay offers a contemporary social science perspective on test validity and the validation process. The instructional piece explores the concepts of test validity, the validation process, validity evidence, and key threats to validity. The essay also includes an in-depth example of a validity argument and validation approach for a test of student argument analysis. In addition to DBERs, this essay should benefit practitioners (e.g., lab directors, faculty members) in the development, evaluation, and/or selection of instruments for their work assessing students or evaluating pedagogical innovations. PMID:26903498

[Modeling in value-based medicine].

PubMed

Neubauer, A S; Hirneiss, C; Kampik, A

2010-03-01

Modeling plays an important role in value-based medicine (VBM). It allows decision support by predicting potential clinical and economic consequences, frequently combining different sources of evidence. Based on relevant publications and examples focusing on ophthalmology the key economic modeling methods are explained and definitions are given. The most frequently applied model types are decision trees, Markov models, and discrete event simulation (DES) models. Model validation includes besides verifying internal validity comparison with other models (external validity) and ideally validation of its predictive properties. The existing uncertainty with any modeling should be clearly stated. This is true for economic modeling in VBM as well as when using disease risk models to support clinical decisions. In economic modeling uni- and multivariate sensitivity analyses are usually applied; the key concepts here are tornado plots and cost-effectiveness acceptability curves. Given the existing uncertainty, modeling helps to make better informed decisions than without this additional information.

Limitations in conduct and reporting of cochrane reviews rarely inhibit the determination of the validity of evidence for clinical decision-making.

PubMed

Alper, Brian S; Fedorowicz, Zbys; van Zuuren, Esther J

2015-08-01

To determine how often clinical conclusions derived from Cochrane Reviews have uncertain validity due to review conduct and reporting deficiencies. We evaluated 5142 clinical conclusions in DynaMed (an evidence-based point-of-care clinical reference) based on 4743 Cochrane Reviews. Clinical conclusions with level 2 evidence due to shortcomings in the review's conduct or reporting (rather than deficiencies in the underlying evidence) were confirmed by a DynaMed editor and two Cochrane Review authors. Thirty-one Cochrane Reviews (0.65%) had confirmed deficiencies in conduct and reporting as the reason for classifying 37 assessed clinical conclusions (0.72%) as level 2 evidence. In all cases, it was not feasible for the assessors to specify a clear criticism of the studies included in the reviews. The deficiencies were specific to not accounting for dropouts (2) or inadequate assessment and reporting of allocation concealment (11), other specific trial quality criteria (14), or all trial quality criteria (4). Cochrane Reviews provide high-quality assessment and synthesis of evidence, with fewer than 1% of Cochrane Reviews having limitations which hinder the summary of best current evidence for clinical decision-making. We expect this will further decrease following recent Cochrane quality initiatives. © 2015 Chinese Cochrane Center, West China Hospital of Sichuan University and Wiley Publishing Asia Pty Ltd.

The Validity of the Comparative Interrupted Time Series Design for Evaluating the Effect of School-Level Interventions.

PubMed

Jacob, Robin; Somers, Marie-Andree; Zhu, Pei; Bloom, Howard

2016-06-01

In this article, we examine whether a well-executed comparative interrupted time series (CITS) design can produce valid inferences about the effectiveness of a school-level intervention. This article also explores the trade-off between bias reduction and precision loss across different methods of selecting comparison groups for the CITS design and assesses whether choosing matched comparison schools based only on preintervention test scores is sufficient to produce internally valid impact estimates. We conduct a validation study of the CITS design based on the federal Reading First program as implemented in one state using results from a regression discontinuity design as a causal benchmark. Our results contribute to the growing base of evidence regarding the validity of nonexperimental designs. We demonstrate that the CITS design can, in our example, produce internally valid estimates of program impacts when multiple years of preintervention outcome data (test scores in the present case) are available and when a set of reasonable criteria are used to select comparison organizations (schools in the present case). © The Author(s) 2016.

An Evidence-Based Systematic Review of Beta-Sitosterol, Sitosterol (22,23- dihydrostigmasterol, 24-ethylcholesterol) by the Natural Standard Research Collaboration.

PubMed

Ulbricht, Catherine E

2016-01-01

An evidence-based systematic review of beta-sitosterol, sitosterol (22,23-dihydrostigmasterol, 24-ethylcholesterol) by the Natural Standard Research Collaboration consolidates the safety and efficacy data available in the scientific literature using a validated, reproducible grading rationale. This article includes written and statistical analysis of clinical trials, plus a compilation of expert opinion, folkloric precedent, history, pharmacology, kinetics/dynamics, interactions, adverse effects, toxicology, and dosing.

Evaluating the accuracy of the Wechsler Memory Scale-Fourth Edition (WMS-IV) logical memory embedded validity index for detecting invalid test performance.

PubMed

Soble, Jason R; Bain, Kathleen M; Bailey, K Chase; Kirton, Joshua W; Marceaux, Janice C; Critchfield, Edan A; McCoy, Karin J M; O'Rourke, Justin J F

2018-01-08

Embedded performance validity tests (PVTs) allow for continuous assessment of invalid performance throughout neuropsychological test batteries. This study evaluated the utility of the Wechsler Memory Scale-Fourth Edition (WMS-IV) Logical Memory (LM) Recognition score as an embedded PVT using the Advanced Clinical Solutions (ACS) for WAIS-IV/WMS-IV Effort System. This mixed clinical sample was comprised of 97 total participants, 71 of whom were classified as valid and 26 as invalid based on three well-validated, freestanding criterion PVTs. Overall, the LM embedded PVT demonstrated poor concordance with the criterion PVTs and unacceptable psychometric properties using ACS validity base rates (42% sensitivity/79% specificity). Moreover, 15-39% of participants obtained an invalid ACS base rate despite having a normatively-intact age-corrected LM Recognition total score. Receiving operating characteristic curve analysis revealed a Recognition total score cutoff of < 61% correct improved specificity (92%) while sensitivity remained weak (31%). Thus, results indicated the LM Recognition embedded PVT is not appropriate for use from an evidence-based perspective, and that clinicians may be faced with reconciling how a normatively intact cognitive performance on the Recognition subtest could simultaneously reflect invalid performance validity.

The Educational Validity and Utility of Single-Case Design Research in Building Evidence-Based Practices in Education

ERIC Educational Resources Information Center

Kourea, Lefki; Lo, Ya-yu

2016-01-01

Improving academic, behavioural, and social outcomes of students through empirical research has been a firm commitment among researchers, policy-makers, and other professionals in education across Europe and the United States (U.S.). To assist in building scientific evidences, executive bodies such as the European Commission and the Institute for…

A Systematic Review of Evidence for the Psychometric Properties of the Strengths and Difficulties Questionnaire

ERIC Educational Resources Information Center

Kersten, Paula; Czuba, Karol; McPherson, Kathryn; Dudley, Margaret; Elder, Hinemoa; Tauroa, Robyn; Vandal, Alain

2016-01-01

This article synthesized evidence for the validity and reliability of the Strengths and Difficulties Questionnaire in children aged 3-5 years. A systematic review using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement guidelines was carried out. Study quality was rated using the Consensus-based Standards for the…

Psychometric evaluation and sex invariance of the Spanish version of the Body and Appearance Self-Conscious Emotions Scale.

PubMed

Alcaraz-Ibáñez, Manuel; Sicilia, Alvaro

2018-06-01

This study examined the psychometric properties of a Spanish translation of the Body and Appearance Self-Conscious Emotions Scale (BASES; Castonguay et al., 2014) in a sample of university Spanish students. A total of 815 participants enrolled in two public universities located in Almería and Elche, Spain, completed the BASES along with measures of social physique anxiety and positive/negative affect. Exploratory and confirmatory factor analyses showed that one item failed to load clearly on the hypothesized factor (guilt). Once it was removed, results supported the hypothesized four-factor structure. Evidence of invariance of the four-factor structure across sex was obtained. Scores on the BASES showed adequate internal consistency and acceptable convergent validity. Compared to men, women reported significantly higher body and appearance-related guilt and shame, and significant lower authentic and hubristic pride. Preliminary evidence supporting the validity and reliability of the Spanish translation of the BASES is provided. Copyright © 2018 Elsevier Ltd. All rights reserved.

An investigation of the validity of the Work Assessment Triage Tool clinical decision support tool for selecting optimal rehabilitation interventions for workers with musculoskeletal injuries.

PubMed

Qin, Ziling; Armijo-Olivo, Susan; Woodhouse, Linda J; Gross, Douglas P

2016-03-01

To evaluate the concurrent validity of a clinical decision support tool (Work Assessment Triage Tool (WATT)) developed to select rehabilitation treatments for injured workers with musculoskeletal conditions. Methodological study with cross-sectional and prospective components. Data were obtained from the Workers' Compensation Board of Alberta rehabilitation facility in Edmonton, Canada. A total of 432 workers' compensation claimants evaluated between November 2011 and June 2012. Percentage agreement between the Work Assessment Triage Tool and clinician recommendations was used to determine concurrent validity. In claimants returning to work, frequencies of matching were calculated and compared between clinician and Work Assessment Triage Tool recommendations and the actual programs undertaken by claimants. The frequency of each intervention recommended by clinicians, Work Assessment Triage Tool, and case managers were also calculated and compared. Percentage agreement between clinician and Work Assessment Triage Tool recommendations was poor (19%) to moderate (46%) and Kappa = 0.37 (95% CI -0.02, 0.76). The Work Assessment Triage Tool did not improve upon clinician recommendations as only 14 out of 31 claimants returning to work had programs that contradicted clinician recommendations, but were consistent with Work Assessment Triage Tool recommendations. Clinicians and case managers were inclined to recommend functional restoration, physical therapy, or no rehabilitation while the Work Assessment Triage Tool recommended additional evidence-based interventions, such as workplace-based interventions. Our findings do not provide evidence of concurrent validity for the Work Assessment Triage Tool compared with clinician recommendations. Based on these results, we cannot recommend further implementation of the Work Assessment Triage Tool. However, the Work Assessment Triage Tool appeared more likely than clinicians to recommend interventions supported by evidence; thus warranting further research. © The Author(s) 2015.

Improving the evidence base for better comparative effectiveness research.

PubMed

Brophy, James M

2015-09-01

The last 20 years has documented that the evidence base for informed clinical decision-making is often suboptimal. It is hoped that high-quality comparative effectiveness research may fill these knowledge gaps. Implicit in these changing paradigms is the underlying assumption that the published evidence, when available, is valid. It is posited here that this assumption is sometimes questionable. However, several recent methods that may improve the design and analysis of comparative effectiveness research have appeared and are discussed here. Examples from the cardiology literature are provided, but it is believed the highlighted principles are applicable to other branches of medicine.

Variation, certainty, evidence, and change in dental education: employing evidence-based dentistry in dental education.

PubMed

Marinho, V C; Richards, D; Niederman, R

2001-05-01

Variation in health care, and more particularly in dental care, was recently chronicled in a Readers Digest investigative report. The conclusions of this report are consistent with sound scientific studies conducted in various areas of health care, including dental care, which demonstrate substantial variation in the care provided to patients. This variation in care parallels the certainty with which clinicians and faculty members often articulate strongly held, but very different opinions. Using a case-based dental scenario, we present systematic evidence-based methods for accessing dental health care information, evaluating this information for validity and importance, and using this information to make informed curricular and clinical decisions. We also discuss barriers inhibiting these systematic approaches to evidence-based clinical decision making and methods for effectively promoting behavior change in health care professionals.

Development of Lesson Plans and Student Worksheets Based Socio-Scientific Issues on Pollution Environmental Topic

NASA Astrophysics Data System (ADS)

Rahayu, S.; Meyliana, M.; Arlingga, A.; Reny, R.; Siahaan, P.; Hernani, H.

2017-09-01

The aim of this study is to develop lesson plans and student worksheets based socio-scientific issues on pollution environmental topic for seventh-grade junior high school students. Environmental pollution topic split into several subtopics namely air pollution, water pollution and soil pollution. The composing of lesson plans were developed based on socio-scientific issues with five stages, namely (1) Motivate; (2) Challenge; (3) Collect scientific evidence; (4) Analyse the evidence; (5) Build knowledge and make connections; and (6) Use evidence. While student worksheets contain articles on socio-scientific issues, practice, and there are a few questions to determine students’ reasoning. The method that is used in this research is research and development (R & D method). Development model used in this study is a model of Plomp that consists of four stages, namely: (1) Initial Research; (2) Design; (3) Realization or Construction; (4) Testing, evaluation and revision; (5) Implementation, while the research was limited to the fourth stage. Lesson plans and student worksheets based on socio-scientific issues was validated through an expert validation. The result showed that lesson plans and student worksheets based socio-scientific issues on pollution theme have a very decent and be able to apply in science classroom.

Montreal-Toulouse Language Assessment Battery: evidence of criterion validity from patients with aphasia.

PubMed

Pagliarin, Karina Carlesso; Ortiz, Karin Zazo; Barreto, Simone dos Santos; Pimenta Parente, Maria Alice de Mattos; Nespoulous, Jean-Luc; Joanette, Yves; Fonseca, Rochele Paz

2015-10-15

The Montreal-Toulouse Language Assessment Battery - Brazilian version (MTL-BR) provides a general description of language processing and related components in adults with brain injury. The present study aimed at verifying the criterion-related validity of the Montreal-Toulouse Language Assessment Battery - Brazilian version (MTL-BR) by assessing its ability to discriminate between individuals with unilateral brain damage with and without aphasia. The investigation was carried out in a Brazilian community-based sample of 104 adults, divided into four groups: 26 participants with left hemisphere damage (LHD) with aphasia, 25 participants with right hemisphere damage (RHD), 28 with LHD non-aphasic, and 25 healthy adults. There were significant differences between patients with aphasia and the other groups on most total and subtotal scores on MTL-BR tasks. The results showed strong criterion-related validity evidence for the MTL-BR Battery, and provided important information regarding hemispheric specialization and interhemispheric cooperation. Future research is required to search for additional evidence of sensitivity, specificity and validity of the MTL-BR in samples with different types of aphasia and degrees of language impairment. Copyright © 2015 Elsevier B.V. All rights reserved.

Validation of an instrument to assess evidence-based practice knowledge, attitudes, access, and confidence in the dental environment.

PubMed

Hendricson, William D; Rugh, John D; Hatch, John P; Stark, Debra L; Deahl, Thomas; Wallmann, Elizabeth R

2011-02-01

This article reports the validation of an assessment instrument designed to measure the outcomes of training in evidence-based practice (EBP) in the context of dentistry. Four EBP dimensions are measured by this instrument: 1) understanding of EBP concepts, 2) attitudes about EBP, 3) evidence-accessing methods, and 4) confidence in critical appraisal. The instrument-the Knowledge, Attitudes, Access, and Confidence Evaluation (KACE)-has four scales, with a total of thirty-five items: EBP knowledge (ten items), EBP attitudes (ten), accessing evidence (nine), and confidence (six). Four elements of validity were assessed: consistency of items within the KACE scales (extent to which items within a scale measure the same dimension), discrimination (capacity to detect differences between individuals with different training or experience), responsiveness (capacity to detect the effects of education on trainees), and test-retest reliability. Internal consistency of scales was assessed by analyzing responses of second-year dental students, dental residents, and dental faculty members using Cronbach coefficient alpha, a statistical measure of reliability. Discriminative validity was assessed by comparing KACE scores for the three groups. Responsiveness was assessed by comparing pre- and post-training responses for dental students and residents. To measure test-retest reliability, the full KACE was completed twice by a class of freshman dental students seventeen days apart, and the knowledge scale was completed twice by sixteen faculty members fourteen days apart. Item-to-scale consistency ranged from 0.21 to 0.78 for knowledge, 0.57 to 0.83 for attitude, 0.70 to 0.84 for accessing evidence, and 0.87 to 0.94 for confidence. For discrimination, ANOVA and post hoc testing by the Tukey-Kramer method revealed significant score differences among students, residents, and faculty members consistent with education and experience levels. For responsiveness to training, dental students and residents demonstrated statistically significant changes, in desired directions, from pre- to post-test. For the student test-retest, Pearson correlations for KACE scales were as follows: knowledge 0.66, attitudes 0.66, accessing evidence 0.74, and confidence 0.76. For the knowledge scale test-retest by faculty members, the Pearson correlation was 0.79. The construct validity of the KACE is equivalent to that of instruments that assess similar EBP dimensions in medicine. Item consistency for the knowledge scale was more variable than for other KACE scales, a finding also reported for medically oriented EBP instruments. We conclude that the KACE has good discriminative validity, responsiveness to training effects, and test-retest reliability.

Measuring quality of dental care: Caries prevention services for children.

PubMed

Herndon, Jill Boylston; Tomar, Scott L; Catalanotto, Frank A; Rudner, Nancy; Huang, I-Chan; Aravamudhan, Krishna; Shenkman, Elizabeth A; Crall, James J

2015-08-01

The authors conducted a study to validate the following 3 evidence-based, process-of-care quality measures focused on dental caries prevention for children with an elevated risk of experiencing caries: sealants for 6- to 9-year-olds, sealants for 10- to 14-year-olds, and topical fluoride. Using evidence-based guidelines, the Dental Quality Alliance developed measures for implementation with administrative data at the plan and program levels. To validate the measures, the authors used data from the Florida and Texas Medicaid programs and Children's Health Insurance Programs and from national commercial dental benefit plans. Data were extracted from 414 randomly selected dental office records to validate the use of administrative data to accurately calculate the measures. The authors also assessed statistically significant variations in overall measure performance. Agreement between administrative data and dental records was 95% for sealants (κ = 0.82) and 90% for topical fluoride (κ = 0.78). Sensitivity and specificity were 90.7% and 88.5% for topical fluoride and 77.8% and 98.8% for sealants, respectively. Variation in overall measure performance was greatest for topical fluoride (χ(2) = 5,887.1; P < .01); 18% to 37% of children with an elevated risk of experiencing caries received at least 2 topical fluoride applications during the reporting year. Although there was greater variation in performance for sealants for 6- to 9-year-olds (range, 21.0-31.3%; χ(2) = 548.6; P < .01) compared with sealants for 10- to 14-year-olds (range, 8.4-11.1%; χ(2) = 22.7; P < .01), overall sealant placement rates were lower for 10- to 14-year-olds. These evidence-based, caries prevention process-of-care quality measures can be implemented feasibly and validly using administrative claims data. The measures can be used to assess, monitor, and improve the proportion of children with an elevated risk of experiencing dental caries who receive evidence-based caries prevention services. Copyright © 2015 American Dental Association. Published by Elsevier Inc. All rights reserved.

Validation of the Portuguese version of the Evidence-Based Practice Questionnaire

PubMed Central

Pereira, Rui Pedro Gomes; Guerra, Ana Cristina Pinheiro; Cardoso, Maria José da Silva Peixoto de Oliveira; dos Santos, Alzira Teresa Vieira Martins Ferreira; de Figueiredo, Maria do Céu Aguiar Barbieri; Carneiro, António Cândido Vaz

2015-01-01

OBJECTIVES: to describe the process of translation and linguistic and cultural validation of the Evidence Based Practice Questionnaire for the Portuguese context: Questionário de Eficácia Clínica e Prática Baseada em Evidências (QECPBE). METHOD: a methodological and cross-sectional study was developed. The translation and back translation was performed according to traditional standards. Principal Components Analysis with orthogonal rotation according to the Varimax method was used to verify the QECPBE's psychometric characteristics, followed by confirmatory factor analysis. Internal consistency was determined by Cronbach's alpha. Data were collected between December 2013 and February 2014. RESULTS: 358 nurses delivering care in a hospital facility in North of Portugal participated in the study. QECPBE contains 20 items and three subscales: Practice (α=0.74); Attitudes (α=0.75); Knowledge/Skills and Competencies (α=0.95), presenting an overall internal consistency of α=0.74. The tested model explained 55.86% of the variance and presented good fit: χ2(167)=520.009; p = 0.0001; χ2df=3.114; CFI=0.908; GFI=0.865; PCFI=0.798; PGFI=0.678; RMSEA=0.077 (CI90%=0.07-0.08). CONCLUSION: confirmatory factor analysis revealed the questionnaire is valid and appropriate to be used in the studied context. PMID:26039307

Validating Trial-Based Functional Analyses in Mainstream Primary School Classrooms

ERIC Educational Resources Information Center

Austin, Jennifer L.; Groves, Emily A.; Reynish, Lisa C.; Francis, Laura L.

2015-01-01

There is growing evidence to support the use of trial-based functional analyses, particularly in classroom settings. However, there currently are no evaluations of this procedure with typically developing children. Furthermore, it is possible that refinements may be needed to adapt trial-based analyses to mainstream classrooms. This study was…

Standard Setting Methods for Pass/Fail Decisions on High-Stakes Objective Structured Clinical Examinations: A Validity Study.

PubMed

Yousuf, Naveed; Violato, Claudio; Zuberi, Rukhsana W

2015-01-01

CONSTRUCT: Authentic standard setting methods will demonstrate high convergent validity evidence of their outcomes, that is, cutoff scores and pass/fail decisions, with most other methods when compared with each other. The objective structured clinical examination (OSCE) was established for valid, reliable, and objective assessment of clinical skills in health professions education. Various standard setting methods have been proposed to identify objective, reliable, and valid cutoff scores on OSCEs. These methods may identify different cutoff scores for the same examinations. Identification of valid and reliable cutoff scores for OSCEs remains an important issue and a challenge. Thirty OSCE stations administered at least twice in the years 2010-2012 to 393 medical students in Years 2 and 3 at Aga Khan University are included. Psychometric properties of the scores are determined. Cutoff scores and pass/fail decisions of Wijnen, Cohen, Mean-1.5SD, Mean-1SD, Angoff, borderline group and borderline regression (BL-R) methods are compared with each other and with three variants of cluster analysis using repeated measures analysis of variance and Cohen's kappa. The mean psychometric indices on the 30 OSCE stations are reliability coefficient = 0.76 (SD = 0.12); standard error of measurement = 5.66 (SD = 1.38); coefficient of determination = 0.47 (SD = 0.19), and intergrade discrimination = 7.19 (SD = 1.89). BL-R and Wijnen methods show the highest convergent validity evidence among other methods on the defined criteria. Angoff and Mean-1.5SD demonstrated least convergent validity evidence. The three cluster variants showed substantial convergent validity with borderline methods. Although there was a high level of convergent validity of Wijnen method, it lacks the theoretical strength to be used for competency-based assessments. The BL-R method is found to show the highest convergent validity evidences for OSCEs with other standard setting methods used in the present study. We also found that cluster analysis using mean method can be used for quality assurance of borderline methods. These findings should be further confirmed by studies in other settings.

Integrating Validity Theory with Use of Measurement Instruments in Clinical Settings

PubMed Central

Kelly, P Adam; O'Malley, Kimberly J; Kallen, Michael A; Ford, Marvella E

2005-01-01

Objective To present validity concepts in a conceptual framework useful for research in clinical settings. Principal Findings We present a three-level decision rubric for validating measurement instruments, to guide health services researchers step-by-step in gathering and evaluating validity evidence within their specific situation. We address construct precision, the capacity of an instrument to measure constructs it purports to measure and differentiate from other, unrelated constructs; quantification precision, the reliability of the instrument; and translation precision, the ability to generalize scores from an instrument across subjects from the same or similar populations. We illustrate with specific examples, such as an approach to validating a measurement instrument for veterans when prior evidence of instrument validity for this population does not exist. Conclusions Validity should be viewed as a property of the interpretations and uses of scores from an instrument, not of the instrument itself: how scores are used and the consequences of this use are integral to validity. Our advice is to liken validation to building a court case, including discovering evidence, weighing the evidence, and recognizing when the evidence is weak and more evidence is needed. PMID:16178998

Psychometric properties of the Late-Life Function and Disability Instrument: a systematic review

PubMed Central

2014-01-01

Background The choice of measure for use as a primary outcome in geriatric research is contingent upon the construct of interest and evidence for its psychometric properties. The Late-Life Function and Disability Instrument (LLFDI) has been widely used to assess functional limitations and disability in studies with older adults. The primary aim of this systematic review was to evaluate the current available evidence for the psychometric properties of the LLFDI. Methods Published studies of any design reporting results based on administration of the original version of the LLFDI in community-dwelling older adults were identified after searches of 9 electronic databases. Data related to construct validity (convergent/divergent and known-groups validity), test-retest reliability and sensitivity to change were extracted. Effect sizes were calculated for within-group changes and summarized graphically. Results Seventy-one studies including 17,301 older adults met inclusion criteria. Data supporting the convergent/divergent and known-groups validity for both the Function and Disability components were extracted from 30 and 18 studies, respectively. High test-retest reliability was found for the Function component, while results for the Disability component were more variable. Sensitivity to change of the LLFDI was confirmed based on findings from 25 studies. The basic lower extremity subscale and overall summary score of the Function component and limitation dimension of the Disability component were associated with the strongest relative effect sizes. Conclusions There is extensive evidence to support the construct validity and sensitivity to change of the LLFDI among various clinical populations of community-dwelling older adults. Further work is needed on predictive validity and values for clinically important change. Findings from this review can be used to guide the selection of the most appropriate LLFDI subscale for use an outcome measure in geriatric research and practice. PMID:24476510

Fiber glass exposure and human respiratory system cancer risk: lack of evidence persists since 2001 IARC re-evaluation.

PubMed

Marsh, Gary M; Buchanich, Jeanine M; Youk, Ada O

2011-06-01

To determine whether IARC's 2001 decision to downgrade the classification of insulation glass wool from Group 2B to Group 3 remains valid in light of epidemiological evidence reported after 2001. We performed a systematic review of epidemiological evidence regarding respiratory cancer risks in relation to man-made vitreous fiber (MMVF) exposure before and after the 2001 IARC re-evaluation with focus on glass wool exposure and respiratory system cancer. Since 2001, three new community-based, case-control studies, two detailed analyses of existing cohort studies and two reviews/meta-analyses were published. These studies revealed no consistent evidence of an increased respiratory system cancer risk in relation to glass wool exposure. From our evaluation of the epidemiological evidence published since 2001, we conclude that IARC's 2001 decision to downgrade insulation glass wool from Group 2B to Group 3 remains valid. Copyright © 2011 Elsevier Inc. All rights reserved.

Exercise program for children and adolescents with leukemia and lymphoma during treatment: A comprehensive review.

PubMed

Zucchetti, Giulia; Rossi, Francesca; Chamorro Vina, Carolina; Bertorello, Nicoletta; Fagioli, Franca

2018-05-01

An exercise program (EP) during cancer treatment seems to be a valid strategy against physiological and quality-of-life impairments, but scientific evidence of benefits among pediatric patients is still limited. This review summarizes the literature focused on randomized controlled trials of EP offered to patients during leukemia and lymphoma treatment. Studies published up to June 2017 were selected from multiple databases and assessed by three independent reviewers for methodological validity. The review identified eight studies, but several types of bias have to be avoided to provide evidence-based recommendations accessible to patients, families, and professionals. © 2018 Wiley Periodicals, Inc.

Achievement Emotions and Achievement Goals in Support of the Convergent, Divergent and Criterion Validity of the Spanish-Cognitive Test Anxiety Scale

ERIC Educational Resources Information Center

Sánchez-Rosas, Javier; Furlan, Luis Alberto

2017-01-01

Based on the control-value theory of achievement emotions and theory of achievement goals, this research provides evidence of convergent, divergent, and criterion validity of the Spanish Cognitive Test Anxiety Scale (S-CTAS). A sample of Argentinean undergraduates responded to several scales administered at three points. At time 1 and 3, the…

Examining construct and predictive validity of the Health-IT Usability Evaluation Scale: confirmatory factor analysis and structural equation modeling results

PubMed Central

Yen, Po-Yin; Sousa, Karen H; Bakken, Suzanne

2014-01-01

Background In a previous study, we developed the Health Information Technology Usability Evaluation Scale (Health-ITUES), which is designed to support customization at the item level. Such customization matches the specific tasks/expectations of a health IT system while retaining comparability at the construct level, and provides evidence of its factorial validity and internal consistency reliability through exploratory factor analysis. Objective In this study, we advanced the development of Health-ITUES to examine its construct validity and predictive validity. Methods The health IT system studied was a web-based communication system that supported nurse staffing and scheduling. Using Health-ITUES, we conducted a cross-sectional study to evaluate users’ perception toward the web-based communication system after system implementation. We examined Health-ITUES's construct validity through first and second order confirmatory factor analysis (CFA), and its predictive validity via structural equation modeling (SEM). Results The sample comprised 541 staff nurses in two healthcare organizations. The CFA (n=165) showed that a general usability factor accounted for 78.1%, 93.4%, 51.0%, and 39.9% of the explained variance in ‘Quality of Work Life’, ‘Perceived Usefulness’, ‘Perceived Ease of Use’, and ‘User Control’, respectively. The SEM (n=541) supported the predictive validity of Health-ITUES, explaining 64% of the variance in intention for system use. Conclusions The results of CFA and SEM provide additional evidence for the construct and predictive validity of Health-ITUES. The customizability of Health-ITUES has the potential to support comparisons at the construct level, while allowing variation at the item level. We also illustrate application of Health-ITUES across stages of system development. PMID:24567081

Reliability and Validity of the Self-Efficacy for Exercise in Epilepsy and the Outcome Expectations for Exercise in Epilepsy Scales.

PubMed

Dustin, Irene; Resnick, Barbara; Galik, Elizabeth; Klinedinst, N Jennifer; Michael, Kathleen; Wiggs, Edythe

2017-04-01

The purpose of this study was to test the psychometric properties of the revised Self-Efficacy for Exercise With Epilepsy (SEE-E) and Outcome Expectations for Exercise with Epilepsy (OEE-E) when used with people with epilepsy. The SEE-E and OEE-E were given in face-to-face interviews to 26 persons with epilepsy in an epilepsy clinic. There was some evidence of validity based on Rasch analysis INFIT and OUTFIT statistics. There was some evidence of reliability for the SEE-E and OEE-E based on person and item separation reliability indexes. These measures can be used to identify persons with epilepsy who have low self-efficacy and outcome expectations for exercise and guide design of interventions to strengthen these expectations and thereby improve exercise behavior.

Recommendations for elaboration, transcultural adaptation and validation process of tests in Speech, Hearing and Language Pathology.

PubMed

Pernambuco, Leandro; Espelt, Albert; Magalhães, Hipólito Virgílio; Lima, Kenio Costa de

2017-06-08

to present a guide with recommendations for translation, adaptation, elaboration and process of validation of tests in Speech and Language Pathology. the recommendations were based on international guidelines with a focus on the elaboration, translation, cross-cultural adaptation and validation process of tests. the recommendations were grouped into two Charts, one of them with procedures for translation and transcultural adaptation and the other for obtaining evidence of validity, reliability and measures of accuracy of the tests. a guide with norms for the organization and systematization of the process of elaboration, translation, cross-cultural adaptation and validation process of tests in Speech and Language Pathology was created.

Clinical Evidence: a useful tool for promoting evidence-based practice?

PubMed

Formoso, Giulio; Moja, Lorenzo; Nonino, Francesco; Dri, Pietro; Addis, Antonio; Martini, Nello; Liberati, Alessandro

2003-12-23

Research has shown that many healthcare professionals have problems with guidelines as they would prefer to be given all relevant information relevant to decision-making rather than being told what they should do. This study assesses doctors' judgement of the validity, relevance, clarity and usability of the Italian translation of Clinical Evidence (CE) after its free distribution launched by the Italian Ministry of Health. Opinions elicited using a standardised questionnaire delivered either by mail or during educational or professional meetings. Twenty percent (n = 1350) doctors participated the study. Most of them found CE's content valid, useful and relevant for their clinical practice, and said CE can foster communications among clinicians, particularly among GPs and specialists. Hospital doctors (63%) more often than GPs (48%) read the detailed presentation of individual chapters. Twenty-nine percent said CE brought changes in their clinical practice. Doctors appreciated CE's nature of an evidence-based information compendium and would have not preferred a collection of practice guidelines. Overall, the pilot initiative launched by the Italian Ministry of Health seems to have been well received and to support the subsequent decision to make the Italian edition of Clinical Evidence concise available to all doctors practising in the country. Local implementation initiatives should be warranted to favour doctor's use of CE.

Development and implementation of a virtual reality laparoscopic colorectal training curriculum.

PubMed

Wynn, Greg; Lykoudis, Panagis; Berlingieri, Pasquale

2017-12-12

Contemporary surgical training can be compromised by fewer practical opportunities. Simulation can fill this gap to optimize skills' development and progress monitoring. A structured virtual reality (VR) laparoscopic sigmoid colectomy curriculum is constructed and its validity and outcomes assessed. Parameters and thresholds were defined by analysing the performance of six expert surgeons completing the relevant module on the LAP Mentor simulator. Fourteen surgical trainees followed the curriculum, performance being recorded and analysed. Evidence of validity was assessed. Time to complete procedure, number of movements of right and left instrument, and total path length of right and left instrument movements demonstrated evidence of validity and clear learning curves, with a median of 14 attempts needed to complete the curriculum. A structured curriculum is proposed for training in laparoscopic sigmoid colectomy in a VR environment based on objective metrics in addition to expert consensus. Validity has been demonstrated for some key metrics. Copyright © 2017 Elsevier Inc. All rights reserved.

Development and Testing of the Nurse Manager EBP Competency Scale.

PubMed

Shuman, Clayton J; Ploutz-Snyder, Robert J; Titler, Marita G

2018-02-01

The purpose of this study was to develop and evaluate the validity and reliability of an instrument to measure nurse manager competencies regarding evidence-based practice (EBP). The Nurse Manager EBP Competency Scale consists of 16 items for respondents to indicate their perceived level of competency on a 0 to 3 Likert-type scale. Content validity was demonstrated through expert panel review and pilot testing. Principal axis factoring and Cronbach's alpha evaluated construct validity and internal consistency reliability, respectively. Eighty-three nurse managers completed the scale. Exploratory factor analysis resulted in a 16-item scale with two subscales, EBP Knowledge ( n = 6 items, α = .90) and EBP Activity ( n = 10 items, α = .94). Cronbach's alpha for the entire scale was .95. The Nurse Manager EBP Competency Scale is a brief measure of nurse manager EBP competency with evidence of validity and reliability. The scale can enhance our understanding in future studies regarding how nurse manager EBP competency affects implementation.

A systematic review of the measurement properties of the Body Image Scale (BIS) in cancer patients.

PubMed

Melissant, Heleen C; Neijenhuijs, Koen I; Jansen, Femke; Aaronson, Neil K; Groenvold, Mogens; Holzner, Bernhard; Terwee, Caroline B; van Uden-Kraan, Cornelia F; Cuijpers, Pim; Verdonck-de Leeuw, Irma M

2018-06-01

Body image is acknowledged as an important aspect of health-related quality of life in cancer patients. The Body Image Scale (BIS) is a patient-reported outcome measure (PROM) to evaluate body image in cancer patients. The aim of this study was to systematically review measurement properties of the BIS among cancer patients. A search in Embase, MEDLINE, PsycINFO, and Web of Science was performed to identify studies that investigated measurement properties of the BIS (Prospero ID 42017057237). Study quality was assessed (excellent, good, fair, poor), and data were extracted and analyzed according to the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology on structural validity, internal consistency, reliability, measurement error, hypothesis testing for construct validity, and responsiveness. Evidence was categorized into sufficient, insufficient, inconsistent, or indeterminate. Nine studies were included. Evidence was sufficient for structural validity (one factor solution), internal consistency (α = 0.86-0.96), and reliability (r > 0.70); indeterminate for measurement error (information on minimal important change lacked) and responsiveness (increasing body image disturbance in only one study); and inconsistent for hypothesis testing (conflicting results). Quality of the evidence was moderate to low. No studies reported on cross-cultural validity. The BIS is a PROM with good structural validity, internal consistency, and test-retest reliability, but good quality studies on the other measurement properties are needed to optimize evidence. It is recommended to include a wider variety of cancer diagnoses and treatment modalities in these future studies.

Evidence on existing caries risk assessment systems: are they predictive of future caries?

PubMed

Tellez, M; Gomez, J; Pretty, I; Ellwood, R; Ismail, A I

2013-02-01

To critically appraise evidence for the prediction of caries using four caries risk assessment (CRA) systems/guidelines (Cariogram, Caries Management by Risk Assessment (CAMBRA), American Dental Association (ADA), and American Academy of Pediatric Dentistry (AAPD)). This review focused on prospective cohort studies or randomized controlled trials. A systematic search strategy was developed to locate papers published in Medline Ovid and Cochrane databases. The search identified 539 scientific reports, and after title and abstract review, 137 were selected for full review and 14 met the following inclusion criteria: (i) used as validating criterion caries incidence/increment, (ii) involved human subjects and natural carious lesions, and (iii) published in peer-reviewed journals. In addition, papers were excluded if they met one or more of the following criteria: (i) incomplete description of sample selection, outcomes, or small sample size and (ii) not meeting the criteria for best evidence under the prognosis category of the Oxford Centre for Evidence-Based Medicine. There are wide variations among the systems in terms of definitions of caries risk categories, type and number of risk factors/markers, and disease indicators. The Cariogram combined sensitivity and specificity for predicting caries in permanent dentition ranges from 110 to 139 and is the only system for which prospective studies have been conducted to assess its validity. The Cariogram had limited prediction utility in preschool children, and a moderate to good performance for sorting out elderly individuals into caries risk groups. One retrospective analysis on CAMBRA's CRA reported higher incidence of cavitated lesions among those assessed as extreme-risk patients when compared with those at low risk. The evidence on the validity for existing systems for CRA is limited. It is unknown if the identification of high-risk individuals can lead to more effective long-term patient management that prevents caries initiation and arrests or reverses the progression of lesions. There is an urgent need to develop valid and reliable methods for caries risk assessment that are based on best evidence for prediction and disease management rather than opinions of experts.

Development of measurable indicators to enhance public health evidence-informed policy-making.

PubMed

Tudisca, Valentina; Valente, Adriana; Castellani, Tommaso; Stahl, Timo; Sandu, Petru; Dulf, Diana; Spitters, Hilde; Van de Goor, Ien; Radl-Karimi, Christina; Syed, Mohamed Ahmed; Loncarevic, Natasa; Lau, Cathrine Juel; Roelofs, Susan; Bertram, Maja; Edwards, Nancy; Aro, Arja R

2018-05-31

Ensuring health policies are informed by evidence still remains a challenge despite efforts devoted to this aim. Several tools and approaches aimed at fostering evidence-informed policy-making (EIPM) have been developed, yet there is a lack of availability of indicators specifically devoted to assess and support EIPM. The present study aims to overcome this by building a set of measurable indicators for EIPM intended to infer if and to what extent health-related policies are, or are expected to be, evidence-informed for the purposes of policy planning as well as formative and summative evaluations. The indicators for EIPM were developed and validated at international level by means of a two-round internet-based Delphi study conducted within the European project 'REsearch into POlicy to enhance Physical Activity' (REPOPA). A total of 82 researchers and policy-makers from the six European countries (Denmark, Finland, Italy, the Netherlands, Romania, the United Kingdom) involved in the project and international organisations were asked to evaluate the relevance and feasibility of an initial set of 23 indicators developed by REPOPA researchers on the basis of literature and knowledge gathered from the previous phases of the project, and to propose new indicators. The first Delphi round led to the validation of 14 initial indicators and to the development of 8 additional indicators based on panellists' suggestions; the second round led to the validation of a further 11 indicators, including 6 proposed by panellists, and to the rejection of 6 indicators. A total of 25 indicators were validated, covering EIPM issues related to human resources, documentation, participation and monitoring, and stressing different levels of knowledge exchange and involvement of researchers and other stakeholders in policy development and evaluation. The study overcame the lack of availability of indicators to assess if and to what extent policies are realised in an evidence-informed manner thanks to the active contribution of researchers and policy-makers. These indicators are intended to become a shared resource usable by policy-makers, researchers and other stakeholders, with a crucial impact on fostering the development of policies informed by evidence.

Assessing quality of maternity care in Hungary: expert validation and testing of the mother-centered prenatal care (MCPC) survey instrument.

PubMed

Rubashkin, Nicholas; Szebik, Imre; Baji, Petra; Szántó, Zsuzsa; Susánszky, Éva; Vedam, Saraswathi

2017-11-16

Instruments to assess quality of maternity care in Central and Eastern European (CEE) region are scarce, despite reports of poor doctor-patient communication, non-evidence-based care, and informal cash payments. We validated and tested an online questionnaire to study maternity care experiences among Hungarian women. Following literature review, we collated validated items and scales from two previous English-language surveys and adapted them to the Hungarian context. An expert panel assessed items for clarity and relevance on a 4-point ordinal scale. We calculated item-level Content Validation Index (CVI) scores. We designed 9 new items concerning informal cash payments, as well as 7 new "model of care" categories based on mode of payment. The final questionnaire (N = 111 items) was tested in two samples of Hungarian women, representative (N = 600) and convenience (N = 657). We conducted bivariate analysis and thematic analysis of open-ended responses. Experts rated pre-existing English-language items as clear and relevant to Hungarian women's maternity care experiences with an average CVI for included questions of 0.97. Significant differences emerged across the model of care categories in terms of informal payments, informed consent practices, and women's perceptions of autonomy. Thematic analysis (N = 1015) of women's responses identified 13 priority areas of the maternity care experience, 9 of which were addressed by the questionnaire. We developed and validated a comprehensive questionnaire that can be used to evaluate respectful maternity care, evidence-based practice, and informal cash payments in CEE region and beyond.

Coherent Governance: A Board-Superintendent Relationship Based on Defined Goals Can Raise Achievement

ERIC Educational Resources Information Center

Dawson, Linda J.; Quinn, Randy

2004-01-01

To defend this nation's chosen system of lay governance of public schools, it is necessary first to assume a direct relationship exists between what happens in the board room and what happens in the classroom. Evidence abounds that the assumption is valid. Unfortunately, much of that evidence is negative. For example, in far too many school…

Human Judgment and the Logic of Evidence: A Critical Examination of Research Methods in Special Education Transition Literature.

ERIC Educational Resources Information Center

Harnisch, Delwyn L.; And Others

This paper describes several common types of research studies in special education transition literature and the threats to their validity. It then describes how the evidential base may be broadened, how diverse sources of evidence can be combined to strengthen causal inferences, and the role of judgment within quasi-experimentation. The paper…

The influence of FMRI lie detection evidence on juror decision-making.

PubMed

McCabe, David P; Castel, Alan D; Rhodes, Matthew G

2011-01-01

In the current study, we report on an experiment examining whether functional magnetic resonance imaging (fMRI) lie detection evidence would influence potential jurors' assessment of guilt in a criminal trial. Potential jurors (N = 330) read a vignette summarizing a trial, with some versions of the vignette including lie detection evidence indicating that the defendant was lying about having committed the crime. Lie detector evidence was based on evidence from the polygraph, fMRI (functional brain imaging), or thermal facial imaging. Results showed that fMRI lie detection evidence led to more guilty verdicts than lie detection evidence based on polygraph evidence, thermal facial imaging, or a control condition that did not include lie detection evidence. However, when the validity of the fMRI lie detection evidence was called into question on cross-examination, guilty verdicts were reduced to the level of the control condition. These results provide important information about the influence of lie detection evidence in legal settings. Copyright © 2011 John Wiley & Sons, Ltd.

Capability beliefs regarding evidence-based practice are associated with application of EBP and research use: validation of a new measure.

PubMed

Wallin, Lars; Boström, Anne-Marie; Gustavsson, J Petter

2012-08-01

Beliefs about capabilities, or self-efficacy, is a construct originating in social cognitive psychology. Capability beliefs have been found to be positively associated with intention and healthcare practice behaviour. A measure of an individual's beliefs about his/her capability to apply the components of evidence-based practice (EBP) has potential to be useful in implementation research. To evaluate the concurrent validity and internal structure of a new scale measuring nurses' capability beliefs regarding EBP. Data were taken from a prospective longitudinal study in Sweden (the Longitudinal Analyses of Nursing Education and Entry in Worklife [LANE]). A cohort of nursing students who graduated in the autumn of 2004 that was followed up 2 years after their graduation was used (n= 1,256). Concurrent validity was tested relating different levels of capability beliefs to extent of research use and application of EBP. An item-response approach was applied in the evaluation of internal structure of the proposed scale (six items). The psychometric analyses indicated that the six items could be summed to reflect a one-dimensional scale. Nurses with the highest level of capability beliefs reported that they used research findings in clinical practice more than twice as often as those with lower levels of capability beliefs. They also participated in the implementation of evidence seven times more often. There is a need for further studies of the construct and predictive validity of the scale. It should also be validated in other groups of health professionals. Learning including mastery experiences, role modelling, social persuasion, and manageable stress could be used in undergraduate education as well as practice development to increase beliefs about capabilities which might open the way to increased application of EBP in healthcare practice. This new measure is well grounded in social cognitive theory, functions as a one-dimensional scale and possesses promising properties of concurrent validity. ©2012 Sigma Theta Tau International.

Appearance motives to tan and not tan: evidence for validity and reliability of a new scale.

PubMed

Cafri, Guy; Thompson, J Kevin; Roehrig, Megan; Rojas, Ariz; Sperry, Steffanie; Jacobsen, Paul B; Hillhouse, Joel

2008-04-01

Risk for skin cancer is increased by UV exposure and decreased by sun protection. Appearance reasons to tan and not tan have consistently been shown to be related to intentions and behaviors to UV exposure and protection. This study was designed to determine the factor structure of appearance motives to tan and not tan, evaluate the extent to which this factor structure is gender invariant, test for mean differences in the identified factors, and evaluate internal consistency, temporal stability, and criterion-related validity. Five-hundred eighty-nine females and 335 male college students were used to test confirmatory factor analysis models within and across gender groups, estimate latent mean differences, and use the correlation coefficient and Cronbach's alpha to further evaluate the reliability and validity of the identified factors. A measurement invariant (i.e., factor-loading invariant) model was identified with three higher-order factors: sociocultural influences to tan (lower order factors: media, friends, family, significant others), appearance reasons to tan (general, acne, body shape), and appearance reasons not to tan (skin aging, immediate skin damage). Females had significantly higher means than males on all higher-order factors. All subscales had evidence of internal consistency, temporal stability, and criterion-related validity. This study offers a framework and measurement instrument that has evidence of validity and reliability for evaluating appearance-based motives to tan and not tan.

Development and Feasibility of a Virtual Reality Task for the Cognitive Assessment of Older Adults: The ECO-VR.

PubMed

Oliveira, Camila R; Lopes Filho, Brandel José P; Sugarman, Michael A; Esteves, Cristiane S; Lima, Margarida Maria B M P; Moret-Tatay, Carmen; Irigaray, Tatiana Q; Argimon, Irani Iracema L

2016-12-13

Cognitive assessment with virtual reality (VR) may have superior ecological validity for older adults compared to traditional pencil-and-paper cognitive assessment. However, few studies have reported the development of VR tasks. The aim of this study was to present the development, feasibility, content validity, and preliminary evidence of construct validity of an ecological task of cognitive assessment for older adults in VR (ECO-VR). The tasks were prepared based on theoretical and clinical backgrounds. We had 29 non-expert judges identify virtual visual stimuli and three-dimensional scenarios, and five expert judges assisted with content analysis and developing instructions. Finally, six older persons participated in three pilot studies and thirty older persons participated in the preliminary study to identify construct validity evidence. Data were analyzed by descriptive statistics and partial correlation. Target stimuli and three-dimensional scenarios were judged adequate and the content analysis demonstrated that ECO-VR evaluates temporo-spatial orientation, memory, language and executive functioning. We made significant changes to the instructions after the pilot studies to increase comprehensibility and reduce the completion time. The total score of ECO-VR was positively correlated mainly with performance in executive function (r = .172, p < .05) and memory tests (r = .488, p ≤ .01). The ECO-VR demonstrated feasibility for cognitive assessment in older adults, as well as content and construct validity evidences.

Do evidence-based active-engagement courses reduce the gender gap in introductory physics?

NASA Astrophysics Data System (ADS)

Karim, Nafis I.; Maries, Alexandru; Singh, Chandralekha

2018-03-01

Prior research suggests that using evidence-based pedagogies can not only improve learning for all students, it can also reduce the gender gap. We describe the impact of physics education research-based pedagogical techniques in flipped and active-engagement non-flipped courses on the gender gap observed with validated conceptual surveys. We compare male and female students’ performance in courses which make significant use of evidence-based active-engagement (EBAE) strategies with courses that primarily use lecture-based (LB) instruction. All courses had large enrolment and often had more than 100 students. The analysis of data for validated conceptual surveys presented here includes data from two-semester sequences of algebra-based and calculus-based introductory physics courses. The conceptual surveys used to assess student learning in the first and second semester courses were the force concept inventory and the conceptual survey of electricity and magnetism, respectively. In the research discussed here, the performance of male and female students in EBAE courses at a particular level is compared with LB courses in two situations: (I) the same instructor taught two courses, one of which was an EBAE course and the other an LB course, while the homework, recitations and final exams were kept the same; (II) student performance in all of the EBAE courses taught by different instructors was averaged and compared with LB courses of the same type also averaged over different instructors. In all cases, on conceptual surveys we find that students in courses which make significant use of active-engagement strategies, on average, outperformed students in courses of the same type using primarily lecture-based instruction even though there was no statistically significant difference on the pre-test before instruction. However, the gender gap persisted even in courses using EBAE methods. We also discuss correlations between the performance of male and female students on the validated conceptual surveys and the final exam, which had a heavy weight on quantitative problem solving.

Development of a Problem-Focused Behavioral Screener Linked to Evidence-Based Intervention

ERIC Educational Resources Information Center

Daniels, Brian; Volpe, Robert J.; Briesch, Amy M.; Fabiano, Gregory A.

2014-01-01

This study examines the factor structure, reliability and validity of a novel school-based screening instrument for academic and disruptive behavior problems commonly experienced by children and adolescents with attention deficit hyperactivity disorder (ADHD). Participants included 39 classroom teachers from two public school districts in the…

Cognitive Integrity Predicts Transitive Inference Performance Bias and Success

ERIC Educational Resources Information Center

Moses, Sandra N.; Villate, Christina; Binns, Malcolm A.; Davidson, Patrick S. R.; Ryan, Jennifer D.

2008-01-01

Transitive inference has traditionally been regarded as a relational proposition-based reasoning task, however, recent investigations question the validity of this assumption. Although some results support the use of a relational proposition-based approach, other studies find evidence for the use of associative learning. We examined whether…

On-the-Job Evidence-Based Medicine Training for Clinician-Scientists of the Next Generation

PubMed Central

Leung, Elaine YL; Malick, Sadia M; Khan, Khalid S

2013-01-01

Clinical scientists are at the unique interface between laboratory science and frontline clinical practice for supporting clinical partnerships for evidence-based practice. In an era of molecular diagnostics and personalised medicine, evidence-based laboratory practice (EBLP) is also crucial in aiding clinical scientists to keep up-to-date with this expanding knowledge base. However, there are recognised barriers to the implementation of EBLP and its training. The aim of this review is to provide a practical summary of potential strategies for training clinician-scientists of the next generation. Current evidence suggests that clinically integrated evidence-based medicine (EBM) training is effective. Tailored e-learning EBM packages and evidence-based journal clubs have been shown to improve knowledge and skills of EBM. Moreover, e-learning is no longer restricted to computer-assisted learning packages. For example, social media platforms such as Twitter have been used to complement existing journal clubs and provide additional post-publication appraisal information for journals. In addition, the delivery of an EBLP curriculum has influence on its success. Although e-learning of EBM skills is effective, having EBM trained teachers available locally promotes the implementation of EBM training. Training courses, such as Training the Trainers, are now available to help trainers identify and make use of EBM training opportunities in clinical practice. On the other hand, peer-assisted learning and trainee-led support networks can strengthen self-directed learning of EBM and research participation among clinical scientists in training. Finally, we emphasise the need to evaluate any EBLP training programme using validated assessment tools to help identify the most crucial ingredients of effective EBLP training. In summary, we recommend on-the-job training of EBM with additional focus on overcoming barriers to its implementation. In addition, future studies evaluating the effectiveness of EBM training should use validated outcome tools, endeavour to achieve adequate power and consider the effects of EBM training on learning environment and patient outcomes. PMID:24151345

On-the-Job Evidence-Based Medicine Training for Clinician-Scientists of the Next Generation.

PubMed

Leung, Elaine Yl; Malick, Sadia M; Khan, Khalid S

2013-08-01

Clinical scientists are at the unique interface between laboratory science and frontline clinical practice for supporting clinical partnerships for evidence-based practice. In an era of molecular diagnostics and personalised medicine, evidence-based laboratory practice (EBLP) is also crucial in aiding clinical scientists to keep up-to-date with this expanding knowledge base. However, there are recognised barriers to the implementation of EBLP and its training. The aim of this review is to provide a practical summary of potential strategies for training clinician-scientists of the next generation. Current evidence suggests that clinically integrated evidence-based medicine (EBM) training is effective. Tailored e-learning EBM packages and evidence-based journal clubs have been shown to improve knowledge and skills of EBM. Moreover, e-learning is no longer restricted to computer-assisted learning packages. For example, social media platforms such as Twitter have been used to complement existing journal clubs and provide additional post-publication appraisal information for journals. In addition, the delivery of an EBLP curriculum has influence on its success. Although e-learning of EBM skills is effective, having EBM trained teachers available locally promotes the implementation of EBM training. Training courses, such as Training the Trainers, are now available to help trainers identify and make use of EBM training opportunities in clinical practice. On the other hand, peer-assisted learning and trainee-led support networks can strengthen self-directed learning of EBM and research participation among clinical scientists in training. Finally, we emphasise the need to evaluate any EBLP training programme using validated assessment tools to help identify the most crucial ingredients of effective EBLP training. In summary, we recommend on-the-job training of EBM with additional focus on overcoming barriers to its implementation. In addition, future studies evaluating the effectiveness of EBM training should use validated outcome tools, endeavour to achieve adequate power and consider the effects of EBM training on learning environment and patient outcomes.

Validity and usability of a professional association's web-based knowledge translation portal: American Physical Therapy Association's PTNow.org.

PubMed

Deutsch, Judith E; Romney, Wendy; Reynolds, Jan; Manal, Tara Jo

2015-10-08

PTNow.org is an evidence-based, on-line portal created by a professional membership association to promote use of evidence in practice and to help decrease unwarranted variation in practice. The site contains synthesis documents designed to promote efficient clinical reasoning. These documents were written and peer-reviewed by teams of content experts and master clinicians. The purpose of this paper is to report on the content and construct validity as well as usability of the site. Physical therapist participants used clinical summaries (available in 3 formats--as a full summary with hyperlinks, "quick takes" with hyperlinks, and a portable two-page version) on the PTNow.org site to answer knowledge acquisition and clinical reasoning questions related to four patient scenarios. They also responded to questions about ease of use related to website navigation and about format and completeness of information using a 1-5 Likert scale. Responses were coded to reflect how participants used the site and then were summarized descriptively. Preferences for clinical summary format were analyzed using an analysis of variance (ANOVA) and a Dunnett T3 post hoc analysis. Seventeen participants completed the study. Clinical relevance and completeness ratings by experienced clinicians, which were used as the measure of content validity, ranged from 3.1 to 4.6 on a 5 point scale. Construct validity based on the information on the PTNow.org site was supported for knowledge acquisition questions 66 % of the time and for clinical reasoning questions 40 % of the time. Usability ratings for the full clinical summary were 4.6 (1.2); for the quick takes, 3.5 (.98); and for the portable clinical summary, 4.0 (.45). Participants preferred the full clinical summary over the other two formats (F = 5.908, P = 0.007). One hundred percent of the participants stated that they would recommend the PTNow site to their colleagues. Prelimary evidence supported both content validity and construct validity of knowledge acquisition, and partially supported construct validity of clinical reasoning for the clinical summaries on the PTNow.org site. Usability was supported, with users preferring the full clinical summary over the other two formats. Iterative design is ongoing.

Evidence-Based Review of Subjective Pediatric Sleep Measures

PubMed Central

Toliver-Sokol, Marisol; Palermo, Tonya M.

2011-01-01

Objective This manuscript provides an evidence-based psychometric review of parent and child-report pediatric sleep measures using criteria developed by the American Psychological Association (APA) Division 54 Evidence-Based Assessment (EBA) Task Force. Methods Twenty-one measures were reviewed: four measures of daytime sleepiness, four measures of sleep habits/hygiene, two measures assessing sleep-related attitudes/cognitions, five measures of sleep initiation/maintenance, and six multidimensional sleep measures. Results Six of the 21 measures met “well-established” evidence-based assessment criteria. An additional eight measures were rated as “approaching well-established” and seven were rated as “promising.” Conclusions Overall, the multidimensional sleep measures received the highest ratings. Strengths and weaknesses of the measures are described. Recommendations for future pediatric sleep assessment are presented including further validation of measures, use of multiple informants, and stability of sleep measures over time. PMID:21227912

Evidence-based assessment in pediatric psychology: family measures.

PubMed

Alderfer, Melissa A; Fiese, Barbara H; Gold, Jeffrey I; Cutuli, J J; Holmbeck, Grayson N; Goldbeck, Lutz; Chambers, Christine T; Abad, Mona; Spetter, Dante; Patterson, Joän

2008-10-01

To provide a review of the evidence base of family measures relevant to pediatric psychology. Twenty-nine family measures were selected based upon endorsement by Division 54 listserv members, expert judgment, and literature review. Spanning observational and self-report methods, the measures fell into three broad assessment categories: Family functioning, Dyadic family relationships, and Family functioning in the context of childhood chronic health conditions. Measures were categorized as: "Well-established", "Approaching well-established", or "Promising." Nineteen measures met "well-established" criteria and the remaining ten were "approaching well-established." "Well-established" measures were documented for each of the broad assessment categories named above. Many measures deemed "well-established" in the general population are proving to be reliable and useful in pediatric samples. More evidence of the validity of family measures is needed in this context. This review should prove helpful to clinicians and researchers as they strive to make evidence-based decisions regarding family measures.

Developing the skills required for evidence-based practice.

PubMed

French, B

1998-01-01

The current health care environment requires practitioners with the skills to find and apply the best currently available evidence for effective health care, to contribute to the development of evidence-based practice protocols, and to evaluate the impact of utilizing validated research findings in practice. Current approaches to teaching research are based mainly on gaining skills by participation in the research process. Emphasis on the requirement for rigour in the process of creating new knowledge is assumed to lead to skill in the process of using research information created by others. This article reflects upon the requirements for evidence-based practice, and the degree to which current approaches to teaching research prepare practitioners who are able to find, evaluate and best use currently available research information. The potential for using the principles of systematic review as a teaching and learning strategy for research is explored, and some of the possible strengths and weakness of this approach are highlighted.

Does the Test Work? Evaluating a Web-Based Language Placement Test

ERIC Educational Resources Information Center

Long, Avizia Y.; Shin, Sun-Young; Geeslin, Kimberly; Willis, Erik W.

2018-01-01

In response to the need for examples of test validation from which everyday language programs can benefit, this paper reports on a study that used Bachman's (2005) assessment use argument (AUA) framework to examine evidence to support claims made about the intended interpretations and uses of scores based on a new web-based Spanish language…

Measurement properties of rheumatoid arthritis-specific quality-of-life questionnaires: systematic review of the literature.

PubMed

Lee, Jiyeon; Kim, Soo Hyun; Moon, Seung Hei; Lee, Eun-Hyun

2014-12-01

This study conducted a systematic review of the methodological quality of the psychometric evaluation process and the quality of measurement properties of rheumatoid arthritis (RA)-specific health-related quality-of-life (HRQOL) questionnaires with the purpose of obtaining the best evidence to help in the selection of the most appropriate instrument for measuring HRQOL in RA patients. A systematic literature search was performed to identify RA-specific HRQOL questionnaires in databases. The methodological quality of the studies was assessed using the Consensus-based Standards for the Selection of Health Measurement Instruments checklist. The quality of the measurement properties was assessed using quality criteria. The evidence regarding the measurement properties was pooled using best-evidence synthesis, with considerations of the number and methodological quality of the studies, and the consistency of their findings in terms of the quality of the measurement properties. The search identified 37 studies describing 9 instruments. Best-evidence synthesis suggested that the Rheumatoid Arthritis Quality of Life (RAQoL) questionnaire had the strongest positive evidence, especially with respect to reliability, measurement error, and content validity, and moderate positive evidence with respect to hypothesis testing and responsiveness. The current evidence suggests that the best-validated instrument among the RA-specific HRQOL measures is the RAQoL questionnaire in terms of both methodological quality in the process of psychometric evaluation and the quality of the measurement properties. However, there is limited evidence regarding internal consistency and structural validity of the RAQoL. Further efforts are warranted to establish the psychometric quality of this questionnaire.

A ubiquitous but ineffective intervention: Signs do not increase hand hygiene compliance.

PubMed

Birnbach, David J; Rosen, Lisa F; Fitzpatrick, Maureen; Everett-Thomas, Ruth; Arheart, Kristopher L

Proper hand hygiene is critical for preventing healthcare-associated infection, but provider compliance remains suboptimal. While signs are commonly used to remind physicians and nurses to perform hand hygiene, the content of these signs is rarely based on specific, validated health behavior theories. This observational study assessed the efficacy of a hand hygiene sign disseminated by the Centers for Disease Control and Prevention in an intensive care unit compared to an optimized evidence-based sign designed by a team of patient safety experts. The optimized sign was developed by four patient safety experts to include known evidence-based components and was subsequently validated by surveying ten physicians and ten nurses using a 10 point Likert scale. Eighty-two physicians and 98 nurses (102 females; 78 males) were observed for hand hygiene (HH) compliance, and the total HH compliance rate was 16%. HH compliance was not significantly different among the signs (Baseline 10% vs. CDC 18% vs. OIS 20%; p=0.280). The findings of this study suggest that even when the content and design of a hand hygiene reminder sign incorporates evidence-based constructs, healthcare providers comply only a fraction of the time. Copyright © 2016 King Saud Bin Abdulaziz University for Health Sciences. Published by Elsevier Ltd. All rights reserved.

Dose-response relationships in multifunctional food design: assembling the evidence.

PubMed

Aggett, Peter J

2012-03-01

Demonstrating single and multiple functions attributable to foods or specific food components is a challenge. The International Life Sciences Institute Europe co-ordinated EU concerted actions, Functional Food Science in Europe (FUFOSE) and the Process for the Assessment of Scientific Support for Claims on Food (PASSCLAIM), respectively, addressed the soundness of the evidence and its coherence with a mechanistic schema comprising valid markers of exposure, intermediate and final outcomes and the quality and integrity of the evidence overall. Demonstrating causality often relies on randomized controlled trials (RCTs). However, in public health and biomedical science there is concern about the suitability of RCTs as sole standards of evidence-based approaches. Alternative and complementary approaches using updated Hill's viewpoints for appraising the evidence can be used in conjunction with evidence-based mechanistic reasoning and the quality criteria proposed in FUFOSE and PASSCLAIM to design studies and to assemble evidence exploring single or multiple benefits from food components and foods.

Reach Out and Write Someone.

ERIC Educational Resources Information Center

Arnold, Vanessa D.; Roach, Terry D.

1993-01-01

Writing letters to elected officials and letters to the editor helps students articulate their thoughts based on sound evidence and valid reasoning, avoiding "sounding off" and emotional appeals. Writing skills, critical thinking, and civic values are reinforced. (SK)

Development and validation of Dutch version of Lasater Clinical Judgment Rubric in hospital practice: An instrument design study.

PubMed

Vreugdenhil, Jettie; Spek, Bea

2018-03-01

Clinical reasoning in patient care is a skill that cannot be observed directly. So far, no reliable, valid instrument exists for the assessment of nursing students' clinical reasoning skills in hospital practice. Lasater's clinical judgment rubric (LCJR), based on Tanner's model "Thinking like a nurse" has been tested, mainly in academic simulation settings. The aim is to develop a Dutch version of the LCJR (D-LCJR) and to test its psychometric properties when used in a hospital traineeship context. A mixed-model approach was used to develop and to validate the instrument. Ten dedicated educational units in a university hospital. A well-mixed group of 52 nursing students, nurse coaches and nurse educators. A Delphi panel developed the D-LCJR. Students' clinical reasoning skills were assessed "live" by nurse coaches, nurse educators and students who rated themselves. The psychometric properties tested during the assessment process are reliability, reproducibility, content validity and construct validity by testing two hypothesis: 1) a positive correlation between assessed and self-reported sum scores (convergent validity) and 2) a linear relation between experience and sum score (clinical validity). The obtained D-LCJR was found to be internally consistent, Cronbach's alpha 0.93. The rubric is also reproducible with intraclass correlations between 0.69 and 0.78. Experts judged it to be content valid. The two hypothesis were both tested significant, supporting evidence for construct validity. The translated and modified LCJR, is a promising tool for the evaluation of nursing students' development in clinical reasoning in hospital traineeships, by students, nurse coaches and nurse educators. More evidence on construct validity is necessary, in particular for students at the end of their hospital traineeship. Based on our research, the D-LCJR applied in hospital traineeships is a usable and reliable tool. Copyright © 2017 Elsevier Ltd. All rights reserved.

A clinical reasoning model focused on clients' behaviour change with reference to physiotherapists: its multiphase development and validation.

PubMed

Elvén, Maria; Hochwälder, Jacek; Dean, Elizabeth; Söderlund, Anne

2015-05-01

A biopsychosocial approach and behaviour change strategies have long been proposed to serve as a basis for addressing current multifaceted health problems. This emphasis has implications for clinical reasoning of health professionals. This study's aim was to develop and validate a conceptual model to guide physiotherapists' clinical reasoning focused on clients' behaviour change. Phase 1 consisted of the exploration of existing research and the research team's experiences and knowledge. Phases 2a and 2b consisted of validation and refinement of the model based on input from physiotherapy students in two focus groups (n = 5 per group) and from experts in behavioural medicine (n = 9). Phase 1 generated theoretical and evidence bases for the first version of a model. Phases 2a and 2b established the validity and value of the model. The final model described clinical reasoning focused on clients' behaviour change as a cognitive, reflective, collaborative and iterative process with multiple interrelated levels that included input from the client and physiotherapist, a functional behavioural analysis of the activity-related target behaviour and the selection of strategies for behaviour change. This unique model, theory- and evidence-informed, has been developed to help physiotherapists to apply clinical reasoning systematically in the process of behaviour change with their clients.

Pediatric bipolar disorder: validity, phenomenology, and recommendations for diagnosis

PubMed Central

Youngstrom, Eric A; Birmaher, Boris; Findling, Robert L

2013-01-01

Objective To find, review, and critically evaluate evidence pertaining to the phenomenology of pediatric bipolar disorder and its validity as a diagnosis. Methods The present qualitative review summarizes and synthesizes available evidence about the phenomenology of bipolar disorder (BD) in youths, including description of the diagnostic sensitivity and specificity of symptoms, clarification about rates of cycling and mixed states, and discussion about chronic versus episodic presentations of mood dysregulation. The validity of the diagnosis of BD in youths is also evaluated based on traditional criteria including associated demographic characteristics, family environmental features, genetic bases, longitudinal studies of youths at risk of developing BD as well as youths already manifesting symptoms on the bipolar spectrum, treatment studies and pharmacologic dissection, neurobiological findings (including morphological and functional data), and other related laboratory findings. Additional sections review impairment and quality of life, personality and temperamental correlates, the clinical utility of a bipolar diagnosis in youths, and the dimensional versus categorical distinction as it applies to mood disorder in youths. Results A schema for diagnosis of BD in youths is developed, including a review of different operational definitions of `bipolar not otherwise specified.' Principal areas of disagreement appear to include the relative role of elated versus irritable mood in assessment, and also the limits of the extent of the bipolar spectrum – when do definitions become so broad that they are no longer describing `bipolar' cases? Conclusions In spite of these areas of disagreement, considerable evidence has amassed supporting the validity of the bipolar diagnosis in children and adolescents. PMID:18199237

Perils of Pragmatic Psychiatry: How We Can Do Better

PubMed Central

Koola, Maju Mathew; Sebastian, Joseph

2016-01-01

Etiologic and pathophysiologic understanding of psychiatric disorders is still in its early stages. The neurobiology of major psychiatric disorders has yet to be fully elucidated. Psychiatric diagnoses are often based on presenting symptoms, lacking reliability and stability. For a variety of reasons, many notable laboratory and clinical observations have not been tested in large trials. Lacking this validation, these potentially valuable practices have not been widely disseminated nor translated into real world practice. Pragmatic practice today requires optimum use of the available resources. This may sometimes require translating novel treatments supported by strong, evidence-based, level II evidence; but still lacking level I evidence into practice and greater utilization of evidence-based approved practices. The purpose of this paper is to highlight some common avoidable pitfalls in practice, and to offer a few psychopharmacological pearls. PMID:26998529

Validity and reliability of portfolio assessment of student competence in two dental school populations: a four-year study.

PubMed

Gadbury-Amyot, Cynthia C; McCracken, Michael S; Woldt, Janet L; Brennan, Robert L

2014-05-01

The purpose of this study was to empirically investigate the validity and reliability of portfolio assessment in two U.S. dental schools using a unified framework for validity. In the process of validation, it is not the test that is validated but rather the claims (interpretations and uses) about test scores that are validated. Kane's argument-based validation framework provided the structure for reporting results where validity claims are followed by evidence to support the argument. This multivariate generalizability theory study found that the greatest source of variance was attributable to faculty raters, suggesting that portfolio assessment would benefit from two raters' evaluating each portfolio independently. The results are generally supportive of holistic scoring, but analytical scoring deserves further research. Correlational analyses between student portfolios and traditional measures of student competence and readiness for licensure resulted in significant correlations between portfolios and National Board Dental Examination Part I (r=0.323, p<0.01) and Part II scores (r=0.268, p<0.05) and small and non-significant correlations with grade point average and scores on the Western Regional Examining Board (WREB) exam. It is incumbent upon the users of portfolio assessment to determine if the claims and evidence arguments set forth in this study support the proposed claims for and decisions about portfolio assessment in their respective institutions.

Development and initial validation of a cognitive-based work-nonwork conflict scale.

PubMed

Ezzedeen, Souha R; Swiercz, Paul M

2007-06-01

Current research related to work and life outside work specifies three types of work-nonwork conflict: time, strain, and behavior-based. Overlooked in these models is a cognitive-based type of conflict whereby individuals experience work-nonwork conflict from cognitive preoccupation with work. Four studies on six different groups (N=549) were undertaken to develop and validate an initial measure of this construct. Structural equation modeling confirmed a two-factor, nine-item scale. Hypotheses regarding cognitive-based conflict's relationship with life satisfaction, work involvement, work-nonwork conflict, and work hours were supported. The relationship with knowledge work was partially supported in that only the cognitive dimension of cognitive-based conflict was related to extent of knowledge work. Hypotheses regarding cognitive-based conflict's relationship with family demands were rejected in that the cognitive dimension correlated positively rather than negatively with number of dependent children and perceived family demands. The study provides encouraging preliminary evidence of scale validity.

Simulation-based assessment to identify critical gaps in safe anesthesia resident performance.

PubMed

Blum, Richard H; Boulet, John R; Cooper, Jeffrey B; Muret-Wagstaff, Sharon L

2014-01-01

Valid methods are needed to identify anesthesia resident performance gaps early in training. However, many assessment tools in medicine have not been properly validated. The authors designed and tested use of a behaviorally anchored scale, as part of a multiscenario simulation-based assessment system, to identify high- and low-performing residents with regard to domains of greatest concern to expert anesthesiology faculty. An expert faculty panel derived five key behavioral domains of interest by using a Delphi process (1) Synthesizes information to formulate a clear anesthetic plan; (2) Implements a plan based on changing conditions; (3) Demonstrates effective interpersonal and communication skills with patients and staff; (4) Identifies ways to improve performance; and (5) Recognizes own limits. Seven simulation scenarios spanning pre-to-postoperative encounters were used to assess performances of 22 first-year residents and 8 fellows from two institutions. Two of 10 trained faculty raters blinded to trainee program and training level scored each performance independently by using a behaviorally anchored rating scale. Residents, fellows, facilitators, and raters completed surveys. Evidence supporting the reliability and validity of the assessment scores was procured, including a high generalizability coefficient (ρ = 0.81) and expected performance differences between first-year resident and fellow participants. A majority of trainees, facilitators, and raters judged the assessment to be useful, realistic, and representative of critical skills required for safe practice. The study provides initial evidence to support the validity of a simulation-based performance assessment system for identifying critical gaps in safe anesthesia resident performance early in training.

The role of simulation in continuing medical education for acute care physicians: a systematic review.

PubMed

Khanduja, P Kristina; Bould, M Dylan; Naik, Viren N; Hladkowicz, Emily; Boet, Sylvain

2015-01-01

We systematically reviewed the effectiveness of simulation-based education, targeting independently practicing qualified physicians in acute care specialties. We also describe how simulation is used for performance assessment in this population. Data source included: DataMEDLINE, Embase, Cochrane Database of Systematic Reviews, Cochrane CENTRAL Database of Controlled Trials, and National Health Service Economic Evaluation Database. The last date of search was January 31, 2013. All original research describing simulation-based education for independently practicing physicians in anesthesiology, critical care, and emergency medicine was reviewed. Data analysis was performed in duplicate with further review by a third author in cases of disagreement until consensus was reached. Data extraction was focused on effectiveness according to Kirkpatrick's model. For simulation-based performance assessment, tool characteristics and sources of validity evidence were also collated. Of 39 studies identified, 30 studies focused on the effectiveness of simulation-based education and nine studies evaluated the validity of simulation-based assessment. Thirteen studies (30%) targeted the lower levels of Kirkpatrick's hierarchy with reliance on self-reporting. Simulation was unanimously described as a positive learning experience with perceived impact on clinical practice. Of the 17 remaining studies, 10 used a single group or "no intervention comparison group" design. The majority (n = 17; 44%) were able to demonstrate both immediate and sustained improvements in educational outcomes. Nine studies reported the psychometric properties of simulation-based performance assessment as their sole objective. These predominantly recruited independent practitioners as a convenience sample to establish whether the tool could discriminate between experienced and inexperienced operators and concentrated on a single aspect of validity evidence. Simulation is perceived as a positive learning experience with limited evidence to support improved learning. Future research should focus on the optimal modality and frequency of exposure, quality of assessment tools and on the impact of simulation-based education beyond the individuals toward improved patient care.

Conducting a Surgical Site Infection Prevention Tracer.

PubMed

Padgette, Polly; Wood, Brittain

2018-05-01

Surgical site infections (SSIs) are the most common health care-associated infections in patients. Approximately half of SSIs are preventable when using evidence-based strategies; however, deviations from evidence-based practice can occur over time. Infection preventionists and perioperative staff members can help prevent these deviations by observing staff member practices using tracer methodology. Tracer methodology uses clinical information to follow patient care, treatment, or services provided throughout the care delivery system. The goal of tracer methodology for SSI prevention is to validate that organizational processes are promoting safer patient care. Using tracers, perioperative and infection prevention staff members can develop strategies to eliminate deviations from evidence-based practice, thereby helping to prevent SSIs and improve patient outcomes. © AORN, Inc, 2018.

Perceived Characteristics of Intervention Scale: Development and Psychometric Properties.

PubMed

Cook, Joan M; Thompson, Richard; Schnurr, Paula P

2015-12-01

The Perceived Characteristics of Intervention Scale (PCIS), a 20-item assessment measure, was developed to assess health care providers' views of interventions. Two hundred and fifteen Department of Veterans Affairs' residential treatment providers from 38 programs across the United States completed an online survey that included the PCIS as well as self-reported use of two evidence-based treatments. The PCIS was anchored to ask about two evidence-based psychotherapies for posttraumatic stress disorder, prolonged exposure, and cognitive processing therapy. The PCIS is a reliable measure of perceived characteristics of interventions, with some preliminary support for its validity. Consideration of providers' perceptions of particular evidence-based treatments may serve as an aid to improve their dissemination, implementation, and sustained use. © The Author(s) 2014.

Organizational readiness to change assessment (ORCA): Development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework

PubMed Central

Helfrich, Christian D; Li, Yu-Fang; Sharp, Nancy D; Sales, Anne E

2009-01-01

Background The Promoting Action on Research Implementation in Health Services, or PARIHS, framework is a theoretical framework widely promoted as a guide to implement evidence-based clinical practices. However, it has as yet no pool of validated measurement instruments that operationalize the constructs defined in the framework. The present article introduces an Organizational Readiness to Change Assessment instrument (ORCA), organized according to the core elements and sub-elements of the PARIHS framework, and reports on initial validation. Methods We conducted scale reliability and factor analyses on cross-sectional, secondary data from three quality improvement projects (n = 80) conducted in the Veterans Health Administration. In each project, identical 77-item ORCA instruments were administered to one or more staff from each facility involved in quality improvement projects. Items were organized into 19 subscales and three primary scales corresponding to the core elements of the PARIHS framework: (1) Strength and extent of evidence for the clinical practice changes represented by the QI program, assessed with four subscales, (2) Quality of the organizational context for the QI program, assessed with six subscales, and (3) Capacity for internal facilitation of the QI program, assessed with nine subscales. Results Cronbach's alpha for scale reliability were 0.74, 0.85 and 0.95 for the evidence, context and facilitation scales, respectively. The evidence scale and its three constituent subscales failed to meet the conventional threshold of 0.80 for reliability, and three individual items were eliminated from evidence subscales following reliability testing. In exploratory factor analysis, three factors were retained. Seven of the nine facilitation subscales loaded onto the first factor; five of the six context subscales loaded onto the second factor; and the three evidence subscales loaded on the third factor. Two subscales failed to load significantly on any factor. One measured resources in general (from the context scale), and one clinical champion role (from the facilitation scale). Conclusion We find general support for the reliability and factor structure of the ORCA. However, there was poor reliability among measures of evidence, and factor analysis results for measures of general resources and clinical champion role did not conform to the PARIHS framework. Additional validation is needed, including criterion validation. PMID:19594942

Organizational readiness to change assessment (ORCA): development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework.

PubMed

Helfrich, Christian D; Li, Yu-Fang; Sharp, Nancy D; Sales, Anne E

2009-07-14

The Promoting Action on Research Implementation in Health Services, or PARIHS, framework is a theoretical framework widely promoted as a guide to implement evidence-based clinical practices. However, it has as yet no pool of validated measurement instruments that operationalize the constructs defined in the framework. The present article introduces an Organizational Readiness to Change Assessment instrument (ORCA), organized according to the core elements and sub-elements of the PARIHS framework, and reports on initial validation. We conducted scale reliability and factor analyses on cross-sectional, secondary data from three quality improvement projects (n = 80) conducted in the Veterans Health Administration. In each project, identical 77-item ORCA instruments were administered to one or more staff from each facility involved in quality improvement projects. Items were organized into 19 subscales and three primary scales corresponding to the core elements of the PARIHS framework: (1) Strength and extent of evidence for the clinical practice changes represented by the QI program, assessed with four subscales, (2) Quality of the organizational context for the QI program, assessed with six subscales, and (3) Capacity for internal facilitation of the QI program, assessed with nine subscales. Cronbach's alpha for scale reliability were 0.74, 0.85 and 0.95 for the evidence, context and facilitation scales, respectively. The evidence scale and its three constituent subscales failed to meet the conventional threshold of 0.80 for reliability, and three individual items were eliminated from evidence subscales following reliability testing. In exploratory factor analysis, three factors were retained. Seven of the nine facilitation subscales loaded onto the first factor; five of the six context subscales loaded onto the second factor; and the three evidence subscales loaded on the third factor. Two subscales failed to load significantly on any factor. One measured resources in general (from the context scale), and one clinical champion role (from the facilitation scale). We find general support for the reliability and factor structure of the ORCA. However, there was poor reliability among measures of evidence, and factor analysis results for measures of general resources and clinical champion role did not conform to the PARIHS framework. Additional validation is needed, including criterion validation.

A serum and platelet-rich plasma serotonin assay using liquid chromatography tandem mass spectrometry for monitoring of neuroendocrine tumor patients.

PubMed

Korse, Catharina M; Buning-Kager, Johanna C G M; Linders, Theodora C; Heijboer, Annemieke C; van den Broek, Daan; Tesselaar, Margot E T; van Tellingen, Olaf; van Rossum, Huub H

2017-06-01

Serotonin is used for the diagnosis and follow-up of neuroendocrine tumors (NET). We describe the analytical and clinical validation of a liquid chromatography tandem mass spectrometry (LC-MS/MS) based serotonin assay for serum and platelet-rich plasma (PRP). An LC-MS/MS based method for serum and PRP serotonin was validated by determination of assay imprecision, carry-over, linearity, interference, recovery, sample stability and a matrix/method comparison of serum and PRP serotonin was made with whole blood serotonin. Furthermore, upper limits of normal were determined and serotonin concentrations of healthy individuals, 14 NET patients without evidence of disease and 51 NET patients with evidence of disease were compared. For serum and PRP fractions, total assay imprecision was <5%. All correlation coefficients were 0.98 and the serum and platelet-rich serotonin upper limit of normal were 5.5nmol/10 9 platelet and 5.1nmol/10 9 platelet, respectively. NET patients with confirmed evidence of disease had significantly higher serum and PRP serotonin levels when compared to NET patients without evidence of disease and healthy volunteers. LC-MS/MS based serum and PRP serotonin assays were developed with suitable analytical characteristics. Furthermore, serum and PRP serotonin was found to be useful for monitoring NET patients. Copyright © 2017 Elsevier B.V. All rights reserved.

Attitudes to, and experience of, pooled sampling for sexually transmitted infection testing: a web-based survey of English sexual health services.

PubMed

Shaw, Jonathan; Saunders, John Michael; Hughes, Gwenda

2018-05-01

Chlamydia trachomatis and Neisseria gonorrhoeae testing guidance recommends extragenital screening with locally validated nucleic acid amplification tests, with anatomical sites tested separately. Evidence supports multi-patient combined aliquot pooled sampling (PS) for population screening; evidence for within-patient PS is sparse. Within-patient PS could be more cost-effective for triple-site testing, but requires distinct clinical pathways and consideration over loss of information to guide risk assessments and treatment. We explored PS attitudes and practices amongst clinicians in England. A cross-sectional web-based survey was distributed to clinical leads of sexual health services throughout England in February 2016. Fifty-two (52/216, 23%) services responded. One service reported current within-patient PS and two were awaiting implementation. Of the 49 services not pooling, five were considering implementation. Concerns raised included the inability to distinguish infection site(s) (36/52, 69%), absence of national guidance (34/52, 65%) and reduced assay performance (18/52, 34%). Only 8/52 (15%) considered the current level of evidence sufficient to support PS, with 40/52 (77%) requesting further validation studies and 39/52 (77%) national guidance. PS was rarely used by respondents to this survey, although the response rate was low. The clinical challenges presented by PS need to be addressed through further development of the evidence base.

Factorial Invariance and Convergent Validity of the Group-Based Medical Mistrust Scale across Gender and Ethnoracial Identity.

PubMed

Wheldon, Christopher W; Kolar, Stephanie K; Hernandez, Natalie D; Daley, Ellen M

2017-01-01

The objective of this study was to assess the factorial invariance and convergent validity of the Group-Based Medical Mistrust Scale (GBMMS) across gender (male and female) and ethnoracial identity (Latino and Black). Minority students (N = 686) attending a southeastern university were surveyed in the fall of 2011. Psychometric analysis of the GBMMS was performed. A three-factor solution fit the data after the omission of two problematic items. This revised version of the GBMMS exhibited sufficient configural, metric, and scalar invariance. Convergence of the GBMMS with conceptually related measures provided further evidence of validity; however, there was variation across ethnoracial identity. The GBMMS has viable psychometric properties across gender and ethnoracial identity in Black and Latino populations.

Scale of attitudes toward alcohol - Spanish version: evidences of validity and reliability 1

PubMed Central

Ramírez, Erika Gisseth León; de Vargas, Divane

2017-01-01

ABSTRACT Objective: validate the Scale of attitudes toward alcohol, alcoholism and individuals with alcohol use disorders in its Spanish version. Method: methodological study, involving 300 Colombian nurses. Adopting the classical theory, confirmatory factor analysis was applied without prior examination, based on the strong historical evidence of the factorial structure of the original scale to determine the construct validity of this Spanish version. To assess the reliability, Cronbach’s Alpha and Mc Donalid’s Omega coefficients were used. Results: the confirmatory factor analysis indicated the good fit of the scale model in a four-factor distribution, with a cut-off point at 3.2, demonstrating 66.7% of sensitivity. Conclusions: the Scale of attitudes toward alcohol, alcoholism and individuals with alcohol use disorders in Spanish presented robust psychometric qualities, affirming that the instrument possesses a solid factorial structure and reliability and is capable of precisely measuring the nurses’ atittudes towards the phenomenon proposed. PMID:28793126

Development and validation of an online interactive, multimedia wound care algorithms program.

PubMed

Beitz, Janice M; van Rijswijk, Lia

2012-01-01

To provide education based on evidence-based and validated wound care algorithms we designed and implemented an interactive, Web-based learning program for teaching wound care. A mixed methods quantitative pilot study design with qualitative components was used to test and ascertain the ease of use, validity, and reliability of the online program. A convenience sample of 56 RN wound experts (formally educated, certified in wound care, or both) participated. The interactive, online program consists of a user introduction, interactive assessment of 15 acute and chronic wound photos, user feedback about the percentage correct, partially correct, or incorrect algorithm and dressing choices and a user survey. After giving consent, participants accessed the online program, provided answers to the demographic survey, and completed the assessment module and photographic test, along with a posttest survey. The construct validity of the online interactive program was strong. Eighty-five percent (85%) of algorithm and 87% of dressing choices were fully correct even though some programming design issues were identified. Online study results were consistently better than previously conducted comparable paper-pencil study results. Using a 5-point Likert-type scale, participants rated the program's value and ease of use as 3.88 (valuable to very valuable) and 3.97 (easy to very easy), respectively. Similarly the research process was described qualitatively as "enjoyable" and "exciting." This digital program was well received indicating its "perceived benefits" for nonexpert users, which may help reduce barriers to implementing safe, evidence-based care. Ongoing research using larger sample sizes may help refine the program or algorithms while identifying clinician educational needs. Initial design imperfections and programming problems identified also underscored the importance of testing all paper and Web-based programs designed to educate health care professionals or guide patient care.

Development and Validation of a Primary Care-Based Family Health History and Decision Support Program (MeTree)

PubMed Central

Orlando, Lori A.; Buchanan, Adam H.; Hahn, Susan E.; Christianson, Carol A.; Powell, Karen P.; Skinner, Celette Sugg; Chesnut, Blair; Blach, Colette; Due, Barbara; Ginsburg, Geoffrey S.; Henrich, Vincent C.

2016-01-01

INTRODUCTION Family health history is a strong predictor of disease risk. To reduce the morbidity and mortality of many chronic diseases, risk-stratified evidence-based guidelines strongly encourage the collection and synthesis of family health history to guide selection of primary prevention strategies. However, the collection and synthesis of such information is not well integrated into clinical practice. To address barriers to collection and use of family health histories, the Genomedical Connection developed and validated MeTree, a Web-based, patient-facing family health history collection and clinical decision support tool. MeTree is designed for integration into primary care practices as part of the genomic medicine model for primary care. METHODS We describe the guiding principles, operational characteristics, algorithm development, and coding used to develop MeTree. Validation was performed through stakeholder cognitive interviewing, a genetic counseling pilot program, and clinical practice pilot programs in 2 community-based primary care clinics. RESULTS Stakeholder feedback resulted in changes to MeTree’s interface and changes to the phrasing of clinical decision support documents. The pilot studies resulted in the identification and correction of coding errors and the reformatting of clinical decision support documents. MeTree’s strengths in comparison with other tools are its seamless integration into clinical practice and its provision of action-oriented recommendations guided by providers’ needs. LIMITATIONS The tool was validated in a small cohort. CONCLUSION MeTree can be integrated into primary care practices to help providers collect and synthesize family health history information from patients with the goal of improving adherence to risk-stratified evidence-based guidelines. PMID:24044145

Development and validation of a preference based measure derived from the Cambridge Pulmonary Hypertension Outcome Review (CAMPHOR) for use in cost utility analyses.

PubMed

McKenna, Stephen P; Ratcliffe, Julie; Meads, David M; Brazier, John E

2008-08-21

Pulmonary Hypertension is a severe and incurable disease with poor prognosis. A suite of new disease-specific measures--the Cambridge Pulmonary Hypertension Outcome Review (CAMPHOR) - was recently developed for use in this condition. The purpose of this study was to develop and validate a preference based measure from the CAMPHOR that could be used in cost-utility analyses. Items were selected that covered major issues covered by the CAMPHOR QoL scale (activities, travelling, dependence and communication). These were used to create 36 health states that were valued by 249 people representative of the UK adult population, using the time trade-off (TTO) technique. Data from the TTO interviews were analysed using both aggregate and individual level modelling. Finally, the original CAMPHOR validation data were used to validate the new preference based model. The predicted health state values ranged from 0.962 to 0.136. The mean level model selected for analyzing the data had good explanatory power (0.936), did not systematically over- or underestimate the observed mean health state values and showed no evidence of auto correlation in the prediction errors. The value of less than 1 reflects a background level of ill health in state 1111, as judged by the respondents. Scores derived from the new measure had excellent test-retest reliability (0.85) and construct validity. The CAMPHOR utility score appears better able to distinguish between WHO functional classes (II and III) than the EQ-5D and SF-6D. The tariff derived in this study can be used to classify an individual into a health state based on their responses to the CAMPHOR. The results of this study widen the evidence base for conducting economic evaluations of interventions designed to improve QoL for patients with PH.

Development and initial validation of the Parental PELICAN Questionnaire (PaPEQu)--an instrument to assess parental experiences and needs during their child's end-of-life care.

PubMed

Zimmermann, Karin; Cignacco, Eva; Eskola, Katri; Engberg, Sandra; Ramelet, Anne-Sylvie; Von der Weid, Nicolas; Bergstraesser, Eva

2015-12-01

To develop and test the Parental PELICAN Questionnaire, an instrument to retrospectively assess parental experiences and needs during their child's end-of-life care. To offer appropriate care for dying children, healthcare professionals need to understand the illness experience from the family perspective. A questionnaire specific to the end-of-life experiences and needs of parents losing a child is needed to evaluate the perceived quality of paediatric end-of-life care. This is an instrument development study applying mixed methods based on recommendations for questionnaire design and validation. The Parental PELICAN Questionnaire was developed in four phases between August 2012-March 2014: phase 1: item generation; phase 2: validity testing; phase 3: translation; phase 4: pilot testing. Psychometric properties were assessed after applying the Parental PELICAN Questionnaire in a sample of 224 bereaved parents in April 2014. Validity testing covered the evidence based on tests of content, internal structure and relations to other variables. The Parental PELICAN Questionnaire consists of approximately 90 items in four slightly different versions accounting for particularities of the four diagnostic groups. The questionnaire's items were structured according to six quality domains described in the literature. Evidence of initial validity and reliability could be demonstrated with the involvement of healthcare professionals and bereaved parents. The Parental PELICAN Questionnaire holds promise as a measure to assess parental experiences and needs and is applicable to a broad range of paediatric specialties and settings. Future validation is needed to evaluate its suitability in different cultures. © 2015 John Wiley & Sons Ltd.

Assessing medical students' self-regulation as aptitude in computer-based learning.

PubMed

Song, Hyuksoon S; Kalet, Adina L; Plass, Jan L

2011-03-01

We developed a Self-Regulation Measure for Computer-based learning (SRMC) tailored toward medical students, by modifying Zimmerman's Self-Regulated Learning Interview Schedule (SRLIS) for K-12 learners. The SRMC's reliability and validity were examined in 2 studies. In Study 1, 109 first-year medical students were asked to complete the SRMC. Bivariate correlation analysis results indicated that the SRMC scores had a moderate degree of correlation with student achievement in a teacher-developed test. In Study 2, 58 third-year clerkship students completed the SRMC. Regression analysis results indicated that the frequency of medical students' usage of self-regulation strategies was associated with their general clinical knowledge measured by a nationally standardized licensing exam. These two studies provided evidence for the reliability and concurrent validity of the SRMC to assess medical students' self-regulation as aptitude. Future work should provide evidence to guide and improve instructional design as well as inform educational policy.

A web-based library consult service for evidence-based medicine: Technical development.

PubMed

Schwartz, Alan; Millam, Gregory

2006-03-16

Incorporating evidence based medicine (EBM) into clinical practice requires clinicians to learn to efficiently gain access to clinical evidence and effectively appraise its validity. Even using current electronic systems, selecting literature-based data to solve a single patient-related problem can require more time than practicing physicians or residents can spare. Clinical librarians, as informationists, are uniquely suited to assist physicians in this endeavor. To improve support for evidence-based practice, we have developed a web-based EBM library consult service application (LCS). Librarians use the LCS system to provide full text evidence-based literature with critical appraisal in response to a clinical question asked by a remote physician. LCS uses an entirely Free/Open Source Software platform and will be released under a Free Software license. In the first year of the LCS project, the software was successfully developed and a reference implementation put into active use. Two years of evaluation of the clinical, educational, and attitudinal impact on physician-users and librarian staff are underway, and expected to lead to refinement and wide dissemination of the system. A web-based EBM library consult model may provide a useful way for informationists to assist clinicians, and is feasible to implement.

Building Capacity for Work-Readiness: Bridging the Cognitive and Affective Domains

ERIC Educational Resources Information Center

Bandaranaike, Suniti; Willison, John

2015-01-01

Teaching for work-integrated learning (WIL) competency is largely directed at delivering knowledge based cognitive skills with little emphasis on affective skills. This study looks at empirical evidence of WIL students through their understanding of the cognitive and affective domains. The research is based on a validated employability framework,…

Advance care planning in dementia: recommendations for healthcare professionals.

PubMed

Piers, Ruth; Albers, Gwenda; Gilissen, Joni; De Lepeleire, Jan; Steyaert, Jan; Van Mechelen, Wouter; Steeman, Els; Dillen, Let; Vanden Berghe, Paul; Van den Block, Lieve

2018-06-21

Advance care planning (ACP) is a continuous, dynamic process of reflection and dialogue between an individual, those close to them and their healthcare professionals, concerning the individual's preferences and values concerning future treatment and care, including end-of-life care. Despite universal recognition of the importance of ACP for people with dementia, who gradually lose their ability to make informed decisions themselves, ACP still only happens infrequently, and evidence-based recommendations on when and how to perform this complex process are lacking. We aimed to develop evidence-based clinical recommendations to guide professionals across settings in the practical application of ACP in dementia care. Following the Belgian Centre for Evidence-Based Medicine's procedures, we 1) performed an extensive literature search to identify international guidelines, articles reporting heterogeneous study designs and grey literature, 2) developed recommendations based on the available evidence and expert opinion of the author group, and 3) performed a validation process using written feedback from experts, a survey for end users (healthcare professionals across settings), and two peer-review groups (with geriatricians and general practitioners). Based on 67 publications and validation from ten experts, 51 end users and two peer-review groups (24 participants) we developed 32 recommendations covering eight domains: initiation of ACP, evaluation of mental capacity, holding ACP conversations, the role and importance of those close to the person with dementia, ACP with people who find it difficult or impossible to communicate verbally, documentation of wishes and preferences, including information transfer, end-of-life decision-making, and preconditions for optimal implementation of ACP. Almost all recommendations received a grading representing low to very low-quality evidence. No high-quality guidelines are available for ACP in dementia care. By combining evidence with expert and user opinions, we have defined a unique set of recommendations for ACP in people living with dementia. These recommendations form a valuable tool for educating healthcare professionals on how to perform ACP across settings.

Using Scratch with Primary School Children: An Evaluation of Games Constructed to Gauge Understanding of Programming Concepts

ERIC Educational Resources Information Center

Wilson, Amanda; Hainey, Thomas; Connolly, Thomas M.

2013-01-01

Newer approaches such as games-based learning (GBL) and games-based construction are being adopted to motivate and engage students within the Curriculum for Excellence (CfE) in Scotland. GBL and games-based construction suffer from a dearth of empirical evidence supporting their validity as teaching and learning approaches. To address this issue…

Examining the ecological validity of the Talent Development Environment Questionnaire.

PubMed

Martindale, Russell J J; Collins, Dave; Douglas, Carl; Whike, Ally

2013-01-01

It is clear that high class expertise and effective practice exists within many talent development environments across the world. However, there is also a general consensus that widespread evidence-based policy and practice is lacking. As such, it is crucial to develop solutions which can facilitate effective dissemination of knowledge and promotion of evidence-based talent development systems. While the Talent Development Environment Questionnaire (Martindale et al., 2010 ) provides a method through which this could be facilitated, its ecological validity has remained untested. As such, this study aimed to investigate the real world applicability of the questionnaire through discriminant function analysis. Athletes across ten distinct regional squads and academies were identified and separated into two broad levels, 'higher quality' (n = 48) and 'lower quality' (n = 51) environments, based on their process quality and productivity. Results revealed that the Talent Development Environment Questionnaire was able to discriminate with 77.8% accuracy. Furthermore, in addition to the questionnaire as a whole, two individual features, 'quality preparation' (P < 0.01) and 'understanding the athlete' (P < 0.01), were found to be significant discriminators. In conclusion, the results indicate robust structural properties and sound ecological validity, allowing the questionnaire to be used with more confidence in applied and research settings.

Reliability and Validity of the Sensory Component of the International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI): A Systematic Review.

PubMed

Hales, M; Biros, E; Reznik, J E

2015-01-01

Since 1982, the International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI) has been used to classify sensation of spinal cord injury (SCI) through pinprick and light touch scores. The absence of proprioception, pain, and temperature within this scale creates questions about its validity and accuracy. To assess whether the sensory component of the ISNCSCI represents a reliable and valid measure of classification of SCI. A systematic review of studies examining the reliability and validity of the sensory component of the ISNCSCI published between 1982 and February 2013 was conducted. The electronic databases MEDLINE via Ovid, CINAHL, PEDro, and Scopus were searched for relevant articles. A secondary search of reference lists was also completed. Chosen articles were assessed according to the Oxford Centre for Evidence-Based Medicine hierarchy of evidence and critically appraised using the McMasters Critical Review Form. A statistical analysis was conducted to investigate the variability of the results given by reliability studies. Twelve studies were identified: 9 reviewed reliability and 3 reviewed validity. All studies demonstrated low levels of evidence and moderate critical appraisal scores. The majority of the articles (~67%; 6/9) assessing the reliability suggested that training was positively associated with better posttest results. The results of the 3 studies that assessed the validity of the ISNCSCI scale were confounding. Due to the low to moderate quality of the current literature, the sensory component of the ISNCSCI requires further revision and investigation if it is to be a useful tool in clinical trials.

Reliability and Validity of the Sensory Component of the International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI): A Systematic Review

PubMed Central

Hales, M.; Biros, E.

2015-01-01

Background: Since 1982, the International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI) has been used to classify sensation of spinal cord injury (SCI) through pinprick and light touch scores. The absence of proprioception, pain, and temperature within this scale creates questions about its validity and accuracy. Objectives: To assess whether the sensory component of the ISNCSCI represents a reliable and valid measure of classification of SCI. Methods: A systematic review of studies examining the reliability and validity of the sensory component of the ISNCSCI published between 1982 and February 2013 was conducted. The electronic databases MEDLINE via Ovid, CINAHL, PEDro, and Scopus were searched for relevant articles. A secondary search of reference lists was also completed. Chosen articles were assessed according to the Oxford Centre for Evidence-Based Medicine hierarchy of evidence and critically appraised using the McMasters Critical Review Form. A statistical analysis was conducted to investigate the variability of the results given by reliability studies. Results: Twelve studies were identified: 9 reviewed reliability and 3 reviewed validity. All studies demonstrated low levels of evidence and moderate critical appraisal scores. The majority of the articles (~67%; 6/9) assessing the reliability suggested that training was positively associated with better posttest results. The results of the 3 studies that assessed the validity of the ISNCSCI scale were confounding. Conclusions: Due to the low to moderate quality of the current literature, the sensory component of the ISNCSCI requires further revision and investigation if it is to be a useful tool in clinical trials. PMID:26363591

Validation of learning assessments: A primer.

PubMed

Peeters, Michael J; Martin, Beth A

2017-09-01

The Accreditation Council for Pharmacy Education's Standards 2016 has placed greater emphasis on validating educational assessments. In this paper, we describe validity, reliability, and validation principles, drawing attention to the conceptual change that highlights one validity with multiple evidence sources; to this end, we recommend abandoning historical (confusing) terminology associated with the term validity. Further, we describe and apply Kane's framework (scoring, generalization, extrapolation, and implications) for the process of validation, with its inferences and conclusions from varied uses of assessment instruments by different colleges and schools of pharmacy. We then offer five practical recommendations that can improve reporting of validation evidence in pharmacy education literature. We describe application of these recommendations, including examples of validation evidence in the context of pharmacy education. After reading this article, the reader should be able to understand the current concept of validation, and use a framework as they validate and communicate their own institution's learning assessments. Copyright © 2017 Elsevier Inc. All rights reserved.

Evidence conflict measure based on OWA operator in open world

PubMed Central

Wang, Shiyu; Liu, Xiang; Zheng, Hanqing; Wei, Boya

2017-01-01

Dempster-Shafer evidence theory has been extensively used in many information fusion systems since it was proposed by Dempster and extended by Shafer. Many scholars have been conducted on conflict management of Dempster-Shafer evidence theory in past decades. However, how to determine a potent parameter to measure evidence conflict, when the given environment is in an open world, namely the frame of discernment is incomplete, is still an open issue. In this paper, a new method which combines generalized conflict coefficient, generalized evidence distance, and generalized interval correlation coefficient based on ordered weighted averaging (OWA) operator, to measure the conflict of evidence is presented. Through ordered weighted average of these three parameters, the combinatorial coefficient can still measure the conflict effectively when one or two parameters are not valid. Several numerical examples demonstrate the effectiveness of the proposed method. PMID:28542271

Confidence in outcome estimates from systematic reviews used in informed consent.

PubMed

Fritz, Robert; Bauer, Janet G; Spackman, Sue S; Bains, Amanjyot K; Jetton-Rangel, Jeanette

2016-12-01

Evidence-based dentistry now guides informed consent in which clinicians are obliged to provide patients with the most current, best evidence, or best estimates of outcomes, of regimens, therapies, treatments, procedures, materials, and equipment or devices when developing personal oral health care, treatment plans. Yet, clinicians require that the estimates provided from systematic reviews be verified to their validity, reliability, and contextualized as to performance competency so that clinicians may have confidence in explaining outcomes to patients in clinical practice. The purpose of this paper was to describe types of informed estimates from which clinicians may have confidence in their capacity to assist patients in competent decision-making, one of the most important concepts of informed consent. Using systematic review methodology, researchers provide clinicians with valid best estimates of outcomes regarding a subject of interest from best evidence. Best evidence is verified through critical appraisals using acceptable sampling methodology either by scoring instruments (Timmer analysis) or checklist (grade), a Cochrane Collaboration standard that allows transparency in open reviews. These valid best estimates are then tested for reliability using large databases. Finally, valid and reliable best estimates are assessed for meaning using quantification of margins and uncertainties. Through manufacturer and researcher specifications, quantification of margins and uncertainties develops a performance competency continuum by which valid, reliable best estimates may be contextualized for their performance competency: at a lowest margin performance competency (structural failure), high margin performance competency (estimated true value of success), or clinically determined critical values (clinical failure). Informed consent may be achieved when clinicians are confident of their ability to provide useful and accurate best estimates of outcomes regarding regimens, therapies, treatments, and equipment or devices to patients in their clinical practices and when developing personal, oral health care, treatment plans. Copyright © 2016 Elsevier Inc. All rights reserved.

The validated sun exposure questionnaire: association of objective and subjective measures of sun exposure in a Danish population-based sample.

PubMed

Køster, B; Søndergaard, J; Nielsen, J B; Allen, M; Olsen, A; Bentzen, J

2017-02-01

Few questionnaires used in monitoring sun-related behaviour have been tested for validity. We established the criteria validity of a questionnaire developed for monitoring population sun-related behaviour. During May-August 2013, 664 Danes wore a personal electronic ultraviolet radiation (UVR) dosimeter for 1 week that measured their outdoor time and dose of erythemal UVR exposure. In the following week, they answered a questionnaire on their sun-related behaviour in the measurement week. Outdoor time measured by dosimetry correlated strongly with both outdoor time and the developed exposure scale measured in the questionnaire. Exposure measured in standard erythema dose (SED) by dosimetry correlated strongly with the exposure scale. In a linear regression model of UVR (SED) received, 41% of the variation was explained by skin type, age, week of participation and exposure scale, with exposure scale as the main contributor. The weekly sunburn fraction correlated strongly with the number of ambient sun hours (r = 0·73, P < 0·001). This criteria-validated questionnaire provides evidence of the exposure that the questionnaire aimed to measure. The evidence provided showed a strong link between the objectively measured behaviour and the behaviour measured by this survey construct. The questionnaire is the first validated tool to measure the UVR exposure in a national population-based sample. © 2016 The Authors. British Journal of Dermatology published by John Wiley & Sons Ltd on behalf of British Association of Dermatologists.

Development and validation of the Simulation Learning Effectiveness Inventory.

PubMed

Chen, Shiah-Lian; Huang, Tsai-Wei; Liao, I-Chen; Liu, Chienchi

2015-10-01

To develop and psychometrically test the Simulation Learning Effectiveness Inventory. High-fidelity simulation helps students develop clinical skills and competencies. Yet, reliable instruments measuring learning outcomes are scant. A descriptive cross-sectional survey was used to validate psychometric properties of the instrument measuring students' perception of stimulation learning effectiveness. A purposive sample of 505 nursing students who had taken simulation courses was recruited from a department of nursing of a university in central Taiwan from January 2010-June 2010. The study was conducted in two phases. In Phase I, question items were developed based on the literature review and the preliminary psychometric properties of the inventory were evaluated using exploratory factor analysis. Phase II was conducted to evaluate the reliability and validity of the finalized inventory using confirmatory factor analysis. The results of exploratory and confirmatory factor analyses revealed the instrument was composed of seven factors, named course arrangement, equipment resource, debriefing, clinical ability, problem-solving, confidence and collaboration. A further second-order analysis showed comparable fits between a three second-order factor (preparation, process and outcome) and the seven first-order factor models. Internal consistency was supported by adequate Cronbach's alphas and composite reliability. Convergent and discriminant validities were also supported by confirmatory factor analysis. The study provides evidence that the Simulation Learning Effectiveness Inventory is reliable and valid for measuring student perception of learning effectiveness. The instrument is helpful in building the evidence-based knowledge of the effect of simulation teaching on students' learning outcomes. © 2015 John Wiley & Sons Ltd.

Applying hospital evidence to paramedicine: issues of indirectness, validity and knowledge translation.

PubMed

Bigham, Blair; Welsford, Michelle

2015-05-01

The practice of emergency medicine (EM) has been intertwined with emergency medical services (EMS) for more than 40 years. In this commentary, we explore the practice of translating hospital based evidence into the prehospital setting. We will challenge both EMS and EM dogma-bringing hospital care to patients in the field is not always better. In providing examples of therapies championed in hospitals that have failed to translate into the field, we will discuss the unique prehospital environment, and why evidence from the hospital setting cannot necessarily be translated to the prehospital field. Paramedicine is maturing so that the capability now exists to conduct practice-specific research that can inform best practices. Before translation from the hospital environment is implemented, evidence must be evaluated by people with expertise in three domains: critical appraisal, EM, and EMS. Scientific evidence should be assessed for: quality and bias; directness, generalizability, and validity to the EMS population; effect size and anticipated benefit from prehospital application; feasibility (including economic evaluation, human resource availability in the mobile environment); and patient and provider safety.

Systematic reviews, systematic error and the acquisition of clinical knowledge

PubMed Central

2010-01-01

Background Since its inception, evidence-based medicine and its application through systematic reviews, has been widely accepted. However, it has also been strongly criticised and resisted by some academic groups and clinicians. One of the main criticisms of evidence-based medicine is that it appears to claim to have unique access to absolute scientific truth and thus devalues and replaces other types of knowledge sources. Discussion The various types of clinical knowledge sources are categorised on the basis of Kant's categories of knowledge acquisition, as being either 'analytic' or 'synthetic'. It is shown that these categories do not act in opposition but rather, depend upon each other. The unity of analysis and synthesis in knowledge acquisition is demonstrated during the process of systematic reviewing of clinical trials. Systematic reviews constitute comprehensive synthesis of clinical knowledge but depend upon plausible, analytical hypothesis development for the trials reviewed. The dangers of systematic error regarding the internal validity of acquired knowledge are highlighted on the basis of empirical evidence. It has been shown that the systematic review process reduces systematic error, thus ensuring high internal validity. It is argued that this process does not exclude other types of knowledge sources. Instead, amongst these other types it functions as an integrated element during the acquisition of clinical knowledge. Conclusions The acquisition of clinical knowledge is based on interaction between analysis and synthesis. Systematic reviews provide the highest form of synthetic knowledge acquisition in terms of achieving internal validity of results. In that capacity it informs the analytic knowledge of the clinician but does not replace it. PMID:20537172

Truth and Evidence in Validity Theory

ERIC Educational Resources Information Center

Borsboom, Denny; Markus, Keith A.

2013-01-01

According to Kane (this issue), "the validity of a proposed interpretation or use depends on how well the evidence supports" the claims being made. Because truth and evidence are distinct, this means that the validity of a test score interpretation could be high even though the interpretation is false. As an illustration, we discuss the case of…

Validity of Factors of the Psychopathy Checklist–Revised in Female Prisoners

PubMed Central

Kennealy, Patrick J.; Hicks, Brian M.; Patrick, Christopher J.

2008-01-01

The validity of the Psychopathy Checklist–Revised (PCL-R) has been examined extensively in men, but its validity for women remains understudied. Specifically, the correlates of the general construct of psychopathy and its components as assessed by PCL-R total, factor, and facet scores have yet to be examined in depth. Based on previous research conducted with male offenders, a large female inmate sample was used to examine the patterns of relations between total, factor, and facet scores on the PCL-R and various criterion variables. These variables include ratings of psychopathy based on Cleckley’s criteria, symptoms of antisocial personality disorder, and measures of substance use and abuse, criminal behavior, institutional misconduct, interpersonal aggression, normal range personality, intellectual functioning, and social background variables. Results were highly consistent with past findings in male samples and provide further evidence for the construct validity of the PCL-R two-factor and four-facet models across genders. PMID:17986651

Development and Psychometric Properties of the Nursing Critical Thinking in Clinical Practice Questionnaire.

PubMed

Zuriguel-Pérez, Esperanza; Falcó-Pegueroles, Anna; Roldán-Merino, Juan; Agustino-Rodriguez, Sandra; Gómez-Martín, Maria Del Carmen; Lluch-Canut, Maria Teresa

2017-08-01

A complex healthcare environment, with greater need for care based on the patient and evidence-based practice, are factors that have contributed to the increased need for critical thinking in professional competence. At the theoretical level, Alfaro-LeFevre () put forward a model of critical thinking made up of four components. And although these explain the construct, instruments for their empirical measurement are lacking. The purpose of the study was to develop and validate the psychometric properties of an instrument, the Nursing Critical Thinking in Clinical Practice Questionnaire (N-CT-4 Practice), designed to evaluate the critical thinking abilities of nurses in the clinical setting. A cross-sectional survey design was used. A pool of items was generated for evaluation by a panel of experts who considered their validity for the new instrument, which was finally made up of 109 items. Following this, validation was carried out using a sample of 339 nurses at a hospital in Barcelona, Spain. Reliability was determined by means of internal consistency and test-retest stability over time, although the validity of the construct was assessed by means of confirmatory factor analysis. The content validity index of the N-CT-4 Practice was .85. Cronbach's alpha coefficient for the whole instrument was .96. The intraclass correlation coefficient was .77. Confirmatory factor analysis showed that the instrument was in line with the four-dimensional model proposed by Alfaro-LeFevre (). The psychometric properties of theN-CT-4 Practice uphold its potential for use in measuring critical thinking and in future research related with the examination of critical thinking. © 2017 The Authors Worldviews on Evidence-Based Nursing published by Wiley Periodicals, Inc. on behalf of Sigma Theta Tau International The Honor Society of Nursing.

Workplace status: The development and validation of a scale.

PubMed

Djurdjevic, Emilija; Stoverink, Adam C; Klotz, Anthony C; Koopman, Joel; da Motta Veiga, Serge P; Yam, Kai Chi; Chiang, Jack Ting-Ju

2017-07-01

Research suggests that employee status, and various status proxies, relate to a number of meaningful outcomes in the workplace. The advancement of the study of status in organizational settings has, however, been stymied by the lack of a validated workplace status measure. The purpose of this manuscript, therefore, is to develop and validate a measure of workplace status based on a theoretically grounded definition of status in organizations. Subject-matter experts were used to examine the content validity of the measure. Then, 2 separate samples were employed to assess the psychometric properties (i.e., factor structure, reliability, convergent and discriminant validity) and nomological network of a 5-item, self-report Workplace Status Scale (WSS). To allow for methodological flexibility, an additional 3 samples were used to extend the WSS to coworker reports of a focal employee's status, provide additional evidence for the validity and reliability of the WSS, and to demonstrate consensus among coworker ratings. Together, these studies provide evidence of the psychometric soundness of the WSS for assessing employee status using either self-reports or other-source reports. The implications of the development of the WSS for the study of status in organizations are discussed, and suggestions for future research using the new measure are offered. (PsycINFO Database Record (c) 2017 APA, all rights reserved).

[Evidence-based cardiology: practical applications from epidemiology. III. Diagnostic capacity of a clinical test].

PubMed

Rodríguez-Escudero, Juan Pablo; López-Jiménez, Francisco; Trejo-Gutiérrez, Jorge F

2011-01-01

This article reviews different characteristics of validity in a clinical diagnostic test. In particular, we emphasize the likelihood ratio as an instrument that facilitates the use of epidemiologic concepts in clinical diagnosis.

Development of a Work Climate Scale in Emergency Health Services

PubMed Central

Sanduvete-Chaves, Susana; Lozano-Lozano, José A.; Chacón-Moscoso, Salvador; Holgado-Tello, Francisco P.

2018-01-01

An adequate work climate fosters productivity in organizations and increases employee satisfaction. Workers in emergency health services (EHS) have an extremely high degree of responsibility and consequent stress. Therefore, it is essential to foster a good work climate in this context. Despite this, scales with a full study of their psychometric properties (i.e., validity evidence based on test content, internal structure and relations to other variables, and reliability) are not available to measure work climate in EHS specifically. For this reason, our objective was to develop a scale to measure the quality of work climates in EHS. We carried out three studies. In Study 1, we used a mixed-method approach to identify the latent conceptual structure of the construct work climate. Thus, we integrated the results found in (a) a previous study, where a content analysis of seven in-depth interviews obtained from EHS professionals in two hospitals in Gibraltar Countryside County was carried out; and (b) the factor analysis of the responses given by 113 EHS professionals from these same centers to 18 items that measured the work climate in health organizations. As a result, we obtained 56 items grouped into four factors (work satisfaction, productivity/achievement of aims, interpersonal relationships, and performance at work). In Study 2, we presented validity evidence based on test content through experts' judgment. Fourteen experts from the methodology and health fields evaluated the representativeness, utility, and feasibility of each of the 56 items with respect to their factor (theoretical dimension). Forty items met the inclusion criterion, which was to obtain an Osterlind index value greater than or equal to 0.5 in the three aspects assessed. In Study 3, 201 EHS professionals from the same centers completed the resulting 40-item scale. This new instrument produced validity evidence based on the internal structure in a second-order factor model with four components (RMSEA = 0.079, GFI = 0.97, AGFI = 0.97, CFI = 0.97; NFI = 0.95, and NNFI = 0.97); absence of Differential Item Functioning (DIF) in 80% of the items; reliability (α = 0.96); and validity evidence based on relations to other variables, specifically the test-criterion relationship (ρ = 0.680). Finally, we discuss further developments of the instrument and its possible implications for EHS workers. PMID:29403417

Development of a Work Climate Scale in Emergency Health Services.

PubMed

Sanduvete-Chaves, Susana; Lozano-Lozano, José A; Chacón-Moscoso, Salvador; Holgado-Tello, Francisco P

2018-01-01

An adequate work climate fosters productivity in organizations and increases employee satisfaction. Workers in emergency health services (EHS) have an extremely high degree of responsibility and consequent stress. Therefore, it is essential to foster a good work climate in this context. Despite this, scales with a full study of their psychometric properties (i.e., validity evidence based on test content, internal structure and relations to other variables, and reliability) are not available to measure work climate in EHS specifically. For this reason, our objective was to develop a scale to measure the quality of work climates in EHS. We carried out three studies. In Study 1, we used a mixed-method approach to identify the latent conceptual structure of the construct work climate . Thus, we integrated the results found in (a) a previous study, where a content analysis of seven in-depth interviews obtained from EHS professionals in two hospitals in Gibraltar Countryside County was carried out; and (b) the factor analysis of the responses given by 113 EHS professionals from these same centers to 18 items that measured the work climate in health organizations. As a result, we obtained 56 items grouped into four factors (work satisfaction, productivity/achievement of aims, interpersonal relationships, and performance at work). In Study 2, we presented validity evidence based on test content through experts' judgment. Fourteen experts from the methodology and health fields evaluated the representativeness, utility, and feasibility of each of the 56 items with respect to their factor (theoretical dimension). Forty items met the inclusion criterion, which was to obtain an Osterlind index value greater than or equal to 0.5 in the three aspects assessed. In Study 3, 201 EHS professionals from the same centers completed the resulting 40-item scale. This new instrument produced validity evidence based on the internal structure in a second-order factor model with four components ( RMSEA = 0.079, GFI = 0.97, AGFI = 0.97, CFI = 0.97; NFI = 0.95, and NNFI = 0.97); absence of Differential Item Functioning (DIF) in 80% of the items; reliability (α = 0.96); and validity evidence based on relations to other variables, specifically the test-criterion relationship (ρ = 0.680). Finally, we discuss further developments of the instrument and its possible implications for EHS workers.

A systematic review of the evidence supporting a causal link between dietary factors and coronary heart disease.

PubMed

Mente, Andrew; de Koning, Lawrence; Shannon, Harry S; Anand, Sonia S

2009-04-13

Although a wealth of literature links dietary factors and coronary heart disease (CHD), the strength of the evidence supporting valid associations has not been evaluated systematically in a single investigation. We conducted a systematic search of MEDLINE for prospective cohort studies or randomized trials investigating dietary exposures in relation to CHD. We used the Bradford Hill guidelines to derive a causation score based on 4 criteria (strength, consistency, temporality, and coherence) for each dietary exposure in cohort studies and examined for consistency with the findings of randomized trials. Strong evidence supports valid associations (4 criteria satisfied) of protective factors, including intake of vegetables, nuts, and "Mediterranean" and high-quality dietary patterns with CHD, and associations of harmful factors, including intake of trans-fatty acids and foods with a high glycemic index or load. Among studies of higher methodologic quality, there was also strong evidence for monounsaturated fatty acids and "prudent" and "western" dietary patterns. Moderate evidence (3 criteria) of associations exists for intake of fish, marine omega-3 fatty acids, folate, whole grains, dietary vitamins E and C, beta carotene, alcohol, fruit, and fiber. Insufficient evidence (< or =2 criteria) of association is present for intake of supplementary vitamin E and ascorbic acid (vitamin C); saturated and polyunsaturated fatty acids; total fat; alpha-linolenic acid; meat; eggs; and milk. Among the dietary exposures with strong evidence of causation from cohort studies, only a Mediterranean dietary pattern is related to CHD in randomized trials. The evidence supports a valid association of a limited number of dietary factors and dietary patterns with CHD. Future evaluation of dietary patterns, including their nutrient and food components, in cohort studies and randomized trials is recommended.

Persistent misunderstandings about evidence-based (sorry: informed!) policy-making.

PubMed

Bédard, Pierre-Olivier; Ouimet, Mathieu

2016-01-01

The field of research on knowledge mobilization and evidence-informed policy-making has seen enduring debates related to various fundamental assumptions such as the definition of 'evidence', the relative validity of various research methods, the actual role of evidence to inform policy-making, etc. In many cases, these discussions serve a useful purpose, but they also stem from serious disagreement on methodological and epistemological issues. This essay reviews the rationale for evidence-informed policy-making by examining some of the common claims made about the aims and practices of this perspective on public policy. Supplementing the existing justifications for evidence-based policy making, we argue in favor of a greater inclusion of research evidence in the policy process but in a structured fashion, based on methodological considerations. In this respect, we present an overview of the intricate relation between policy questions and appropriate research designs. By closely examining the relation between research questions and research designs, we claim that the usual points of disagreement are mitigated. For instance, when focusing on the variety of research designs that can answer a range of policy questions, the common critical claim about 'RCT-based policy-making' seems to lose some, if not all of its grip.

Using a matrix-analytical approach to synthesizing evidence solved incompatibility problem in the hierarchy of evidence.

PubMed

Walach, Harald; Loef, Martin

2015-11-01

The hierarchy of evidence presupposes linearity and additivity of effects, as well as commutativity of knowledge structures. It thereby implicitly assumes a classical theoretical model. This is an argumentative article that uses theoretical analysis based on pertinent literature and known facts to examine the standard view of methodology. We show that the assumptions of the hierarchical model are wrong. The knowledge structures gained by various types of studies are not sequentially indifferent, that is, do not commute. External validity and internal validity are at least partially incompatible concepts. Therefore, one needs a different theoretical structure, typical of quantum-type theories, to model this situation. The consequence of this situation is that the implicit assumptions of the hierarchical model are wrong, if generalized to the concept of evidence in total. The problem can be solved by using a matrix-analytical approach to synthesizing evidence. Here, research methods that produce different types of evidence that complement each other are synthesized to yield the full knowledge. We show by an example how this might work. We conclude that the hierarchical model should be complemented by a broader reasoning in methodology. Copyright © 2015 Elsevier Inc. All rights reserved.

Child maltreatment prevention: a systematic review of reviews.

PubMed

Mikton, Christopher; Butchart, Alexander

2009-05-01

To synthesize recent evidence from systematic and comprehensive reviews on the effectiveness of universal and selective child maltreatment prevention interventions, evaluate the methodological quality of the reviews and outcome evaluation studies they are based on, and map the geographical distribution of the evidence. A systematic review of reviews was conducted. The quality of the systematic reviews was evaluated with a tool for the assessment of multiple systematic reviews (AMSTAR), and the quality of the outcome evaluations was assessed using indicators of internal validity and of the construct validity of outcome measures. The review focused on seven main types of interventions: home visiting, parent education, child sex abuse prevention, abusive head trauma prevention, multi-component interventions, media-based interventions, and support and mutual aid groups. Four of the seven - home-visiting, parent education, abusive head trauma prevention and multi-component interventions - show promise in preventing actual child maltreatment. Three of them - home visiting, parent education and child sexual abuse prevention - appear effective in reducing risk factors for child maltreatment, although these conclusions are tentative due to the methodological shortcomings of the reviews and outcome evaluation studies they draw on. An analysis of the geographical distribution of the evidence shows that outcome evaluations of child maltreatment prevention interventions are exceedingly rare in low- and middle-income countries and make up only 0.6% of the total evidence base. Evidence for the effectiveness of four of the seven main types of interventions for preventing child maltreatment is promising, although it is weakened by methodological problems and paucity of outcome evaluations from low- and middle-income countries.

Evidence of Construct Validity in Published Achievement Tests.

ERIC Educational Resources Information Center

Nolet, Victor; Tindal, Gerald

Valid interpretation of test scores is the shared responsibility of the test designer and the test user. Test publishers must provide evidence of the validity of the decisions their tests are intended to support, while test users are responsible for analyzing this evidence and subsequently using the test in the manner indicated by the publisher.…

Toward validation of a structural approach to conceptualizing psychopathology: A special section of the Journal of Abnormal Psychology.

PubMed

Krueger, Robert F; Tackett, Jennifer L; MacDonald, Angus

2016-11-01

Traditionally, psychopathology has been conceptualized in terms of polythetic categories derived from committee deliberations and enshrined in authoritative psychiatric nosologies-most notably the Diagnostic and Statistical Manual of Mental Disorders (DSM; American Psychiatric Association [APA], 2013). As the limitations of this form of classification have become evident, empirical data have been increasingly relied upon to investigate the structure of psychopathology. These efforts have borne fruit in terms of an increasingly consistent set of psychopathological constructs closely connected with similar personality constructs. However, the work of validating these constructs using convergent sources of data is an ongoing enterprise. This special section collects several new efforts to use structural approaches to study the validity of this empirically based organizational scheme for psychopathology. Inasmuch as a structural approach reflects the natural organization of psychopathology, it has great potential to facilitate comprehensive organization of information on the correlates of psychopathology, providing evidence for the convergent and discriminant validity of an empirical approach to classification. Here, we highlight several themes that emerge from this burgeoning literature. (PsycINFO Database Record (c) 2016 APA, all rights reserved).

Hardiness scales in Iranian managers: evidence of incremental validity in relationships with the five factor model and with organizational and psychological adjustment.

PubMed

Ghorbani, Nima; Watson, P J

2005-06-01

This study examined the incremental validity of Hardiness scales in a sample of Iranian managers. Along with measures of the Five Factor Model and of Organizational and Psychological Adjustment, Hardiness scales were administered to 159 male managers (M age = 39.9, SD = 7.5) who had worked in their organizations for 7.9 yr. (SD=5.4). Hardiness predicted greater Job Satisfaction, higher Organization-based Self-esteem, and perceptions of the work environment as being less stressful and constraining. Hardiness also correlated positively with Assertiveness, Emotional Stability, Extraversion, Openness to Experience, Agreeableness, and Conscientiousness and negatively with Depression, Anxiety, Perceived Stress, Chance External Control, and a Powerful Others External Control. Evidence of incremental validity was obtained when the Hardiness scales supplemented the Five Factor Model in predicting organizational and psychological adjustment. These data documented the incremental validity of the Hardiness scales in a non-Western sample and thus confirmed once again that Hardiness has a relevance that extends beyond the culture in which it was developed.

Macro- and Micro-Validation: Beyond the "Five Sources" Framework for Classifying Validation Evidence and Analysis

ERIC Educational Resources Information Center

Newton, Paul E.

2016-01-01

This paper argues that the dominant framework for conceptualizing validation evidence and analysis--the "five sources" framework from the 1999 "Standards"--is seriously limited. Its limitation raises a significant barrier to understanding the nature of comprehensive validation, and this presents a significant threat to…

Beliefs and implementation of evidence-based practice among community health nurses: A cross-sectional descriptive study.

PubMed

Pereira, Filipa; Pellaux, Victoria; Verloo, Henk

2018-03-08

To describe beliefs about evidence-based practice and record levels of implementation among community health nurses working independently and in community healthcare centres in the canton of Valais, Switzerland. In many settings, evidence-based practice is considered a key means of delivering better and secure health care. However, there is a paucity of published studies on the implementation of evidence-based practice in community health care. Cross-sectional descriptive study (n = 100). Beliefs about evidence-based practice and levels of implementation were measured using validated scales developed by Melnyk et al. (Worldviews on Evidence-Based Nursing, 5, 2008, 208). Information on respondents' sociodemographic and professional characteristics was collected. Data were analysed using descriptive and inferential statistics. The final response rate was 32.3% (n = 100). More than half of respondents had previously heard about evidence-based practice; most believed in the value of using evidence to guide their practice and were prepared to improve their skills to be able to do so. However, the rate of implementation of evidence-based practice in daily practice in the 8 weeks before the survey was poor. Statistically significant positive associations were found between beliefs about evidence-based practice and how respondents had heard about it and between implementation rates and whether they had heard about evidence-based practice and how they had done so. Evidence-based practices requiring scientific knowledge and skills were implemented less frequently. Greater professional community healthcare experience and management roles did not increase implementation of evidence-based practice. The systematic implementation of evidence-based practice by community health nurses working independently and in healthcare centres in Valais was rare, despite their positive beliefs about it. These results revealed the level of implementation of evidence-based practice by nurses in community healthcare settings in Valais. Further research is required to better understand their needs and expectations and to develop suitable strategies that will allow the integration of evidence-based practice into nurses' daily practice. © 2018 The Authors Journal of Clinical Nursing Published by John Wiley & Sons Ltd.

Validity and feasibility of the EMG direct observation tool (EMG-DOT).

PubMed

Leep Hunderfund, Andrea N; Rubin, Devon I; Laughlin, Ruple S; Sorenson, Eric J; Watson, James C; Jones, Lyell K; Juul, Dorthea; Park, Yoon Soo

2016-04-26

To develop a new workplace-based EMG direct observation tool (EMG-DOT) and gather validity evidence supporting its use for assessing electrodiagnostic skills among postgraduate medical trainees. The EMG-DOT was developed by experts using an iterative process. Validity evidence from content, response process, internal structure, relations to other variables, and consequences of testing was collected during the 2013-2014 academic year. Of 3,412 studies performed by trainees during the study period, 299 (9%) were assessed using the EMG-DOT. Of these, 203 (68%) involved a physician rater and 96 (32%) involved a technician rater. The 14-item EMG-DOT had excellent internal-consistency reliability (Cronbach α 0.94). Correlations between individual items and criterion-referenced global ratings of performance ranged from 0.36 to 0.72 (all p < 0.001). Mean total scores increased from 70% to 80% over 4 months of the EMG rotation (p < 0.001) despite a corresponding significant increase in case complexity (0.21-0.74 on a 3-point rating scale; p < 0.001). Trainees reported that the observational assessment exercise improved their knowledge or skills in 82% of encounters (188/230) and that feedback generated by the EMG-DOT improved the quality of care provided to patients in 58% (133/230). Trainees were "satisfied" or "very satisfied" with the observational assessment exercise in 96% of encounters (234/243). This study provides validity evidence supporting the use of EMG-DOT scores to assess electrodiagnostic skills of residents and fellows. The EMG-DOT can be used to inform milestone-based assessments of trainee performance in neurology, child neurology, physical medicine and rehabilitation, neuromuscular, and clinical neurophysiology training programs. © 2016 American Academy of Neurology.

When is good, good enough? Methodological pragmatism for sustainable guideline development.

PubMed

Browman, George P; Somerfield, Mark R; Lyman, Gary H; Brouwers, Melissa C

2015-03-06

Continuous escalation in methodological and procedural rigor for evidence-based processes in guideline development is associated with increasing costs and production delays that threaten sustainability. While health research methodologists are appropriately responsible for promoting increasing rigor in guideline development, guideline sponsors are responsible for funding such processes. This paper acknowledges that other stakeholders in addition to methodologists should be more involved in negotiating trade-offs between methodological procedures and efficiency in guideline production to produce guidelines that are 'good enough' to be trustworthy and affordable under specific circumstances. The argument for reasonable methodological compromise to meet practical circumstances is consistent with current implicit methodological practice. This paper proposes a conceptual tool as a framework to be used by different stakeholders in negotiating, and explicitly reporting, reasonable compromises for trustworthy as well as cost-worthy guidelines. The framework helps fill a transparency gap in how methodological choices in guideline development are made. The principle, 'when good is good enough' can serve as a basis for this approach. The conceptual tool 'Efficiency-Validity Methodological Continuum' acknowledges trade-offs between validity and efficiency in evidence-based guideline development and allows for negotiation, guided by methodologists, of reasonable methodological compromises among stakeholders. Collaboration among guideline stakeholders in the development process is necessary if evidence-based guideline development is to be sustainable.

Diagnostic tools for post-gastric bypass hypoglycaemia.

PubMed

Emous, M; Ubels, F L; van Beek, A P

2015-10-01

In spite of its evident success, several late complications can occur after gastric bypass surgery. One of these is post-gastric bypass hypoglycaemia. No evidence-based guidelines exist in the literature on how to confirm the presence of this syndrome. This study aims to describe and compare the tests aimed at making a diagnosis of post-gastric bypass hypoglycaemia and to provide a diagnostic approach based upon the available evidence. A search was conducted in PubMed, Cochrane and Embase. A few questionnaires have been developed to measure the severity of symptoms in post-gastric bypass hypoglycaemia but none has been validated. The gold standard for provocation of a hypoglycaemic event is the oral glucose tolerance test or the liquid mixed meal tolerance test. Both show a high prevalence of hypoglycaemia in post-gastric bypass patients with and without hypoglycaemic complaints as well as in healthy volunteers. No uniformly established cut-off values for glucose concentrations are defined in the literature for the diagnosis of post-gastric bypass hypoglycaemia. For establishing an accurate diagnosis of post-gastric bypass hypoglycaemia, a validated questionnaire, in connection with the diagnostic performance of provocation tests, is the most important thing missing. Given these shortcomings, we provide recommendations based upon the current literature. © 2015 World Obesity.

Assessing child and adolescent pragmatic language competencies: toward evidence-based assessments.

PubMed

Russell, Robert L; Grizzle, Kenneth L

2008-06-01

Using language appropriately and effectively in social contexts requires pragmatic language competencies (PLCs). Increasingly, deficits in PLCs are linked to child and adolescent disorders, including autism spectrum, externalizing, and internalizing disorders. As the role of PLCs expands in diagnosis and treatment of developmental psychopathology, psychologists and educators will need to appraise and select clinical and research PLC instruments for use in assessments and/or studies. To assist in this appraisal, 24 PLC instruments, containing 1,082 items, are assessed by addressing four questions: (1) Can PLC domains targeted by assessment items be reliably identified?, (2) What are the core PLC domains that emerge across the 24 instruments?, (3) Do PLC questionnaires and tests assess similar PLC domains?, and (4) Do the instruments achieve content, structural, diagnostic, and ecological validity? Results indicate that test and questionnaire items can be reliably categorized into PLC domains, that PLC domains featured in questionnaires and tests significantly differ, and that PLC instruments need empirical confirmation of their dimensional structure, content validity across all developmental age bands, and ecological validity. Progress in building a better evidence base for PLC assessments should be a priority in future research.

Psychological Autopsy Studies as Diagnostic Tools: Are they Methodologically Flawed?

PubMed Central

Hjelmeland, Heidi; Dieserud, Gudrun; Dyregrov, Kari; Knizek, Birthe L.; Leenaars, Antoon A.

2012-01-01

One of the most established “truths” in suicidology is that almost all (90 % or more) of those who kill themselves suffer from one or more mental disorders, and a causal link between the two is implied. Psychological autopsy (PA) studies constitute one main evidence base for this conclusion. However, there has been little reflection on the reliability and validity of this method. For example, psychiatric diagnoses are assigned to people who have died by suicide by interviewing a few of the relatives and/or friends, often many years after the suicide. In this article, we scrutinize PA studies with particular focus on the diagnostic process and demonstrate that they cannot constitute a valid evidence base for a strong relationship between mental disorders and suicide. We show that most questions asked to assign a diagnosis are impossible to answer reliably by proxies, and thus, one cannot validly make conclusions. Thus, as a diagnostic tool psychological autopsies should now be abandoned. Instead, we recommend qualitative approaches focusing on the understanding of suicide beyond mental disorders, where narratives from a relatively high number of informants around each suicide are systematically analyzed in terms of the informants’ relationships with the deceased. PMID:24563941

A multi-source feedback tool for measuring a subset of Pediatrics Milestones.

PubMed

Schwartz, Alan; Margolis, Melissa J; Multerer, Sara; Haftel, Hilary M; Schumacher, Daniel J

2016-10-01

The Pediatrics Milestones Assessment Pilot employed a new multisource feedback (MSF) instrument to assess nine Pediatrics Milestones among interns and subinterns in the inpatient context. To report validity evidence for the MSF tool for informing milestone classification decisions. We obtained MSF instruments by different raters per learner per rotation. We present evidence for validity based on the unified validity framework. One hundred and ninety two interns and 41 subinterns at 18 Pediatrics residency programs received a total of 1084 MSF forms from faculty (40%), senior residents (34%), nurses (22%), and other staff (4%). Variance in ratings was associated primarily with rater (32%) and learner (22%). The milestone factor structure fit data better than simpler structures. In domains except professionalism, ratings by nurses were significantly lower than those by faculty and ratings by other staff were significantly higher. Ratings were higher when the rater observed the learner for longer periods and had a positive global opinion of the learner. Ratings of interns and subinterns did not differ, except for ratings by senior residents. MSF-based scales correlated with summative milestone scores. We obtain moderately reliable MSF ratings of interns and subinterns in the inpatient context to inform some milestone assignments.

Social marketing: should it be used to promote evidence-based health information?

PubMed

Formoso, Giulio; Marata, Anna Maria; Magrini, Nicola

2007-02-01

The implementation of public health knowledge is a complex process; researchers focus on organizational barriers but generally give little attention to the format and validity of relevant information. Primary and secondary papers and practice guidelines should represent valid and relevant sources of knowledge for clinicians and others involved in public health. However, this information is usually targeted at researchers rather than practitioners; it is often not completely intelligible, does not explain what it really adds to existing knowledge or which clinical/organizational context to place it in, and often lacks 'appeal' for those who are less informed. Moreover, this information is sometimes founded on biased research, shaped by sponsors to give scientific plausibility to market-driven messages. A "social marketing" approach can help public health researchers make evidence-based information clear and appealing. The validity and relevance of this information can be explained to target readers in light of their own knowledge levels and in terms of how this information could help their practice. In this paper we analyse the barriers to knowledge transfer that are often inherent in the format of the information, and propose a more user-friendly, enriched and non-research-article format.

Drive: Theory and Construct Validation

PubMed Central

Petrides, K. V.

2016-01-01

This article explicates the theory of drive and describes the development and validation of two measures. A representative set of drive facets was derived from an extensive corpus of human attributes (Study 1). Operationalised using an International Personality Item Pool version (the Drive:IPIP), a three-factor model was extracted from the facets in two samples and confirmed on a third sample (Study 2). The multi-item IPIP measure showed congruence with a short form, based on single-item ratings of the facets, and both demonstrated cross-informant reliability. Evidence also supported the measures’ convergent, discriminant, concurrent, and incremental validity (Study 3). Based on very promising findings, the authors hope to initiate a stream of research in what is argued to be a rather neglected niche of individual differences and non-cognitive assessment. PMID:27409773

Services Received and Parental Perception of Quality of Life for Children with Autism Spectrum Disorder

ERIC Educational Resources Information Center

Cholewicki, Judith Marie

2015-01-01

With the rapid increase in the rate of children diagnosed with Autism Spectrum Disorder (ASD), there has been a surge in treatment interventions and outcome measures. Treatment interventions consist of evidence-based practices and programs that lack scientific validation. Parents' selection of a treatment or multiple treatments is often based on…

Developing a Knowledge Base for Educational Leadership and Management in East Asia

ERIC Educational Resources Information Center

Hallinger, Philip

2011-01-01

The role of school leadership in educational reform has reached the status of a truism, and led to major changes in school leader recruitment, selection, training and appraisal. While similar policy trends are evident in East Asia, the empirical knowledge base underlying these measures is distorted and lacking in validation. This paper begins by…

Utilizing the M-Scan to Measure Standards-Based Mathematics Teaching Practices: Affordances and Limitations

ERIC Educational Resources Information Center

Walkowiak, Temple A.; Berry, Robert Q.; Pinter, Holly H.; Jacobson, Erik D.

2018-01-01

The Mathematics Scan (M-Scan), a content-specific observational measure, was utilized to examine the extent to which "standards-based mathematics teaching practices" were present in three focal lessons. While previous studies have provided evidence of validity of the inferences drawn from M-Scan data, no prior work has investigated the…

Early Validation Evidence of a Canadian Practitioner-Based Assessment of Physical Literacy in Physical Education: Passport for Life

ERIC Educational Resources Information Center

Lodewyk, Ken R.; Mandigo, James L.

2017-01-01

Physical and Health Education Canada has developed and implemented a formative, criterion-referenced, and practitioner-based national (Canadian) online educational assessment and support resource called Passport for Life (PFL). It was developed to support the awareness and advancement of physical literacy among PE students and teachers. PFL…

Technical Adequacy and Acceptability of Curriculum-Based Measurement and the Measures of Academic Progress

ERIC Educational Resources Information Center

January, Stacy-Ann A.; Ardoin, Scott P.

2015-01-01

Curriculum-based measurement in reading (CBM-R) and the Measures of Academic Progress (MAP) are assessment tools widely employed for universal screening in schools. Although a large body of research supports the validity of CBM-R, limited empirical evidence exists supporting the technical adequacy of MAP or the acceptability of either measure for…

Initial Validation of the Prekindergarten Classroom Observation Tool and Goal Setting System for Data-Based Coaching

ERIC Educational Resources Information Center

Crawford, April D.; Zucker, Tricia A.; Williams, Jeffrey M.; Bhavsar, Vibhuti; Landry, Susan H.

2013-01-01

Although coaching is a popular approach for enhancing the quality of Tier 1 instruction, limited research has addressed observational measures specifically designed to focus coaching on evidence-based practices. This study explains the development of the prekindergarten (pre-k) Classroom Observation Tool (COT) designed for use in a data-based…

Construct and Predictive Validity Evidence for Curriculum-Based Measures of Early Literacy and Numeracy Skills in Kindergarten

ERIC Educational Resources Information Center

Betts, Joseph; Pickart, Mary; Heistad, Dave

2009-01-01

The assessment of early literacy and numeracy skills can provide useful and important information in pursuance of the goal to increase student academic achievement. At present, there have been promising results using curriculum-based measurement (CBM) for evaluating early literacy and early numeracy. There has been little research investigating…

Validation and reliability of the VF-14 questionnaire in a German population.

PubMed

Chiang, Peggy Pei-Chia; Fenwick, Eva; Marella, Manjula; Finger, Robert; Lamoureux, Ecosse

2011-11-21

To evaluate the validity, reliability, and measurement characteristics of the Visual Function 14 (VF-14) in a German sample using Rasch analysis. This was a clinic-based, cross-sectional study with 184 patients with low vision recruited from an outpatient clinic at a German eye hospital. Participants underwent a clinical examination and completed the German VF-14 scale. The validity of the VF-14 scale was assessed using Rasch analysis. The main outcome measure was the overall functional score provided by the VF-14. After collapsing two response categories for items 13 and 14, the VF-14 scale satisfied fundamental criteria to achieve fit to the Rasch model, namely, ordered thresholds, the ability to distinguish between different strata of participant ability, absence of misfitting items, no evidence of unidimensionality, and no significant differential item functioning for key sociodemographic covariates. The VF-14 is able to discriminate between participants with different levels of vision impairment and across different cultural groups. The VF-14 is a valid, reliable, and unidimensional questionnaire for use in a German population. These findings contribute to the growing evidence base for second generation patient reported outcome measures in ophthalmology, and support the use of the German VF-14 in tertiary eye clinics in Germany to capture the impact of visual impairment on visual function from the patient's perspective and to inform low vision rehabilitation and interventions.

A new test for the assessment of working memory in clinical settings: Validation and norming of a month ordering task.

PubMed

Buekenhout, Imke; Leitão, José; Gomes, Ana A

2018-05-24

Month ordering tasks have been used in experimental settings to obtain measures of working memory (WM) capacity in older/clinical groups based solely on their face validity. We sought to assess the appropriateness of using a month ordering task in other contexts, including clinical settings, as a psychometrically sound WM assessment. To this end, we constructed a month ordering task (ucMOT), studied its reliability (internal consistency and temporal stability), and gathered construct-related and criterion-related validity evidence for its use as a WM assessment. The ucMOT proved to be internally consistent and temporally stable, and analyses of the criterion-related validity evidence revealed that its scores predicted the efficiency of language comprehension processes known to depend crucially on WM resources, namely, processes involved in pronoun interpretation. Furthermore, all ucMOT items discriminated between younger and older age groups; the global scores were significantly correlated with scores on well-established WM tasks and presented lower correlations with instruments that evaluate different (although related) processes, namely, inhibition and processing speed. We conclude that the ucMOT possesses solid psychometric properties. Accordingly, we acquired normative data for the Portuguese population, which we present as a regression-based algorithm that yields z scores adjusted for age, gender, and years of formal education. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Improving the governance of patient safety in emergency care: a systematic review of interventions

PubMed Central

Hesselink, Gijs; Berben, Sivera; Beune, Thimpe

2016-01-01

Objectives To systematically review interventions that aim to improve the governance of patient safety within emergency care on effectiveness, reliability, validity and feasibility. Design A systematic review of the literature. Methods PubMed, EMBASE, Cumulative Index to Nursing and Allied Health Literature, the Cochrane Database of Systematic Reviews and PsychInfo were searched for studies published between January 1990 and July 2014. We included studies evaluating interventions relevant for higher management to oversee and manage patient safety, in prehospital emergency medical service (EMS) organisations and hospital-based emergency departments (EDs). Two reviewers independently selected candidate studies, extracted data and assessed study quality. Studies were categorised according to study quality, setting, sample, intervention characteristics and findings. Results Of the 18 included studies, 13 (72%) were non-experimental. Nine studies (50%) reported data on the reliability and/or validity of the intervention. Eight studies (44%) reported on the feasibility of the intervention. Only 4 studies (22%) reported statistically significant effects. The use of a simulation-based training programme and well-designed incident reporting systems led to a statistically significant improvement of safety knowledge and attitudes by ED staff and an increase of incident reports within EDs, respectively. Conclusions Characteristics of the interventions included in this review (eg, anonymous incident reporting and validation of incident reports by an independent party) could provide useful input for the design of an effective tool to govern patient safety in EMS organisations and EDs. However, executives cannot rely on a robust set of evidence-based and feasible tools to govern patient safety within their emergency care organisation and in the chain of emergency care. Established strategies from other high-risk sectors need to be evaluated in emergency care settings, using an experimental design with valid outcome measures to strengthen the evidence base. PMID:26826151

Study design elements for rigorous quasi-experimental comparative effectiveness research.

PubMed

Maciejewski, Matthew L; Curtis, Lesley H; Dowd, Bryan

2013-03-01

Quasi-experiments are likely to be the workhorse study design used to generate evidence about the comparative effectiveness of alternative treatments, because of their feasibility, timeliness, affordability and external validity compared with randomized trials. In this review, we outline potential sources of discordance in results between quasi-experiments and experiments, review study design choices that can improve the internal validity of quasi-experiments, and outline innovative data linkage strategies that may be particularly useful in quasi-experimental comparative effectiveness research. There is an urgent need to resolve the debate about the evidentiary value of quasi-experiments since equal consideration of rigorous quasi-experiments will broaden the base of evidence that can be brought to bear in clinical decision-making and governmental policy-making.

Examining perceptions of academic stress and its sources among university students: The Perception of Academic Stress Scale

PubMed Central

Bedewy, Dalia

2015-01-01

The development of a scale to measure perceived sources of academic stress among university students. Based on empirical evidence and recent literature review, we developed an 18-item scale to measure perceptions of academic stress and its sources. Experts (n = 12) participated in the content validation process of the instrument before it was administered to (n = 100) students. The developed instrument has internal consistency reliability of 0.7 (Cronbach’s alpha), there was evidence for content validity, and factor analysis resulted in four correlated and theoretically meaningful factors. We developed and tested a scale to measure academic stress and its sources. This scale takes 5 minutes to complete. PMID:28070363

The Role of Ursodeoxycholic Acid in Acute Viral Hepatitis: an Evidence-based Case Report.

PubMed

Wijaya, Indra

2015-10-01

to review the role of ursodeoxycholic acid in acute viral hepatitis. following literature searching according to the clinical question on Pubmed and Cochrane Library. After filtered with our inclusion and exclusion criteria, one meta-analysis and two randomized clinical trials are obtained. Through critical appraisal, it was concluded that the articles meet the criteria for validity and relevance. the article found that there is a positive effect of ursodeoxycholic acid on the activity of serum transaminases and cholestasis indexes. However, there is insufficient evidence to support or to refute effects of ursodeoxycholic acid on disease's course as well as the viral load. better method of clinical trials are needed to obtain a valid and applicable result for daily practice.

Visual judgment of similarity across shape transformations: evidence for a compositional model of articulated objects.

PubMed

Barenholtz, Elan; Tarr, Michael J

2008-06-01

A single biological object, such as a hand, can assume multiple, very different shapes, due to the articulation of its parts. Yet we are able to recognize all of these shapes as examples of the same object. How is this invariance to pose achieved? Here, we present evidence that the visual system maintains a model of object transformation that is based on rigid, convex parts articulating at extrema of negative curvature, i.e., part boundaries. We compared similarity judgments in a task in which subjects had to decide which of the two transformed versions of a 'base' shape-one a 'biologically valid' articulation and one a geometrically similar but 'biologically invalid' articulation-was more similar to the base shape. Two types of comparisons were made: in the figure/ground-reversal, the invalid articulation consisted of exactly the same contour transformation as the valid one with reversed figural polarity. In the axis-of-rotation reversal, the valid articulation consisted of a part rotated around its concave part boundaries, while the invalid articulation consisted of the same part rotated around the endpoints on the opposite side of the part. In two separate 2AFC similarity experiments-one in which the base and transformed shapes were presented simultaneously and one in which they were presented sequentially-subjects were more likely to match the base shape to a transform when it corresponded to a legitimate articulation. These results suggest that the visual system maintains expectations about the way objects will transform, based on their static geometry.

Evidence-based Assessment in Pediatric Psychology: Family Measures

PubMed Central

Fiese, Barbara H.; Gold, Jeffrey I.; Cutuli, J. J.; Holmbeck, Grayson N.; Goldbeck, Lutz; Chambers, Christine T.; Abad, Mona; Spetter, Dante; Patterson, Joän

2008-01-01

Objective To provide a review of the evidence base of family measures relevant to pediatric psychology. Method Twenty-nine family measures were selected based upon endorsement by Division 54 listserv members, expert judgment, and literature review. Spanning observational and self-report methods, the measures fell into three broad assessment categories: Family functioning, Dyadic family relationships, and Family functioning in the context of childhood chronic health conditions. Measures were categorized as: “Well-established”, “Approaching well-established”, or “Promising.” Results Nineteen measures met “well-established” criteria and the remaining ten were “approaching well-established.” “Well-established” measures were documented for each of the broad assessment categories named above. Conclusions Many measures deemed “well-established” in the general population are proving to be reliable and useful in pediatric samples. More evidence of the validity of family measures is needed in this context. This review should prove helpful to clinicians and researchers as they strive to make evidence-based decisions regarding family measures. PMID:17905801

Evidence-based Assessment in Pediatric Psychology: Measures of Psychosocial Adjustment and Psychopathology

PubMed Central

Thill, Azure Welborn; Bachanas, Pamela; Garber, Judy; Miller, Karen Bearman; Abad, Mona; Bruno, Elizabeth Franks; Carter, Jocelyn Smith; David-Ferdon, Corinne; Jandasek, Barbara; Mennuti-Washburn, Jean E.; O’Mahar, Kerry; Zukerman, Jill

2008-01-01

Objective To provide an evidence-based review of measures of psychosocial adjustment and psychopathology, with a specific focus on their use in the field of pediatric psychology. Methods As part of a larger survey of pediatric psychologists from the Society of Pediatric Psychology e-mail listserv (American Psychological Association, APA, Division 54), 37 measures were selected for this psychometric review. Measures that qualified for the review fell into one of the following three categories: (a) internalizing or externalizing rating scales, (b) broad-band rating scales, and (c) self-related rating scales. Results Psychometric characteristics (i.e., three types of reliability, two types of validity) were strong for the majority of measures reviewed, with 34 of the 37 measures meeting “well-established” evidence-based assessment (EBA) criteria. Strengths and weaknesses of existing measures were noted. Conclusions Recommendations for future work in this area of assessment are presented, including suggestions that more fine-grained EBA criteria be developed and that evidence-based “profiles” be devised for each measure. PMID:17728305

Examining construct and predictive validity of the Health-IT Usability Evaluation Scale: confirmatory factor analysis and structural equation modeling results.

PubMed

Yen, Po-Yin; Sousa, Karen H; Bakken, Suzanne

2014-10-01

In a previous study, we developed the Health Information Technology Usability Evaluation Scale (Health-ITUES), which is designed to support customization at the item level. Such customization matches the specific tasks/expectations of a health IT system while retaining comparability at the construct level, and provides evidence of its factorial validity and internal consistency reliability through exploratory factor analysis. In this study, we advanced the development of Health-ITUES to examine its construct validity and predictive validity. The health IT system studied was a web-based communication system that supported nurse staffing and scheduling. Using Health-ITUES, we conducted a cross-sectional study to evaluate users' perception toward the web-based communication system after system implementation. We examined Health-ITUES's construct validity through first and second order confirmatory factor analysis (CFA), and its predictive validity via structural equation modeling (SEM). The sample comprised 541 staff nurses in two healthcare organizations. The CFA (n=165) showed that a general usability factor accounted for 78.1%, 93.4%, 51.0%, and 39.9% of the explained variance in 'Quality of Work Life', 'Perceived Usefulness', 'Perceived Ease of Use', and 'User Control', respectively. The SEM (n=541) supported the predictive validity of Health-ITUES, explaining 64% of the variance in intention for system use. The results of CFA and SEM provide additional evidence for the construct and predictive validity of Health-ITUES. The customizability of Health-ITUES has the potential to support comparisons at the construct level, while allowing variation at the item level. We also illustrate application of Health-ITUES across stages of system development. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Introducing a Chair-Side Novel Approach to Reach Evidence-based Periodontal Information in the Daily Periodontal Practice

PubMed Central

Dannan, Aous

2009-01-01

Background Evidence-based healthcare is not an easier approach to patient management, but should provide both clinicians and patients with greater confidence and trust in their mutual relationship. The intellectual embrace of evidence-based methods, coupled with clinical expertise and consideration of the patients individual uniqueness and requirements, is needed for all periodontal therapists if optimum care is the goal. One important element of evidence-based decision making in periodontology is the systematic review. Systematic reviews usually provide the periodontist with the highest level of evidence which should be taken into consideration when constructing any treatment plan in the dental clinic. However, reaching systematic reviews might be a time-consuming procedure that needs further personal skills. Methods In this paper, a chair-side novel approach to facilitate the incorporation of systematic reviews into daily periodontal practice is presented. It is based on three simple tools, namely, a list of suitable periodontics-related key words, a data bank of all up-to-date published systematic reviews in periodontology, and hand-made paper sheets to match the key words with their related systematic review statements. Results and Conclusions A primary validation of this method indicated the simplicity in learning and application. Keywords Chair-side; Evidence-based medicine; Periodontology; Systematic review PMID:22461868

Synthesizing Quantitative Evidence for Evidence-based Nursing: Systematic Review.

PubMed

Oh, Eui Geum

2016-06-01

As evidence-based practice has become an important issue in healthcare settings, the educational needs for knowledge and skills for the generation and utilization of healthcare evidence are increasing. Systematic review (SR), a way of evidence generation, is a synthesis of primary scientific evidence, which summarizes the best evidence on a specific clinical question using a transparent, a priori protocol driven approach. SR methodology requires a critical appraisal of primary studies, data extraction in a reliable and repeatable way, and examination for validity of the results. SRs are considered hierarchically as the highest form of evidence as they are a systematic search, identification, and summarization of the available evidence to answer a focused clinical question with particular attention to the methodological quality of studies or the credibility of opinion and text. The purpose of this paper is to introduce an overview of the fundamental knowledge, principals and processes in SR. The focus of this paper is on SR especially for the synthesis of quantitative data from primary research studies that examines the effectiveness of healthcare interventions. To activate evidence-based nursing care in various healthcare settings, the best and available scientific evidence are essential components. This paper will include some examples to promote understandings. Copyright © 2016. Published by Elsevier B.V.

Application of evidence-based dentistry: from research to clinical periodontal practice.

PubMed

Kwok, Vivien; Caton, Jack G; Polson, Alan M; Hunter, Paul G

2012-06-01

Dentists need to make daily decisions regarding patient care, and these decisions should essentially be scientifically sound. Evidence-based dentistry is meant to empower clinicians to provide the most contemporary treatment. The benefits of applying the evidence-based method in clinical practice include application of the most updated treatment and stronger reasoning to justify the treatment. A vast amount of information is readily accessible with today's digital technology, and a standardized search protocol can be developed to ensure that a literature search is valid, specific and repeatable. It involves developing a preset question (population, intervention, comparison and outcome; PICO) and search protocol. It is usually used academically to perform commissioned reviews, but it can also be applied to answer simple clinical queries. The scientific evidence thus obtained can then be considered along with patient preferences and values, clinical patient circumstances and the practitioner's experience and judgment in order to make the treatment decision. This paper describes how clinicians can incorporate evidence-based methods into patient care and presents a clinical example to illustrate the process. © 2012 John Wiley & Sons A/S.

Design process and preliminary psychometric study of a video game to detect cognitive impairment in senior adults.

PubMed

Valladares-Rodriguez, Sonia; Perez-Rodriguez, Roberto; Facal, David; Fernandez-Iglesias, Manuel J; Anido-Rifon, Luis; Mouriño-Garcia, Marcos

2017-01-01

Assessment of episodic memory has been traditionally used to evaluate potential cognitive impairments in senior adults. Typically, episodic memory evaluation is based on personal interviews and pen-and-paper tests. This article presents the design, development and a preliminary validation of a novel digital game to assess episodic memory intended to overcome the limitations of traditional methods, such as the cost of its administration, its intrusive character, the lack of early detection capabilities, the lack of ecological validity, the learning effect and the existence of confounding factors. Our proposal is based on the gamification of the California Verbal Learning Test (CVLT) and it has been designed to comply with the psychometric characteristics of reliability and validity. Two qualitative focus groups and a first pilot experiment were carried out to validate the proposal. A more ecological, non-intrusive and better administrable tool to perform cognitive assessment was developed. Initial evidence from the focus groups and pilot experiment confirmed the developed game's usability and offered promising results insofar its psychometric validity is concerned. Moreover, the potential of this game for the cognitive classification of senior adults was confirmed, and administration time is dramatically reduced with respect to pen-and-paper tests. Additional research is needed to improve the resolution of the game for the identification of specific cognitive impairments, as well as to achieve a complete validation of the psychometric properties of the digital game. Initial evidence show that serious games can be used as an instrument to assess the cognitive status of senior adults, and even to predict the onset of mild cognitive impairments or Alzheimer's disease.

Design process and preliminary psychometric study of a video game to detect cognitive impairment in senior adults

PubMed Central

Perez-Rodriguez, Roberto; Facal, David; Fernandez-Iglesias, Manuel J.; Anido-Rifon, Luis; Mouriño-Garcia, Marcos

2017-01-01

Introduction Assessment of episodic memory has been traditionally used to evaluate potential cognitive impairments in senior adults. Typically, episodic memory evaluation is based on personal interviews and pen-and-paper tests. This article presents the design, development and a preliminary validation of a novel digital game to assess episodic memory intended to overcome the limitations of traditional methods, such as the cost of its administration, its intrusive character, the lack of early detection capabilities, the lack of ecological validity, the learning effect and the existence of confounding factors. Materials and Methods Our proposal is based on the gamification of the California Verbal Learning Test (CVLT) and it has been designed to comply with the psychometric characteristics of reliability and validity. Two qualitative focus groups and a first pilot experiment were carried out to validate the proposal. Results A more ecological, non-intrusive and better administrable tool to perform cognitive assessment was developed. Initial evidence from the focus groups and pilot experiment confirmed the developed game’s usability and offered promising results insofar its psychometric validity is concerned. Moreover, the potential of this game for the cognitive classification of senior adults was confirmed, and administration time is dramatically reduced with respect to pen-and-paper tests. Limitations Additional research is needed to improve the resolution of the game for the identification of specific cognitive impairments, as well as to achieve a complete validation of the psychometric properties of the digital game. Conclusion Initial evidence show that serious games can be used as an instrument to assess the cognitive status of senior adults, and even to predict the onset of mild cognitive impairments or Alzheimer’s disease. PMID:28674661

Validation of the GreenLight™ Simulator and development of a training curriculum for photoselective vaporisation of the prostate.

PubMed

Aydin, Abdullatif; Muir, Gordon H; Graziano, Manuela E; Khan, Muhammad Shamim; Dasgupta, Prokar; Ahmed, Kamran

2015-06-01

To assess face, content and construct validity, and feasibility and acceptability of the GreenLight™ Simulator as a training tool for photoselective vaporisation of the prostate (PVP), and to establish learning curves and develop an evidence-based training curriculum. This prospective, observational and comparative study, recruited novice (25 participants), intermediate (14) and expert-level urologists (seven) from the UK and Europe at the 28th European Association of Urological Surgeons Annual Meeting 2013. A group of novices (12 participants) performed 10 sessions of subtask training modules followed by a long operative case, whereas a second group (13) performed five sessions of a given case module. Intermediate and expert groups performed all training modules once, followed by one operative case. The outcome measures for learning curves and construct validity were time to task, coagulation time, vaporisation time, average sweep speed, average laser distance, blood loss, operative errors, and instrument cost. Face and content validity, feasibility and acceptability were addressed through a quantitative survey. Construct validity was demonstrated in two of five training modules (P = 0.038; P = 0.018) and in a considerable number of case metrics (P = 0.034). Learning curves were seen in all five training modules (P < 0.001) and significant reduction in case operative time (P < 0.001) and error (P = 0.017) were seen. An evidence-based training curriculum, to help trainees acquire transferable skills, was produced using the results. This study has shown the GreenLight Simulator to be a valid and useful training tool for PVP. It is hoped that by using the training curriculum for the GreenLight Simulator, novice trainees can acquire skills and knowledge to a predetermined level of proficiency. © 2014 The Authors. BJU International © 2014 BJU International.

Educating change agents: a qualitative descriptive study of graduates of a Master's program in evidence-based practice.

PubMed

Hole, Grete Oline; Brenna, Sissel Johansson; Graverholt, Birgitte; Ciliska, Donna; Nortvedt, Monica Wammen

2016-02-25

Health care professionals are expected to build decisions upon evidence. This implies decisions based on the best available, current, valid and relevant evidence, informed by clinical expertise and patient values. A multi-professional master's program in evidence-based practice was developed and offered. The aims of this study were to explore how students in this program viewed their ability to apply evidence-based practice and their perceptions of what constitute necessary conditions to implement evidence-based practice in health care organizations, one year after graduation. A qualitative descriptive design was chosen to examine the graduates' experiences. All students in the first two cohorts of the program were invited to participate. Six focus-group interviews, with a total of 21 participants, and a telephone interview of one participant were conducted. The data was analyzed thematically, using the themes from the interview guide as the starting point. The graduates reported that an overall necessary condition for evidence-based practice to occur is the existence of a "readiness for change" both at an individual level and at the organizational level. They described that they gained personal knowledge and skills to be "change-agents" with "self-efficacy, "analytic competence" and "tools" to implement evidence based practice in clinical care. An organizational culture of a "learning organization" was also required, where leaders have an "awareness of evidence- based practice", and see the need for creating "evidence-based networks". One year after graduation the participants saw themselves as "change agents" prepared to improve clinical care within a learning organization. The results of this study provides useful information for facilitating the implementation of EBP both from educational and health care organizational perspectives.

Reliability and concurrent and construct validity of the Strategies for Weight Management measure for adults.

PubMed

Kolodziejczyk, Julia K; Norman, Gregory J; Rock, Cheryl L; Arredondo, Elva M; Roesch, Scott C; Madanat, Hala; Patrick, Kevin

2016-01-01

This study evaluates the reliability and validity of the strategies for weight management (SWM) measure, a questionnaire that assesses weight management strategies for adults. The SWM includes 20 items that are categorized within the following subscales: (1) energy intake, (2) energy expenditure, (3) self-monitoring, and (4) self-regulation. Baseline and 6-month data were collected from 404 overweight/obese adults (mean age=22±3.8 years, 68% ethnic minority) enrolled in a randomized controlled trial aiming to reduce weight by improving diet and physical activity behaviours. Reliability and validity were assessed for each subscale separately. Cronbach alpha was conducted to assess reliability. Concurrent, construct I (sensitivity to the study treatment condition), and construct II (relationship to the outcomes) validity were assessed using linear regressions with the following outcome measures: weight, self-reported diet, and weekly energy expenditure. All subscales showed strong internal consistency. The strength of the validity evidence depended on subscale and validity type. The strongest validity evidence was concurrent validity of the energy intake and energy expenditure subscales; construct I validity of the energy intake and self-monitoring subscales; and construct II validity of the energy intake, energy expenditure, and self-regulation subscales. Results indicate that the SWM can be used to assess weight management strategies among an ethnically diverse sample of adults as each subscale showed evidence of reliability and select types of validity. As validity is an accumulation of evidence over multiple studies, this study provides initial reliability and validity evidence in one population segment. Copyright © 2015 Asia Oceania Association for the Study of Obesity. Published by Elsevier Ltd. All rights reserved.

Federal Public Health Workforce Development: An Evidence-Based Approach for Defining Competencies.

PubMed

Mumford, Karen; Young, Andrea C; Nawaz, Saira

2016-01-01

This study reports the use of exploratory factor analysis to describe essential skills and knowledge for an important segment of the domestic public health workforce-Centers for Disease Control and Prevention (CDC) project officers-using an evidence-based approach to competency development and validation. A multicomponent survey was conducted. Exploratory factor analysis was used to examine the underlying domains and relationships between competency domains and key behaviors. The Cronbach α coefficient determined the reliability of the overall scale and identified factors. All domestic (US state, tribe, local, and territorial) grantees who received funding from the CDC during fiscal year 2011 to implement nonresearch prevention or intervention programs were invited to participate in a Web-based questionnaire. A total of 34 key behaviors representing knowledge, skills, and abilities, grouped in 7 domains-communication, grant administration and management, public health applied science and knowledge, program planning and development, program management, program monitoring and improvement, and organizational consultation-were examined. There were 795 responses (58% response rate). A total of 6 factors were identified with loadings of 0.40 or more for all 34 behavioral items. The Cronbach α coefficient was 0.95 overall and ranged between 0.73 and 0.91 for the factors. This study provides empirical evidence for the construct validity of 6 competencies and 34 key behaviors important for CDC project officers and serves as an important first step to evidence-driven workforce development efforts in public health.

Occupational therapy-based and evidence-supported recommendations for assessment and exercises in hand osteoarthritis.

PubMed

Kjeken, Ingvild

2011-12-01

The aims of this study were to develop recommendations for occupational therapy assessment and design of hand exercise programmes in patients with hand osteoarthritis. An expert group followed a Delphi procedure to reach consensus for up to 10 recommendations for assessment and exercises, respectively. Thereafter, an evidence-based approach was used to identify and appraise research evidence supporting each recommendation, before the recommendations were validated by the expert group. The process resulted in 10 recommendations for assessment and eight for design of exercise programmes. The literature search revealed that there is a paucity of clinical trials to guide recommendations for hand osteoarthritis, and the evidence for the majority of the recommendations was based on expert opinions. Also, even if a systematic review demonstrates some evidence for the efficacy of strength training exercises in hand OA, the evidence for any specific exercise is limited to expert opinions. A first set of recommendations for assessment and exercise in hand osteoarthritis has been developed. For many of the recommendations there is a paucity of research evidence. High-quality studies are therefore needed to establish a high level of evidence concerning functional assessment and the effect of hand exercises in hand osteoarthritis.

Introduction to the special section on developing guidelines for the evidence-based assessment (EBA) of adult disorders.

PubMed

Hunsley, John; Mash, Eric J

2005-09-01

The goal of this special section is to encourage greater awareness of evidence-based assessment (EBA) in the development of a scientifically supported clinical psychology. In this introductory article, the authors describe the elements that authors in this special section were asked to consider in their focused reviews (including the scope of available psychometric evidence, advancements in psychopathology research, and evidence of attention to factors such as gender, age, and ethnicity in measure validation). The authors then present central issues evident in the articles that deal with anxiety, depression, personality disorders, and couple distress and in the accompanying commentaries. The authors conclude by presenting key themes emerging from the articles in this special section, including gaps in psychometric information, limited information about the utility of assessment, the discrepancy between recommended EBAs and current training and practice, and the need for further data on the process of clinical assessment.

Validity Evidence and Scoring Guidelines for Standardized Patient Encounters and Patient Notes From a Multisite Study of Clinical Performance Examinations in Seven Medical Schools.

PubMed

Park, Yoon Soo; Hyderi, Abbas; Heine, Nancy; May, Win; Nevins, Andrew; Lee, Ming; Bordage, Georges; Yudkowsky, Rachel

2017-11-01

To examine validity evidence of local graduation competency examination scores from seven medical schools using shared cases and to provide rater training protocols and guidelines for scoring patient notes (PNs). Between May and August 2016, clinical cases were developed, shared, and administered across seven medical schools (990 students participated). Raters were calibrated using training protocols, and guidelines were developed collaboratively across sites to standardize scoring. Data included scores from standardized patient encounters for history taking, physical examination, and PNs. Descriptive statistics were used to examine scores from the different assessment components. Generalizability studies (G-studies) using variance components were conducted to estimate reliability for composite scores. Validity evidence was collected for response process (rater perception), internal structure (variance components, reliability), relations to other variables (interassessment correlations), and consequences (composite score). Student performance varied by case and task. In the PNs, justification of differential diagnosis was the most discriminating task. G-studies showed that schools accounted for less than 1% of total variance; however, for the PNs, there were differences in scores for varying cases and tasks across schools, indicating a school effect. Composite score reliability was maximized when the PN was weighted between 30% and 40%. Raters preferred using case-specific scoring guidelines with clear point-scoring systems. This multisite study presents validity evidence for PN scores based on scoring rubric and case-specific scoring guidelines that offer rigor and feedback for learners. Variability in PN scores across participating sites may signal different approaches to teaching clinical reasoning among medical schools.

Examining validity evidence for a simulation-based assessment tool for basic robotic surgical skills.

PubMed

Havemann, Maria Cecilie; Dalsgaard, Torur; Sørensen, Jette Led; Røssaak, Kristin; Brisling, Steffen; Mosgaard, Berit Jul; Høgdall, Claus; Bjerrum, Flemming

2018-05-14

Increasing focus on patient safety makes it important to ensure surgical competency among surgeons before operating on patients. The objective was to gather validity evidence for a virtual-reality simulator test for robotic surgical skills and evaluate its potential as a training tool. Surgeons with varying experience in robotic surgery were recruited: novices (zero procedures), intermediates (1-50), experienced (> 50). Five experienced surgeons rated five exercises on the da Vinci Skills Simulator. Participants were tested using the five exercises. Participants were invited back 3 times and completed a total of 10 attempts per exercise. The outcome was the average simulator performance score for the 5 exercises. 32 participants from 5 surgical specialties were included. 38 participants completed all 4 sessions. A moderate correlation between the average total score and robotic experience was identified for the first attempt (Spearman r = 0.58; p = 0.0004). A difference in average total score was observed between novices and intermediates [median score 61% (IQR 52-66) vs. 83% (IQR 75-91), adjusted p < 0.0001], as well as novices and experienced [median score 61% (IQR 52-66) vs. 80 (IQR 69-85), adjusted p = 0.002]. All three groups improved their performance between the 1st and 10th attempts (p < 0.00). This study describes validity evidence for a virtual-reality simulator for basic robotic surgical skills, which can be used for assessment of basic competency and as a training tool. However, more validity evidence is needed before it can be used for certification or high-stakes assessment.

Questioning the Validity of the 4+/5+ Binge or Heavy Drinking Criterion in College and Clinical Populations

PubMed Central

Pearson, Matthew R.; Kirouac, Megan; Witkiewitz, Katie

2015-01-01

Background and Aims The terms “binge drinking” and “heavy drinking” are both typically operationalized as 4+/5+ standard drinks per occasion for women/men and are commonly used as a proxy for non-problematic (<4/<5) versus problematic (4+/5+) drinking in multiple research contexts. The Food and Drug Administration in the United States (US) recently proposed the 4+/5+ criterion as a primary efficacy endpoint in their guidance for trials examining new medications for alcohol use disorders (AUDs). Internationally, similar cut-offs have been proposed, with the European Medicines Agency having identified reductions in the number of heavy drinking days (defined as 40/60g pure alcohol in women/men) as a primary endpoint for efficacy trials with a harm reduction goal. Analysis and Evidence We question the validity of the 4+/5+ cutoff (and other similar cutoffs) on multiple accounts. The 4+/5+ cutoff has not been shown to have unique predictive validity or clinical utility. The cutoff has been created based on retrospective self-reports and its use demonstrates ecological bias. Given strong evidence that the relationship between alcohol consumption and problems related to drinking is at least monotonic, if not linear, there is little existing evidence to support the 4+/5+ cutoff as a valid marker of problematic alcohol use. Conclusions There is little empirical evidence for the 4+/5+ units per occasion threshold for “binge” or “heavy” drinking in indexing treatment efficacy. Further consideration of an appropriate threshold seems to be warranted. PMID:27605077

Measuring Decision-Making During Thyroidectomy: Validity Evidence for a Web-Based Assessment Tool.

PubMed

Madani, Amin; Gornitsky, Jordan; Watanabe, Yusuke; Benay, Cassandre; Altieri, Maria S; Pucher, Philip H; Tabah, Roger; Mitmaker, Elliot J

2018-02-01

Errors in judgment during thyroidectomy can lead to recurrent laryngeal nerve injury and other complications. Despite the strong link between patient outcomes and intraoperative decision-making, methods to evaluate these complex skills are lacking. The purpose of this study was to develop objective metrics to evaluate advanced cognitive skills during thyroidectomy and to obtain validity evidence for them. An interactive online learning platform was developed ( www.thinklikeasurgeon.com ). Trainees and surgeons from four institutions completed a 33-item assessment, developed based on a cognitive task analysis and expert Delphi consensus. Sixteen items required subjects to make annotations on still frames of thyroidectomy videos, and accuracy scores were calculated based on an algorithm derived from experts' responses ("visual concordance test," VCT). Seven items were short answer (SA), requiring users to type their answers, and scores were automatically calculated based on their similarity to a pre-populated repertoire of correct responses. Test-retest reliability, internal consistency, and correlation of scores with self-reported experience and training level (novice, intermediate, expert) were calculated. Twenty-eight subjects (10 endocrine surgeons and otolaryngologists, 18 trainees) participated. There was high test-retest reliability (intraclass correlation coefficient = 0.96; n = 10) and internal consistency (Cronbach's α = 0.93). The assessment demonstrated significant differences between novices, intermediates, and experts in total score (p < 0.01), VCT score (p < 0.01) and SA score (p < 0.01). There was high correlation between total case number and total score (ρ = 0.95, p < 0.01), between total case number and VCT score (ρ = 0.93, p < 0.01), and between total case number and SA score (ρ = 0.83, p < 0.01). This study describes the development of novel metrics and provides validity evidence for an interactive Web-based platform to objectively assess decision-making during thyroidectomy.

Validating the Implementation Climate Scale (ICS) in Child Welfare Organizations

PubMed Central

Ehrhart, Mark G.; Torres, Elisa M.; Wright, Lisa A.; Martinez, Sandra Y.; Aarons, Gregory A.

2015-01-01

There is increasing emphasis on the use of evidence-based practices (EBPs) in child welfare settings and growing recognition of the importance of the organizational environment, and the organization’s climate in particular, for how employees perceive and support EBP implementation. Recently, Ehrhart, Aarons, and Farahnak (2014) reported on the development and validation of a measure of EBP implementation climate, the Implementation Climate Scale (ICS), in a sample of mental health clinicians. The ICS consists of 18 items and measures six critical dimensions of implementation climate: focus on EBP, educational support for EBP, recognition for EBP, rewards for EBP, selection or EBP, and selection for openness. The goal of the current study is to extend this work by providing evidence for the factor structure, reliability, and validity of the ICS in a sample of child welfare service providers. Survey data were collected from 215 child welfare providers across three states, 12 organizations, and 43 teams. Confirmatory factor analysis demonstrated good fit to the six-factor model and the alpha reliabilities for the overall measure and its subscales was acceptable. In addition, there was general support for the invariance of the factor structure across the child welfare and mental health sectors. In conclusion, this study provides evidence for the factor structure, reliability, and validity of the ICS measure for use in child welfare service organizations. PMID:26563643

Validating the Implementation Climate Scale (ICS) in child welfare organizations.

PubMed

Ehrhart, Mark G; Torres, Elisa M; Wright, Lisa A; Martinez, Sandra Y; Aarons, Gregory A

2016-03-01

There is increasing emphasis on the use of evidence-based practices (EBPs) in child welfare settings and growing recognition of the importance of the organizational environment, and the organization's climate in particular, for how employees perceive and support EBP implementation. Recently, Ehrhart, Aarons, and Farahnak (2014) reported on the development and validation of a measure of EBP implementation climate, the Implementation Climate Scale (ICS), in a sample of mental health clinicians. The ICS consists of 18 items and measures six critical dimensions of implementation climate: focus on EBP, educational support for EBP, recognition for EBP, rewards for EBP, selection or EBP, and selection for openness. The goal of the current study is to extend this work by providing evidence for the factor structure, reliability, and validity of the ICS in a sample of child welfare service providers. Survey data were collected from 215 child welfare providers across three states, 12 organizations, and 43 teams. Confirmatory factor analysis demonstrated good fit to the six-factor model and the alpha reliabilities for the overall measure and its subscales was acceptable. In addition, there was general support for the invariance of the factor structure across the child welfare and mental health sectors. In conclusion, this study provides evidence for the factor structure, reliability, and validity of the ICS measure for use in child welfare service organizations. Copyright © 2015 Elsevier Ltd. All rights reserved.

Mapping the Moral Domain

PubMed Central

Graham, Jesse; Nosek, Brian A.; Haidt, Jonathan; Iyer, Ravi; Koleva, Spassena; Ditto, Peter H.

2010-01-01

The moral domain is broader than the empathy and justice concerns assessed by existing measures of moral competence, and it is not just a subset of the values assessed by value inventories. To fill the need for reliable and theoretically-grounded measurement of the full range of moral concerns, we developed the Moral Foundations Questionnaire (MFQ) based on a theoretical model of five universally available (but variably developed) sets of moral intuitions: Harm/care, Fairness/reciprocity, Ingroup/loyalty, Authority/respect, and Purity/sanctity. We present evidence for the internal and external validity of the scale and the model, and in doing so present new findings about morality: 1. Comparative model fitting of confirmatory factor analyses provides empirical justification for a five-factor structure of moral concerns. 2. Convergent/discriminant validity evidence suggests that moral concerns predict personality features and social group attitudes not previously considered morally relevant. 3. We establish pragmatic validity of the measure in providing new knowledge and research opportunities concerning demographic and cultural differences in moral intuitions. These analyses provide evidence for the usefulness of Moral Foundations Theory in simultaneously increasing the scope and sharpening the resolution of psychological views of morality. PMID:21244182

Clinical Prediction Models for Patients With Nontraumatic Knee Pain in Primary Care: A Systematic Review and Internal Validation Study.

PubMed

Panken, Guus; Verhagen, Arianne P; Terwee, Caroline B; Heymans, Martijn W

2017-08-01

Study Design Systematic review and validation study. Background Many prognostic models of knee pain outcomes have been developed for use in primary care. Variability among published studies with regard to patient population, outcome measures, and relevant prognostic factors hampers the generalizability and implementation of these models. Objectives To summarize existing prognostic models in patients with knee pain in a primary care setting and to develop and internally validate new summary prognostic models. Methods After a sensitive search strategy, 2 reviewers independently selected prognostic models for patients with nontraumatic knee pain and assessed the methodological quality of the included studies. All predictors of the included studies were evaluated, summarized, and classified. The predictors assessed in multiple studies of sufficient quality are presented in this review. Using data from the Musculoskeletal System Study (BAS) cohort of patients with a new episode of knee pain, recruited consecutively by Dutch general medical practitioners (n = 372), we used predictors with a strong level of evidence to develop new prognostic models for each outcome measure and internally validated these models. Results Sixteen studies were eligible for inclusion. We considered 11 studies to be of sufficient quality. None of these studies validated their models. Five predictors with strong evidence were related to function and 6 to recovery, and were used to compose 2 prognostic models for patients with knee pain at 1 year. Running these new models in another data set showed explained variances (R 2 ) of 0.36 (function) and 0.33 (recovery). The area under the curve of the recovery model was 0.79. After internal validation, the adjusted R 2 values of the models were 0.30 (function) and 0.20 (recovery), and the area under the curve was 0.73. Conclusion We developed 2 valid prognostic models for function and recovery for patients with nontraumatic knee pain, based on predictors with strong evidence. A longer duration of complaints predicted poorer function but did not adequately predict chance of recovery. Level of Evidence Prognosis, levels 1a and 1b. J Orthop Sports Phys Ther 2017;47(8):518-529. Epub 16 Jun 2017. doi:10.2519/jospt.2017.7142.

Mindfulness: A systematic review of instruments to measure an emergent patientreported outcome (PRO)

PubMed Central

Park, Taehwan; Reilly-Spong, Maryanne

2013-01-01

Purpose Mindfulness has emerged as an important health concept based on evidence that mindfulness interventions reduce symptoms and improve health-related quality of life. The objectives of this study were to systematically assess and compare the properties of instruments to measure self-reported mindfulness. Methods Ovid Medline®, CINAHL®, and PsycINFO® were searched through May 2012, and articles were selected if their primary purpose was development or evaluation of the measurement properties (validity, reliability, responsiveness) of a self-report mindfulness scale. Two reviewers independently evaluated the methodological quality of the selected studies using the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist. Discrepancies were discussed with a third reviewer, and scored by consensus. Finally, a level of evidence approach was used to synthesize results and study quality. Results Our search strategy identified a total of 2,588 articles. Forty-six articles, reporting 79 unique studies, met inclusion criteria. Ten instruments quantifying mindfulness as a unidimensional scale (n=5) or as a set of 2 to 5 subscales (n=5) were reviewed. The Mindful Attention Awareness Scale (MAAS) was evaluated by the most studies (n=27), and had positive overall quality ratings for most of the psychometric properties reviewed. The Five Facet Mindfulness Questionnaire (FFMQ) received the highest possible rating (“consistent findings in multiple studies of good methodological quality”) for two properties, internal consistency and construct validation by hypothesis testing. However, none of the instruments had sufficient evidence of content validity. Comprehensiveness of construct coverage had not been assessed; qualitative methods to confirm understanding and relevance were absent. In addition, estimates of test-retest reliability, responsiveness, or measurement error to guide users in protocol development or interpretation of scores were lacking. Conclusions Current mindfulness scales have important conceptual differences, and none can be strongly recommended based solely on superior psychometric properties. Important limitations in the field are the absence of qualitative evaluations and accepted external referents to support construct validity. Investigators need to proceed cautiously before optimizing any mindfulness intervention based on the existing scales. PMID:23539467

4: A STUDY ON THE RATE OF INFORMATION LITERACY OF FACULTY MEMBERS AND PHD STUDENTS OF FACULTY OF NURSING AND MIDWIFERY, TABRIZ UNIVERSITY OF MEDICAL SCIENCES, BASED ON SUCCESSFUL EVIDENCE HEALTHCARE

PubMed Central

Razmkhah, Maryam; Moghadam, Hadi Sharif; Ziaei, Soraya; Zarea, Vahideh; Narimani, Mohammad Reza

2017-01-01

Background and aims Evidence based care is an approach to clinical problem-solving in which merging the results of several studies and information on specialty clinical care as well as patients' wishes and values leads to effective decision making, to avoid seeking frequent care facilitating the patient cares, empowering healthcare workers, maintaining and improving the health of patients and the families. Results of the conducted studies suggest that using such an approach requires information literacy skills. Therefore, the present study aimed to assess information literacy of the faculty members and PhD students of Nursing and Midwifery School of Tabriz University of Medical Sciences about evidence based care. Methods In this cross-sectional survey 53 PhD students and faculty members were selected using census sampling method. Data gathering tool was a researcher-made questionnaire. This inventory was developed regarding valid scientific literature on information literacy and evidence-based care with 68 items and 5 standards of literacy prepared within some steps. After confirming the validity, its reliability was concluded by Cranach's Alpha (0.89). Data was analyzed using SPSS/22. Results Average information literacy skill level for faculty members and students related to evidence-based care and information literacy standards was higher than the average index, except for “information exchange” standard (50±10). The highest and lowest mean scores in evidence based care were for, respectively, questions formation (respectively, 96.18±18.6.17 and 48.51±14.69) and evaluation results (respectively 95.56±6.66 and 45.94±14.08). For information literacy standards there were calculated for (respectively) finding information as the highest score for (respectively, 95.56±6.66 and 72.44±13.62) and the lowest for information exchange (respectively, 74.19±11.83 and 48.51±11.35). Conclusion According to the results of this study and also regarding to this subject that PhD students' and faculty members' information literacy level was above the average; it is recommended to develop optimal measures to promote evidence based decision making.

A web-based library consult service for evidence-based medicine: Technical development

PubMed Central

Schwartz, Alan; Millam, Gregory

2006-01-01

Background Incorporating evidence based medicine (EBM) into clinical practice requires clinicians to learn to efficiently gain access to clinical evidence and effectively appraise its validity. Even using current electronic systems, selecting literature-based data to solve a single patient-related problem can require more time than practicing physicians or residents can spare. Clinical librarians, as informationists, are uniquely suited to assist physicians in this endeavor. Results To improve support for evidence-based practice, we have developed a web-based EBM library consult service application (LCS). Librarians use the LCS system to provide full text evidence-based literature with critical appraisal in response to a clinical question asked by a remote physician. LCS uses an entirely Free/Open Source Software platform and will be released under a Free Software license. In the first year of the LCS project, the software was successfully developed and a reference implementation put into active use. Two years of evaluation of the clinical, educational, and attitudinal impact on physician-users and librarian staff are underway, and expected to lead to refinement and wide dissemination of the system. Conclusion A web-based EBM library consult model may provide a useful way for informationists to assist clinicians, and is feasible to implement. PMID:16542453

A contemporary approach to validity arguments: a practical guide to Kane's framework.

PubMed

Cook, David A; Brydges, Ryan; Ginsburg, Shiphra; Hatala, Rose

2015-06-01

Assessment is central to medical education and the validation of assessments is vital to their use. Earlier validity frameworks suffer from a multiplicity of types of validity or failure to prioritise among sources of validity evidence. Kane's framework addresses both concerns by emphasising key inferences as the assessment progresses from a single observation to a final decision. Evidence evaluating these inferences is planned and presented as a validity argument. We aim to offer a practical introduction to the key concepts of Kane's framework that educators will find accessible and applicable to a wide range of assessment tools and activities. All assessments are ultimately intended to facilitate a defensible decision about the person being assessed. Validation is the process of collecting and interpreting evidence to support that decision. Rigorous validation involves articulating the claims and assumptions associated with the proposed decision (the interpretation/use argument), empirically testing these assumptions, and organising evidence into a coherent validity argument. Kane identifies four inferences in the validity argument: Scoring (translating an observation into one or more scores); Generalisation (using the score[s] as a reflection of performance in a test setting); Extrapolation (using the score[s] as a reflection of real-world performance), and Implications (applying the score[s] to inform a decision or action). Evidence should be collected to support each of these inferences and should focus on the most questionable assumptions in the chain of inference. Key assumptions (and needed evidence) vary depending on the assessment's intended use or associated decision. Kane's framework applies to quantitative and qualitative assessments, and to individual tests and programmes of assessment. Validation focuses on evaluating the key claims, assumptions and inferences that link assessment scores with their intended interpretations and uses. The Implications and associated decisions are the most important inferences in the validity argument. © 2015 John Wiley & Sons Ltd.

Curriculum-Based Handwriting Programs: A Systematic Review With Effect Sizes

PubMed Central

Engel, Courtney; Lillie, Kristin; Zurawski, Sarah; Travers, Brittany G.

2018-01-01

Challenges with handwriting can have a negative impact on academic performance, and these challenges are commonly addressed by occupational therapy practitioners in school settings. This systematic review examined the efficacy of curriculum-based interventions to address children’s handwriting difficulties in the classroom (preschool to second grade). We reviewed and computed effect sizes for 13 studies (11 Level II, 2 Level III) identified through a comprehensive database search. The evidence shows that curriculum-based handwriting interventions resulted in small- to medium-sized improvements in legibility, a commonly reported challenge in this age group. The evidence for whether these interventions improved speed is mixed, and the evidence for whether they improved fluency is insufficient. No clear support was found for one handwriting program over another. These results suggest that curriculum-based interventions can lead to improvements in handwriting legibility, but Level I research is needed to validate the efficacy of these curricula. PMID:29689170

An evidence-based virtual reality training program for novice laparoscopic surgeons.

PubMed

Aggarwal, Rajesh; Grantcharov, Teodor P; Eriksen, Jens R; Blirup, Dorthe; Kristiansen, Viggo B; Funch-Jensen, Peter; Darzi, Ara

2006-08-01

To develop an evidence-based virtual reality laparoscopic training curriculum for novice laparoscopic surgeons to achieve a proficient level of skill prior to participating in live cases. Technical skills for laparoscopic surgery must be acquired within a competency-based curriculum that begins in the surgical skills laboratory. Implementation of this program necessitates the definition of the validity, learning curves and proficiency criteria on the training tool. The study recruited 40 surgeons, classified into experienced (performed >100 laparoscopic cholecystectomies) or novice groups (<10 laparoscopic cholecystectomies). Ten novices and 10 experienced surgeons were tested on basic tasks, and 11 novices and 9 experienced surgeons on a procedural module for dissection of Calot triangle. Performance of the 2 groups was assessed using time, error, and economy of movement parameters. All basic tasks demonstrated construct validity (Mann-Whitney U test, P < 0.05), and learning curves for novices plateaued at a median of 7 repetitions (Friedman's test, P < 0.05). Expert surgeons demonstrated a learning rate at a median of 2 repetitions (P < 0.05). Performance on the dissection module demonstrated significant differences between experts and novices (P < 0.002); learning curves for novice subjects plateaued at the fourth repetition (P < 0.05). Expert benchmark criteria were defined for validated parameters on each task. A competency-based training curriculum for novice laparoscopic surgeons has been defined. This can serve to ensure that junior trainees have acquired prerequisite levels of skill prior to entering the operating room, and put them directly into practice.

Are we really measuring what we say we're measuring? Using video techniques to supplement traditional construct validation procedures.

PubMed

Podsakoff, Nathan P; Podsakoff, Philip M; Mackenzie, Scott B; Klinger, Ryan L

2013-01-01

Several researchers have persuasively argued that the most important evidence to consider when assessing construct validity is whether variations in the construct of interest cause corresponding variations in the measures of the focal construct. Unfortunately, the literature provides little practical guidance on how researchers can go about testing this. Therefore, the purpose of this article is to describe how researchers can use video techniques to test whether their scales measure what they purport to measure. First, we discuss how researchers can develop valid manipulations of the focal construct that they hope to measure. Next, we explain how to design a study to use this manipulation to test the validity of the scale. Finally, comparing and contrasting traditional and contemporary perspectives on validation, we discuss the advantages and limitations of video-based validation procedures. PsycINFO Database Record (c) 2013 APA, all rights reserved.

Evaluation of the measurement properties of self-reported health-related work-functioning instruments among workers with common mental disorders.

PubMed

Abma, Femke I; van der Klink, Jac J L; Terwee, Caroline B; Amick, Benjamin C; Bültmann, Ute

2012-01-01

During the past decade, common mental disorders (CMD) have emerged as a major public and occupational health problem in many countries. Several instruments have been developed to measure the influence of health on functioning at work. To select appropriate instruments for use in occupational health practice and research, the measurement properties (eg, reliability, validity, responsiveness) must be evaluated. The objective of this study is to appraise critically and compare the measurement properties of self-reported health-related work-functioning instruments among workers with CMD. A systematic review was performed searching three electronic databases. Papers were included that: (i) mainly focused on the development and/or evaluation of the measurement properties of a self-reported health-related work-functioning instrument; (ii) were conducted in a CMD population; and (iii) were fulltext original papers. Quality appraisal was performed using the consensus-based standards for the selection of health status measurement instruments (COSMIN) checklist. Five papers evaluating measurement properties of five self-reported health-related work-functioning instruments in CMD populations were included. There is little evidence available for the measurement properties of the identified instruments in this population, mainly due to low methodological quality of the included studies. The available evidence on measurement properties is based on studies of poor-to-fair methodological quality. Information on a number of measurement properties, such as measurement error, content validity, and cross-cultural validity is still lacking. Therefore, no evidence-based decisions and recommendations can be made for the use of health-related work functioning instruments. Studies of high methodological quality are needed to properly assess the existing instruments' measurement properties.

The Evidence-based Practice Attitude Scale-36 (EBPAS-36): a brief and pragmatic measure of attitudes to evidence-based practice validated in US and Norwegian samples.

PubMed

Rye, Marte; Torres, Elisa M; Friborg, Oddgeir; Skre, Ingunn; Aarons, Gregory A

2017-04-04

Short and valid instruments for measuring factors facilitating or hindering implementation efforts are called for. This article describes (1) the adaptation of a shorter version of the Evidence-based Practice Attitude Scale (EBPAS-50 items), and (2) the psychometric properties of the shortened version in both US and Norwegian data. The US participants were mental health service providers (N = 418) recruited from clinics providing mental health services in San Diego County, California. The Norwegian participants were psychologists, psychiatric nurses, and psychology students (N = 838) recruited from the Norwegian Psychological Association and the Norwegian Nurses Organization. A confirmatory factor analysis (CFA) approach was used. The reduction resulted in 36 items named EBPAS-36, and the original 12 factor model was maintained. The EBPAS-36 had acceptable model fit, as indicated by a low degree of misspecification errors in both the US (RMSEA = .045 (CI 90% .040-.049); SRMR = .05) and the Norwegian data (RMSEA = .052 (CI 90% .047-.056, SRMR = .07). Incremental model fit was fair in the US (CFI = .93, TLI = .91) and in the Norwegian samples (CFI = .91, TLI = .89). The internal consistency (Cronbach's α) in the US and the Norwegian samples were good for the total EBPAS-36 score (.79 and .86, respectively) and were ranged from adequate to excellent for the subscales (US .60-.91 and Norway .61-.92). The EBPAS-36 has adequate psychometric properties both in US and Norwegian samples, hence indicating cross-cultural validity. It is a brief, pragmatic, and more user-friendly instrument than the EBPAS-50, yet maintains a broad scope by retaining the original 12 measurement domains.

Mass-casualty triage: time for an evidence-based approach.

PubMed

Jenkins, Jennifer Lee; McCarthy, Melissa L; Sauer, Lauren M; Green, Gary B; Stuart, Stephanie; Thomas, Tamara L; Hsu, Edbert B

2008-01-01

Mass-casualty triage has developed from a wartime necessity to a civilian tool to ensure that constrained medical resources are directed at achieving the greatest good for the most number of people. Several primary and secondary triage tools have been developed, including Simple Treatment and Rapid Transport (START), JumpSTART, Care Flight Triage, Triage Sieve, Sacco Triage Method, Secondary Assessment of Victim Endpoint (SAVE), and Pediatric Triage Tape. Evidence to support the use of one triage algorithm over another is limited, and the development of effective triage protocols is an important research priority. The most widely recognized mass-casualty triage algorithms in use today are not evidence-based, and no studies directly address these issues in the mass-casualty setting. Furthermore, no studies have evaluated existing mass-casualty triage algorithms regarding ease of use, reliability, and validity when biological, chemical, or radiological agents are introduced. Currently, the lack of a standardized mass-casualty triage system that is well validated, reliable, and uniformly accepted, remains an important gap. Future research directed at triage is recognized as a necessity, and the development of a practical, universal, triage algorithm that incorporates requirements for decontamination or special precautions for infectious agents would facilitate a more organized mass-casualty medical response.

[Evidence-based medicine - the current self-reflection of an individualised approach to medicine as an action science].

PubMed

Behrens, Johann

2010-01-01

Evidence-based Medicine (EbM) is the ongoing self-reflection of an individualised approach to medicine in terms of a science that originates from and focuses on clinical decision-making (pragmatic science="Handlungswissenschaft"). EbM is particularly suitable for self-reflecting individualised medicine on the basis of decision-oriented pragmatic science because it consistently distinguishes between external evidence (i.e., other subjects' experience gained through "qualitative" and "quantitative" scientific methods) and internal evidence, i.e., the individual user's, or patient's, own experience manifesting and developing in the individual contact between therapist and patient. Therefore, internal evidence is completely different from the individual clinical experience, expertise, and conviction which therapists contribute to the encounter with clients. A deeper understanding of internal evidence as a result of this encounter has emerged only in the past 15 years. However, it is an integral part of the logic of evidence-based professional decision-making. Scientifically justified beneficial and effective treatment in the individual case cannot be deduced from external evidence but can only be gathered from internal evidence for which the best external evidence available has been utilised. In the past 15 years nursing science has not only carved out the decision-oriented scientific core of evidence-based practice but has also tried to increase the validity of studies on external evidence by employing a combination of 'qualitative' social science studies and clinical epidemiological methods. Copyright © 2010. Published by Elsevier GmbH.

A critical look at descriptive epidemiology of sexual dysfunction in Asia compared to the rest of the world - a call for evidence-based data

PubMed Central

2013-01-01

Recently evidence-based medicine has been applied to comparative epidemiological papers regarding sexual dysfunction that have appeared in the literature. This review is intended to focus the readers on a validated and standardized methodological evidence-based process for preparing such articles. It reviews four key articles that have been published in the English language that have obtained a high evidence-based score for reliability that have included descriptive epidemiology of sexual dysfunctions in men and women in Asia compared to the rest of the world. These four papers are analyzed in detail in order to provide stress of what constitutes evidence-based studies in descriptive epidemiology for sexual function. As can be seen there has not yet been a perfect article that compares the prevalence of sexual function in Asia compared to the rest of the world since there are key methodological problems in the collection of the data. In addition, there is a paucity of incidence studies for sexual dysfunction in Asian populations. The readers are encouraged to use this data in preparation of future descriptive epidemiological studies that involve Asian countries. PMID:26816724

A critical look at descriptive epidemiology of sexual dysfunction in Asia compared to the rest of the world - a call for evidence-based data.

PubMed

Lewis, Ronald W

2013-03-01

Recently evidence-based medicine has been applied to comparative epidemiological papers regarding sexual dysfunction that have appeared in the literature. This review is intended to focus the readers on a validated and standardized methodological evidence-based process for preparing such articles. It reviews four key articles that have been published in the English language that have obtained a high evidence-based score for reliability that have included descriptive epidemiology of sexual dysfunctions in men and women in Asia compared to the rest of the world. These four papers are analyzed in detail in order to provide stress of what constitutes evidence-based studies in descriptive epidemiology for sexual function. As can be seen there has not yet been a perfect article that compares the prevalence of sexual function in Asia compared to the rest of the world since there are key methodological problems in the collection of the data. In addition, there is a paucity of incidence studies for sexual dysfunction in Asian populations. The readers are encouraged to use this data in preparation of future descriptive epidemiological studies that involve Asian countries.

Scientific Research in Homeopathic Medicine: Validation, Methodology and Perspectives

PubMed Central

2007-01-01

Verona's School of Homeopathic Medicine (www.omeopatia.org) organized a day of full immersion in the field of homeopathy, focusing on the validity of this much-debated discipline. There is widespread consensus in the medical community that evidence-based medicine is the best standard for assessing efficacy and safety of healthcare practices, and systematic reviews with strict protocols are essential to establish proof for various therapies. Students, homeopathic practitioners, academic and business representatives, who are interested in or curious about homeopathic practices attended the conference.

The Safety Culture Enactment Questionnaire (SCEQ): Theoretical model and empirical validation.

PubMed

de Castro, Borja López; Gracia, Francisco J; Tomás, Inés; Peiró, José M

2017-06-01

This paper presents the Safety Culture Enactment Questionnaire (SCEQ), designed to assess the degree to which safety is an enacted value in the day-to-day running of nuclear power plants (NPPs). The SCEQ is based on a theoretical safety culture model that is manifested in three fundamental components of the functioning and operation of any organization: strategic decisions, human resources practices, and daily activities and behaviors. The extent to which the importance of safety is enacted in each of these three components provides information about the pervasiveness of the safety culture in the NPP. To validate the SCEQ and the model on which it is based, two separate studies were carried out with data collection in 2008 and 2014, respectively. In Study 1, the SCEQ was administered to the employees of two Spanish NPPs (N=533) belonging to the same company. Participants in Study 2 included 598 employees from the same NPPs, who completed the SCEQ and other questionnaires measuring different safety outcomes (safety climate, safety satisfaction, job satisfaction and risky behaviors). Study 1 comprised item formulation and examination of the factorial structure and reliability of the SCEQ. Study 2 tested internal consistency and provided evidence of factorial validity, validity based on relationships with other variables, and discriminant validity between the SCEQ and safety climate. Exploratory Factor Analysis (EFA) carried out in Study 1 revealed a three-factor solution corresponding to the three components of the theoretical model. Reliability analyses showed strong internal consistency for the three scales of the SCEQ, and each of the 21 items on the questionnaire contributed to the homogeneity of its theoretically developed scale. Confirmatory Factor Analysis (CFA) carried out in Study 2 supported the internal structure of the SCEQ; internal consistency of the scales was also supported. Furthermore, the three scales of the SCEQ showed the expected correlation patterns with the measured safety outcomes. Finally, results provided evidence of discriminant validity between the SCEQ and safety climate. We conclude that the SCEQ is a valid, reliable instrument supported by a theoretical framework, and it is useful to measure the enactment of safety culture in NPPs. Copyright © 2017 Elsevier Ltd. All rights reserved.

The Nature of Science Instrument-Elementary (NOSI-E): the end of the road?

PubMed

Peoples, Shelagh M; O'Dwyer, Laura M

2014-01-01

This research continues prior work published in this journal (Peoples, O'Dwyer, Shields and Wang, 2013). The first paper described the scale development, psychometric analyses and part-validation of a theoretically-grounded Rasch-based instrument, the Nature of Science Instrument-Elementary (NOSI-E). The NOSI-E was designed to measure elementary students' understanding of the Nature of Science (NOS). In the first paper, evidence was provided for three of the six validity aspects (content, substantive and generalizability) needed to support the construct validity of the NOSI-E. The research described in this paper examines two additional validity aspects (structural and external). The purpose of this study was to determine which of three competing internal models provides reliable, interpretable, and responsive measures of students' understanding of NOS. One postulate is that the NOS construct is unidimensional;. alternatively, the NOS construct is composed of five independent unidimensional constructs (the consecutive approach). Lastly, the NOS construct is multidimensional and composed of five inter-related but separate dimensions. The vast body of evidence supported the claim that the NOS construct is multidimensional. Measures from the multidimensional model were positively related to student science achievement and students' perceptions of their classroom environment; this provided supporting evidence for the external validity aspect of the NOS construct. As US science education moves toward students learning science through engaging in authentic scientific practices and building learning progressions (NRC, 2012), it will be important to assess whether this new approach to teaching science is effective, and the NOSI-E may be used as a measure of the impact of this reform.

Psychometric Evaluation of an Instrument for Measuring Organizational Climate for Quality: Evidence From a National Sample of Infection Preventionists.

PubMed

Pogorzelska-Maziarz, Monika; Nembhard, Ingrid M; Schnall, Rebecca; Nelson, Shanelle; Stone, Patricia W

2016-09-01

In recent years, there has been increased interest in measuring the climate for infection prevention; however, reliable and valid instruments are lacking. This study tested the psychometric properties of the Leading a Culture of Quality for Infection Prevention (LCQ-IP) instrument measuring the infection prevention climate in a sample of 972 infection preventionists from acute care hospitals. An exploratory principal component analysis showed that the instrument had structural validity and captured 4 factors related to the climate for infection prevention: Psychological Safety, Prioritization of Quality, Supportive Work Environment, and Improvement Orientation. LCQ-IP exhibited excellent internal consistency, with a Cronbach α of .926. Criterion validity was supported with overall LCQ-IP scores, increasing with the number of evidence-based prevention policies in place (P = .047). This psychometrically sound instrument may be helpful to researchers and providers in assessing climate for quality related to infection prevention. © The Author(s) 2015.

Utilizing evidence-based assessment instruments to detect well-being and distress in English- and Spanish-speaking caregivers of individuals affected by dementia.

PubMed

Burke, Shanna L; Burgess, Aaron; Cadet, Tamara

2017-01-01

Objective The purpose of this study was to examine the most effective and available English and Spanish language caregiver assessments for providers and caregivers. Methods Assessments were included if they screened for caregiving-related concerns, including stress, depression, and caregiving burden and could be administered directly to caregivers in person or online. Results Eighteen assessments are designed to assess caregiver burden, distress, depression, and grief. Six did not have psychometric data to support efficacy but are widely used in clinical and research settings. Six were validated in Spanish, and one other is available in Spanish but not validated. Conclusion As many as 80% of care recipients are cared for in the home by family members who act as informal caregivers. Caregivers of persons with dementia may experience depression symptoms, high caregiver burden, and feelings of being constrained. Due to the lack of psychometric evidence available, the validity of some assessments is questionable.

Can we really use available scales for child and adolescent psychopathology across cultures? A systematic review of cross-cultural measurement invariance data.

PubMed

Stevanovic, Dejan; Jafari, Peyman; Knez, Rajna; Franic, Tomislav; Atilola, Olayinka; Davidovic, Nikolina; Bagheri, Zahra; Lakic, Aneta

2017-02-01

In this systematic review, we assessed available evidence for cross-cultural measurement invariance of assessment scales for child and adolescent psychopathology as an indicator of cross-cultural validity. A literature search was conducted using the Medline, PsychInfo, Scopus, Web of Science, and Google Scholar databases. Cross-cultural measurement invariance data was available for 26 scales. Based on the aggregation of the evidence from the studies under review, none of the evaluated scales have strong evidence for cross-cultural validity and suitability for cross-cultural comparison. A few of the studies showed a moderate level of measurement invariance for some scales (such as the Fear Survey Schedule for Children-Revised, Multidimensional Anxiety Scale for Children, Revised Child Anxiety and Depression Scale, Revised Children's Manifest Anxiety Scale, Mood and Feelings Questionnaire, and Disruptive Behavior Rating Scale), which may make them suitable in cross-cultural comparative studies. The remainder of the scales either showed weak or outright lack of measurement invariance. This review showed only limited testing for measurement invariance across cultural groups of scales for pediatric psychopathology, with evidence of cross-cultural validity for only a few scales. This study also revealed a need to improve practices of statistical analysis reporting in testing measurement invariance. Implications for future research are discussed.

Comparing Cognitive Models of Domain Mastery and Task Performance in Algebra: Validity Evidence for a State Assessment

ERIC Educational Resources Information Center

Warner, Zachary B.

2013-01-01

This study compared an expert-based cognitive model of domain mastery with student-based cognitive models of task performance for Integrated Algebra. Interpretations of student test results are limited by experts' hypotheses of how students interact with the items. In reality, the cognitive processes that students use to solve each item may be…

Practice-Based Measures of Elementary Science Teachers' Content Knowledge for Teaching: Initial Item Development and Validity Evidence. Research Report. ETS RR-17-43

ERIC Educational Resources Information Center

Mikeska, Jamie N.; Phelps, Geoffrey; Croft, Andrew J.

2017-01-01

This report describes efforts by a group of science teachers, teacher educators, researchers, and content specialists to conceptualize, develop, and pilot practice-based assessment items designed to measure elementary science teachers' content knowledge for teaching (CKT). The report documents the framework used to specify the content-specific…

The Peer Attitudes toward Children Who Stutter Scale: Reliability, Known Groups Validity, and Negativity of Elementary School-Age Children's Attitudes

ERIC Educational Resources Information Center

Langevin, Marilyn

2009-01-01

Psychometric properties of the Peer Attitudes Toward Children who Stutter (PATCS) scale (Langevin, M., & Hagler, P. (2004). Development of a scale to measure peer attitudes toward children who stutter. In A.K. Bothe (Ed.), Evidence-based treatment of stuttering: empirical bases and clinical applications (pp. 139-171). Mahwah, NJ: Lawrence…

Yaruss, Coleman, and Hammer (2006): an exemplar of non-evidence-based practice in stuttering treatment.

PubMed

Ingham, Roger J

2007-07-01

This letter is a response to a recent report by J. S. Yaruss, C. Coleman, and D. Hammer (2006) that described a treatment program for preschool children who stutter. Problems with the Yaruss et al. study fall into four domains: (a) failure to provide clinicians with replicable procedures, (b) failure to collect valid and reliable speech performance data, (c) failure to control for predictable improvement in children who have been stuttering for less than 15 months, and (d) the advocacy of procedures for which there is no credible research evidence. The claims made for the efficacy of this treatment are problematic and essentially violate the principles of evidence-based practice as recommended by the American Speech-Language-Hearing Association (ASHA).

Evidence-based hypnotherapy for depression.

PubMed

Alladin, Assen

2010-04-01

Cognitive hypnotherapy (CH) is a comprehensive evidence-based hypnotherapy for clinical depression. This article describes the major components of CH, which integrate hypnosis with cognitive-behavior therapy as the latter provides an effective host theory for the assimilation of empirically supported treatment techniques derived from various theoretical models of psychotherapy and psychopathology. CH meets criteria for an assimilative model of psychotherapy, which is considered to be an efficacious model of psychotherapy integration. The major components of CH for depression are described in sufficient detail to allow replication, verification, and validation of the techniques delineated. CH for depression provides a template that clinicians and investigators can utilize to study the additive effects of hypnosis in the management of other psychological or medical disorders. Evidence-based hypnotherapy and research are encouraged; such a movement is necessary if clinical hypnosis is to integrate into mainstream psychotherapy.

Challenging evidence-based decision-making: a hypothetical case study about return to work.

PubMed

Aas, Randi W; Alexanderson, Kristina

2012-03-01

A hypothetical case study about return to work was used to explore the process of translating research into practice. The method involved constructing a case study derived from the characteristics of a typical, sick-listed employee with non-specific low back pain in Norway. Next, the five-step evidence-based process, including the Patient, Intervention, Co-Interventions and Outcome framework (PICO), was applied to the case study. An inductive analysis produced 10 technical and more fundamental challenges to incorporate research into intervention decisions for an individual with comorbidity. A more dynamic, interactive approach to the evidence-based practice process is proposed. It is recommended that this plus the 10 challenges are validated with real life cases, as the hypothetical case study may not be replicable. Copyright © 2011 John Wiley & Sons, Ltd.

Defining lactation acuity to improve patient safety and outcomes.

PubMed

Mannel, Rebecca

2011-05-01

While substantial evidence exists identifying risks factors associated with premature weaning from breastfeeding, there are no previously published definitions of patient acuity in the lactation field. This article defines evidence-based levels of lactation acuity based on maternal and infant characteristics. Patient acuity, matching severity of illness to intensity of care required, is an important determinant of patient safety and outcomes. It is often used as part of a patient classification system to determine staffing needs and acceptable workloads in health care settings. As acuity increases, more resources, including more skilled clinicians, are needed to provide optimal care. Developing an evidence-based definition of lactation acuity can help to standardize terminology, more effectively distribute health care staff resources, encourage research to verify the validity and reliability of lactation acuity, and potentially improve breastfeeding initiation and duration rates.

Instruments to assess self-care among healthy children: A systematic review of measurement properties.

PubMed

Urpí-Fernández, Ana-María; Zabaleta-Del-Olmo, Edurne; Montes-Hidalgo, Javier; Tomás-Sábado, Joaquín; Roldán-Merino, Juan-Francisco; Lluch-Canut, María-Teresa

2017-12-01

To identify, critically appraise and summarize the measurement properties of instruments to assess self-care in healthy children. Assessing self-care is a proper consideration for nursing practice and nursing research. No systematic review summarizes instruments of measurement validated in healthy children. Psychometric review in accordance with the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) panel. MEDLINE, CINAHL, PsycINFO, Web of Science and Open Grey were searched from their inception to December 2016. Validation studies with a healthy child population were included. Search was not restricted by language. Two reviewers independently assessed the methodological quality of included studies using the COSMIN checklist. Eleven studies were included in the review assessing the measurement properties of ten instruments. There was a maximum of two studies per instrument. None of the studies evaluated the properties of test-retest reliability, measurement error, criterion validity and responsiveness. Internal consistency and structural validity were rated as "excellent" or "good" in four studies. Four studies were rated as "excellent" in content validity. Cross-cultural validity was rated as "poor" in the two studies (three instruments) which cultural adaptation was carried out. The evidence available does not allow firm conclusions about the instruments identified in terms of reliability and validity. Future research should focus on generate evidence about a wider range of measurement properties of these instruments using a rigorous methodology, as well as instrument testing on different countries and child population. © 2017 John Wiley & Sons Ltd.

Educational Milestone Development in the First 7 Specialties to Enter the Next Accreditation System

PubMed Central

Swing, Susan R.; Beeson, Michael S.; Carraccio, Carol; Coburn, Michael; Iobst, William; Selden, Nathan R.; Stern, Peter J.; Vydareny, Kay

2013-01-01

Background The Accreditation Council for Graduate Medical Education (ACGME) Outcome Project introduced 6 general competencies relevant to medical practice but fell short of its goal to create a robust assessment system that would allow program accreditation based on outcomes. In response, the ACGME, the specialty boards, and other stakeholders collaborated to develop educational milestones, observable steps in residents' professional development that describe progress from entry to graduation and beyond. Objectives We summarize the development of the milestones, focusing on 7 specialties, moving to the next accreditation system in July 2013, and offer evidence of their validity. Methods Specialty workgroups with broad representation used a 5-level developmental framework and incorporated information from literature reviews, specialty curricula, dialogue with constituents, and pilot testing. Results The workgroups produced richly diverse sets of milestones that reflect the community's consideration of attributes of competence relevant to practice in the given specialty. Both their development process and the milestones themselves establish a validity argument, when contemporary views of validity for complex performance assessment are used. Conclusions Initial evidence for validity emerges from the development processes and the resulting milestones. Further advancing a validity argument will require research on the use of milestone data in resident assessment and program accreditation. PMID:24404235

Refining knowledge, attitude and practice of evidence-based medicine (EBM) among pharmacy students for professional challenges.

PubMed

Abu-Gharbieh, Eman; Khalidi, Doaa Al; Baig, Mirza R; Khan, Saeed A

2015-04-01

Practicing evidence based medicine (EBM) is a professional need for the future clinical pharmacist in UAE and around the world. An attempt was made to evaluate pharmacy student's knowledge, attitude and proficiency in the practice of EBM. A within-subject study design with pre and post survey and skill test were conducted using case based practice of EBM through a validated questionnaire. The results were tabulated and there was a statistically significant increase in pharmacy students' perceived ability to go through steps of EBM, namely: formulating PICO questions (95.3%), searching for evidence (97%), appraising the evidence (81%), understanding statistics (78.1%), and applying evidence at point of care (81.2%). In this study, workshops and (Problem Based Learning) PBLs were used as a module of EBM teaching and practices, which has been shown to be an effective educational method in terms of improving students' skills, knowledge and attitude toward EBM. Incorporating hands on experience, PBLs will become an impetus for developing EBM skills and critical appraisal of research evidence alongside routine clinical practice. This integration would constitute the cornerstone in lifting EBM in UAE up to the needed standards and would enable pharmacy students to become efficient pharmacists that rely on evidence in their health practice.

Assessing Medical Students’ Self-regulation as Aptitude in Computer-based Learning

PubMed Central

Song, Hyuksoon S.; Kalet, Adina L.; Plass, Jan L.

2013-01-01

We developed a Self-Regulation Measure for Computer-based learning (SRMC) tailored toward medical students, by modifying Zimmerman’s Self-Regulated Learning Interview Schedule (SRLIS) for K-12 learners. The SRMC’s reliability and validity were examined in 2 studies. In Study 1, 109 first-year medical students were asked to complete the SRMC. Bivariate correlation analysis results indicated that the SRMC scores had a moderate degree of correlation with student achievement in a teacher-developed test. In Study 2, 58 third-year clerkship students completed the SRMC. Regression analysis results indicated that the frequency of medical students’ usage of self-regulation strategies was associated with their general clinical knowledge measured by a nationally standardized licensing exam. These two studies provided evidence for the reliability and concurrent validity of the SRMC to assess medical students’ self-regulation as aptitude. Future work should provide evidence to guide and improve instructional design as well as inform educational policy. PMID:20872071

Does classroom-based Crew Resource Management training improve patient safety culture? A systematic review

PubMed Central

de Bruijne, Martine C; Zwijnenberg, Nicolien C; Jansma, Elise P; van Dyck, Cathy; Wagner, Cordula

2014-01-01

Aim: To evaluate the evidence of the effectiveness of classroom-based Crew Resource Management training on safety culture by a systematic review of literature. Methods: Studies were identified in PubMed, Cochrane Library, PsycINFO, and Educational Resources Information Center up to 19 December 2012. The Methods Guide for Comparative Effectiveness Reviews was used to assess the risk of bias in the individual studies. Results: In total, 22 manuscripts were included for review. Training settings, study designs, and evaluation methods varied widely. Most studies reporting only a selection of culture dimensions found mainly positive results, whereas studies reporting all safety culture dimensions of the particular survey found mixed results. On average, studies were at moderate risk of bias. Conclusion: Evidence of the effectiveness of Crew Resource Management training in health care on safety culture is scarce and the validity of most studies is limited. The results underline the necessity of more valid study designs, preferably using triangulation methods. PMID:26770720

Voice care knowledge among clinicians and people with healthy voices or dysphonia.

PubMed

Fletcher, Helen M; Drinnan, Michael J; Carding, Paul N

2007-01-01

An important clinical component in the prevention and treatment of voice disorders is voice care and hygiene. Research in voice care knowledge has mainly focussed on specific groups of professional voice users with limited reporting on the tool and evidence base used. In this study, a questionnaire to measure voice care knowledge was developed based on "best evidence." The questionnaire was validated by measuring specialist voice clinicians' agreement. Preliminary data are then presented using the voice care knowledge questionnaire with 17 subjects with nonorganic dysphonia and 17 with healthy voices. There was high (89%) agreement among the clinicians. There was a highly significant difference between the dysphonic and the healthy group scores (P = 0.00005). Furthermore, the dysphonic subjects (63% agreement) presented with less voice care knowledge than the subjects with healthy voices (72% agreement). The questionnaire provides a useful and valid tool to investigate voice care knowledge. The findings have implications for clinical intervention, voice therapy, and health prevention.

Evidence-Based School Behavior Assessment of Externalizing Behavior in Young Children.

PubMed

Bagner, Daniel M; Boggs, Stephen R; Eyberg, Sheila M

2010-02-01

This study examined the psychometric properties of the Revised Edition of the School Observation Coding System (REDSOCS). Participants were 68 children ages 3 to 6 who completed parent-child interaction therapy for Oppositional Defiant Disorder as part of a larger efficacy trial. Interobserver reliability on REDSOCS categories was moderate to high, with percent agreement ranging from 47% to 90% (M = 67%) and Cohen's kappa coefficients ranging from .69 to .95 (M = .82). Convergent validity of the REDSOCS categories was supported by significant correlations with the Intensity Scale of the Sutter-Eyberg Student Behavior Inventory-Revised and related subscales of the Conners' Teacher Rating Scale-Revised: Long Version (CTRS-R: L). Divergent validity was indicated by nonsignificant correlations between REDSOCS categories and scales on the CTRS-R: L expected not to relate to disruptive classroom behavior. Treatment sensitivity was demonstrated for two of the three primary REDSOCS categories by significant pre to posttreatment changes. This study provides psychometric support for the designation of REDSOCS as an evidence-based assessment procedure for young children.

Exploring the Benefits of Respite Services to Family Caregivers: Methodological Issues and Current Findings

PubMed Central

Zarit, Steven H.; Liu, Yin; Bangerter, Lauren R.; Rovine, Michael J.

2017-01-01

Objectives There is growing emphasis on empirical validation of the efficacy of community-based services for older people and their families, but research on services such as respite care faces methodological challenges that have limited the growth of outcome studies. We identify problems associated with the usual research approaches for studying respite care, with the goal of stimulating use of novel and more appropriate research designs that can lead to improved studies of community-based services. Method Using the concept of research validity, we evaluate the methodological approaches in the current literature on respite services, including adult day services, in-home respite and overnight respite. Results Although randomized control trials (RCTs) are possible in community settings, validity is compromised by practical limitations of randomization and other problems. Quasi-experimental and interrupted time series designs offer comparable validity to RCTs and can be implemented effectively in community settings. Conclusion An emphasis on RCTs by funders and researchers is not supported by scientific evidence. Alternative designs can lead to development of a valid body of research on community services such as respite. PMID:26729467

Exploring the benefits of respite services to family caregivers: methodological issues and current findings.

PubMed

Zarit, Steven H; Bangerter, Lauren R; Liu, Yin; Rovine, Michael J

2017-03-01

There is growing emphasis on empirical validation of the efficacy of community-based services for older people and their families, but research on services such as respite care faces methodological challenges that have limited the growth of outcome studies. We identify problems associated with the usual research approaches for studying respite care, with the goal of stimulating use of novel and more appropriate research designs that can lead to improved studies of community-based services. Using the concept of research validity, we evaluate the methodological approaches in the current literature on respite services, including adult day services, in-home respite and overnight respite. Although randomized control trials (RCTs) are possible in community settings, validity is compromised by practical limitations of randomization and other problems. Quasi-experimental and interrupted time series designs offer comparable validity to RCTs and can be implemented effectively in community settings. An emphasis on RCTs by funders and researchers is not supported by scientific evidence. Alternative designs can lead to development of a valid body of research on community services such as respite.

Indirect Measurement of Sexual Orientation: Comparison of the Implicit Relational Assessment Procedure, Viewing Time, and Choice Reaction Time Tasks.

PubMed

Rönspies, Jelena; Schmidt, Alexander F; Melnikova, Anna; Krumova, Rosina; Zolfagari, Asadeh; Banse, Rainer

2015-07-01

The present study was conducted to validate an adaptation of the Implicit Relational Assessment Procedure (IRAP) as an indirect latency-based measure of sexual orientation. Furthermore, reliability and criterion validity of the IRAP were compared to two established indirect measures of sexual orientation: a Choice Reaction Time task (CRT) and a Viewing Time (VT) task. A sample of 87 heterosexual and 35 gay men completed all three indirect measures in an online study. The IRAP and the VT predicted sexual orientation nearly perfectly. Both measures also showed a considerable amount of convergent validity. Reliabilities (internal consistencies) reached satisfactory levels. In contrast, the CRT did not tap into sexual orientation in the present study. In sum, the VT measure performed best, with the IRAP showing only slightly lower reliability and criterion validity, whereas the CRT did not yield any evidence of reliability or criterion validity in the present research. The results were discussed in the light of specific task properties of the indirect latency-based measures (task-relevance vs. task-irrelevance).

Patterns of Cognitive Strengths and Weaknesses: Identification Rates, Agreement, and Validity for Learning Disabilities Identification

PubMed Central

Miciak, Jeremy; Fletcher, Jack M.; Stuebing, Karla; Vaughn, Sharon; Tolar, Tammy D.

2014-01-01

Purpose Few empirical investigations have evaluated LD identification methods based on a pattern of cognitive strengths and weaknesses (PSW). This study investigated the reliability and validity of two proposed PSW methods: the concordance/discordance method (C/DM) and cross battery assessment (XBA) method. Methods Cognitive assessment data for 139 adolescents demonstrating inadequate response to intervention was utilized to empirically classify participants as meeting or not meeting PSW LD identification criteria using the two approaches, permitting an analysis of: (1) LD identification rates; (2) agreement between methods; and (3) external validity. Results LD identification rates varied between the two methods depending upon the cut point for low achievement, with low agreement for LD identification decisions. Comparisons of groups that met and did not meet LD identification criteria on external academic variables were largely null, raising questions of external validity. Conclusions This study found low agreement and little evidence of validity for LD identification decisions based on PSW methods. An alternative may be to use multiple measures of academic achievement to guide intervention. PMID:24274155

Assessing impact of physical activity-based youth development programs: validation of the Life Skills Transfer Survey (LSTS).

PubMed

Weiss, Maureen R; Bolter, Nicole D; Kipp, Lindsay E

2014-09-01

A signature characteristic of positive youth development (PYD) programs is the opportunity to develop life skills, such as social, behavioral, and moral competencies, that can be generalized to domains beyond the immediate activity. Although context-specific instruments are available to assess developmental outcomes, a measure of life skills transfer would enable evaluation of PYD programs in successfully teaching skills that youth report using in other domains. The purpose of our studies was to develop and validate a measure of perceived life skills transfer, based on data collected with The First Tee, a physical activity-based PYD program. In 3 studies, we conducted a series of steps to provide content and construct validity and internal consistency reliability for the Life Skills Transfer Survey (LSTS), a measure of perceived life skills transfer. Study 1 provided content validity for the LSTS that included 8 life skills and 50 items. Study 2 revealed construct validity (structural validity) through a confirmatory factor analysis and convergent validity by correlating scores on the LSTS with scores on an assessment tool that measures a related construct. Study 3 offered additional construct validity by reassessing youth 1 year later and showing that scores during both time periods were invariant in factor pattern, loadings, and variances and covariances. Studies 2 and 3 demonstrated internal consistency reliability of the LSTS. RESULTS from 3 studies provide evidence of content and construct validity and internal consistency reliability for the LSTS, which can be used in evaluation research with youth development programs.

Clinical evidence supporting pharmacogenomic biomarker testing provided in US Food and Drug Administration drug labels.

PubMed

Wang, Bo; Canestaro, William J; Choudhry, Niteesh K

2014-12-01

Genetic biomarkers that predict a drug's efficacy or likelihood of toxicity are assuming increasingly important roles in the personalization of pharmacotherapy, but concern exists that evidence that links use of some biomarkers to clinical benefit is insufficient. Nevertheless, information about the use of biomarkers appears in the labels of many prescription drugs, which may add confusion to the clinical decision-making process. To evaluate the evidence that supports pharmacogenomic biomarker testing in drug labels and how frequently testing is recommended. Publicly available US Food and Drug Administration databases. We identified drug labels that described the use of a biomarker and evaluated whether the label contained or referenced convincing evidence of its clinical validity (ie, the ability to predict phenotype) and clinical utility (ie, the ability to improve clinical outcomes) using guidelines published by the Evaluation of Genomic Applications in Practice and Prevention Working Group. We graded the completeness of the citation of supporting studies and determined whether the label recommended incorporation of biomarker test results in therapeutic decision making. Of the 119 drug-biomarker combinations, only 43 (36.1%) had labels that provided convincing clinical validity evidence, whereas 18 (15.1%) provided convincing evidence of clinical utility. Sixty-one labels (51.3%) made recommendations about how clinical decisions should be based on the results of a biomarker test; 36 (30.3%) of these contained convincing clinical utility data. A full description of supporting studies was included in 13 labels (10.9%). Fewer than one-sixth of drug labels contained or referenced convincing evidence of clinical utility of biomarker testing, whereas more than half made recommendations based on biomarker test results. It may be premature to include biomarker testing recommendations in drug labels when convincing data that link testing to patient outcomes do not exist.

Child maltreatment prevention: a systematic review of reviews

PubMed Central

Butchart, Alexander

2009-01-01

Abstract Objective To synthesize recent evidence from systematic and comprehensive reviews on the effectiveness of universal and selective child maltreatment prevention interventions, evaluate the methodological quality of the reviews and outcome evaluation studies they are based on, and map the geographical distribution of the evidence. Methods A systematic review of reviews was conducted. The quality of the systematic reviews was evaluated with a tool for the assessment of multiple systematic reviews (AMSTAR), and the quality of the outcome evaluations was assessed using indicators of internal validity and of the construct validity of outcome measures. Findings The review focused on seven main types of interventions: home visiting, parent education, child sex abuse prevention, abusive head trauma prevention, multi-component interventions, media-based interventions, and support and mutual aid groups. Four of the seven – home-visiting, parent education, abusive head trauma prevention and multi-component interventions – show promise in preventing actual child maltreatment. Three of them – home visiting, parent education and child sexual abuse prevention – appear effective in reducing risk factors for child maltreatment, although these conclusions are tentative due to the methodological shortcomings of the reviews and outcome evaluation studies they draw on. An analysis of the geographical distribution of the evidence shows that outcome evaluations of child maltreatment prevention interventions are exceedingly rare in low- and middle-income countries and make up only 0.6% of the total evidence base. Conclusion Evidence for the effectiveness of four of the seven main types of interventions for preventing child maltreatment is promising, although it is weakened by methodological problems and paucity of outcome evaluations from low- and middle-income countries. PMID:19551253

Validity evidence for the measurement of the strength of motivation for medical school.

PubMed

Kusurkar, Rashmi; Croiset, Gerda; Kruitwagen, Cas; ten Cate, Olle

2011-05-01

The Strength of Motivation for Medical School (SMMS) questionnaire is designed to determine the strength of motivation of students particularly for medical study. This research was performed to establish the validity evidence for measuring strength of motivation for medical school. Internal structure and relations to other variables were used as the sources of validity evidence. The SMMS questionnaire was filled out by 1,494 medical students in different years of medical curriculum. The validity evidence for the internal structure was analyzed by principal components analysis with promax rotation. Validity evidence for relations to other variables was tested by comparing the SMMS scores with scores on the Academic Motivation Scale (AMS) and the exhaustion scale of Maslach Burnout Inventory-Student Survey (MBI-SS) for measuring study stress. Evidence for internal consistency was determined through the Cronbach's alpha for reliability. The analysis showed that the SMMS had a 3-factor structure. The validity in relations to other variables was established as both, the subscales and full scale scores significantly correlated positively with the intrinsic motivation scores and with the more autonomous forms of extrinsic motivation, the correlation decreasing and finally becoming negative towards the extrinsic motivation end of the spectrum. They also had significant negative correlations with amotivation scale of the AMS and exhaustion scale of MBI-SS. The Cronbach's alpha for reliability of the three subscales and full SMMS scores was 0.70, 0.67, 0.55 and 0.79. The strength of motivation for medical school has a three factor structure and acceptable validity evidence was found in our study.

Measuring organizational readiness for knowledge translation in chronic care.

PubMed

Gagnon, Marie-Pierre; Labarthe, Jenni; Légaré, France; Ouimet, Mathieu; Estabrooks, Carole A; Roch, Geneviève; Ghandour, El Kebir; Grimshaw, Jeremy

2011-07-13

Knowledge translation (KT) is an imperative in order to implement research-based and contextualized practices that can answer the numerous challenges of complex health problems. The Chronic Care Model (CCM) provides a conceptual framework to guide the implementation process in chronic care. Yet, organizations aiming to improve chronic care require an adequate level of organizational readiness (OR) for KT. Available instruments on organizational readiness for change (ORC) have shown limited validity, and are not tailored or adapted to specific phases of the knowledge-to-action (KTA) process. We aim to develop an evidence-based, comprehensive, and valid instrument to measure OR for KT in healthcare. The OR for KT instrument will be based on core concepts retrieved from existing literature and validated by a Delphi study. We will specifically test the instrument in chronic care that is of an increasing importance for the health system. Phase one: We will conduct a systematic review of the theories and instruments assessing ORC in healthcare. The retained theoretical information will be synthesized in a conceptual map. A bibliography and database of ORC instruments will be prepared after appraisal of their psychometric properties according to the standards for educational and psychological testing. An online Delphi study will be carried out among decision makers and knowledge users across Canada to assess the importance of these concepts and measures at different steps in the KTA process in chronic care.Phase two: A final OR for KT instrument will be developed and validated both in French and in English and tested in chronic disease management to measure OR for KT regarding the adoption of comprehensive, patient-centered, and system-based CCMs. This study provides a comprehensive synthesis of current knowledge on explanatory models and instruments assessing OR for KT. Moreover, this project aims to create more consensus on the theoretical underpinnings and the instrumentation of OR for KT in chronic care. The final product--a comprehensive and valid OR for KT instrument--will provide the chronic care settings with an instrument to assess their readiness to implement evidence-based chronic care.

Measuring organizational readiness for knowledge translation in chronic care

PubMed Central

2011-01-01

Background Knowledge translation (KT) is an imperative in order to implement research-based and contextualized practices that can answer the numerous challenges of complex health problems. The Chronic Care Model (CCM) provides a conceptual framework to guide the implementation process in chronic care. Yet, organizations aiming to improve chronic care require an adequate level of organizational readiness (OR) for KT. Available instruments on organizational readiness for change (ORC) have shown limited validity, and are not tailored or adapted to specific phases of the knowledge-to-action (KTA) process. We aim to develop an evidence-based, comprehensive, and valid instrument to measure OR for KT in healthcare. The OR for KT instrument will be based on core concepts retrieved from existing literature and validated by a Delphi study. We will specifically test the instrument in chronic care that is of an increasing importance for the health system. Methods Phase one: We will conduct a systematic review of the theories and instruments assessing ORC in healthcare. The retained theoretical information will be synthesized in a conceptual map. A bibliography and database of ORC instruments will be prepared after appraisal of their psychometric properties according to the standards for educational and psychological testing. An online Delphi study will be carried out among decision makers and knowledge users across Canada to assess the importance of these concepts and measures at different steps in the KTA process in chronic care. Phase two: A final OR for KT instrument will be developed and validated both in French and in English and tested in chronic disease management to measure OR for KT regarding the adoption of comprehensive, patient-centered, and system-based CCMs. Discussion This study provides a comprehensive synthesis of current knowledge on explanatory models and instruments assessing OR for KT. Moreover, this project aims to create more consensus on the theoretical underpinnings and the instrumentation of OR for KT in chronic care. The final product--a comprehensive and valid OR for KT instrument--will provide the chronic care settings with an instrument to assess their readiness to implement evidence-based chronic care. PMID:21752264

Reconsidering vocational interests for personnel selection: the validity of an interest-based selection test in relation to job knowledge, job performance, and continuance intentions.

PubMed

Van Iddekinge, Chad H; Putka, Dan J; Campbell, John P

2011-01-01

Although vocational interests have a long history in vocational psychology, they have received extremely limited attention within the recent personnel selection literature. We reconsider some widely held beliefs concerning the (low) validity of interests for predicting criteria important to selection researchers, and we review theory and empirical evidence that challenge such beliefs. We then describe the development and validation of an interests-based selection measure. Results of a large validation study (N = 418) reveal that interests predicted a diverse set of criteria—including measures of job knowledge, job performance, and continuance intentions—with corrected, cross-validated Rs that ranged from .25 to .46 across the criteria (mean R = .31). Interests also provided incremental validity beyond measures of general cognitive aptitude and facets of the Big Five personality dimensions in relation to each criterion. Furthermore, with a couple exceptions, the interest scales were associated with small to medium subgroup differences, which in most cases favored women and racial minorities. Taken as a whole, these results appear to call into question the prevailing thought that vocational interests have limited usefulness for selection.

Reliability, Validity, and Factor Structure of the Current Assessment Practice Evaluation-Revised (CAPER) in a National Sample.

PubMed

Lyon, Aaron R; Pullmann, Michael D; Dorsey, Shannon; Martin, Prerna; Grigore, Alexandra A; Becker, Emily M; Jensen-Doss, Amanda

2018-05-11

Measurement-based care (MBC) is an increasingly popular, evidence-based practice, but there are no tools with established psychometrics to evaluate clinician use of MBC practices in mental health service delivery. The current study evaluated the reliability, validity, and factor structure of scores generated from a brief, standardized tool to measure MBC practices, the Current Assessment Practice Evaluation-Revised (CAPER). Survey data from a national sample of 479 mental health clinicians were used to conduct exploratory and confirmatory factor analyses, as well as reliability and validity analyses (e.g., relationships between CAPER subscales and clinician MBC attitudes). Analyses revealed competing two- and three-factor models. Regardless of the model used, scores from CAPER subscales demonstrated good reliability and convergent and divergent validity with MBC attitudes in the expected directions. The CAPER appears to be a psychometrically sound tool for assessing clinician MBC practices. Future directions for development and application of the tool are discussed.

Minimal clinically important difference of the Modified Fatigue Impact Scale in Parkinson's disease.

PubMed

Kluger, Benzi M; Garimella, Sanjana; Garvan, Cynthia

2017-10-01

Fatigue is a common and debilitating symptom of Parkinson's disease (PD) with no evidence-based treatments. While several fatigue scales are partially validated in PD the minimal clinically important difference (MCID) is unknown for any scale but is an important psychometric value to design and interpret therapeutic trials. We thus sought to determine the MCID for the Modified Fatigue Impact Scale (MFIS). This is a secondary data analysis from 94 PD participants in an acupuncture trial for PD fatigue. Standard psychometric approaches were used to establish validity and an anchor-based approach was used to determine the MCID. The MFIS demonstrated good concurrent validity with other outcome measures and high internal consistency. MCIDs values were found to be 13.8, 6.8 and 6.2 for the MFIS total, MFIS cognitive, and MFIS physical subscores respectively. The MFIS is a valid multidimensional measure of fatigue in PD with demonstrable MCID. Copyright © 2017 Elsevier Ltd. All rights reserved.

Rule base system for identification of patients with specific critical care syndromes: The "sniffer" for acute lung injury.

PubMed

Herasevich, V; Yilmaz, M; Khan, H; Chute, C G; Gajic, O

2007-10-11

Early detection of specific critical care syndromes, such as sepsis or acute lung injury (ALI)is essential for timely implementation of evidence based therapies. Using a near-real time copy of the electronic medical records ("ICU data mart") we developed and validated custom electronic alert (ALI"sniffer") in a cohort of 485 critically ill medical patients. Compared with the gold standard of prospective screening, ALI "sniffer" demonstrated good sensitivity, 93% (95% CI 90 to 95) and specificity, 90% (95% CI 87 to 92). It is not known if the bedside implementation of ALI "sniffer" will improve the adherence to evidence-based therapies and outcome of patients with ALI.

Are Health-Related Tweets Evidence Based? Review and Analysis of Health-Related Tweets on Twitter.

PubMed

Alnemer, Khalid A; Alhuzaim, Waleed M; Alnemer, Ahmed A; Alharbi, Bader B; Bawazir, Abdulrahman S; Barayyan, Omar R; Balaraj, Faisal K

2015-10-29

Health care professionals are utilizing Twitter to communicate, develop disease surveillance systems, and mine health-related information. The immediate users of this health information is the general public, including patients. This necessitates the validation of health-related tweets by health care professionals to ensure they are evidence based and to avoid the use of noncredible information as a basis for critical decisions. The aim of this study was to evaluate health-related tweets on Twitter for validity (evidence based) and to create awareness in the community regarding the importance of evidence-based health-related tweets. All tweets containing health-related information in the Arabic language posted April 1-5, 2015, were mined from Twitter. The tweets were classified based on popularity, activity, interaction, and frequency to obtain 25 Twitter accounts (8 physician accounts, 10 nonofficial health institute accounts, 4 dietitian accounts, and 3 government institute accounts) and 625 tweets. These tweets were evaluated by 3 American Board-certified medical consultants and a score was generated (true/false) and interobserver agreement was calculated. A total of 625 health-related Arabic-language tweets were identified from 8 physician accounts, 10 nonofficial health institute accounts, 4 dietician accounts, and 3 government institute accounts. The reviewers labeled 320 (51.2%) tweets as false and 305 (48.8%) tweets as true. Comparative analysis of tweets by account type showed 60 of 75 (80%) tweets by government institutes, 124 of 201 (61.7%) tweets by physicians, and 42 of 101 (41.6%) tweets by dieticians were true. The interobserver agreement was moderate (range 0.78-0.22). More than half of the health-related tweets (169/248, 68.1%) from nonofficial health institutes and dietician accounts (59/101, 58.4%) were false. Tweets by the physicians were more likely to be rated "true" compared to other groups (P<.001). Approximately half of the medical tweets from professional accounts on Twitter were found to be false based on expert review. Furthermore, most of the evidence-based health-related tweets are posted by government institutes and physicians.

The IDEA Assessment Tool: Assessing the Reporting, Diagnostic Reasoning, and Decision-Making Skills Demonstrated in Medical Students' Hospital Admission Notes.

PubMed

Baker, Elizabeth A; Ledford, Cynthia H; Fogg, Louis; Way, David P; Park, Yoon Soo

2015-01-01

Construct: Clinical skills are used in the care of patients, including reporting, diagnostic reasoning, and decision-making skills. Written comprehensive new patient admission notes (H&Ps) are a ubiquitous part of student education but are underutilized in the assessment of clinical skills. The interpretive summary, differential diagnosis, explanation of reasoning, and alternatives (IDEA) assessment tool was developed to assess students' clinical skills using written comprehensive new patient admission notes. The validity evidence for assessment of clinical skills using clinical documentation following authentic patient encounters has not been well documented. Diagnostic justification tools and postencounter notes are described in the literature (1,2) but are based on standardized patient encounters. To our knowledge, the IDEA assessment tool is the first published tool that uses medical students' H&Ps to rate students' clinical skills. The IDEA assessment tool is a 15-item instrument that asks evaluators to rate students' reporting, diagnostic reasoning, and decision-making skills based on medical students' new patient admission notes. This study presents validity evidence in support of the IDEA assessment tool using Messick's unified framework, including content (theoretical framework), response process (interrater reliability), internal structure (factor analysis and internal-consistency reliability), and relationship to other variables. Validity evidence is based on results from four studies conducted between 2010 and 2013. First, the factor analysis (2010, n = 216) yielded a three-factor solution, measuring patient story, IDEA, and completeness, with reliabilities of .79, .88, and .79, respectively. Second, an initial interrater reliability study (2010) involving two raters demonstrated fair to moderate consensus (κ = .21-.56, ρ =.42-.79). Third, a second interrater reliability study (2011) with 22 trained raters also demonstrated fair to moderate agreement (intraclass correlations [ICCs] = .29-.67). There was moderate reliability for all three skill domains, including reporting skills (ICC = .53), diagnostic reasoning skills (ICC = .64), and decision-making skills (ICC = .63). Fourth, there was a significant correlation between IDEA rating scores (2010-2013) and final Internal Medicine clerkship grades (r = .24), 95% confidence interval (CI) [.15, .33]. The IDEA assessment tool is a novel tool with validity evidence to support its use in the assessment of students' reporting, diagnostic reasoning, and decision-making skills. The moderate reliability achieved supports formative or lower stakes summative uses rather than high-stakes summative judgments.

miRTarBase update 2018: a resource for experimentally validated microRNA-target interactions.

PubMed

Chou, Chih-Hung; Shrestha, Sirjana; Yang, Chi-Dung; Chang, Nai-Wen; Lin, Yu-Ling; Liao, Kuang-Wen; Huang, Wei-Chi; Sun, Ting-Hsuan; Tu, Siang-Jyun; Lee, Wei-Hsiang; Chiew, Men-Yee; Tai, Chun-San; Wei, Ting-Yen; Tsai, Tzi-Ren; Huang, Hsin-Tzu; Wang, Chung-Yu; Wu, Hsin-Yi; Ho, Shu-Yi; Chen, Pin-Rong; Chuang, Cheng-Hsun; Hsieh, Pei-Jung; Wu, Yi-Shin; Chen, Wen-Liang; Li, Meng-Ju; Wu, Yu-Chun; Huang, Xin-Yi; Ng, Fung Ling; Buddhakosai, Waradee; Huang, Pei-Chun; Lan, Kuan-Chun; Huang, Chia-Yen; Weng, Shun-Long; Cheng, Yeong-Nan; Liang, Chao; Hsu, Wen-Lian; Huang, Hsien-Da

2018-01-04

MicroRNAs (miRNAs) are small non-coding RNAs of ∼ 22 nucleotides that are involved in negative regulation of mRNA at the post-transcriptional level. Previously, we developed miRTarBase which provides information about experimentally validated miRNA-target interactions (MTIs). Here, we describe an updated database containing 422 517 curated MTIs from 4076 miRNAs and 23 054 target genes collected from over 8500 articles. The number of MTIs curated by strong evidence has increased ∼1.4-fold since the last update in 2016. In this updated version, target sites validated by reporter assay that are available in the literature can be downloaded. The target site sequence can extract new features for analysis via a machine learning approach which can help to evaluate the performance of miRNA-target prediction tools. Furthermore, different ways of browsing enhance user browsing specific MTIs. With these improvements, miRTarBase serves as more comprehensively annotated, experimentally validated miRNA-target interactions databases in the field of miRNA related research. miRTarBase is available at http://miRTarBase.mbc.nctu.edu.tw/. © The Author(s) 2017. Published by Oxford University Press on behalf of Nucleic Acids Research.

Emergency department triage scales and their components: a systematic review of the scientific evidence.

PubMed

Farrohknia, Nasim; Castrén, Maaret; Ehrenberg, Anna; Lind, Lars; Oredsson, Sven; Jonsson, Håkan; Asplund, Kjell; Göransson, Katarina E

2011-06-30

Emergency department (ED) triage is used to identify patients' level of urgency and treat them based on their triage level. The global advancement of triage scales in the past two decades has generated considerable research on the validity and reliability of these scales. This systematic review aims to investigate the scientific evidence for published ED triage scales. The following questions are addressed: 1. Does assessment of individual vital signs or chief complaints affect mortality during the hospital stay or within 30 days after arrival at the ED?2. What is the level of agreement between clinicians' triage decisions compared to each other or to a gold standard for each scale (reliability)? 3. How valid is each triage scale in predicting hospitalization and hospital mortality? A systematic search of the international literature published from 1966 through March 31, 2009 explored the British Nursing Index, Business Source Premier, CINAHL, Cochrane Library, EMBASE, and PubMed. Inclusion was limited to controlled studies of adult patients (≥ 15 years) visiting EDs for somatic reasons. Outcome variables were death in ED or hospital and need for hospitalization (validity). Methodological quality and clinical relevance of each study were rated as high, medium, or low. The results from the studies that met the inclusion criteria and quality standards were synthesized applying the internationally developed GRADE system. Each conclusion was then assessed as having strong, moderately strong, limited, or insufficient scientific evidence. If studies were not available, this was also noted.We found ED triage scales to be supported, at best, by limited and often insufficient evidence.The ability of the individual vital signs included in the different scales to predict outcome is seldom, if at all, studied in the ED setting. The scientific evidence to assess interrater agreement (reliability) was limited for one triage scale and insufficient or lacking for all other scales. Two of the scales yielded limited scientific evidence, and one scale yielded insufficient evidence, on which to assess the risk of early death or hospitalization in patients assigned to the two lowest triage levels on a 5-level scale (validity).

Emergency Department Triage Scales and Their Components: A Systematic Review of the Scientific Evidence

PubMed Central

2011-01-01

Emergency department (ED) triage is used to identify patients' level of urgency and treat them based on their triage level. The global advancement of triage scales in the past two decades has generated considerable research on the validity and reliability of these scales. This systematic review aims to investigate the scientific evidence for published ED triage scales. The following questions are addressed: 1. Does assessment of individual vital signs or chief complaints affect mortality during the hospital stay or within 30 days after arrival at the ED? 2. What is the level of agreement between clinicians' triage decisions compared to each other or to a gold standard for each scale (reliability)? 3. How valid is each triage scale in predicting hospitalization and hospital mortality? A systematic search of the international literature published from 1966 through March 31, 2009 explored the British Nursing Index, Business Source Premier, CINAHL, Cochrane Library, EMBASE, and PubMed. Inclusion was limited to controlled studies of adult patients (≥15 years) visiting EDs for somatic reasons. Outcome variables were death in ED or hospital and need for hospitalization (validity). Methodological quality and clinical relevance of each study were rated as high, medium, or low. The results from the studies that met the inclusion criteria and quality standards were synthesized applying the internationally developed GRADE system. Each conclusion was then assessed as having strong, moderately strong, limited, or insufficient scientific evidence. If studies were not available, this was also noted. We found ED triage scales to be supported, at best, by limited and often insufficient evidence. The ability of the individual vital signs included in the different scales to predict outcome is seldom, if at all, studied in the ED setting. The scientific evidence to assess interrater agreement (reliability) was limited for one triage scale and insufficient or lacking for all other scales. Two of the scales yielded limited scientific evidence, and one scale yielded insufficient evidence, on which to assess the risk of early death or hospitalization in patients assigned to the two lowest triage levels on a 5-level scale (validity). PMID:21718476

Characteristics of knowledge content in a curated online evidence library.

PubMed

Varada, Sowmya; Lacson, Ronilda; Raja, Ali S; Ip, Ivan K; Schneider, Louise; Osterbur, David; Bain, Paul; Vetrano, Nicole; Cellini, Jacqueline; Mita, Carol; Coletti, Margaret; Whelan, Julia; Khorasani, Ramin

2018-05-01

To describe types of recommendations represented in a curated online evidence library, report on the quality of evidence-based recommendations pertaining to diagnostic imaging exams, and assess underlying knowledge representation. The evidence library is populated with clinical decision rules, professional society guidelines, and locally developed best practice guidelines. Individual recommendations were graded based on a standard methodology and compared using chi-square test. Strength of evidence ranged from grade 1 (systematic review) through grade 5 (recommendations based on expert opinion). Finally, variations in the underlying representation of these recommendations were identified. The library contains 546 individual imaging-related recommendations. Only 15% (16/106) of recommendations from clinical decision rules were grade 5 vs 83% (526/636) from professional society practice guidelines and local best practice guidelines that cited grade 5 studies (P < .0001). Minor head trauma, pulmonary embolism, and appendicitis were topic areas supported by the highest quality of evidence. Three main variations in underlying representations of recommendations were "single-decision," "branching," and "score-based." Most recommendations were grade 5, largely because studies to test and validate many recommendations were absent. Recommendation types vary in amount and complexity and, accordingly, the structure and syntax of statements they generate. However, they can be represented in single-decision, branching, and score-based representations. In a curated evidence library with graded imaging-based recommendations, evidence quality varied widely, with decision rules providing the highest-quality recommendations. The library may be helpful in highlighting evidence gaps, comparing recommendations from varied sources on similar clinical topics, and prioritizing imaging recommendations to inform clinical decision support implementation.

When fast logic meets slow belief: Evidence for a parallel-processing model of belief bias.

PubMed

Trippas, Dries; Thompson, Valerie A; Handley, Simon J

2017-05-01

Two experiments pitted the default-interventionist account of belief bias against a parallel-processing model. According to the former, belief bias occurs because a fast, belief-based evaluation of the conclusion pre-empts a working-memory demanding logical analysis. In contrast, according to the latter both belief-based and logic-based responding occur in parallel. Participants were given deductive reasoning problems of variable complexity and instructed to decide whether the conclusion was valid on half the trials or to decide whether the conclusion was believable on the other half. When belief and logic conflict, the default-interventionist view predicts that it should take less time to respond on the basis of belief than logic, and that the believability of a conclusion should interfere with judgments of validity, but not the reverse. The parallel-processing view predicts that beliefs should interfere with logic judgments only if the processing required to evaluate the logical structure exceeds that required to evaluate the knowledge necessary to make a belief-based judgment, and vice versa otherwise. Consistent with this latter view, for the simplest reasoning problems (modus ponens), judgments of belief resulted in lower accuracy than judgments of validity, and believability interfered more with judgments of validity than the converse. For problems of moderate complexity (modus tollens and single-model syllogisms), the interference was symmetrical, in that validity interfered with belief judgments to the same degree that believability interfered with validity judgments. For the most complex (three-term multiple-model syllogisms), conclusion believability interfered more with judgments of validity than vice versa, in spite of the significant interference from conclusion validity on judgments of belief.

Training, Simulation, the Learning Curve, and How to Reduce Complications in Urology.

PubMed

Brunckhorst, Oliver; Volpe, Alessandro; van der Poel, Henk; Mottrie, Alexander; Ahmed, Kamran

2016-04-01

Urology is at the forefront of minimally invasive surgery to a great extent. These procedures produce additional learning challenges and possess a steep initial learning curve. Training and assessment methods in surgical specialties such as urology are known to lack clear structure and often rely on differing operative flow experienced by individuals and institutions. This article aims to assess current urology training modalities, to identify the role of simulation within urology, to define and identify the learning curves for various urologic procedures, and to discuss ways to decrease complications in the context of training. A narrative review of the literature was conducted through December 2015 using the PubMed/Medline, Embase, and Cochrane Library databases. Evidence of the validity of training methods in urology includes observation of a procedure, mentorship and fellowship, e-learning, and simulation-based training. Learning curves for various urologic procedures have been recommended based on the available literature. The importance of structured training pathways is highlighted, with integration of modular training to ensure patient safety. Valid training pathways are available in urology. The aim in urology training should be to combine all of the available evidence to produce procedure-specific curricula that utilise the vast array of training methods available to ensure that we continue to improve patient outcomes and reduce complications. The current evidence for different training methods available in urology, including simulation-based training, was reviewed, and the learning curves for various urologic procedures were critically analysed. Based on the evidence, future pathways for urology curricula have been suggested to ensure that patient safety is improved. Copyright © 2016 European Association of Urology. Published by Elsevier B.V. All rights reserved.

[Approaches, knowledge and capabilities of nurses and physicians regarding evidence-based clinical practice in the Imbadura province (Ecuador)].

PubMed

Molina Mula, Jesús; Muñoz Navarro, Paulina; Vaca Auz, Janeth; Cabascango Cabascango, Carmita; Cabascango Cabascango, Katty

2015-01-01

The research raises the need to increase understanding of organizational and personal factors that influence the attitude and aptitude of each professional, with respect to evidence-based clinical practice. The aim of this study is to describe the transfer of knowledge into clinical practice in hospital units in Imbabura (Ecuador) identifying the obstacles to implementing evidence-based clinical practice validated questionnaire EBPQ-19. A cross-sectional observational study was conducted in hospitals of the Ministry of Public Health of Imbabura of Ecuador took place, including a total of 281 nurses and physicians. Nurses and physicians showed positive attitudes toward evidence-based clinical practice (EBCP) and their use to support clinical decision-making. This research evidences perceptions of professionals on strategies for knowledge transfer and obstacles to carry it out. Significant differences between the perception of the use of EBCP strategies between nurses and physicians are observed. Physicians consider they use them frequently, while nurses acknowledge using them less (chi-square: 105.254, P=.018). In conclusion, we can say that these factors should be considered as necessary to improve the quality of care that is provided to users based on the best available evidence. It is necessary to start developing change interventions in this regard to remedy the current situation of clinical practice based not on evidence, but rather on experience only. Experimental studies demonstrating the effectiveness of strategies to eliminate barriers to scientific evidence-based clinical practice should be conducted. Copyright © 2015 Elsevier España, S.L.U. All rights reserved.

A Systematic Literature Review of the Information-Seeking Behavior of Dentists in Developed Countries.

PubMed

Isham, Amy; Bettiol, Silvana; Hoang, Ha; Crocombe, Leonard

2016-05-01

Understanding the information-seeking behavior of dentists may inform ways to increase the dentist uptake of evidence-based research for clinical decision making and the practice of evidence-based dentistry, but no systematic review of dentist information-seeking behavior has been conducted. This review aimed to synthesize the best available evidence on where and how dentists seek information. A literature search of Web of Science, Scopus, PubMed, and reference lists of English language studies from the Organization for Economic Cooperation and Development countries of dentists' information-seeking behavior published between 2002 and 2014 was conducted. Selected articles were assessed using mixed methods analysis, and the data extracted were thematically synthesized. Nine studies met the inclusion criteria, and four main themes were identified: dentists' difficulty translating evidence-based resources into clinical practice; dentists' preference for face-to-face meetings, collegial discussion, and print materials over evidence-based resources; dentists' perceptions of the validity of evidence-based resources and the role of specialist and experienced dentists as information sources for general and less experienced dentists; and differences between early and late adopters of research evidence. Dentists in these studies tended to adopt new materials/techniques after discussion with a colleague, a dental specialist, or a respected dental expert. These dentists also reported lacking time, experience, skills, and confidence to find and use evidence-based resources. Many of the dentists studied were cautious about making decisions based on documentary sources like literature reviews and preferred to seek advice from an experienced or specialist colleague or to participate in face-to-face meetings.

Changing paradigms from a historical DSM-III and DSM-IV view toward an evidence-based definition of premature ejaculation. Part I--validity of DSM-IV-TR.

PubMed

Waldinger, Marcel D; Schweitzer, Dave H

2006-07-01

In former days, information obtained from randomized well-controlled clinical trials and epidemiological studies on premature ejaculation (PE) was not available, thereby hampering the efforts of the consecutive DSM Work Groups on Sexual Disorders to formulate an evidence-based definition of PE. The current DSM-IV-TR definition of PE is still nonevidence based. In addition, the requirement that persistent self-perceived PE, distress, and interpersonal difficulties, in absence of a quantified ejaculation time, are necessary to establish the diagnosis remains disputable. To investigate the validity and reliability of DSM and ICD diagnosis of premature ejaculation. The historical development of DSM and ICD classification of mental disorders is critically reviewed, and two studies using the DSM-IV-TR definition of PE is critically reanalyzed. Reanalysis of two studies using the DSM-IV-TR definition of PE has shown that DSM-diagnosed PE can be accompanied by long intravaginal ejaculation latency time (IELT) values. The reanalysis revealed a low positive predictive value for the DSM-IV-TR definition when used as a diagnostic test. A similar situation pertains to the American Urological Association (AUA) definition of PE, which is practically a copy of the DSM-IV-TR definition. It should be emphasized that any evidence-based definition of PE needs objectively collected patient-reported outcome (PRO) data from epidemiological studies, as well as reproducible quantifications of the IELT.

A constructive Indian country response to the evidence-based program mandate.

PubMed

Walker, R Dale; Bigelow, Douglas A

2011-01-01

Over the last 20 years governmental mandates for preferentially funding evidence-based "model" practices and programs has become doctrine in some legislative bodies, federal agencies, and state agencies. It was assumed that what works in small sample, controlled settings would work in all community settings, substantially improving safety, effectiveness, and value-for-money. The evidence-based "model" programs mandate has imposed immutable "core components," fidelity testing, alien programming and program developers, loss of familiar programs, and resource capacity requirements upon tribes, while infringing upon their tribal sovereignty and consultation rights. Tribal response in one state (Oregon) went through three phases: shock and rejection; proposing an alternative approach using criteria of cultural appropriateness, aspiring to evaluability; and adopting logic modeling. The state heard and accepted the argument that the tribal way of knowing is different and valid. Currently, a state-authorized tribal logic model and a review panel process are used to approve tribal best practices for state funding. This constructive response to the evidence-based program mandate elevates tribal practices in the funding and regulatory world, facilitates continuing quality improvement and evaluation, while ensuring that practices and programs remain based on local community context and culture. This article provides details of a model that could well serve tribes facing evidence-based model program mandates throughout the country.

Brain Stretchers Book 4--Advanced.

ERIC Educational Resources Information Center

Anderson, Carolyn

This book provides puzzles, games, and mathematical activities for students in elementary grades. Number concepts and arithmetic are common topics. These classic math, logic, and word-problem activities encourage students to become flexible, creative thinkers while teaching them to draw valid conclusions based on logic and evidence. Each activity…

Familiarizing Students with the Empirically Supported Treatment Approaches for Childhood Problems.

ERIC Educational Resources Information Center

Wilkins, Victoria; Chambliss, Catherine

The clinical research literature exploring the efficacy of particular treatment approaches is reviewed with the intent to facilitate the training of counseling students. Empirically supported treatments (ESTs) is defined operationally as evidence-based treatments following the listing of empirically validated psychological treatments reported by…

Reaction time as an indicator of insufficient effort: Development and validation of an embedded performance validity parameter.

PubMed

Stevens, Andreas; Bahlo, Simone; Licha, Christina; Liske, Benjamin; Vossler-Thies, Elisabeth

2016-11-30

Subnormal performance in attention tasks may result from various sources including lack of effort. In this report, the derivation and validation of a performance validity parameter for reaction time is described, using a set of malingering-indices ("Slick-criteria"), and 3 independent samples of participants (total n =893). The Slick-criteria yield an estimate of the probability of malingering based on the presence of an external incentive, evidence from neuropsychological testing, from self-report and clinical data. In study (1) a validity parameter is derived using reaction time data of a sample, composed of inpatients with recent severe brain lesions not involved in litigation and of litigants with and without brain lesion. In study (2) the validity parameter is tested in an independent sample of litigants. In study (3) the parameter is applied to an independent sample comprising cooperative and non-cooperative testees. Logistic regression analysis led to a derived validity parameter based on median reaction time and standard deviation. It performed satisfactorily in studies (2) and (3) (study 2 sensitivity=0.94, specificity=1.00; study 3 sensitivity=0.79, specificity=0.87). The findings suggest that median reaction time and standard deviation may be used as indicators of negative response bias. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

Evidence-Based Redesign of the COMLEX-USA Series.

PubMed

Gimpel, John R; Horber, Dorothy; Sandella, Jeanne M; Knebl, Janice A; Thornburg, John E

2017-04-01

To ensure that the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) reflects the evolving practice of osteopathic medicine, the National Board of Osteopathic Medical Examiners has developed new content and format specifications for an enhanced, competency-based examination program to be implemented with COMLEX-USA Level 3 in 2018. This article summarizes the evidence-based design processes that served as the foundation for blueprint development and the evidence supporting its validity. An overview is provided of the blueprint's 2 dimensions: Competency Domains and Clinical Presentations. The authors focus on the evidence that supports interpretation of test scores for the primary and intended purpose of COMLEX-USA, which is osteopathic physician licensure. Important secondary uses and the educational and catalytic effect of assessments are also described. This article concludes with the National Board of Osteopathic Medical Examiners' plans to ensure that the COMLEX-USA series remains current and meets the needs of its stakeholders-the patients who seek care from osteopathic physicians.

Information systems: the key to evidence-based health practice.

PubMed Central

Rodrigues, R. J.

2000-01-01

Increasing prominence is being given to the use of best current evidence in clinical practice and health services and programme management decision-making. The role of information in evidence-based practice (EBP) is discussed, together with questions of how advanced information systems and technology (IS&T) can contribute to the establishment of a broader perspective for EBP. The author examines the development, validation and use of a variety of sources of evidence and knowledge that go beyond the well-established paradigm of research, clinical trials, and systematic literature review. Opportunities and challenges in the implementation and use of IS&T and knowledge management tools are examined for six application areas: reference databases, contextual data, clinical data repositories, administrative data repositories, decision support software, and Internet-based interactive health information and communication. Computerized and telecommunications applications that support EBP follow a hierarchy in which systems, tasks and complexity range from reference retrieval and the processing of relatively routine transactions, to complex "data mining" and rule-driven decision support systems. PMID:11143195

Focused Evidence Review: Psychometric Properties of Patient-Reported Outcome Measures for Chronic Musculoskeletal Pain.

PubMed

Goldsmith, Elizabeth S; Taylor, Brent C; Greer, Nancy; Murdoch, Maureen; MacDonald, Roderick; McKenzie, Lauren; Rosebush, Christina E; Wilt, Timothy J

2018-05-01

Developing successful interventions for chronic musculoskeletal pain requires valid, responsive, and reliable outcome measures. The Minneapolis VA Evidence-based Synthesis Program completed a focused evidence review on key psychometric properties of 17 self-report measures of pain severity and pain-related functional impairment suitable for clinical research on chronic musculoskeletal pain. Pain experts of the VA Pain Measurement Outcomes Workgroup identified 17 pain measures to undergo systematic review. In addition to a MEDLINE search on these 17 measures (1/2000-1/2017), we hand-searched (without publication date limits) the reference lists of all included studies, prior systematic reviews, and-when available-Web sites dedicated to each measure (PROSPERO registration CRD42017056610). Our primary outcome was the measure's minimal important difference (MID). Secondary outcomes included responsiveness, validity, and test-retest reliability. Outcomes were synthesized through evidence mapping and qualitative comparison. Of 1635 abstracts identified, 331 articles underwent full-text review, and 43 met inclusion criteria. Five measures (Oswestry Disability Index (ODI), Roland-Morris Disability Questionnaire (RMDQ), SF-36 Bodily Pain Scale (SF-36 BPS), Numeric Rating Scale (NRS), and Visual Analog Scale (VAS)) had data reported on MID, responsiveness, validity, and test-retest reliability. Seven measures had data reported on three of the four psychometric outcomes. Eight measures had reported MIDs, though estimation methods differed substantially and often were not clinically anchored. In this focused evidence review, the most evidence on key psychometric properties in chronic musculoskeletal pain populations was found for the ODI, RMDQ, SF-36 BPS, NRS, and VAS. Key limitations in the field include substantial variation in methods of estimating psychometric properties, defining chronic musculoskeletal pain, and reporting patient demographics. Registered in the PROSPERO database: CRD42017056610.

Patient-Reported Outcome Measures for Hand and Wrist Trauma: Is There Sufficient Evidence of Reliability, Validity, and Responsiveness?

PubMed

Dacombe, Peter Jonathan; Amirfeyz, Rouin; Davis, Tim

2016-03-01

Patient-reported outcome measures (PROMs) are important tools for assessing outcomes following injuries to the hand and wrist. Many commonly used PROMs have no evidence of reliability, validity, and responsiveness in a hand and wrist trauma population. This systematic review examines the PROMs used in the assessment of hand and wrist trauma patients, and the evidence for reliability, validity, and responsiveness of each measure in this population. A systematic review of Pubmed, Medline, and CINAHL searching for randomized controlled trials of patients with traumatic injuries to the hand and wrist was carried out to identify the PROMs. For each identified PROM, evidence of reliability, validity, and responsiveness was identified using a further systematic review of the Pubmed, Medline, CINAHL, and reverse citation trail audit procedure. The PROM used most often was the Disabilities of the Arm, Shoulder and Hand (DASH) questionnaire; the Patient-Rated Wrist Evaluation (PRWE), Gartland and Werley score, Michigan Hand Outcomes score, Mayo Wrist Score, and Short Form 36 were also commonly used. Only the DASH and PRWE have evidence of reliability, validity, and responsiveness in patients with traumatic injuries to the hand and wrist; other measures either have incomplete evidence or evidence gathered in a nontraumatic population. The DASH and PRWE both have evidence of reliability, validity, and responsiveness in a hand and wrist trauma population. Other PROMs used to assess hand and wrist trauma patients do not. This should be considered when selecting a PROM for patients with traumatic hand and wrist pathology.

[Validity evidence of the Health-Related Quality of Life for Drug Abusers Test based on the Biaxial Model of Addiction].

PubMed

Lozano, Oscar M; Rojas, Antonio J; Pérez, Cristino; González-Sáiz, Francisco; Ballesta, Rosario; Izaskun, Bilbao

2008-05-01

The aim of this work is to show evidence of the validity of the Health-Related Quality of Life for Drug Abusers Test (HRQoLDA Test). This test was developed to measure specific HRQoL for drugs abusers, within the theoretical addiction framework of the biaxial model. The sample comprised 138 patients diagnosed with opiate drug dependence. In this study, the following constructs and variables of the biaxial model were measured: severity of dependence, physical health status, psychological adjustment and substance consumption. Results indicate that the HRQoLDA Test scores are related to dependency and consumption-related problems. Multiple regression analysis reveals that HRQoL can be predicted from drug dependence, physical health status and psychological adjustment. These results contribute empirical evidence of the theoretical relationships established between HRQoL and the biaxial model, and they support the interpretation of the HRQoLDA Test to measure HRQoL in drug abusers, thus providing a test to measure this specific construct in this population.

Computerized neurocognitive testing in the management of sport-related concussion: an update.

PubMed

Resch, Jacob E; McCrea, Michael A; Cullum, C Munro

2013-12-01

Since the late nineties, computerized neurocognitive testing has become a central component of sport-related concussion (SRC) management at all levels of sport. In 2005, a review of the available evidence on the psychometric properties of four computerized neuropsychological test batteries concluded that the tests did not possess the necessary criteria to warrant clinical application. Since the publication of that review, several more computerized neurocognitive tests have entered the market place. The purpose of this review is to summarize the body of published studies on psychometric properties and clinical utility of computerized neurocognitive tests available for use in the assessment of SRC. A review of the literature from 2005 to 2013 was conducted to gather evidence of test-retest reliability and clinical validity of these instruments. Reviewed articles included both prospective and retrospective studies of primarily sport-based adult and pediatric samples. Summaries are provided regarding the available evidence of reliability and validity for the most commonly used computerized neurocognitive tests in sports settings.

20 CFR 219.31 - Evidence of a valid ceremonial marriage.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 20 Employees' Benefits 1 2010-04-01 2010-04-01 false Evidence of a valid ceremonial marriage. 219... marriage. (a) Preferred evidence. Preferred evidence of a ceremonial marriage is— (1) A copy of the public record of the marriage, certified by the custodian of the record or by a Board employee; (2) A copy of a...

Assessing the capacity of ministries of health to use research in decision-making: conceptual framework and tool.

PubMed

Rodríguez, Daniela C; Hoe, Connie; Dale, Elina M; Rahman, M Hafizur; Akhter, Sadika; Hafeez, Assad; Irava, Wayne; Rajbangshi, Preety; Roman, Tamlyn; Ţîrdea, Marcela; Yamout, Rouham; Peters, David H

2017-08-01

The capacity to demand and use research is critical for governments if they are to develop policies that are informed by evidence. Existing tools designed to assess how government officials use evidence in decision-making have significant limitations for low- and middle-income countries (LMICs); they are rarely tested in LMICs and focus only on individual capacity. This paper introduces an instrument that was developed to assess Ministry of Health (MoH) capacity to demand and use research evidence for decision-making, which was tested for reliability and validity in eight LMICs (Bangladesh, Fiji, India, Lebanon, Moldova, Pakistan, South Africa, Zambia). Instrument development was based on a new conceptual framework that addresses individual, organisational and systems capacities, and items were drawn from existing instruments and a literature review. After initial item development and pre-testing to address face validity and item phrasing, the instrument was reduced to 54 items for further validation and item reduction. In-country study teams interviewed a systematic sample of 203 MoH officials. Exploratory factor analysis was used in addition to standard reliability and validity measures to further assess the items. Thirty items divided between two factors representing organisational and individual capacity constructs were identified. South Africa and Zambia demonstrated the highest level of organisational capacity to use research, whereas Pakistan and Bangladesh were the lowest two. In contrast, individual capacity was highest in Pakistan, followed by South Africa, whereas Bangladesh and Lebanon were the lowest. The framework and related instrument represent a new opportunity for MoHs to identify ways to understand and improve capacities to incorporate research evidence in decision-making, as well as to provide a basis for tracking change.

Adapting detection sensitivity based on evidence of irregular sinus arrhythmia to improve atrial fibrillation detection in insertable cardiac monitors.

PubMed

Pürerfellner, Helmut; Sanders, Prashanthan; Sarkar, Shantanu; Reisfeld, Erin; Reiland, Jerry; Koehler, Jodi; Pokushalov, Evgeny; Urban, Luboš; Dekker, Lukas R C

2017-10-03

Intermittent change in p-wave discernibility during periods of ectopy and sinus arrhythmia is a cause of inappropriate atrial fibrillation (AF) detection in insertable cardiac monitors (ICM). To address this, we developed and validated an enhanced AF detection algorithm. Atrial fibrillation detection in Reveal LINQ ICM uses patterns of incoherence in RR intervals and absence of P-wave evidence over a 2-min period. The enhanced algorithm includes P-wave evidence during RR irregularity as evidence of sinus arrhythmia or ectopy to adaptively optimize sensitivity for AF detection. The algorithm was developed and validated using Holter data from the XPECT and LINQ Usability studies which collected surface electrocardiogram (ECG) and continuous ICM ECG over a 24-48 h period. The algorithm detections were compared with Holter annotations, performed by multiple reviewers, to compute episode and duration detection performance. The validation dataset comprised of 3187 h of valid Holter and LINQ recordings from 138 patients, with true AF in 37 patients yielding 108 true AF episodes ≥2-min and 449 h of AF. The enhanced algorithm reduced inappropriately detected episodes by 49% and duration by 66% with <1% loss in true episodes or duration. The algorithm correctly identified 98.9% of total AF duration and 99.8% of total sinus or non-AF rhythm duration. The algorithm detected 97.2% (99.7% per-patient average) of all AF episodes ≥2-min, and 84.9% (95.3% per-patient average) of detected episodes involved AF. An enhancement that adapts sensitivity for AF detection reduced inappropriately detected episodes and duration with minimal reduction in sensitivity. © The Author 2017. Published by Oxford University Press on behalf of the European Society of Cardiology

Validity of the SAT® for Predicting First-Year Grades: 2010 SAT Validity Sample. Statistical Report 2013-2

ERIC Educational Resources Information Center

Patterson, Brian F.; Mattern, Krista D.

2013-01-01

The continued accumulation of validity evidence for the core uses of educational assessments is critical to ensure that proper inferences will be made for those core purposes. To that end, the College Board has continued to follow previous cohorts of college students and this report provides updated validity evidence for using the SAT to predict…

The key-features approach to assess clinical decisions: validity evidence to date.

PubMed

Bordage, G; Page, G

2018-05-17

The key-features (KFs) approach to assessment was initially proposed during the First Cambridge Conference on Medical Education in 1984 as a more efficient and effective means of assessing clinical decision-making skills. Over three decades later, we conducted a comprehensive, systematic review of the validity evidence gathered since then. The evidence was compiled according to the Standards for Educational and Psychological Testing's five sources of validity evidence, namely, Content, Response process, Internal structure, Relations to other variables, and Consequences, to which we added two other types related to Cost-feasibility and Acceptability. Of the 457 publications that referred to the KFs approach between 1984 and October 2017, 164 are cited here; the remaining 293 were either redundant or the authors simply mentioned the KFs concept in relation to their work. While one set of articles reported meeting the validity standards, another set examined KFs test development choices and score interpretation. The accumulated validity evidence for the KFs approach since its inception supports the decision-making construct measured and its use to assess clinical decision-making skills at all levels of training and practice and with various types of exam formats. Recognizing that gathering validity evidence is an ongoing process, areas with limited evidence, such as item factor analyses or consequences of testing, are identified as well as new topics needing further clarification, such as the use of the KFs approach for formative assessment and its place within a program of assessment.

Measurement properties of the QuickDASH (disabilities of the arm, shoulder and hand) outcome measure and cross-cultural adaptations of the QuickDASH: a systematic review.

PubMed

Kennedy, Carol A; Beaton, Dorcas E; Smith, Peter; Van Eerd, Dwayne; Tang, Kenneth; Inrig, Taucha; Hogg-Johnson, Sheilah; Linton, Denise; Couban, Rachel

2013-11-01

To identify and synthesize evidence for the measurement properties of the QuickDASH, a shortened version of the 30-item DASH (Disabilities of the Arm, Shoulder and Hand) instrument. This systematic review used a best evidence synthesis approach to critically appraise the measurement properties [using COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN)] of the QuickDASH and cross-cultural adaptations. A standard search strategy was conducted between 2005 (year of first publication of QuickDASH) and March 2011 in MEDLINE, EMBASE and CINAHL. The search identified 14 studies to include in the best evidence synthesis of the QuickDASH. A further 11 studies were identified on eight cross-cultural adaptation versions. Many measurement properties of the QuickDASH have been evaluated in multiple studies and across most of the measurement properties. The best evidence synthesis of the QuickDASH English version suggests that this tool is performing well with strong positive evidence for reliability and validity (hypothesis testing), and moderate positive evidence for structural validity testing. Strong negative evidence was found for responsiveness due to lower correlations with global estimates of change. Information about the measurement properties of the cross-cultural adaptation versions is still lacking, or the available information is of poor overall methodological quality.

What is the evidence for conducting palliative care family meetings? A systematic review.

PubMed

Cahill, Philippa J; Lobb, Elizabeth A; Sanderson, Christine; Phillips, Jane L

2017-03-01

Structured family meeting procedures and guidelines suggest that these forums enhance family-patient-team communication in the palliative care inpatient setting. However, the vulnerability of palliative patients and the resources required to implement family meetings in accordance with recommended guidelines make better understanding about the effectiveness of this type of intervention an important priority. Aim and design: This systematic review examines the evidence supporting family meetings as a strategy to address the needs of palliative patients and their families. The review conforms to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Statement. Six medical and psychosocial databases and "CareSearch," a palliative care-specific database, were used to identify studies reporting empirical data, published in English in peer-reviewed journals from 1980 to March 2015. Book chapters, expert opinion, and gray literature were excluded. The Cochrane Collaboration Tool assessed risk of bias. Of the 5051 articles identified, 13 met the inclusion criteria: 10 quantitative and 3 qualitative studies. There was low-level evidence to support family meetings. Only two quantitative pre- and post-studies used a validated palliative care family outcome measure with both studies reporting significant results post-family meetings. Four other quantitative studies reported significant results using non-validated measures. Despite the existence of consensus-based family meeting guidelines, there is a paucity of evidence to support family meetings in the inpatient palliative care setting. Further research using more robust designs, validated outcome measures, and an economic analysis are required to build the family meeting evidence before they are routinely adopted into clinical practice.

A rule-based named-entity recognition method for knowledge extraction of evidence-based dietary recommendations

PubMed Central

2017-01-01

Evidence-based dietary information represented as unstructured text is a crucial information that needs to be accessed in order to help dietitians follow the new knowledge arrives daily with newly published scientific reports. Different named-entity recognition (NER) methods have been introduced previously to extract useful information from the biomedical literature. They are focused on, for example extracting gene mentions, proteins mentions, relationships between genes and proteins, chemical concepts and relationships between drugs and diseases. In this paper, we present a novel NER method, called drNER, for knowledge extraction of evidence-based dietary information. To the best of our knowledge this is the first attempt at extracting dietary concepts. DrNER is a rule-based NER that consists of two phases. The first one involves the detection and determination of the entities mention, and the second one involves the selection and extraction of the entities. We evaluate the method by using text corpora from heterogeneous sources, including text from several scientifically validated web sites and text from scientific publications. Evaluation of the method showed that drNER gives good results and can be used for knowledge extraction of evidence-based dietary recommendations. PMID:28644863

Evidence of the Validity of "Teaching Strategies GOLD[R]" Assessment Tool for English Language Learners and Children with Disabilities

ERIC Educational Resources Information Center

Kim, Do-Hong; Lambert, Richard G.; Burts, Diane C.

2013-01-01

Research Findings: This study examined the measurement equivalence of the "Teaching Strategies GOLD[R]" assessment system across subgroups of children based on their primary language and disability status. This study is based on teacher-collected assessment data for 3-, 4-, and 5-year-old children for the fall of 2010, winter of 2010, and spring…

Self-reported attitudes, skills and use of evidence-based practice among Canadian doctors of chiropractic: a national survey

PubMed Central

Bussières, André E.; Terhorst, Lauren; Leach, Matthew; Stuber, Kent; Evans, Roni; Schneider, Michael J.

2015-01-01

Objectives: To identify Canadian chiropractors’ attitudes, skills and use of evidence based practice (EBP), as well as their level of awareness of previously published chiropractic clinical practice guidelines (CPGs). Methods: 7,200 members of the Canadian Chiropractic Association were invited by e-mail to complete an online version of the Evidence Based practice Attitude & utilisation SurvEy (EBASE); a valid and reliable measure of participant attitudes, skills and use of EBP. Results: Questionnaires were completed by 554 respondents. Most respondents (>75%) held positive attitudes toward EBP. Over half indicated a high level of self-reported skills in EBP, and over 90% expressed an interest in improving these skills. A majority of respondents (65%) reported over half of their practice was based on evidence from clinical research, and only half (52%) agreed that chiropractic CPGs significantly impacted on their practice. Conclusions: While most Canadian chiropractors held positive attitudes towards EBP, believed EBP was useful, and were interested in improving their skills in EBP, many did not use research evidence or CPGs to guide clinical decision making. Our findings should be interpreted cautiously due to the low response rate. PMID:26816412

Validity of the Brazilian version of the Godin-Shephard Leisure-Time Physical Activity Questionnaire.

PubMed

João, Thaís Moreira São; Rodrigues, Roberta Cunha Matheus; Gallani, Maria Cecília Bueno Jayme; Miura, Cinthya Tamie Passos; Domingues, Gabriela de Barros Leite; Amireault, Steve; Godin, Gaston

2015-09-01

This study provides evidence of construct validity for the Brazilian version of the Godin-Shephard Leisure-Time Physical Activity Questionnaire (GSLTPAQ), a 1-item instrument used among 236 participants referred for cardiopulmonary exercise testing. The Baecke Habitual Physical Activity Questionnaire (Baecke-HPA) was used to evaluate convergent and divergent validity. The self-reported measure of walking (QCAF) evaluated the convergent validity. Cardiorespiratory fitness assessed convergent validity by the Veterans Specific Activity Questionnaire (VSAQ), peak measured (VO2peak) and maximum predicted (VO2pred) oxygen uptake. Partial adjusted correlation coefficients between the GSLTPAQ, Baecke-HPA, QCAF, VO2pred and VSAQ provided evidence for convergent validity; while divergent validity was supported by the absence of correlations between the GSLTPAQ and the Occupational Physical Activity domain (Baecke-HPA). The GSLTPAQ presents level 3 of evidence of construct validity and may be useful to assess leisure-time physical activity among patients with cardiovascular disease and healthy individuals.

Montreal Battery of Evaluation of Amusia: Validity evidence and norms for adolescents in Belo Horizonte, Minas Gerais, Brazil

PubMed Central

Nunes-Silva, Marília; Haase, Vitor Geraldi

2012-01-01

The Montreal Battery of Evaluation of Amusia (MBEA) is a battery of tests that assesses six music processing components: scale, contour, interval, rhythm, metric, and music memory. The present study sought to verify the psychometric characteristics of the MBEA in a sample of 150 adolescents aged 14-18 years in the city of Belo Horizonte, Minas Gerais, Brazil, and to develop specific norms for this population. We used statistical procedures that explored the dimensional structure of the MBEA and its items, evaluating their adequacy from empirical data, verifying their reliability, and providing evidence of validity. The results for the difficult levels for each test indicated a trend toward higher scores, corroborating previous studies. From the analysis of the criterion groups, almost all of the items were considered discriminatory. The global score of the MBEA was shown to be valid and reliable (rK-R20=0.896) for assessing the musical ability of normal teenagers. Based on the analysis of the items, we proposed a short version of the MBEA. Further studies with larger samples and amusic individuals are necessary to provide evidence of the validity of the MBEA in the Brazilian milieu. The present study brings to the Brazilian context a tool for diagnosing deficits in musical skills and will serve as a basis for comparisons with single case studies and studies of populations with specific neuropsychological syndromes. PMID:29213804

FUNCTIONAL PERFORMANCE TESTING OF THE HIP IN ATHLETES: A SYSTEMATIC REVIEW FOR RELIABILITY AND VALIDITY

PubMed Central

Martin, RobRoy L.

2012-01-01

Purpose/Background: The purpose of this study was to systematically review the literature for functional performance tests with evidence of reliability and validity that could be used for a young, athletic population with hip dysfunction. Methods: A search of PubMed and SPORTDiscus databases were performed to identify movement, balance, hop/jump, or agility functional performance tests from the current peer-reviewed literature used to assess function of the hip in young, athletic subjects. Results: The single-leg stance, deep squat, single-leg squat, and star excursion balance tests (SEBT) demonstrated evidence of validity and normative data for score interpretation. The single-leg stance test and SEBT have evidence of validity with association to hip abductor function. The deep squat test demonstrated evidence as a functional performance test for evaluating femoroacetabular impingement. Hop/Jump tests and agility tests have no reported evidence of reliability or validity in a population of subjects with hip pathology. Conclusions: Use of functional performance tests in the assessment of hip dysfunction has not been well established in the current literature. Diminished squat depth and provocation of pain during the single-leg balance test have been associated with patients diagnosed with FAI and gluteal tendinopathy, respectively. The SEBT and single-leg squat tests provided evidence of convergent validity through an analysis of kinematics and muscle function in normal subjects. Reliability of functional performance tests have not been established on patients with hip dysfunction. Further study is needed to establish reliability and validity of functional performance tests that can be used in a young, athletic population with hip dysfunction. Level of Evidence: 2b (Systematic Review of Literature) PMID:22893860

Practice-Based Evidence in Community Guide Systematic Reviews.

PubMed

Vaidya, Namita; Thota, Anilkrishna B; Proia, Krista K; Jamieson, Sara; Mercer, Shawna L; Elder, Randy W; Yoon, Paula; Kaufmann, Rachel; Zaza, Stephanie

2017-03-01

To assess the relative contributions and quality of practice-based evidence (PBE) and research-based evidence (RBE) in The Guide to Community Preventive Services (The Community Guide). We developed operational definitions for PBE and RBE in which the main distinguishing feature was whether allocation of participants to intervention and comparison conditions was under the control of researchers (RBE) or not (PBE). We conceptualized a continuum between RBE and PBE. We then categorized 3656 studies in 202 reviews completed since The Community Guide began in 1996. Fifty-four percent of studies were PBE and 46% RBE. Community-based and policy reviews had more PBE. Health care system and programmatic reviews had more RBE. The majority of both PBE and RBE studies were of high quality according to Community Guide scoring methods. The inclusion of substantial PBE in Community Guide reviews suggests that evidence of adequate rigor to inform practice is being produced. This should increase stakeholders' confidence that The Community Guide provides recommendations with real-world relevance. Limitations in some PBE studies suggest a need for strengthening practice-relevant designs and external validity reporting standards.

Procedure-specific assessment tool for flexible pharyngo-laryngoscopy: gathering validity evidence and setting pass-fail standards.

PubMed

Melchiors, Jacob; Petersen, K; Todsen, T; Bohr, A; Konge, Lars; von Buchwald, Christian

2018-06-01

The attainment of specific identifiable competencies is the primary measure of progress in the modern medical education system. The system, therefore, requires a method for accurately assessing competence to be feasible. Evidence of validity needs to be gathered before an assessment tool can be implemented in the training and assessment of physicians. This evidence of validity must according to the contemporary theory on validity be gathered from specific sources in a structured and rigorous manner. The flexible pharyngo-laryngoscopy (FPL) is central to the otorhinolaryngologist. We aim to evaluate the flexible pharyngo-laryngoscopy assessment tool (FLEXPAT) created in a previous study and to establish a pass-fail level for proficiency. Eighteen physicians with different levels of experience (novices, intermediates, and experienced) were recruited to the study. Each performed an FPL on two patients. These procedures were video recorded, blinded, and assessed by two specialists. The score was expressed as the percentage of a possible max score. Cronbach's α was used to analyze internal consistency of the data, and a generalizability analysis was performed. The scores of the three different groups were explored, and a pass-fail level was determined using the contrasting groups' standard setting method. Internal consistency was strong with a Cronbach's α of 0.86. We found a generalizability coefficient of 0.72 sufficient for moderate stakes assessment. We found a significant difference between the novice and experienced groups (p < 0.001) and strong correlation between experience and score (Pearson's r = 0.75). The pass/fail level was established at 72% of the maximum score. Applying this pass-fail level in the test population resulted in half of the intermediary group receiving a failing score. We gathered validity evidence for the FLEXPAT according to the contemporary framework as described by Messick. Our results support a claim of validity and are comparable to other studies exploring clinical assessment tools. The high rate of physicians underperforming in the intermediary group demonstrates the need for continued educational intervention. Based on our work, we recommend the use of the FLEXPAT in clinical assessment of FPL and the application of a pass-fail level of 72% for proficiency.

Multidimensional Diagnostic Criteria for Chronic Pain: Introduction to the ACTTION-American Pain Society Pain Taxonomy (AAPT).

PubMed

Dworkin, Robert H; Bruehl, Stephen; Fillingim, Roger B; Loeser, John D; Terman, Gregory W; Turk, Dennis C

2016-09-01

A variety of approaches have been used to develop diagnostic criteria for chronic pain. The published evidence of the reliability and validity of existing diagnostic criteria is limited, and these criteria have typically not been used in clinical practice. The availability of a widely accepted, consistently applied, and evidence-based taxonomy of diagnostic criteria would improve the quality of clinical research on chronic pain and would be of great value in clinical practice. To address the need for evidence-based diagnostic criteria for the major chronic pain conditions, the Analgesic, Anesthetic, and Addiction Clinical Trial Translations, Innovations, Opportunities, and Networks (ACTTION) public-private partnership with the US Food and Drug Administration and the American Pain Society (APS) have collaborated on the development of the ACTTION-APS Pain Taxonomy (AAPT). AAPT provides a multidimensional framework that is applied systematically in the development of diagnostic criteria. This article (1) describes the background and rationale for AAPT; (2) presents the AAPT taxonomy and the specific conditions for which diagnostic criteria have been developed (to be published separately); (3) briefly reviews the 5 dimensions that constitute the AAPT multidimensional framework and describes the 7 accompanying articles that discuss these dimensions and other important issues involving AAPT; and (4) provides an overview of next steps, specifically, the general processes by which the initial set of diagnostic criteria (for which the evidence base has been drawn from the literature, systematic reviews, and secondary analyses of existing databases) will undergo additional assessments of reliability and validity. To address the need for evidence-based diagnostic criteria for the major chronic pain conditions, the AAPT provides a multidimensional framework that is applied systematically in the development of diagnostic criteria. The long-term objective of AAPT is to advance the scientific understanding of chronic pain and its treatment. Copyright © 2016 American Pain Society. Published by Elsevier Inc. All rights reserved.

Perceived experiences of atheist discrimination: Instrument development and evaluation.

PubMed

Brewster, Melanie E; Hammer, Joseph; Sawyer, Jacob S; Eklund, Austin; Palamar, Joseph

2016-10-01

The present 2 studies describe the development and initial psychometric evaluation of a new instrument, the Measure of Atheist Discrimination Experiences (MADE), which may be used to examine the minority stress experiences of atheist people. Items were created from prior literature, revised by a panel of expert researchers, and assessed psychometrically. In Study 1 (N = 1,341 atheist-identified people), an exploratory factor analysis with 665 participants suggested the presence of 5 related dimensions of perceived discrimination. However, bifactor modeling via confirmatory factor analysis and model-based reliability estimates with data from the remaining 676 participants affirmed the presence of a strong "general" factor of discrimination and mixed to poor support for substantive subdimensions. In Study 2 (N = 1,057 atheist-identified people), another confirmatory factor analysis and model-based reliability estimates strongly supported the bifactor model from Study 1 (i.e., 1 strong "general" discrimination factor) and poor support for subdimensions. Across both studies, the MADE general factor score demonstrated evidence of good reliability (i.e., Cronbach's alphas of .94 and .95; omega hierarchical coefficients of .90 and .92), convergent validity (i.e., with stigma consciousness, β = .56; with awareness of public devaluation, β = .37), and preliminary evidence for concurrent validity (i.e., with loneliness β = .18; with psychological distress β = .27). Reliability and validity evidence for the MADE subscale scores was not sufficient to warrant future use of the subscales. Limitations and implications for future research and clinical work with atheist individuals are discussed. (PsycINFO Database Record (c) 2016 APA, all rights reserved).

Memory for murder. A psychological perspective on dissociative amnesia in legal contexts.

PubMed

Porter, S; Birt, A R; Yuille, J C; Hervé, H F

2001-01-01

There is currently a complex and inconsistent state in the law relating to dissociation and dissociative amnesia (McSherry, 1998). Although dissociative amnesia in defendants is relevant to both competency to stand trial and criminal responsibility in principle, courts have typically assumed a skeptical stance toward such claims in practice. However, there is considerable evidence from both nonoffender and offender populations to support the validity of dissociative amnesia in defendants. Further, there is information available to aid in the evaluation of amnesia, such as the quality of the report itself and characteristics of the person reporting the amnesia (e.g., psychopathy). When consideration is given to the legal response to reports of dissociative amnesia by complainants, the situation becomes even more complex. While some courts have rejected recovered memory evidence, others have convicted defendants of historical offenses based on such evidence. In some cases, judges have argued that jurors should be left to decide on the validity of recovered memories based on their common sense and experience. The uncritical acceptance of the validity of repressed memories in complainants by many courts stands in stark contrast to the response to claims of amnesia from defendants. It seems apparent that the courts need better guidelines around the issue of dissociative amnesia in both populations. We think that the increasing scientific understanding of memory in the past decade (see Schacter, 1999) can meaningfully contribute to the development of such guidelines. Responsible, nonpartisan expert testimony from mental health professionals would be one step in the direction of rectifying the current state of law in regards to dissociation.

Systematic review and proposal of a field-based physical fitness-test battery in preschool children: the PREFIT battery.

PubMed

Ortega, Francisco B; Cadenas-Sánchez, Cristina; Sánchez-Delgado, Guillermo; Mora-González, José; Martínez-Téllez, Borja; Artero, Enrique G; Castro-Piñero, Jose; Labayen, Idoia; Chillón, Palma; Löf, Marie; Ruiz, Jonatan R

2015-04-01

Physical fitness is a powerful health marker in childhood and adolescence, and it is reasonable to think that it might be just as important in younger children, i.e. preschoolers. At the moment, researchers, clinicians and sport practitioners do not have enough information about which fitness tests are more reliable, valid and informative from the health point of view to be implemented in preschool children. Our aim was to systematically review the studies conducted in preschool children using field-based fitness tests, and examine their (1) reliability, (2) validity, and (3) relationship with health outcomes. Our ultimate goal was to propose a field-based physical fitness-test battery to be used in preschool children. PubMed and Web of Science. Studies conducted in healthy preschool children that included field-based fitness tests. When using PubMed, we included Medical Subject Heading (MeSH) terms to enhance the power of the search. A set of fitness-related terms were combined with 'child, preschool' [MeSH]. The same strategy and terms were used for Web of Science (except for the MeSH option). Since no previous reviews with a similar aim were identified, we searched for all articles published up to 1 April 2014 (no starting date). A total of 2,109 articles were identified, of which 22 articles were finally selected for this review. Most studies focused on reliability of the fitness tests (n = 21, 96%), while very few focused on validity (0 criterion-related validity and 4 (18%) convergent validity) or relationship with health outcomes (0 longitudinal and 1 (5%) cross-sectional study). Motor fitness, particularly balance, was the most studied fitness component, while cardiorespiratory fitness was the least studied. After analyzing the information retrieved in the current systematic review about fitness testing in preschool children, we propose the PREFIT battery, field-based FITness testing in PREschool children. The PREFIT battery is composed of the following tests: the 20 m shuttle-run test for assessing cardiorespiratory fitness, the handgrip-strength and the standing long-jump tests for assessing musculoskeletal fitness, and the 4 × 10 m shuttle run and the one-leg-stance tests for assessing motor fitness, i.e. speed/agility and balance, respectively. The rationale for the selection of each of the tests included in the PREFIT battery is provided in this review, as well as directions for future research. Levels of evidence based on quality assessment of selected studies could not be constructed due to the limited number of studies identified for each test. The present systematic review has identified a need for further research on the validity of fitness tests in preschool children, as well as on their relationship with health. Due to this limited information, the PREFIT battery hereby proposed is based on the output of the current systematic review in preschool children, together with existing evidence in older children and adolescents. While we wait for more evidence to be accumulated in preschool children, the PREFIT battery hereby proposed is a useful tool for assessing physical fitness in children aged 3-5 years.

Reliability and validity of the revised Gibson Test of Cognitive Skills, a computer-based test battery for assessing cognition across the lifespan.

PubMed

Moore, Amy Lawson; Miller, Terissa M

2018-01-01

The purpose of the current study is to evaluate the validity and reliability of the revised Gibson Test of Cognitive Skills, a computer-based battery of tests measuring short-term memory, long-term memory, processing speed, logic and reasoning, visual processing, as well as auditory processing and word attack skills. This study included 2,737 participants aged 5-85 years. A series of studies was conducted to examine the validity and reliability using the test performance of the entire norming group and several subgroups. The evaluation of the technical properties of the test battery included content validation by subject matter experts, item analysis and coefficient alpha, test-retest reliability, split-half reliability, and analysis of concurrent validity with the Woodcock Johnson III Tests of Cognitive Abilities and Tests of Achievement. Results indicated strong sources of evidence of validity and reliability for the test, including internal consistency reliability coefficients ranging from 0.87 to 0.98, test-retest reliability coefficients ranging from 0.69 to 0.91, split-half reliability coefficients ranging from 0.87 to 0.91, and concurrent validity coefficients ranging from 0.53 to 0.93. The Gibson Test of Cognitive Skills-2 is a reliable and valid tool for assessing cognition in the general population across the lifespan.

Development of a Fall-Risk Self-Assessment for Community-Dwelling Seniors

PubMed Central

Vivrette, Rebecca L.; Rubenstein, Laurence Z.; Martin, Jennifer L.; Josephson, Karen R.; Kramer, B. Josea

2012-01-01

Objective To determine seniors’ beliefs about falls and design a fall-risk self-assessment and educational materials to promote early identification of evidence-based fall risks and encourage prevention behaviors. Methods Focus groups with community-dwelling seniors, conducted in two phases to identify perceptions about fall risks and risk reduction and to assess face validity of the fall-risk self-assessment and acceptability of educational materials. Results Lay perception of fall risks was in general concordance with evidence-based research. Maintaining independence and positive tone were perceived as key motivators for fall prevention. Seniors intended to use information in the educational tool to stimulate discussions about falls with health care providers. Implications An evidence-based, educational fall-risk self-assessment acceptable to older adults can build on existing lay knowledge about fall risks and perception that falls are a relevant problem and can educate seniors about their specific risks and how to minimize them. PMID:21285473

Measurement properties of quality-of-life measurement instruments for infants, children and adolescents with eczema: a systematic review.

PubMed

Heinl, D; Prinsen, C A C; Sach, T; Drucker, A M; Ofenloch, R; Flohr, C; Apfelbacher, C

2017-04-01

Quality of life (QoL) is one of the core outcome domains identified by the Harmonising Outcome Measures for Eczema (HOME) initiative to be assessed in every eczema trial. There is uncertainty about the most appropriate QoL instrument to measure this domain in infants, children and adolescents. To systematically evaluate the measurement properties of existing measurement instruments developed and/or validated for the measurement of QoL in infants, children and adolescents with eczema. A systematic literature search in PubMed and Embase, complemented by a thorough hand search of reference lists, retrieved studies on measurement properties of eczema QoL instruments for infants, children and adolescents. For all eligible studies, we judged the adequacy of the measurement properties and the methodological study quality with the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. Results from different studies were summarized in a best-evidence synthesis and formed the basis to assign four degrees of recommendation. Seventeen articles, three of which were found by hand search, were included. These 17 articles reported on 24 instruments. No instrument can be recommended for use in all eczema trials because none fulfilled all required adequacy criteria. With adequate internal consistency, reliability and hypothesis testing, the U.S. version of the Childhood Atopic Dermatitis Impact Scale (CADIS), a proxy-reported instrument, has the potential to be recommended depending on the results of further validation studies. All other instruments, including all self-reported ones, lacked significant validation data. Currently, no QoL instrument for infants, children and adolescents with eczema can be highly recommended. Future validation research should primarily focus on the CADIS, but also attempt to broaden the evidence base for the validity of self-reported instruments. © 2016 British Association of Dermatologists.

Reliability, validity and responsiveness of the Arabic version of the Disability of Arm, Shoulder and Hand (DASH-Arabic).

PubMed

Alotaibi, Naser M; Aljadi, Sameera H; Alrowayeh, Hesham N

2016-12-01

To investigate the psychometric properties (reliability, validity and responsiveness) of the DASH-Arabic in a cohort of Arabic patients presenting with various upper extremity conditions. Participants were 139 patients with various upper extremity conditions, who completed the DASH-Arabic at the baseline, 2-5 days later and 30-36 days later. Participants completed demographic data forms, the SF-36 and VAS at baseline, and a Global Rating of Change scale at first and second follow-ups. Cronbach's alpha of the DASH-Arabic was 0.94. Test-retest reliability was excellent with an ICC of 0.97. The SEM was 3.50 and the MDC95 was 9.28. Construct validity of the DASH-Arabic with the SF-36 subscales and VAS scores ranged from r -0.32 to -0.57, all statistically significant (p < 0.001). The effect size (ES) for the DASH-Arabic was 1.39 and its standard response mean was 1.51. The area under the curve was 0.82 (95% CI = 0.72-0.92, p < 0.001). The optimally efficient cutoff for an improvement was found to be a difference of 15 DASH points. The DASH-Arabic is a reliable, valid and responsive upper extremity outcome measure for patients whose primary language is Arabic; it can be used to document patient status and outcomes and support evidence-based practice. Implications for Rehabilitation The DASH-Arabic demonstrated sound psychometric properties of reliability, validity and responsiveness. It is an effective patient status and outcome tool that will support evidence-based practice. This tool is recommended for evaluating upper extremity work-related injuries and tracking therapeutic outcomes.

Cross-cultural adaptation of the Oral Health Impact Profile (OHIP) for the Malaysian adult population.

PubMed

Saub, R; Locker, D; Allison, P; Disman, M

2007-09-01

The aim of this project was to develop an oral health related-quality of life measure for the Malaysian adult population aged 18 and above by the cross-cultural adaption the Oral Health Impact Profile (OHIP). The adaptation of the OHIP was based on the framework proposed by Herdman et al (1998). The OHIP was translated into the Malay language using a forward-backward translation technique. Thirty-six patients were interviewed to assess the conceptual equivalence and relevancy of each item. Based on the translation process and interview results a Malaysian version of the OHIP questionnaire was produced that contained 45 items. It was designated as the OHIP(M). This questionnaire was pre-tested on 20 patients to assess its face validity. A short 14-item version of the questionnaire was completed by 171 patients to assess the suitability of the Likert-type response format. Field-testing was conducted in order to assess the suitability of two modes of administration (mail and interview) and to establish the psychometric properties of the adapted measure. The pre-testing revealed that the OHIP(M) has good face validity. It was found that the five-point frequency Likert scale could be used for the Malaysian population. The OHIP(M) was reliable, where the scale Cronbach's alpha was 0.95 and the ICC value for test-retest reliability was 0.79. Three out four construct validity hypotheses tested were confirmed. OHIP(M) works equally well as the English version. OHIP(M) was found to be reliable and valid regardless of the mode of administration. However, this study only provides initial evidence for the reliability and validity of the measure. Further study is recommended to collect more evidence to support these results.

Enhancing students' learning in problem based learning: validation of a self-assessment scale for active learning and critical thinking.

PubMed

Khoiriyah, Umatul; Roberts, Chris; Jorm, Christine; Van der Vleuten, C P M

2015-08-26

Problem based learning (PBL) is a powerful learning activity but fidelity to intended models may slip and student engagement wane, negatively impacting learning processes, and outcomes. One potential solution to solve this degradation is by encouraging self-assessment in the PBL tutorial. Self-assessment is a central component of the self-regulation of student learning behaviours. There are few measures to investigate self-assessment relevant to PBL processes. We developed a Self-assessment Scale on Active Learning and Critical Thinking (SSACT) to address this gap. We wished to demonstrated evidence of its validity in the context of PBL by exploring its internal structure. We used a mixed methods approach to scale development. We developed scale items from a qualitative investigation, literature review, and consideration of previous existing tools used for study of the PBL process. Expert review panels evaluated its content; a process of validation subsequently reduced the pool of items. We used structural equation modelling to undertake a confirmatory factor analysis (CFA) of the SSACT and coefficient alpha. The 14 item SSACT consisted of two domains "active learning" and "critical thinking." The factorial validity of SSACT was evidenced by all items loading significantly on their expected factors, a good model fit for the data, and good stability across two independent samples. Each subscale had good internal reliability (>0.8) and strongly correlated with each other. The SSACT has sufficient evidence of its validity to support its use in the PBL process to encourage students to self-assess. The implementation of the SSACT may assist students to improve the quality of their learning in achieving PBL goals such as critical thinking and self-directed learning.

Understanding the State of the Art for Measurement in Chemistry Education Research: Examining the Psychometric Evidence

ERIC Educational Resources Information Center

Arjoon, Janelle A.; Xu, Xiaoying; Lewis, Jennifer E.

2013-01-01

education community are relatively new. Because psychometric evidence dictates the validity of interpretations made from test scores, gathering and reporting validity and reliability evidence is of utmost importance. Therefore, the purpose of this study was to investigate what…

Scaling the Information Processing Demands of Occupations

ERIC Educational Resources Information Center

Haase, Richard F.; Jome, LaRae M.; Ferreira, Joaquim Armando; Santos, Eduardo J. R.; Connacher, Christopher C.; Sendrowitz, Kerrin

2011-01-01

The purpose of this study was to provide additional validity evidence for a model of person-environment fit based on polychronicity, stimulus load, and information processing capacities. In this line of research the confluence of polychronicity and information processing (e.g., the ability of individuals to process stimuli from the environment…

Item-Based Psychometrics of the Preschool Behavioral and Emotional Rating Scale

ERIC Educational Resources Information Center

Cress, Cynthia J.; Lambert, Matthew C.; Epstein, Michael H.

2014-01-01

The Preschool Behavioral and Emotional Rating Scale (PreBERS) is an assessment of emotional and behavioral strengths in preschoolers with well-established reliability and validity for educational and clinical application in children with and without disabilities. The present study provides further evidence of psychometric rigor for items and…

Phonological and Non-Phonological Language Skills as Predictors of Early Reading Performance

ERIC Educational Resources Information Center

Batson-Magnuson, LuAnn

2010-01-01

Accurate prediction of early childhood reading performance could help identify at-risk students, aid in the development of evidence-based intervention strategies, and further our theoretical understanding of reading development. This study assessed the validity of the Developmental Indicator for the Assessment of Learning (DIAL) language-based…

Assessing Young Adolescents' Personality with the Five-Factor Personality Inventory

ERIC Educational Resources Information Center

Hendriks, A. A. Jolijn; Kuyper, Hans; Offringa, G. Johan; Van der Werf, Margaretha P. C.

2008-01-01

The Five-Factor Personality Inventory (FFPI) assesses a person's position on the (Dutch) psycholexically based Big Five factors: Extraversion, Agreeableness, Conscientiousness, Emotional Stability, and Autonomy. FFPI factor scores are reliable and valid if ratings are made by adults. The present study yields preliminary evidence of whether young…

Examining Changes in Certification/Licensure Requirements and the International Medical Graduate Examinee Pool

ERIC Educational Resources Information Center

McKinley, Danette W.; Hess, Brian J.; Boulet, John R.; Lipner, Rebecca S.

2014-01-01

Changes in certification requirements and examinee characteristics are likely to influence the validity of the evidence associated with interpretations made based on test data. We examined whether changes in Educational Commission for Foreign Medical Graduates (ECFMG) certification requirements over time were associated with changes in internal…

[Psychosomatics in rheumatology].

PubMed

Eich, W; Blumenstiel, K; Lensche, H; Fiehn, C; Bieber, C

2004-04-01

Psychosocial factors influence the course and the outcome of chronic somatic diseases. This is also valid for rheumatic diseases like rheumatoid arthritis, spondyloarthropathies, systemic collagen vascular diseases, and fibromyalgia syndrome. The article summarises the evidence-based findings and it illustrates possibilities of psychosomatic treatment in rheumatic diseases by means of three case reports.

Teachers Engaging Parents as Tutors to Improve Oral Reading Fluency

ERIC Educational Resources Information Center

Kupzyk, Sara S.

2012-01-01

This dissertation examined the application of evidence-based tutoring for oral reading fluency (ORF) to a natural setting, using teachers as parent trainers. Measures used to determine the impact of parent tutoring included treatment integrity, student reading outcomes, attitudes towards involvement and reading, and social validity. Six teachers…

A Dynamic Theory of Reading.

ERIC Educational Resources Information Center

Wark, David M.

The initial means for arriving at a dynamic model of reading were suggested in the form of "behaviormetric" research. A review of valid reading models noted those of Smith and Carrigan, Delacato, and Holmes as eminent, and it distinguished between models based on concrete evidence and metaphors of the reading process which are basically…

Executive summary: biomarkers of nutrition for development: building a consensus

USDA-ARS?s Scientific Manuscript database

The ability to develop evidence-based clinical guidance and effective programs and policies to achieve global health promotion and disease prevention goals depends on the availability of valid and reliable data. With specific regard to the role of food and nutrition in achieving those goals, relevan...

Physics in the Courtroom

NASA Astrophysics Data System (ADS)

Vosk, Ted

2011-10-01

The principles, methods and technologies of physics can provide a powerful tool for the discovery of truth in the criminal justice system. Accordingly, physics based forensic evidence is relied upon in criminal prosecutions around the country every day. Infrared spectroscopy for the determination of the alcohol concentration of an individual's breath, force, momentum and multi-body dynamics for purposes of accident reconstruction and the basic application of sound metrological (measurement) practices constitute but a few examples. In many cases, a jury's determination of guilt or innocence, upon which the liberty of a Citizen rests, may in fact be determined by such evidence. Society may well place a high degree of confidence in the integrity of verdicts so obtained when ``the physics'' has been applied in a valid manner. Unfortunately, as concluded by the National Academy of Sciences, ``The law's greatest dilemma in its heavy reliance on forensic evidence--concerns the question of whether---and to what extent-- -there is science in any given `forensic science' discipline.'' Even where valid physical principles are relied upon, their improper application by forensic practitioners who have little physics training, background and/or understanding calls into question the validity of results or conclusions obtained. This presentation provides examples of the application of physics in the courtroom, where problems have been discovered and how they can be addressed by the physics community.

[Evidence-based medicine and 'The Cochrane Collaboration'].

PubMed

Kawamura, T; Tamakoshi, A; Wakai, K; Ohno, Y

1999-06-01

In Evidence-Based Medicine (EBM), a clinical decision is based neither on pathophysiological theories nor personal experience but on the results derived from scientifically designed clinical epidemiological studies (i.e., evidence). EBM is used in various clinical applications, such as therapy, diagnosis, and prognosis prediction. The process includes (1) asking a clinical question consisting of the three elements of "patient", "exposure", and "outcome"; (2) searching for the best evidence using MEDLINE or Cochrane Library; (3) appraising critically the validity of the method and the magnitude and probability of the result; and finally (4) applying the evidence of the patient. In actual clinical practice, clinical expertise and patient preferences should be as much regarded as research evidence. 'The Cochrane Collaboration' supplies systematic reviews of clinical trials carried out all over the world to its consumers. Its fruit, 'The Cochrane Library (CD-ROM),' is a highly valuable resource. 'The Cochrane Collaboration' serves as the infrastructure for EBM. EBM, which was originally developed for the individual patient care, can also be applicable to community- or workplace-healthcare and policy making by governments. Thus, EBM is both a philosophy and a method to provide people with the most appropriate medical practice.

Screening, diagnosis, and treatment of post-traumatic stress disorder.

PubMed

Wisco, Blair E; Marx, Brian P; Keane, Terence M

2012-08-01

Post-traumatic stress disorder (PTSD) is a prevalent problem among military personnel and veterans. Identification of effective screening tools, diagnostic technologies, and treatments for PTSD is essential to ensure that all individuals in need of treatment are offered interventions with proven efficacy. Well-validated methods for screening and diagnosing PTSD are now available, and effective pharmacological and psychological treatments can be offered. Despite these advances, many military personnel and veterans do not receive evidence-based care. We review the literature on screening, diagnosis, and treatment of PTSD in military populations, and discuss the challenges to implementing the best evidence-based practices in clinical settings.

Evaluating the Predictive Validity of the Computerized Comprehension Task: Comprehension Predicts Production

PubMed Central

Friend, Margaret; Schmitt, Sara A.; Simpson, Adrianne M.

2017-01-01

Until recently, the challenges inherent in measuring comprehension have impeded our ability to predict the course of language acquisition. The present research reports on a longitudinal assessment of the convergent and predictive validity of the CDI: Words and Gestures and the Computerized Comprehension Task (CCT). The CDI: WG and the CCT evinced good convergent validity however the CCT better predicted subsequent parent reports of language production. Language sample data in the third year confirm this finding: the CCT accounted for 24% of the variance in unique word use. These studies provide evidence for the utility of a behavior-based approach to predicting the course of language acquisition into production. PMID:21928878

Development of the CarMen-Q Questionnaire for mental workload assessment.

PubMed

Rubio-Valdehita, Susana; López-Núñez, María I; López-Higes, Ramón; Díaz-Ramiro, Eva M

2017-11-01

Mental workload has emerged as one of the most important occupational risk factors present in most psychological and physical diseases caused by work. In view of the lack of specific tools to assess mental workload, the objective of this research was to assess the construct validity and reliability of a new questionnaire for mental workload assessment (CarMen-Q). The sample was composed of 884 workers from several professional sectors, between 18 and 65 years old, 53.4% men and 46.6% women. To evaluate the validity based on relationships with other measures, the NASA-TLX scale was also administered. Confirmatory factor analysis showed an internal structure made up of four dimensions: cognitive, temporal and emotional demands and performance requirement. The results show satisfactory evidence of validity based on relationships with NASA-TLX and good reliability. The questionnaire has good psychometric properties and can be an easy, brief, useful tool for mental workload diagnosis and prevention.

The Development and Validation of an Instrument to Measure Preservice Teachers' Self-Efficacy in Regard to The Teaching of Science as Inquiry

NASA Astrophysics Data System (ADS)

Dira Smolleck, Lori; Zembal-Saul, Carla; Yoder, Edgar P.

2006-06-01

The purpose of this study was to develop, validate, and establish the reliability of an instrument that measures preservice teachers' self-efficacy in regard to the teaching of science as inquiry. The instrument, Teaching Science as Inquiry (TSI), is based upon the work of Bandura (1977, 1981, 1982, 1986, 1989, 1995, 1997), Riggs (1988), and Enochs and Riggs (1990). Self-efficacy in regard to the teaching of science as inquiry was measured through the use of a 69-item Likert-type scale instrument designed by the author of the study. Based on the standardized development processes used and the associated evidence, the TSI appears to be a content and construct valid instrument with high internal reliability for use with preservice elementary teachers to assess self-efficacy beliefs in regard to the teaching of science as inquiry.

The Development and Validation of an Instrument to Measure Preservice Teachers' Self-Efficacy in Regard to The Teaching of Science as Inquiry

NASA Astrophysics Data System (ADS)

Smolleck, Lori Dira; Zembal-Saul, Carla; Yoder, Edgar P.

2006-06-01

The purpose of this study was to develop, validate, and establish the reliability of an instrument that measures preservice teachers' self-efficacy in regard to the teaching of science as inquiry. The instrument, Teaching Science as Inquiry (TSI), is based upon the work of Bandura (1977, 1981, 1982, 1986, 1989, 1995, 1997), Riggs (1988), and Enochs and Riggs (1990). Self-efficacy in regard to the teaching of science as inquiry was measured through the use of a 69-item Likert-type scale instrument designed by the author of the study. Based on the standardized development processes used and the associated evidence, the TSI appears to be a content and construct valid instrument with high internal reliability for use with preservice elementary teachers to assess self-efficacy beliefs in regard to the teaching of science as inquiry.

Factor structure of a standards-based inventory of competencies in social work with groups.

PubMed

Macgowan, Mark J; Dillon, Frank R; Spadola, Christine E

2018-01-01

This study extends previous findings on a measure of competencies based on Standards for Social Work Practice with Groups. The Inventory of Competencies in Social Work with Groups (ICSWG) measures confidence in performing the Standards. This study examines the latent structure of the Inventory, while illuminating the underlying structure of the Standards. A multinational sample of 586 persons completed the ICSWG. Exploratory factor analysis (EFA), reliability estimates, standard error of measurement estimates, and a range of validity tests were conducted. The EFA yielded a six-factor solution consisting of core values, mutuality/connectivity, collaboration, and three phases of group development (planning, beginnings/middles, endings). The alphas were .98 for the scale and ranged from .85 to .95 for the subscales. Correlations between the subscales and validators supported evidence of construct validity. The findings suggest key group work domains that should be taught and practiced in social work with groups.

Measurement issues in research on social support and health.

PubMed Central

Dean, K; Holst, E; Kreiner, S; Schoenborn, C; Wilson, R

1994-01-01

STUDY OBJECTIVE--The aims were: (1) to identify methodological problems that may explain the inconsistencies and contradictions in the research evidence on social support and health, and (2) to validate a frequently used measure of social support in order to determine whether or not it could be used in multivariate analyses of population data in research on social support and health. DESIGN AND METHODS--Secondary analysis of data collected in a cross sectional survey of a multistage cluster sample of the population of the United States, designed to study relationships in behavioural, social support and health variables. Statistical models based on item response theory and graph theory were used to validate the measure of social support to be used in subsequent analyses. PARTICIPANTS--Data on 1755 men and women aged 20 to 64 years were available for the scale validation. RESULTS--Massive evidence of item bias was found for all items of a group membership subscale. The most serious problems were found in relationship to an item measuring membership in work related groups. Using that item in the social network scale in multivariate analyses would distort findings on the statistical effects of education, employment status, and household income. Evidence of item bias was also found for a sociability subscale. When marital status was included to create what is called an intimate contacts subscale, the confounding grew worse. CONCLUSIONS--The composite measure of social network is not valid and would seriously distort the findings of analyses attempting to study relationships between the index and other variables. The findings show that valid measurement is a methodological issue that must be addressed in scientific research on population health. PMID:8189179

A Virtual Reality Training Curriculum for Laparoscopic Colorectal Surgery.

PubMed

Beyer-Berjot, Laura; Berdah, Stéphane; Hashimoto, Daniel A; Darzi, Ara; Aggarwal, Rajesh

Training within a competency-based curriculum (CBC) outside the operating room enhances performance during real basic surgical procedures. This study aimed to design and validate a virtual reality CBC for an advanced laparoscopic procedure: sigmoid colectomy. This was a multicenter randomized study. Novice (surgeons who had performed <5 laparoscopic colorectal resections as primary operator), intermediate (between 10 and 20), and experienced surgeons (>50) were enrolled. Validity evidence for the metrics given by the virtual reality simulator, the LAP Mentor, was based on the second attempt of each task in between groups. The tasks assessed were 3 modules of a laparoscopic sigmoid colectomy (medial dissection [MD], lateral dissection [LD], and anastomosis) and a full procedure (FP). Novice surgeons were randomized to 1 of 2 groups to perform 8 further attempts of all 3 modules or FP, for learning curve analysis. Two academic tertiary care centers-division of surgery of St. Mary's campus, Imperial College Healthcare NHS Trust, London and Nord Hospital, Assistance Publique-Hôpitaux de Marseille, Aix-Marseille Université, Marseille, were involved. Novice surgeons were residents in digestive surgery at St. Mary's and Nord Hospitals. Intermediate and experienced surgeons were board-certified academic surgeons. A total of 20 novice surgeons, 7 intermediate surgeons, and 6 experienced surgeons were enrolled. Evidence for validity based on experience was identified in MD, LD, and FP for time (p = 0.005, p = 0.003, and p = 0.001, respectively), number of movements (p = 0.013, p = 0.005, and p = 0.001, respectively), and path length (p = 0.03, p = 0.017, and p = 0.001, respectively), and only for time (p = 0.03) and path length (p = 0.013) in the anastomosis module. Novice surgeons' performance significantly improved through repetition for time, movements, and path length in MD, LD, and FP. Experienced surgeons' benchmark criteria were defined for all construct metrics showing validity evidence. A CBC in laparoscopic colorectal surgery has been designed. Such training may reduce the learning curve during real colorectal resections in the operating room. Copyright © 2016 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Healthcare professionals' behavior, skills, knowledge and attitudes on evidence-based health practice: a protocol of cross-sectional study.

PubMed

Mariano, Arielly Souza; Souza, Nathan Mendes; Cavaco, Afonso; Lopes, Luciane Cruz

2018-06-04

In Brazil, as in most countries nowadays, there is a pursuit for healthcare quality improvement and sustainability in public and private systems. Healthcare professionals' perceptions, knowledge and attitudes determine evidence-based practice (EBP), which remain uncertain among Brazilian practitioners. A standardised national instrument whose wide use will identify gaps and flaws in establishing an EBP could contribute to an effective resources allocation from health professionals willing to use an EBP. To present a study protocol on the development and validation of an instrument to measure Brazilian healthcare professionals' behaviour, skills, self-efficacy, knowledge and attitudes towards EBP. This is a validation study with Brazilian healthcare professionals to develop a valid and reliable questionnaire, including selection of domains and formulation of questions. Construct and content validity will be assess by a panel of experts, with data collection and analysis following a Delphi-like methodology. Further, a pilot survey will be accomplished with a representative sample of different healthcare professionals from all main Brazilian regions. An exploratory factor analysis and a confirmatory factor analysis will be conducted afterwards. The ratio of χ 2 and df (χ 2 /df), comparative fit index, goodness of fit index and root mean square error of approximation will be used for assessing the model fit. In addition, the reliability of the instrument will be estimated by test-retest reproducibility and Cronbach's alpha coefficient (α). This study has received ethical approval from the Pharmaceutical Sciences Faculty of the São Paulo State University (1.425.808). The use among a wide national sample is expected to promote an extensive view of evidence-based decision-making, identifying the knowledge gaps in this area. Study findings will be circulated to healthcare professionals and scientists in the field through the publication in peer-reviewed journals and conference presentations. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

Evaluating Existing and New Validity Evidence for the Academic Motivation Scale

ERIC Educational Resources Information Center

Fairchild, Amanda J.; Horst, S. Jeanne; Finney, Sara J.; Barron, Kenneth E.

2005-01-01

The current study evaluates existing and new validity evidence for the Academic Motivation Scale (AMS; Vallerand et al., 1992). We first provide a narrative review synthesizing past research, and then conduct a validity investigation of the scores from the measure. Data analysis using a sample of 1406 American college students provided construct…

Validity of Childhood Career Development Scale Scores in South Africa

ERIC Educational Resources Information Center

Stead, Graham B.; Schultheiss, Donna E. Palladino

2010-01-01

The purpose of this study was to provide evidence of the construct and concurrent validity of the Childhood Career Development Scale's (CCDS) scores among South African primary school children. Using a sample of 808 children in grades four through seven, evidence for the CCDS's construct validity was provided using confirmatory factor analysis,…

Functional performance testing of the hip in athletes: a systematic review for reliability and validity.

PubMed

Kivlan, Benjamin R; Martin, Robroy L

2012-08-01

The purpose of this study was to systematically review the literature for functional performance tests with evidence of reliability and validity that could be used for a young, athletic population with hip dysfunction. A search of PubMed and SPORTDiscus databases were performed to identify movement, balance, hop/jump, or agility functional performance tests from the current peer-reviewed literature used to assess function of the hip in young, athletic subjects. The single-leg stance, deep squat, single-leg squat, and star excursion balance tests (SEBT) demonstrated evidence of validity and normative data for score interpretation. The single-leg stance test and SEBT have evidence of validity with association to hip abductor function. The deep squat test demonstrated evidence as a functional performance test for evaluating femoroacetabular impingement. Hop/Jump tests and agility tests have no reported evidence of reliability or validity in a population of subjects with hip pathology. Use of functional performance tests in the assessment of hip dysfunction has not been well established in the current literature. Diminished squat depth and provocation of pain during the single-leg balance test have been associated with patients diagnosed with FAI and gluteal tendinopathy, respectively. The SEBT and single-leg squat tests provided evidence of convergent validity through an analysis of kinematics and muscle function in normal subjects. Reliability of functional performance tests have not been established on patients with hip dysfunction. Further study is needed to establish reliability and validity of functional performance tests that can be used in a young, athletic population with hip dysfunction. 2b (Systematic Review of Literature).

An Experimental Study of the Internal Consistency of Judgments Made in Bookmark Standard Setting

ERIC Educational Resources Information Center

Clauser, Brian E.; Baldwin, Peter; Margolis, Melissa J.; Mee, Janet; Winward, Marcia

2017-01-01

Validating performance standards is challenging and complex. Because of the difficulties associated with collecting evidence related to external criteria, validity arguments rely heavily on evidence related to internal criteria--especially evidence that expert judgments are internally consistent. Given its importance, it is somewhat surprising…

Social anxiety disorder: questions and answers for the DSM-V.

PubMed

Bögels, Susan M; Alden, Lynn; Beidel, Deborah C; Clark, Lee Anna; Pine, Daniel S; Stein, Murray B; Voncken, Marisol

2010-02-01

This review evaluates the DSM-IV criteria of social anxiety disorder (SAD), with a focus on the generalized specifier and alternative specifiers, the considerable overlap between the DSM-IV diagnostic criteria for SAD and avoidant personality disorder, and developmental issues. A literature review was conducted, using the validators provided by the DSM-V Spectrum Study Group. This review presents a number of options and preliminary recommendations to be considered for DSM-V. Little supporting evidence was found for the current specifier, generalized SAD. Rather, the symptoms of individuals with SAD appear to fall along a continuum of severity based on the number of fears. Available evidence suggested the utility of a specifier indicating a "predominantly performance" variety of SAD. A specifier based on "fear of showing anxiety symptoms" (e.g., blushing) was considered. However, a tendency to show anxiety symptoms is a core fear in SAD, similar to acting or appearing in a certain way. More research is needed before considering subtyping SAD based on core fears. SAD was found to be a valid diagnosis in children and adolescents. Selective mutism could be considered in part as a young child's avoidance response to social fears. Pervasive test anxiety may belong not only to SAD, but also to generalized anxiety disorder. The data are equivocal regarding whether to consider avoidant personality disorder simply a severe form of SAD. Secondary data analyses, field trials, and validity tests are needed to investigate the recommendations and options.

The reliability and validity of a portfolio designed as a programmatic assessment of performance in an integrated clinical placement.

PubMed

Roberts, Chris; Shadbolt, Narelle; Clark, Tyler; Simpson, Phillip

2014-09-20

Little is known about the technical adequacy of portfolios in reporting multiple complex academic and performance-based assessments. We explored, first, the influencing factors on the precision of scoring within a programmatic assessment of student learning outcomes within an integrated clinical placement. Second, the degree to which validity evidence supported interpretation of student scores. Within generalisability theory, we estimated the contribution that each wanted factor (i.e. student capability) and unwanted factors (e.g. the impact of assessors) made to the variation in portfolio task scores. Relative and absolute standard errors of measurement provided a confidence interval around a pre-determined pass/fail standard for all six tasks. Validity evidence was sought through demonstrating the internal consistency of the portfolio and exploring the relationship of student scores with clinical experience. The mean portfolio mark for 257 students, across 372 raters, based on six tasks, was 75.56 (SD, 6.68). For a single student on one assessment task, 11% of the variance in scores was due to true differences in student capability. The most significant interaction was context specificity (49%), the tendency for one student to engage with one task and not engage with another task. Rater subjectivity was 29%. An absolute standard error of measurement of 4.74%, gave a 95% CI of +/- 9.30%, and a 68% CI of +/- 4.74% around a pass/fail score of 57%. Construct validity was supported by demonstration of an assessment framework, the internal consistency of the portfolio tasks, and higher scores for students who did the clinical placement later in the academic year. A portfolio designed as a programmatic assessment of an integrated clinical placement has sufficient evidence of validity to support a specific interpretation of student scores around passing a clinical placement. It has modest precision in assessing students' achievement of a competency standard. There were identifiable areas for reducing measurement error and providing more certainty around decision-making. Reducing the measurement error would require engaging with the student body on the value of the tasks, more focussed academic and clinical supervisor training, and revisiting the rubric of the assessment in the light of feedback.

Mapping cognitive overlaps between practice-based learning and improvement and evidence-based medicine: an operational definition for assessing resident physician competence.

PubMed

Chatterji, Madhabi; Graham, Mark J; Wyer, Peter C

2009-12-01

The complex competency labeled practice-based learning and improvement (PBLI) by the Accreditation Council for Graduate Medical Education (ACGME) incorporates core knowledge in evidence-based medicine (EBM). The purpose of this study was to operationally define a "PBLI-EBM" domain for assessing resident physician competence. The authors used an iterative design process to first content analyze and map correspondences between ACGME and EBM literature sources. The project team, including content and measurement experts and residents/fellows, parsed, classified, and hierarchically organized embedded learning outcomes using a literature-supported cognitive taxonomy. A pool of 141 items was produced from the domain and assessment specifications. The PBLI-EBM domain and resulting items were content validated through formal reviews by a national panel of experts. The final domain represents overlapping PBLI and EBM cognitive dimensions measurable through written, multiple-choice assessments. It is organized as 4 subdomains of clinical action: Therapy, Prognosis, Diagnosis, and Harm. Four broad cognitive skill branches (Ask, Acquire, Appraise, and Apply) are subsumed under each subdomain. Each skill branch is defined by enabling skills that specify the cognitive processes, content, and conditions pertinent to demonstrable competence. Most items passed content validity screening criteria and were prepared for test form assembly and administration. The operational definition of PBLI-EBM competence is based on a rigorously developed and validated domain and item pool, and substantially expands conventional understandings of EBM. The domain, assessment specifications, and procedures outlined may be used to design written assessments to tap important cognitive dimensions of the overall PBLI competency, as given by ACGME. For more comprehensive coverage of the PBLI competency, such instruments need to be complemented with performance assessments.

Mapping Cognitive Overlaps Between Practice-Based Learning and Improvement and Evidence-Based Medicine: An Operational Definition for Assessing Resident Physician Competence

PubMed Central

Chatterji, Madhabi; Graham, Mark J.; Wyer, Peter C.

2009-01-01

Purpose The complex competency labeled practice-based learning and improvement (PBLI) by the Accreditation Council for Graduate Medical Education (ACGME) incorporates core knowledge in evidence-based medicine (EBM). The purpose of this study was to operationally define a “PBLI-EBM” domain for assessing resident physician competence. Method The authors used an iterative design process to first content analyze and map correspondences between ACGME and EBM literature sources. The project team, including content and measurement experts and residents/fellows, parsed, classified, and hierarchically organized embedded learning outcomes using a literature-supported cognitive taxonomy. A pool of 141 items was produced from the domain and assessment specifications. The PBLI-EBM domain and resulting items were content validated through formal reviews by a national panel of experts. Results The final domain represents overlapping PBLI and EBM cognitive dimensions measurable through written, multiple-choice assessments. It is organized as 4 subdomains of clinical action: Therapy, Prognosis, Diagnosis, and Harm. Four broad cognitive skill branches (Ask, Acquire, Appraise, and Apply) are subsumed under each subdomain. Each skill branch is defined by enabling skills that specify the cognitive processes, content, and conditions pertinent to demonstrable competence. Most items passed content validity screening criteria and were prepared for test form assembly and administration. Conclusions The operational definition of PBLI-EBM competence is based on a rigorously developed and validated domain and item pool, and substantially expands conventional understandings of EBM. The domain, assessment specifications, and procedures outlined may be used to design written assessments to tap important cognitive dimensions of the overall PBLI competency, as given by ACGME. For more comprehensive coverage of the PBLI competency, such instruments need to be complemented with performance assessments. PMID:21975994

Instrumental and statistical methods for the comparison of class evidence

NASA Astrophysics Data System (ADS)

Liszewski, Elisa Anne

Trace evidence is a major field within forensic science. Association of trace evidence samples can be problematic due to sample heterogeneity and a lack of quantitative criteria for comparing spectra or chromatograms. The aim of this study is to evaluate different types of instrumentation for their ability to discriminate among samples of various types of trace evidence. Chemometric analysis, including techniques such as Agglomerative Hierarchical Clustering, Principal Components Analysis, and Discriminant Analysis, was employed to evaluate instrumental data. First, automotive clear coats were analyzed by using microspectrophotometry to collect UV absorption data. In total, 71 samples were analyzed with classification accuracy of 91.61%. An external validation was performed, resulting in a prediction accuracy of 81.11%. Next, fiber dyes were analyzed using UV-Visible microspectrophotometry. While several physical characteristics of cotton fiber can be identified and compared, fiber color is considered to be an excellent source of variation, and thus was examined in this study. Twelve dyes were employed, some being visually indistinguishable. Several different analyses and comparisons were done, including an inter-laboratory comparison and external validations. Lastly, common plastic samples and other polymers were analyzed using pyrolysis-gas chromatography/mass spectrometry, and their pyrolysis products were then analyzed using multivariate statistics. The classification accuracy varied dependent upon the number of classes chosen, but the plastics were grouped based on composition. The polymers were used as an external validation and misclassifications occurred with chlorinated samples all being placed into the category containing PVC.

Implementing the Science Assessment Standards: Developing and validating a set of laboratory assessment tasks in high school biology

NASA Astrophysics Data System (ADS)

Saha, Gouranga Chandra

Very often a number of factors, especially time, space and money, deter many science educators from using inquiry-based, hands-on, laboratory practical tasks as alternative assessment instruments in science. A shortage of valid inquiry-based laboratory tasks for high school biology has been cited. Driven by this need, this study addressed the following three research questions: (1) How can laboratory-based performance tasks be designed and developed that are doable by students for whom they are designed/written? (2) Do student responses to the laboratory-based performance tasks validly represent at least some of the intended process skills that new biology learning goals want students to acquire? (3) Are the laboratory-based performance tasks psychometrically consistent as individual tasks and as a set? To answer these questions, three tasks were used from the six biology tasks initially designed and developed by an iterative process of trial testing. Analyses of data from 224 students showed that performance-based laboratory tasks that are doable by all students require careful and iterative process of development. Although the students demonstrated more skill in performing than planning and reasoning, their performances at the item level were very poor for some items. Possible reasons for the poor performances have been discussed and suggestions on how to remediate the deficiencies have been made. Empirical evidences for validity and reliability of the instrument have been presented both from the classical and the modern validity criteria point of view. Limitations of the study have been identified. Finally implications of the study and directions for further research have been discussed.

Assessing fidelity in individual and family therapy for adolescent substance abuse.

PubMed

Hogue, Aaron; Dauber, Sarah; Chinchilla, Priscilla; Fried, Adam; Henderson, Craig; Inclan, Jaime; Reiner, Robert H; Liddle, Howard A

2008-09-01

This study introduces an observational measure of fidelity in evidence-based practices for adolescent substance abuse treatment. The Therapist Behavior Rating Scale-Competence (TBRS-C) measures adherence and competence in individual cognitive-behavioral therapy and multidimensional family therapy for adolescent substance abuse. The TBRS-C assesses fidelity to the core therapeutic goals of each approach and also contains global ratings of therapist competence. Study participants were 136 clinically referred adolescents and their families observed in 437 treatment sessions. The TBRS-C demonstrated strong interrater reliability for goal-specific ratings of treatment adherence, and modest reliability for goal-specific and global ratings of therapist competence, evidence of construct validity, and discriminant validity with an observational measure of therapeutic alliance. The utility of the TBRS-C for evaluating treatment fidelity in field settings is discussed.