Journal Reviewer Ratings: Issues of Particularistic Bias, Agreement, and Predictive Validity within the Manuscript Review Process

ERIC Educational Resources Information Center

Vecchio, Robert P.

2006-01-01

Reviewer evaluations and recommendations for 853 manuscript submissions, over a span of 4 years, are analyzed for evidence of particularistic bias, reviewer agreement, and predictive validity for forecasting a published manuscript's citation impact. Attributes of the submitters, their affiliated institutions, and the reviewers have little…

Predictive validity evidence for medical education research study quality instrument scores: quality of submissions to JGIM's Medical Education Special Issue.

PubMed

Reed, Darcy A; Beckman, Thomas J; Wright, Scott M; Levine, Rachel B; Kern, David E; Cook, David A

2008-07-01

Deficiencies in medical education research quality are widely acknowledged. Content, internal structure, and criterion validity evidence support the use of the Medical Education Research Study Quality Instrument (MERSQI) to measure education research quality, but predictive validity evidence has not been explored. To describe the quality of manuscripts submitted to the 2008 Journal of General Internal Medicine (JGIM) medical education issue and determine whether MERSQI scores predict editorial decisions. Cross-sectional study of original, quantitative research studies submitted for publication. Study quality measured by MERSQI scores (possible range 5-18). Of 131 submitted manuscripts, 100 met inclusion criteria. The mean (SD) total MERSQI score was 9.6 (2.6), range 5-15.5. Most studies used single-group cross-sectional (54%) or pre-post designs (32%), were conducted at one institution (78%), and reported satisfaction or opinion outcomes (56%). Few (36%) reported validity evidence for evaluation instruments. A one-point increase in MERSQI score was associated with editorial decisions to send manuscripts for peer review versus reject without review (OR 1.31, 95%CI 1.07-1.61, p = 0.009) and to invite revisions after review versus reject after review (OR 1.29, 95%CI 1.05-1.58, p = 0.02). MERSQI scores predicted final acceptance versus rejection (OR 1.32; 95% CI 1.10-1.58, p = 0.003). The mean total MERSQI score of accepted manuscripts was significantly higher than rejected manuscripts (10.7 [2.5] versus 9.0 [2.4], p = 0.003). MERSQI scores predicted editorial decisions and identified areas of methodological strengths and weaknesses in submitted manuscripts. Researchers, reviewers, and editors might use this instrument as a measure of methodological quality.

Validation and Use of a Predictive Modeling Tool: Employing Scientific Findings to Improve Responsible Conduct of Research Education.

PubMed

Mulhearn, Tyler J; Watts, Logan L; Todd, E Michelle; Medeiros, Kelsey E; Connelly, Shane; Mumford, Michael D

2017-01-01

Although recent evidence suggests ethics education can be effective, the nature of specific training programs, and their effectiveness, varies considerably. Building on a recent path modeling effort, the present study developed and validated a predictive modeling tool for responsible conduct of research education. The predictive modeling tool allows users to enter ratings in relation to a given ethics training program and receive instantaneous evaluative information for course refinement. Validation work suggests the tool's predicted outcomes correlate strongly (r = 0.46) with objective course outcomes. Implications for training program development and refinement are discussed.

A systematic review of validated sinus surgery simulators.

PubMed

Stew, B; Kao, S S-T; Dharmawardana, N; Ooi, E H

2018-06-01

Simulation provides a safe and effective opportunity to develop surgical skills. A variety of endoscopic sinus surgery (ESS) simulators has been described in the literature. Validation of these simulators allows for effective utilisation in training. To conduct a systematic review of the published literature to analyse the evidence for validated ESS simulation. Pubmed, Embase, Cochrane and Cinahl were searched from inception of the databases to 11 January 2017. Twelve thousand five hundred and sixteen articles were retrieved of which 10 112 were screened following the removal of duplicates. Thirty-eight full-text articles were reviewed after meeting search criteria. Evidence of face, content, construct, discriminant and predictive validity was extracted. Twenty articles were included in the analysis describing 12 ESS simulators. Eleven of these simulators had undergone validation: 3 virtual reality, 7 physical bench models and 1 cadaveric simulator. Seven of the simulators were shown to have face validity, 7 had construct validity and 1 had predictive validity. None of the simulators demonstrated discriminate validity. This systematic review demonstrates that a number of ESS simulators have been comprehensively validated. Many of the validation processes, however, lack standardisation in outcome reporting, thus limiting a meta-analysis comparison between simulators. © 2017 John Wiley & Sons Ltd.

Assessing young children's intention-reading in authentic communicative contexts: preliminary evidence and clinical utility.

PubMed

Greenslade, Kathryn J; Coggins, Truman E

2014-01-01

Identifying what a communication partner is looking at (referential intention) and why (social intention) is essential to successful social communication, and may be challenging for children with social communication deficits. This study explores a clinical task that assesses these intention-reading abilities within an authentic context. To gather evidence of the task's reliability and validity, and to discuss its clinical utility. The intention-reading task was administered to twenty 4-7-year-olds with typical development (TD) and ten with autism spectrum disorder (ASD). Task items were embedded in an authentic activity, and they targeted the child's ability to identify the examiner's referential and social intentions, which were communicated through joint attention behaviours. Reliability and construct validity evidence were addressed using established psychometric methods. Reliability and validity evidence supported the use of task scores for identifying children whose intention-reading warranted concern. Evidence supported the reliability of task administration and coding, and item-level codes were highly consistent with overall task performance. Supporting task validity, group differences aligned with predictions, with children with ASD exhibiting poorer and more variable task scores than children with TD. Also, as predicted, task scores correlated significantly with verbal mental age and ratings of parental concerns regarding social communication abilities. The evidence provides preliminary support for the reliability and validity of the clinical task's scores in assessing young children's real-time intention-reading abilities, which are essential for successful interactions in school and beyond. © 2014 Royal College of Speech and Language Therapists.

Examining construct and predictive validity of the Health-IT Usability Evaluation Scale: confirmatory factor analysis and structural equation modeling results

PubMed Central

Yen, Po-Yin; Sousa, Karen H; Bakken, Suzanne

2014-01-01

Background In a previous study, we developed the Health Information Technology Usability Evaluation Scale (Health-ITUES), which is designed to support customization at the item level. Such customization matches the specific tasks/expectations of a health IT system while retaining comparability at the construct level, and provides evidence of its factorial validity and internal consistency reliability through exploratory factor analysis. Objective In this study, we advanced the development of Health-ITUES to examine its construct validity and predictive validity. Methods The health IT system studied was a web-based communication system that supported nurse staffing and scheduling. Using Health-ITUES, we conducted a cross-sectional study to evaluate users’ perception toward the web-based communication system after system implementation. We examined Health-ITUES's construct validity through first and second order confirmatory factor analysis (CFA), and its predictive validity via structural equation modeling (SEM). Results The sample comprised 541 staff nurses in two healthcare organizations. The CFA (n=165) showed that a general usability factor accounted for 78.1%, 93.4%, 51.0%, and 39.9% of the explained variance in ‘Quality of Work Life’, ‘Perceived Usefulness’, ‘Perceived Ease of Use’, and ‘User Control’, respectively. The SEM (n=541) supported the predictive validity of Health-ITUES, explaining 64% of the variance in intention for system use. Conclusions The results of CFA and SEM provide additional evidence for the construct and predictive validity of Health-ITUES. The customizability of Health-ITUES has the potential to support comparisons at the construct level, while allowing variation at the item level. We also illustrate application of Health-ITUES across stages of system development. PMID:24567081

Applicability Analysis of Validation Evidence for Biomedical Computational Models

DOE PAGES

Pathmanathan, Pras; Gray, Richard A.; Romero, Vicente J.; ...

2017-09-07

Computational modeling has the potential to revolutionize medicine the way it transformed engineering. However, despite decades of work, there has only been limited progress to successfully translate modeling research to patient care. One major difficulty which often occurs with biomedical computational models is an inability to perform validation in a setting that closely resembles how the model will be used. For example, for a biomedical model that makes in vivo clinically relevant predictions, direct validation of predictions may be impossible for ethical, technological, or financial reasons. Unavoidable limitations inherent to the validation process lead to challenges in evaluating the credibilitymore » of biomedical model predictions. Therefore, when evaluating biomedical models, it is critical to rigorously assess applicability, that is, the relevance of the computational model, and its validation evidence to the proposed context of use (COU). However, there are no well-established methods for assessing applicability. In this paper, we present a novel framework for performing applicability analysis and demonstrate its use with a medical device computational model. The framework provides a systematic, step-by-step method for breaking down the broad question of applicability into a series of focused questions, which may be addressed using supporting evidence and subject matter expertise. The framework can be used for model justification, model assessment, and validation planning. While motivated by biomedical models, it is relevant to a broad range of disciplines and underlying physics. Finally, the proposed applicability framework could help overcome some of the barriers inherent to validation of, and aid clinical implementation of, biomedical models.« less

Applicability Analysis of Validation Evidence for Biomedical Computational Models

DOE Office of Scientific and Technical Information (OSTI.GOV)

Pathmanathan, Pras; Gray, Richard A.; Romero, Vicente J.

Computational modeling has the potential to revolutionize medicine the way it transformed engineering. However, despite decades of work, there has only been limited progress to successfully translate modeling research to patient care. One major difficulty which often occurs with biomedical computational models is an inability to perform validation in a setting that closely resembles how the model will be used. For example, for a biomedical model that makes in vivo clinically relevant predictions, direct validation of predictions may be impossible for ethical, technological, or financial reasons. Unavoidable limitations inherent to the validation process lead to challenges in evaluating the credibilitymore » of biomedical model predictions. Therefore, when evaluating biomedical models, it is critical to rigorously assess applicability, that is, the relevance of the computational model, and its validation evidence to the proposed context of use (COU). However, there are no well-established methods for assessing applicability. In this paper, we present a novel framework for performing applicability analysis and demonstrate its use with a medical device computational model. The framework provides a systematic, step-by-step method for breaking down the broad question of applicability into a series of focused questions, which may be addressed using supporting evidence and subject matter expertise. The framework can be used for model justification, model assessment, and validation planning. While motivated by biomedical models, it is relevant to a broad range of disciplines and underlying physics. Finally, the proposed applicability framework could help overcome some of the barriers inherent to validation of, and aid clinical implementation of, biomedical models.« less

Comparison of predictive equations for resting metabolic rate in healthy nonobese and obese adults: a systematic review.

PubMed

Frankenfield, David; Roth-Yousey, Lori; Compher, Charlene

2005-05-01

An assessment of energy needs is a necessary component in the development and evaluation of a nutrition care plan. The metabolic rate can be measured or estimated by equations, but estimation is by far the more common method. However, predictive equations might generate errors large enough to impact outcome. Therefore, a systematic review of the literature was undertaken to document the accuracy of predictive equations preliminary to deciding on the imperative to measure metabolic rate. As part of a larger project to determine the role of indirect calorimetry in clinical practice, an evidence team identified published articles that examined the validity of various predictive equations for resting metabolic rate (RMR) in nonobese and obese people and also in individuals of various ethnic and age groups. Articles were accepted based on defined criteria and abstracted using evidence analysis tools developed by the American Dietetic Association. Because these equations are applied by dietetics practitioners to individuals, a key inclusion criterion was research reports of individual data. The evidence was systematically evaluated, and a conclusion statement and grade were developed. Four prediction equations were identified as the most commonly used in clinical practice (Harris-Benedict, Mifflin-St Jeor, Owen, and World Health Organization/Food and Agriculture Organization/United Nations University [WHO/FAO/UNU]). Of these equations, the Mifflin-St Jeor equation was the most reliable, predicting RMR within 10% of measured in more nonobese and obese individuals than any other equation, and it also had the narrowest error range. No validation work concentrating on individual errors was found for the WHO/FAO/UNU equation. Older adults and US-residing ethnic minorities were underrepresented both in the development of predictive equations and in validation studies. The Mifflin-St Jeor equation is more likely than the other equations tested to estimate RMR to within 10% of that measured, but noteworthy errors and limitations exist when it is applied to individuals and possibly when it is generalized to certain age and ethnic groups. RMR estimation errors would be eliminated by valid measurement of RMR with indirect calorimetry, using an evidence-based protocol to minimize measurement error. The Expert Panel advises clinical judgment regarding when to accept estimated RMR using predictive equations in any given individual. Indirect calorimetry may be an important tool when, in the judgment of the clinician, the predictive methods fail an individual in a clinically relevant way. For members of groups that are greatly underrepresented by existing validation studies of predictive equations, a high level of suspicion regarding the accuracy of the equations is warranted.

Evidence on existing caries risk assessment systems: are they predictive of future caries?

PubMed

Tellez, M; Gomez, J; Pretty, I; Ellwood, R; Ismail, A I

2013-02-01

To critically appraise evidence for the prediction of caries using four caries risk assessment (CRA) systems/guidelines (Cariogram, Caries Management by Risk Assessment (CAMBRA), American Dental Association (ADA), and American Academy of Pediatric Dentistry (AAPD)). This review focused on prospective cohort studies or randomized controlled trials. A systematic search strategy was developed to locate papers published in Medline Ovid and Cochrane databases. The search identified 539 scientific reports, and after title and abstract review, 137 were selected for full review and 14 met the following inclusion criteria: (i) used as validating criterion caries incidence/increment, (ii) involved human subjects and natural carious lesions, and (iii) published in peer-reviewed journals. In addition, papers were excluded if they met one or more of the following criteria: (i) incomplete description of sample selection, outcomes, or small sample size and (ii) not meeting the criteria for best evidence under the prognosis category of the Oxford Centre for Evidence-Based Medicine. There are wide variations among the systems in terms of definitions of caries risk categories, type and number of risk factors/markers, and disease indicators. The Cariogram combined sensitivity and specificity for predicting caries in permanent dentition ranges from 110 to 139 and is the only system for which prospective studies have been conducted to assess its validity. The Cariogram had limited prediction utility in preschool children, and a moderate to good performance for sorting out elderly individuals into caries risk groups. One retrospective analysis on CAMBRA's CRA reported higher incidence of cavitated lesions among those assessed as extreme-risk patients when compared with those at low risk. The evidence on the validity for existing systems for CRA is limited. It is unknown if the identification of high-risk individuals can lead to more effective long-term patient management that prevents caries initiation and arrests or reverses the progression of lesions. There is an urgent need to develop valid and reliable methods for caries risk assessment that are based on best evidence for prediction and disease management rather than opinions of experts.

Predicting implementation from organizational readiness for change: a study protocol

PubMed Central

2011-01-01

Background There is widespread interest in measuring organizational readiness to implement evidence-based practices in clinical care. However, there are a number of challenges to validating organizational measures, including inferential bias arising from the halo effect and method bias - two threats to validity that, while well-documented by organizational scholars, are often ignored in health services research. We describe a protocol to comprehensively assess the psychometric properties of a previously developed survey, the Organizational Readiness to Change Assessment. Objectives Our objective is to conduct a comprehensive assessment of the psychometric properties of the Organizational Readiness to Change Assessment incorporating methods specifically to address threats from halo effect and method bias. Methods and Design We will conduct three sets of analyses using longitudinal, secondary data from four partner projects, each testing interventions to improve the implementation of an evidence-based clinical practice. Partner projects field the Organizational Readiness to Change Assessment at baseline (n = 208 respondents; 53 facilities), and prospectively assesses the degree to which the evidence-based practice is implemented. We will conduct predictive and concurrent validities using hierarchical linear modeling and multivariate regression, respectively. For predictive validity, the outcome is the change from baseline to follow-up in the use of the evidence-based practice. We will use intra-class correlations derived from hierarchical linear models to assess inter-rater reliability. Two partner projects will also field measures of job satisfaction for convergent and discriminant validity analyses, and will field Organizational Readiness to Change Assessment measures at follow-up for concurrent validity (n = 158 respondents; 33 facilities). Convergent and discriminant validities will test associations between organizational readiness and different aspects of job satisfaction: satisfaction with leadership, which should be highly correlated with readiness, versus satisfaction with salary, which should be less correlated with readiness. Content validity will be assessed using an expert panel and modified Delphi technique. Discussion We propose a comprehensive protocol for validating a survey instrument for assessing organizational readiness to change that specifically addresses key threats of bias related to halo effect, method bias and questions of construct validity that often go unexplored in research using measures of organizational constructs. PMID:21777479

Why Do Fine Motor Skills Predict Mathematics? Construct Validity of the Design Copying Task

ERIC Educational Resources Information Center

Murrah, William M.; Chen, Wei-Bing; Cameron, Claire E.

2013-01-01

Recent educational studies have found evidence that measures of fine motor skills are predictive of educational outcomes. However, the precise nature of fine motor skills has received little attention in these studies. With evidence mounting that fine motor skills are an important indicator of school readiness, investigating the nature of this…

Analytic Validation of Immunohistochemical Assays: A Comparison of Laboratory Practices Before and After Introduction of an Evidence-Based Guideline.

PubMed

Fitzgibbons, Patrick L; Goldsmith, Jeffrey D; Souers, Rhona J; Fatheree, Lisa A; Volmar, Keith E; Stuart, Lauren N; Nowak, Jan A; Astles, J Rex; Nakhleh, Raouf E

2017-09-01

- Laboratories must demonstrate analytic validity before any test can be used clinically, but studies have shown inconsistent practices in immunohistochemical assay validation. - To assess changes in immunohistochemistry analytic validation practices after publication of an evidence-based laboratory practice guideline. - A survey on current immunohistochemistry assay validation practices and on the awareness and adoption of a recently published guideline was sent to subscribers enrolled in one of 3 relevant College of American Pathologists proficiency testing programs and to additional nonsubscribing laboratories that perform immunohistochemical testing. The results were compared with an earlier survey of validation practices. - Analysis was based on responses from 1085 laboratories that perform immunohistochemical staining. Of 1057 responses, 65.4% (691) were aware of the guideline recommendations before this survey was sent and 79.9% (550 of 688) of those have already adopted some or all of the recommendations. Compared with the 2010 survey, a significant number of laboratories now have written validation procedures for both predictive and nonpredictive marker assays and specifications for the minimum numbers of cases needed for validation. There was also significant improvement in compliance with validation requirements, with 99% (100 of 102) having validated their most recently introduced predictive marker assay, compared with 74.9% (326 of 435) in 2010. The difficulty in finding validation cases for rare antigens and resource limitations were cited as the biggest challenges in implementing the guideline. - Dissemination of the 2014 evidence-based guideline validation practices had a positive impact on laboratory performance; some or all of the recommendations have been adopted by nearly 80% of respondents.

Construct validity of the helplessness/hopelessness/haplessness scale: correlations with perfectionism and depression.

PubMed

Leenaars, Lindsey; Lester, David

2007-02-01

In a sample of 117 undergraduates, helplessness scores and the discrepancy scores on a measure of perfectionism predicted depression scores, providing evidence for construct validity for the hopelessness, helplessness, and haplessness scales.

Translation and validation of the Spanish version of the "Echelle de Satisfaction des Besoins Psychologiques" in the sports context.

PubMed

Domínguez, Evelia; Martín, Patricia; Martín-Albo, José; Núñez, Juan L; León, Jaime

2010-11-01

The aim of the present research was to translate and to analyze the psychometric properties of the Spanish version of the Satisfaction of Psychological Needs Scale, using a sample of 284 athletes (204 male and 78 female). Results of the confirmatory factor analysis confirmed the correlated three-factor structure of the scale. Furthermore, the results showed evidence of convergence validity with the Basic Psychological Needs in Exercise Scale. The predictive validity was tested using a structural equation model in which task orientation climate predicted the three basic psychological needs and these, in turn, intrinsic motivation. Likewise, we documented evidence of reliability, analyzed as internal consistency and temporal stability. Results partially support the use of the Spanish version of the scale in sports.

Project on the Good Physician: Further Evidence for the Validity of a Moral Intuitionist Model of Virtuous Caring.

PubMed

Leffel, G Michael; Oakes Mueller, Ross A; Ham, Sandra A; Karches, Kyle E; Curlin, Farr A; Yoon, John D

2018-01-19

In the Project on the Good Physician, the authors propose a moral intuitionist model of virtuous caring that places the virtues of Mindfulness, Empathic Compassion, and Generosity at the heart of medical character education. Hypothesis 1a: The virtues of Mindfulness, Empathic Compassion, and Generosity will be positively associated with one another (convergent validity). Hypothesis 1b: The virtues of Mindfulness and Empathic Compassion will explain variance in the action-related virtue of Generosity beyond that predicted by Big Five personality traits alone (discriminant validity). Hypothesis 1c: Virtuous students will experience greater well-being ("flourishing"), as measured by four indices of well-being: life meaning, life satisfaction, vocational identity, and vocational calling (predictive validity). Hypothesis 1d: Students who self-report higher levels of the virtues will be nominated by their peers for the Gold Humanism Award (predictive validity). Hypothesis 2a-2c: Neuroticism and Burnout will be positively associated with each other and inversely associated with measures of virtue and well-being. The authors used data from a 2011 nationally representative sample of U.S. medical students (n = 499) in which medical virtues (Mindfulness, Empathic Compassion, and Generosity) were measured using scales adapted from existing instruments with validity evidence. Supporting the predictive validity of the model, virtuous students were recognized by their peers to be exemplary doctors, and they were more likely to have higher ratings on measures of student well-being. Supporting the discriminant validity of the model, virtues predicted prosocial behavior (Generosity) more than personality traits alone, and students higher in the virtue of Mindfulness were less likely to be high in Neuroticism and Burnout. Data from this descriptive-correlational study offered additional support for the validity of the moral intuitionist model of virtuous caring. Applied to medical character education, medical school programs should consider designing educational experiences that intentionally emphasize the cultivation of virtue.

Validity of the Brazilian version of the Godin-Shephard Leisure-Time Physical Activity Questionnaire.

PubMed

João, Thaís Moreira São; Rodrigues, Roberta Cunha Matheus; Gallani, Maria Cecília Bueno Jayme; Miura, Cinthya Tamie Passos; Domingues, Gabriela de Barros Leite; Amireault, Steve; Godin, Gaston

2015-09-01

This study provides evidence of construct validity for the Brazilian version of the Godin-Shephard Leisure-Time Physical Activity Questionnaire (GSLTPAQ), a 1-item instrument used among 236 participants referred for cardiopulmonary exercise testing. The Baecke Habitual Physical Activity Questionnaire (Baecke-HPA) was used to evaluate convergent and divergent validity. The self-reported measure of walking (QCAF) evaluated the convergent validity. Cardiorespiratory fitness assessed convergent validity by the Veterans Specific Activity Questionnaire (VSAQ), peak measured (VO2peak) and maximum predicted (VO2pred) oxygen uptake. Partial adjusted correlation coefficients between the GSLTPAQ, Baecke-HPA, QCAF, VO2pred and VSAQ provided evidence for convergent validity; while divergent validity was supported by the absence of correlations between the GSLTPAQ and the Occupational Physical Activity domain (Baecke-HPA). The GSLTPAQ presents level 3 of evidence of construct validity and may be useful to assess leisure-time physical activity among patients with cardiovascular disease and healthy individuals.

Recidivism in female offenders: PCL-R lifestyle factor and VRAG show predictive validity in a German sample.

PubMed

Eisenbarth, Hedwig; Osterheider, Michael; Nedopil, Norbert; Stadtland, Cornelis

2012-01-01

A clear and structured approach to evidence-based and gender-specific risk assessment of violence in female offenders is high on political and mental health agendas. However, most data on the factors involved in risk-assessment instruments are based on data of male offenders. The aim of the present study was to validate the use of the Psychopathy Checklist Revised (PCL-R), the HCR-20 and the Violence Risk Appraisal Guide (VRAG) for the prediction of recidivism in German female offenders. This study is part of the Munich Prognosis Project (MPP). It focuses on a subsample of female delinquents (n = 80) who had been referred for forensic-psychiatric evaluation prior to sentencing. The mean time at risk was 8 years (SD = 5 years; range: 1-18 years). During this time, 31% (n = 25) of the female offenders were reconvicted, 5% (n = 4) for violent and 26% (n = 21) for non-violent re-offenses. The predictive validity of the PCL-R for general recidivism was calculated. Analysis with receiver-operating characteristics revealed that the PCL-R total score, the PCL-R antisocial lifestyle factor, the PCL-R lifestyle factor and the PCL-R impulsive and irresponsible behavioral style factor had a moderate predictive validity for general recidivism (area under the curve, AUC = 0.66, p = 0.02). The VRAG has also demonstrated predictive validity (AUC = 0.72, p = 0.02), whereas the HCR-20 showed no predictive validity. These results appear to provide the first evidence that the PCL-R total score and the antisocial lifestyle factor are predictive for general female recidivism, as has been shown consistently for male recidivists. The implications of these findings for crime prevention, prognosis in women, and future research are discussed. Copyright © 2012 John Wiley & Sons, Ltd.

Assessing mental health clinicians' intentions to adopt evidence-based treatments: reliability and validity testing of the evidence-based treatment intentions scale.

PubMed

Williams, Nathaniel J

2016-05-05

Intentions play a central role in numerous empirically supported theories of behavior and behavior change and have been identified as a potentially important antecedent to successful evidence-based treatment (EBT) implementation. Despite this, few measures of mental health clinicians' EBT intentions exist and available measures have not been subject to thorough psychometric evaluation or testing. This paper evaluates the psychometric properties of the evidence-based treatment intentions (EBTI) scale, a new measure of mental health clinicians' intentions to adopt EBTs. The study evaluates the reliability and validity of inferences made with the EBTI using multi-method, multi-informant criterion variables collected over 12 months from a sample of 197 mental health clinicians delivering services in 13 mental health agencies. Structural, predictive, and discriminant validity evidence is assessed. Findings support the EBTI's factor structure (χ (2) = 3.96, df = 5, p = .556) and internal consistency reliability (α = .80). Predictive validity evidence was provided by robust and significant associations between EBTI scores and clinicians' observer-reported attendance at a voluntary EBT workshop at a 1-month follow-up (OR = 1.92, p < .05), self-reported EBT adoption at a 12-month follow-up (R (2) = .17, p < .001), and self-reported use of EBTs with clients at a 12-month follow-up (R (2) = .25, p < .001). Discriminant validity evidence was provided by small associations with clinicians' concurrently measured psychological work climate perceptions of functionality (R (2) = .06, p < .05), engagement (R (2) = .06, p < .05), and stress (R (2) = .00, ns). The EBTI is a practical and theoretically grounded measure of mental health clinicians' EBT intentions. Scores on the EBTI provide a basis for valid inferences regarding mental health clinicians' intentions to adopt EBTs. Discussion focuses on research and practice applications.

Assessing Attachment Security With the Attachment Q Sort: Meta-Analytic Evidence for the Validity of the Observer AQS

ERIC Educational Resources Information Center

van I Jzendoorn,Marinus H.; Vereijken, Carolus M.J.L.; Bakermans-Kranenburg, Marian J.; Riksen-Walraven, Marianne J.

2004-01-01

The reliability and validity of the Attachment Q Sort (AQS; Waters & Deane, 1985) was tested in a series of meta-analyses on 139 studies with 13,835 children. The observer AQS security score showed convergent validity with Strange Situation procedure (SSP) security (r=31) and excellent predictive validity with sensitivity measures (r=39). Its…

Examining construct and predictive validity of the Health-IT Usability Evaluation Scale: confirmatory factor analysis and structural equation modeling results.

PubMed

Yen, Po-Yin; Sousa, Karen H; Bakken, Suzanne

2014-10-01

In a previous study, we developed the Health Information Technology Usability Evaluation Scale (Health-ITUES), which is designed to support customization at the item level. Such customization matches the specific tasks/expectations of a health IT system while retaining comparability at the construct level, and provides evidence of its factorial validity and internal consistency reliability through exploratory factor analysis. In this study, we advanced the development of Health-ITUES to examine its construct validity and predictive validity. The health IT system studied was a web-based communication system that supported nurse staffing and scheduling. Using Health-ITUES, we conducted a cross-sectional study to evaluate users' perception toward the web-based communication system after system implementation. We examined Health-ITUES's construct validity through first and second order confirmatory factor analysis (CFA), and its predictive validity via structural equation modeling (SEM). The sample comprised 541 staff nurses in two healthcare organizations. The CFA (n=165) showed that a general usability factor accounted for 78.1%, 93.4%, 51.0%, and 39.9% of the explained variance in 'Quality of Work Life', 'Perceived Usefulness', 'Perceived Ease of Use', and 'User Control', respectively. The SEM (n=541) supported the predictive validity of Health-ITUES, explaining 64% of the variance in intention for system use. The results of CFA and SEM provide additional evidence for the construct and predictive validity of Health-ITUES. The customizability of Health-ITUES has the potential to support comparisons at the construct level, while allowing variation at the item level. We also illustrate application of Health-ITUES across stages of system development. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

The Predictive Validity of Measures of Teacher Candidate Programs and Performance: Toward an Evidence-Based Approach to Teacher Preparation

ERIC Educational Resources Information Center

Henry, Gary T.; Campbell, Shanyce L.; Thompson, Charles L.; Patriarca, Linda A.; Luterbach, Kenneth J.; Lys, Diana B.; Covington, Vivian Martin

2013-01-01

Calls for evidence-based reform of teacher preparation programs (TPPs) suggest the question: Do the current indicators of progress and performance used by TPPs predict effectiveness of their graduates when they become teachers? In this study, the indicators of progress and performance used by one program are examined for their ability to predict…

Testing the Predictive Validity and Construct of Pathological Video Game Use

PubMed Central

Groves, Christopher L.; Gentile, Douglas; Tapscott, Ryan L.; Lynch, Paul J.

2015-01-01

Three studies assessed the construct of pathological video game use and tested its predictive validity. Replicating previous research, Study 1 produced evidence of convergent validity in 8th and 9th graders (N = 607) classified as pathological gamers. Study 2 replicated and extended the findings of Study 1 with college undergraduates (N = 504). Predictive validity was established in Study 3 by measuring cue reactivity to video games in college undergraduates (N = 254), such that pathological gamers were more emotionally reactive to and provided higher subjective appraisals of video games than non-pathological gamers and non-gamers. The three studies converged to show that pathological video game use seems similar to other addictions in its patterns of correlations with other constructs. Conceptual and definitional aspects of Internet Gaming Disorder are discussed. PMID:26694472

A Multimethod Multitrait Validity Assessment of Self-Construal in Japan, Korea, and the United States

ERIC Educational Resources Information Center

Bresnahan, Mary J.; Levine, Timothy R.; Shearman, Sachiyo Morinaga; Lee, Sun Young; Park, Cheong-Yi; Kiyomiya, Toru

2005-01-01

A large number of previous studies have used self-construal to predict communication outcomes. Recent evidence, however, suggests that validity problems may exist in self-construal measurement. The current study conducted a multimethod multitrait (Campbell & Fiske, 1959) validation study of self-construal measures with data (total N = 578)…

Validity Evidence for the Security Scale as a Measure of Perceived Attachment Security in Adolescence

ERIC Educational Resources Information Center

Van Ryzin, Mark J.; Leve, Leslie D.

2012-01-01

In this study, the validity of a self-report measure of children's perceived attachment security (the Kerns Security Scale) was tested using adolescents. With regards to predictive validity, the Security Scale was significantly associated with (1) observed mother-adolescent interactions during conflict and (2) parent- and teacher-rated social…

New Perspectives on the Validity of the "GRE"® General Test for Predicting Graduate School Grades. ETS GRE® Board Research Report. ETS GRE®-14-03. ETS Research Report. RR-14-26

ERIC Educational Resources Information Center

Klieger, David M.; Cline, Frederick A.; Holtzman, Steven L.; Minsky, Jennifer L.; Lorenz, Florian

2014-01-01

Given the serious consequences of making ill-fated admissions and funding decisions for applicants to graduate and professional school, it is important to rely on sound evidence to optimize such judgments. Previous meta-analytic research has demonstrated the generalizable validity of the "GRE"® General Test for predicting academic…

On the incremental validity of irrational beliefs to predict subjective well-being while controlling for personality factors.

PubMed

Spörrle, Matthias; Strobel, Maria; Tumasjan, Andranik

2010-11-01

This research examines the incremental validity of irrational thinking as conceptualized by Albert Ellis to predict diverse aspects of subjective well-being while controlling for the influence of personality factors. Rational-emotive behavior therapy (REBT) argues that irrational beliefs result in maladaptive emotions leading to reduced well-being. Although there is some early scientific evidence for this relation, it has never been investigated whether this connection would still persist when statistically controlling for the Big Five personality factors, which were consistently found to be important determinants of well-being. Regression analyses revealed significant incremental validity of irrationality over personality factors when predicting life satisfaction, but not when predicting subjective happiness. Results are discussed with respect to conceptual differences between these two aspects of subjective well-being.

Predictive validity of the classroom strategies scale-observer form on statewide testing scores: an initial investigation.

PubMed

Reddy, Linda A; Fabiano, Gregory A; Dudek, Christopher M; Hsu, Louis

2013-12-01

The present study examined the validity of a teacher observation measure, the Classroom Strategies Scale--Observer Form (CSS), as a predictor of student performance on statewide tests of mathematics and English language arts. The CSS is a teacher practice observational measure that assesses evidence-based instructional and behavioral management practices in elementary school. A series of two-level hierarchical generalized linear models were fitted to data of a sample of 662 third- through fifth-grade students to assess whether CSS Part 2 Instructional Strategy and Behavioral Management Strategy scale discrepancy scores (i.e., ∑ |recommended frequency--frequency ratings|) predicted statewide mathematics and English language arts proficiency scores when percentage of minority students in schools was controlled. Results indicated that the Instructional Strategy scale discrepancy scores significantly predicted mathematics and English language arts proficiency scores: Relatively larger discrepancies on observer ratings of what teachers did versus what should have been done were associated with lower proficiency scores. Results offer initial evidence of the predictive validity of the CSS Part 2 Instructional Strategy discrepancy scores on student academic outcomes. PsycINFO Database Record (c) 2013 APA, all rights reserved.

Validity of the SAT® for Predicting First-Year Grades: 2011 SAT Validity Sample. Statistical Report 2013-3

ERIC Educational Resources Information Center

Patterson, Brian F.; Mattern, Krista D.

2013-01-01

The continued accumulation of validity evidence for the intended uses of educational assessments is critical to ensure that proper inferences will be made for those purposes. To that end, the College Board has continued to collect college outcome data to evaluate the relationship between SAT® scores and college success. This report provides…

Validity of the SAT® for Predicting First-Year Grades: 2012 SAT Validity Sample. Statistical Report 2015 2

ERIC Educational Resources Information Center

Beard, Jonathan; Marini, Jessica P.

2015-01-01

The continued accumulation of validity evidence for the intended uses of educational assessment scores is critical to ensure that inferences made using the scores are sound. To that end, the College Board has continued to collect college outcome data to evaluate the relationship between SAT® scores and college success. This report provides updated…

Clinical Prediction Models for Patients With Nontraumatic Knee Pain in Primary Care: A Systematic Review and Internal Validation Study.

PubMed

Panken, Guus; Verhagen, Arianne P; Terwee, Caroline B; Heymans, Martijn W

2017-08-01

Study Design Systematic review and validation study. Background Many prognostic models of knee pain outcomes have been developed for use in primary care. Variability among published studies with regard to patient population, outcome measures, and relevant prognostic factors hampers the generalizability and implementation of these models. Objectives To summarize existing prognostic models in patients with knee pain in a primary care setting and to develop and internally validate new summary prognostic models. Methods After a sensitive search strategy, 2 reviewers independently selected prognostic models for patients with nontraumatic knee pain and assessed the methodological quality of the included studies. All predictors of the included studies were evaluated, summarized, and classified. The predictors assessed in multiple studies of sufficient quality are presented in this review. Using data from the Musculoskeletal System Study (BAS) cohort of patients with a new episode of knee pain, recruited consecutively by Dutch general medical practitioners (n = 372), we used predictors with a strong level of evidence to develop new prognostic models for each outcome measure and internally validated these models. Results Sixteen studies were eligible for inclusion. We considered 11 studies to be of sufficient quality. None of these studies validated their models. Five predictors with strong evidence were related to function and 6 to recovery, and were used to compose 2 prognostic models for patients with knee pain at 1 year. Running these new models in another data set showed explained variances (R 2 ) of 0.36 (function) and 0.33 (recovery). The area under the curve of the recovery model was 0.79. After internal validation, the adjusted R 2 values of the models were 0.30 (function) and 0.20 (recovery), and the area under the curve was 0.73. Conclusion We developed 2 valid prognostic models for function and recovery for patients with nontraumatic knee pain, based on predictors with strong evidence. A longer duration of complaints predicted poorer function but did not adequately predict chance of recovery. Level of Evidence Prognosis, levels 1a and 1b. J Orthop Sports Phys Ther 2017;47(8):518-529. Epub 16 Jun 2017. doi:10.2519/jospt.2017.7142.

The Diagnostic Accuracy of the Berg Balance Scale in Predicting Falls.

PubMed

Park, Seong-Hi; Lee, Young-Shin

2017-11-01

This study aimed to evaluate the predictive validity of the Berg Balance Scale (BBS) as a screening tool for fall risks among those with varied levels of balance. A total of 21 studies reporting predictive validity of the BBS of fall risk were meta-analyzed. With regard to the overall predictive validity of the BBS, the pooled sensitivity and specificity were 0.72 and 0.73, respectively; the accuracy curve area was 0.84. The findings showed statistical heterogeneity among studies. Among the sub-groups, the age group of those younger than 65 years, those with neuromuscular disease, those with 2+ falls, and those with a cutoff point of 45 to 49 showed better sensitivity with statistically less heterogeneity. The empirical evidence indicates that the BBS is a suitable tool to screen for the risk of falls and shows good predictability when used with the appropriate criteria and applied to those with neuromuscular disease.

Implicit Sex Guilt Predicts Sexual Behaviors: Evidence for the Validity of the Sex Guilt Implicit Association Test.

PubMed

Totonchi, Delaram A; Derlega, Valerian J; Janda, Louis H

2018-05-14

Self-report measures of sexuality may be influenced by people's conscious concerns about confidentiality and social desirability. Alternatively, non-conscious measures (e.g., implicit association tests; IATs) are designed to minimize these validity concerns. We constructed an IAT measure of sex guilt using 154 male and female university students. The sex guilt IAT demonstrated convergent validity as it correlated with various sexual behaviors and incremental validity as it improved the prediction of several sexual behaviors beyond that provided by the Mosher sex guilt scale. We conclude that a non-conscious measure of sex guilt may complement the use of self-reports in studying sexual behaviors.

Validation of the Self-Regulation Strategy Inventory-Parent Rating Scale

ERIC Educational Resources Information Center

Lubin, Audrey Ruth

2015-01-01

The current dissertation gathered empirical evidence of convergent and predictive validity for the Self-Regulation Strategies Inventory-Parent Rating Scale (SRSI-PRS), which measures parents' perception of their child's use of self-regulated learning (SRL) strategies during mathematics activities. The SRSI-PRS, which is part of the larger SRSI…

Commentary: Evidential Validity Versus Predictive Validity--The Need for Both

ERIC Educational Resources Information Center

Montgomery, Alyssa; Dumont, Ron; Willis, John O.

2017-01-01

The articles presented in this Special Issue provide evidence for many statistically significant relationships among error scores obtained from the Kaufman Test of Educational Achievement, Third Edition (KTEA)-3 between various groups of students with and without disabilities. The data reinforce the importance of examiners looking beyond the…

An Action Assembly Approach to Predicting Emotional Responses to Frightening Mass Media.

ERIC Educational Resources Information Center

Sparks, Glenn G.

1986-01-01

Assesses the validity of a 20-item scale that purportedly measures long term memory records--in this case, frightening mass media. Evidence for validity emerged in that subjects' scale scores were related to negative emotion, negative cognitions, and skin conductance during film clips of scary movies. (NKA)

Additional Evidence for the Reliability and Validity of the Student Risk Screening Scale at the High School Level: A Replication and Extension

ERIC Educational Resources Information Center

Lane, Kathleen Lynne; Oakes, Wendy P.; Ennis, Robin Parks; Cox, Meredith Lucille; Schatschneider, Christopher; Lambert, Warren

2013-01-01

This study reports findings from a validation study of the Student Risk Screening Scale for use with 9th- through 12th-grade students (N = 1854) attending a rural fringe school. Results indicated high internal consistency, test-retest stability, and inter-rater reliability. Predictive validity was established across two academic years, with Spring…

Evaluating the complementary roles of an SJT and academic assessment for entry into clinical practice.

PubMed

Cousans, Fran; Patterson, Fiona; Edwards, Helena; Walker, Kim; McLachlan, John C; Good, David

2017-05-01

Although there is extensive evidence confirming the predictive validity of situational judgement tests (SJTs) in medical education, there remains a shortage of evidence for their predictive validity for performance of postgraduate trainees in their first role in clinical practice. Moreover, to date few researchers have empirically examined the complementary roles of academic and non-academic selection methods in predicting in-role performance. This is an important area of enquiry as despite it being common practice to use both types of methods within a selection system, there is currently no evidence that this approach translates into increased predictive validity of the selection system as a whole, over that achieved by the use of a single selection method. In this preliminary study, the majority of the range of scores achieved by successful applicants to the UK Foundation Programme provided a unique opportunity to address both of these areas of enquiry. Sampling targeted high (>80th percentile) and low (<20th percentile) scorers on the SJT. Supervisors rated 391 trainees' in-role performance, and incidence of remedial action was collected. SJT and academic performance scores correlated with supervisor ratings (r = .31 and .28, respectively). The relationship was stronger between the SJT and in-role performance for the low scoring group (r = .33, high scoring group r = .11), and between academic performance and in-role performance for the high scoring group (r = .29, low scoring group r = .11). Trainees with low SJT scores were almost five times more likely to receive remedial action. Results indicate that an SJT for entry into trainee physicians' first role in clinical practice has good predictive validity of supervisor-rated performance and incidence of remedial action. In addition, an SJT and a measure of academic performance appeared to be complementary to each other. These initial findings suggest that SJTs may be more predictive at the lower end of a scoring distribution, and academic attainment more predictive at the higher end.

Predictive validity of the Biomedical Admissions Test: an evaluation and case study.

PubMed

McManus, I C; Ferguson, Eamonn; Wakeford, Richard; Powis, David; James, David

2011-01-01

There has been an increase in the use of pre-admission selection tests for medicine. Such tests need to show good psychometric properties. Here, we use a paper by Emery and Bell [2009. The predictive validity of the Biomedical Admissions Test for pre-clinical examination performance. Med Educ 43:557-564] as a case study to evaluate and comment on the reporting of psychometric data in the field of medical student selection (and the comments apply to many papers in the field). We highlight pitfalls when reliability data are not presented, how simple zero-order associations can lead to inaccurate conclusions about the predictive validity of a test, and how biases need to be explored and reported. We show with BMAT that it is the knowledge part of the test which does all the predictive work. We show that without evidence of incremental validity it is difficult to assess the value of any selection tests for medicine.

The Predictive Validity of the Short-Term Assessment of Risk and Treatability (START) for Multiple Adverse Outcomes in a Secure Psychiatric Inpatient Setting.

PubMed

O'Shea, Laura E; Picchioni, Marco M; Dickens, Geoffrey L

2016-04-01

The Short-Term Assessment of Risk and Treatability (START) aims to assist mental health practitioners to estimate an individual's short-term risk for a range of adverse outcomes via structured consideration of their risk ("Vulnerabilities") and protective factors ("Strengths") in 20 areas. It has demonstrated predictive validity for aggression but this is less established for other outcomes. We collated START assessments for N = 200 adults in a secure mental health hospital and ascertained 3-month risk event incidence using the START Outcomes Scale. The specific risk estimates, which are the tool developers' suggested method of overall assessment, predicted aggression, self-harm/suicidality, and victimization, and had incremental validity over the Strength and Vulnerability scales for these outcomes. The Strength scale had incremental validity over the Vulnerability scale for aggressive outcomes; therefore, consideration of protective factors had demonstrable value in their prediction. Further evidence is required to support use of the START for the full range of outcomes it aims to predict. © The Author(s) 2015.

The 2002 NIMH Provisional Diagnostic Criteria for Depression of Alzheimer's Disease (PDC-dAD): Gauging their Validity over a Decade Later.

PubMed

Sepehry, Amir A; Lee, Philip E; Hsiung, Ging-Yuek R; Beattie, B Lynn; Feldman, Howard H; Jacova, Claudia

2017-01-01

Presented herein is evidence for criterion, content, and convergent/discriminant validity of the NIMH-Provisional Diagnostic Criteria for depression of Alzheimer's Disease (PDC-dAD) that were formulated to address depression in Alzheimer's disease (AD). Using meta-analytic and systematic review methods, we examined criterion validity evidence in epidemiological and clinical studies comparing the PDC-dAD to Diagnostic and Statistical Manual of Mental Disorders fourth edition (DSM-IV), and International Classification of Disease (ICD 9) depression diagnostic criteria. We estimated prevalence of depression by PDC, DSM, and ICD with an omnibus event rate effect-size. We also examined diagnostic agreement between PDC and DSM. To gauge content validity, we reviewed rates of symptom endorsement for each diagnostic approach. Finally, we examined the PDC's relationship with assessment scales (global cognition, neuropsychiatric, and depression definition) for convergent validity evidence. The aggregate evidence supports the validity of the PDC-dAD. Our findings suggest that depression in AD differs from other depressive disorders including Major Depressive Disorder (MDD) in that dAD is more prevalent, with generally a milder presentation and with unique features not captured by the DSM. Although the PDC are the current standard for diagnosis of depression in AD, we identified the need for their further optimization based on predictive validity evidence.

Responsiveness and predictive validity of the tablet-based symbol digit modalities test in patients with stroke.

PubMed

Hsiao, Pei-Chi; Yu, Wan-Hui; Lee, Shih-Chieh; Chen, Mei-Hsiang; Hsieh, Ching-Lin

2018-06-14

The responsiveness and predictive validity of the Tablet-based Symbol Digit Modalities Test (T-SDMT) are unknown, which limits the utility of the T-SDMT in both clinical and research settings. The purpose of this study was to examine the responsiveness and predictive validity of the T-SDMT in inpatients with stroke. A follow-up, repeated-assessments design. One rehabilitation unit at a local medical center. A total of 50 inpatients receiving rehabilitation completed T-SDMT assessments at admission to and discharge from a rehabilitation ward. The median follow-up period was 14 days. The Barthel index (BI) was assessed at discharge and was used as the criterion of the predictive validity. The mean changes in the T-SDMT scores between admission and discharge were statistically significant (paired t-test = 3.46, p = 0.001). The T-SDMT scores showed a nearly moderate standardized response mean (0.49). A moderate association (Pearson's r = 0.47) was found between the scores of the T-SDMT at admission and those of the BI at discharge, indicating good predictive validity of the T-SDMT. Our results support the responsiveness and predictive validity of the T-SDMT in patients with stroke receiving rehabilitation in hospitals. This study provides empirical evidence supporting the use of the T-SDMT as an outcome measure for assessing processingspeed in inpatients with stroke. The scores of the T-SDMT could be used to predict basic activities of daily living function in inpatients with stroke.

The predictive validity of selection for entry into postgraduate training in general practice: evidence from three longitudinal studies

PubMed Central

Patterson, Fiona; Lievens, Filip; Kerrin, Máire; Munro, Neil; Irish, Bill

2013-01-01

Background The selection methodology for UK general practice is designed to accommodate several thousand applicants per year and targets six core attributes identified in a multi-method job-analysis study Aim To evaluate the predictive validity of selection methods for entry into postgraduate training, comprising a clinical problem-solving test, a situational judgement test, and a selection centre. Design and setting A three-part longitudinal predictive validity study of selection into training for UK general practice. Method In sample 1, participants were junior doctors applying for training in general practice (n = 6824). In sample 2, participants were GP registrars 1 year into training (n = 196). In sample 3, participants were GP registrars sitting the licensing examination after 3 years, at the end of training (n = 2292). The outcome measures include: assessor ratings of performance in a selection centre comprising job simulation exercises (sample 1); supervisor ratings of trainee job performance 1 year into training (sample 2); and licensing examination results, including an applied knowledge examination and a 12-station clinical skills objective structured clinical examination (OSCE; sample 3). Results Performance ratings at selection predicted subsequent supervisor ratings of job performance 1 year later. Selection results also significantly predicted performance on both the clinical skills OSCE and applied knowledge examination for licensing at the end of training. Conclusion In combination, these longitudinal findings provide good evidence of the predictive validity of the selection methods, and are the first reported for entry into postgraduate training. Results show that the best predictor of work performance and training outcomes is a combination of a clinical problem-solving test, a situational judgement test, and a selection centre. Implications for selection methods for all postgraduate specialties are considered. PMID:24267856

The predictive validity of selection for entry into postgraduate training in general practice: evidence from three longitudinal studies.

PubMed

Patterson, Fiona; Lievens, Filip; Kerrin, Máire; Munro, Neil; Irish, Bill

2013-11-01

The selection methodology for UK general practice is designed to accommodate several thousand applicants per year and targets six core attributes identified in a multi-method job-analysis study To evaluate the predictive validity of selection methods for entry into postgraduate training, comprising a clinical problem-solving test, a situational judgement test, and a selection centre. A three-part longitudinal predictive validity study of selection into training for UK general practice. In sample 1, participants were junior doctors applying for training in general practice (n = 6824). In sample 2, participants were GP registrars 1 year into training (n = 196). In sample 3, participants were GP registrars sitting the licensing examination after 3 years, at the end of training (n = 2292). The outcome measures include: assessor ratings of performance in a selection centre comprising job simulation exercises (sample 1); supervisor ratings of trainee job performance 1 year into training (sample 2); and licensing examination results, including an applied knowledge examination and a 12-station clinical skills objective structured clinical examination (OSCE; sample 3). Performance ratings at selection predicted subsequent supervisor ratings of job performance 1 year later. Selection results also significantly predicted performance on both the clinical skills OSCE and applied knowledge examination for licensing at the end of training. In combination, these longitudinal findings provide good evidence of the predictive validity of the selection methods, and are the first reported for entry into postgraduate training. Results show that the best predictor of work performance and training outcomes is a combination of a clinical problem-solving test, a situational judgement test, and a selection centre. Implications for selection methods for all postgraduate specialties are considered.

Hardiness scales in Iranian managers: evidence of incremental validity in relationships with the five factor model and with organizational and psychological adjustment.

PubMed

Ghorbani, Nima; Watson, P J

2005-06-01

This study examined the incremental validity of Hardiness scales in a sample of Iranian managers. Along with measures of the Five Factor Model and of Organizational and Psychological Adjustment, Hardiness scales were administered to 159 male managers (M age = 39.9, SD = 7.5) who had worked in their organizations for 7.9 yr. (SD=5.4). Hardiness predicted greater Job Satisfaction, higher Organization-based Self-esteem, and perceptions of the work environment as being less stressful and constraining. Hardiness also correlated positively with Assertiveness, Emotional Stability, Extraversion, Openness to Experience, Agreeableness, and Conscientiousness and negatively with Depression, Anxiety, Perceived Stress, Chance External Control, and a Powerful Others External Control. Evidence of incremental validity was obtained when the Hardiness scales supplemented the Five Factor Model in predicting organizational and psychological adjustment. These data documented the incremental validity of the Hardiness scales in a non-Western sample and thus confirmed once again that Hardiness has a relevance that extends beyond the culture in which it was developed.

GMAT versus Alternatives: Predictive Validity Evidence from Central Europe and the Middle East

ERIC Educational Resources Information Center

Koys, Daniel

2010-01-01

The author found that the GPA at the end of the MBA program is most accurately predicted by the Graduate Management Admission Test (GMAT) and the Test of English as a Foreign Language (TOEFL). MBA GPA is also predicted, though less accurately, by the Scholastic Level Exam, a mathematics test, undergraduate GPA, and previous career progression. If…

Placement of Students into First-Year Writing Courses

ERIC Educational Resources Information Center

Elliot, Norbert; Deess, Perry; Rudniy, Alex; Joshi, Kamal

2012-01-01

The purpose of the present study is to examine concurrent and predictive evidence used in the validation of ACCUPLACER, a purchased test used to place first-year students into writing courses at an urban, public research university devoted to science and technology education. Concurrent evidence was determined by correlations between ACCUPLACER…

Self-perceived Coparenting of Nonresident Fathers: Scale Development and Validation.

PubMed

Dyer, W Justin; Fagan, Jay; Kaufman, Rebecca; Pearson, Jessica; Cabrera, Natasha

2017-11-16

This study reports on the development and validation of the Fatherhood Research and Practice Network coparenting perceptions scale for nonresident fathers. Although other measures of coparenting have been developed, this is the first measure developed specifically for low-income, nonresident fathers. Focus groups were conducted to determine various aspects of coparenting. Based on this, a scale was created and administered to 542 nonresident fathers. Participants also responded to items used to examine convergent and predictive validity (i.e., parental responsibility, contact with the mother, father self-efficacy and satisfaction, child behavior problems, and contact and engagement with the child). Factor analyses and reliability tests revealed three distinct and reliable perceived coparenting factors: undermining, alliance, and gatekeeping. Validity tests suggest substantial overlap between the undermining and alliance factors, though undermining was uniquely related to child behavior problems. The alliance and gatekeeping factors showed strong convergent validity and evidence for predictive validity. Taken together, results suggest this relatively short measure (11 items) taps into three coparenting dimensions significantly predictive of aspects of individual and family life. © 2017 Family Process Institute.

In 'big bang' major incidents do triage tools accurately predict clinical priority?: a systematic review of the literature.

PubMed

Kilner, T M; Brace, S J; Cooke, M W; Stallard, N; Bleetman, A; Perkins, G D

2011-05-01

The term "big bang" major incidents is used to describe sudden, usually traumatic,catastrophic events, involving relatively large numbers of injured individuals, where demands on clinical services rapidly outstrip the available resources. Triage tools support the pre-hospital provider to prioritise which patients to treat and/or transport first based upon clinical need. The aim of this review is to identify existing triage tools and to determine the extent to which their reliability and validity have been assessed. A systematic review of the literature was conducted to identify and evaluate published data validating the efficacy of the triage tools. Studies using data from trauma patients that report on the derivation, validation and/or reliability of the specific pre-hospital triage tools were eligible for inclusion.Purely descriptive studies, reviews, exercises or reports (without supporting data) were excluded. The search yielded 1982 papers. After initial scrutiny of title and abstract, 181 papers were deemed potentially applicable and from these 11 were identified as relevant to this review (in first figure). There were two level of evidence one studies, three level of evidence two studies and six level of evidence three studies. The two level of evidence one studies were prospective validations of Clinical Decision Rules (CDR's) in children in South Africa, all the other studies were retrospective CDR derivation, validation or cohort studies. The quality of the papers was rated as good (n=3), fair (n=7), poor (n=1). There is limited evidence for the validity of existing triage tools in big bang major incidents.Where evidence does exist it focuses on sensitivity and specificity in relation to prediction of trauma death or severity of injury based on data from single or small number patient incidents. The Sacco system is unique in combining survivability modelling with the degree by which the system is overwhelmed in the triage decision system. The practicalities, training implications, performance characteristics and reliance on computer technology during a mass casualty incident require further evaluation. 2010 Elsevier Ltd. All rights reserved.

Structural Validity of the MACI Psychopathy and Narcissism Scales: Evidence of Multidimensionality and Implications for Use in Research and Screening

ERIC Educational Resources Information Center

Penney, Stephanie R.; Moretti, Marlene M.; Da Silva, Kimberley S.

2008-01-01

This study investigated the psychometric properties and predictive validity of three self-report scales (the Psychopathy Content Scale, the Psychopathy-16 scale, and the Egotistic scale) derived from the Millon Adolescent Clinical Inventory (MACI) to screen for the presence of psychopathic and narcissistic personality characteristics. Exploratory…

Predictive Validity of a Cigarette Purchase Task in a Randomized Controlled Trial of Contingent Vouchers for Smoking in Individuals With Substance Use Disorders

PubMed Central

Mackillop, James; Murphy, Cara M.; Martin, Rosemarie A.; Stojek, Monika; Tidey, Jennifer W.; Colby, Suzanne M.

2016-01-01

Abstract Introduction: A cigarette purchase task (CPT) is a behavioral economic measure of the reinforcing value of smoking in monetary terms (ie, cigarette demand). This study investigated whether cigarette demand predicted response to contingent monetary rewards for abstinence among individuals with substance use disorders. It also sought to replicate evidence for greater price sensitivity at whole-dollar pack price transitions (ie, left-digit effects). Methods: Participants ( N = 338) were individuals in residential substance use disorder treatment who participated in a randomized controlled trial that compared contingent vouchers to noncontingent vouchers for smoking abstinence. Baseline demand indices were used to predict number of abstinent days during the 14-day voucher period (after the reduction lead-in) and at 1 and 3 months afterward. Results: Demand indices correlated with measures of smoking and nicotine dependence. As measured by elasticity, intensity and Omax , higher demand significantly predicted fewer abstinent exhaled carbon monoxide readings during voucher period for individuals in the noncontingent vouchers condition. Breakpoint exhibited a trend-level association with abstinent exhaled carbon monoxide readings. Demand indices did not predict abstinence in the contingent vouchers group, and did not predict abstinence at 1- and 3-month follow-ups. Left-digit price transitions were associated with significantly greater reductions in consumption. Conclusions: The association of cigarette demand with smoking behavior only in the group for whom abstinence was not incentivized indicates that CPT assesses the value of smoking more than the value of money per se and that vouchers counteract the effects of the intrinsic reinforcing value of cigarettes. Results provide initial short-term evidence of predictive validity for the CPT indices. Implications: This study provides the first evidence of the validity of the CPT for predicting early response to brief advice for smoking cessation plus nicotine replacement in smokers with substance dependence. However, demand for cigarettes did not predict voucher-based treatment response, indicating that incentives serve as a powerful motivator not to smoke that acts in opposition to the intrinsic reinforcing value of cigarettes and that the indices reflect the value of smoking more than the value of money per se. PMID:26498173

The validity of upper-limb neurodynamic tests for detecting peripheral neuropathic pain.

PubMed

Nee, Robert J; Jull, Gwendolen A; Vicenzino, Bill; Coppieters, Michel W

2012-05-01

The validity of upper-limb neurodynamic tests (ULNTs) for detecting peripheral neuropathic pain (PNP) was assessed by reviewing the evidence on plausibility, the definition of a positive test, reliability, and concurrent validity. Evidence was identified by a structured search for peer-reviewed articles published in English before May 2011. The quality of concurrent validity studies was assessed with the Quality Assessment of Diagnostic Accuracy Studies tool, where appropriate. Biomechanical and experimental pain data support the plausibility of ULNTs. Evidence suggests that a positive ULNT should at least partially reproduce the patient's symptoms and that structural differentiation should change these symptoms. Data indicate that this definition of a positive ULNT is reliable when used clinically. Limited evidence suggests that the median nerve test, but not the radial nerve test, helps determine whether a patient has cervical radiculopathy. The median nerve test does not help diagnose carpal tunnel syndrome. These findings should be interpreted cautiously, because diagnostic accuracy might have been distorted by the investigators' definitions of a positive ULNT. Furthermore, patients with PNP who presented with increased nerve mechanosensitivity rather than conduction loss might have been incorrectly classified by electrophysiological reference standards as not having PNP. The only evidence for concurrent validity of the ulnar nerve test was a case study on cubital tunnel syndrome. We recommend that researchers develop more comprehensive reference standards for PNP to accurately assess the concurrent validity of ULNTs and continue investigating the predictive validity of ULNTs for prognosis or treatment response.

An Evaluation of the Predictive Validity of Confidence Ratings in Identifying Functional Behavioral Assessment Hypothesis Statements

ERIC Educational Resources Information Center

Borgmeier, Chris; Horner, Robert H.

2006-01-01

Faced with limited resources, schools require tools that increase the accuracy and efficiency of functional behavioral assessment. Yarbrough and Carr (2000) provided evidence that informant confidence ratings of the likelihood of problem behavior in specific situations offered a promising tool for predicting the accuracy of function-based…

RBANS memory percentage retention: No evidence of incremental validity beyond RBANS scores for diagnostic classification of mild cognitive impairment and dementia and for prediction of daily function.

PubMed

Jodouin, Kara A; O'Connell, Megan E; Morgan, Debra G

2017-01-01

RBANS percentage retention scores may be useful for diagnosis, but their incremental validity is unclear. Percentage retention versus RBANS immediate and delayed memory subtests and delayed index scores were compared for diagnostic classification and for prediction of function. Data from 173 memory clinic patients with an interdisciplinary diagnosis (no cognitive impairment, amnestic mild cognitive impairment [aMCI], and dementia due to Alzheimer's disease [AD]) and complete RBANS data were analyzed. Across diagnostic contrasts, list percentage retention classification accuracy was similar to List Learning delayed recall, but below the Delayed Memory Index (DMI). Similarly, for classifying no cognitive impairment versus aMCI or dementia due to AD, story percentage retention was similar to Story Memory subtests and below the DMI. For classifying aMCI versus AD; however, Story Memory exceeded the DMI, but was similar to Story Memory subtest scores. Similarly, for prediction of function percentage retention measures did not predict variance beyond that predicted by the RBANS subtest or index scores. In sum, there is no evidence that calculation of percentage retention for RBANS adds clinical utility beyond those provided by the standard RBANS scores.

Validation and Trustworthiness of Multiscale Models of Cardiac Electrophysiology

PubMed Central

Pathmanathan, Pras; Gray, Richard A.

2018-01-01

Computational models of cardiac electrophysiology have a long history in basic science applications and device design and evaluation, but have significant potential for clinical applications in all areas of cardiovascular medicine, including functional imaging and mapping, drug safety evaluation, disease diagnosis, patient selection, and therapy optimisation or personalisation. For all stakeholders to be confident in model-based clinical decisions, cardiac electrophysiological (CEP) models must be demonstrated to be trustworthy and reliable. Credibility, that is, the belief in the predictive capability, of a computational model is primarily established by performing validation, in which model predictions are compared to experimental or clinical data. However, there are numerous challenges to performing validation for highly complex multi-scale physiological models such as CEP models. As a result, credibility of CEP model predictions is usually founded upon a wide range of distinct factors, including various types of validation results, underlying theory, evidence supporting model assumptions, evidence from model calibration, all at a variety of scales from ion channel to cell to organ. Consequently, it is often unclear, or a matter for debate, the extent to which a CEP model can be trusted for a given application. The aim of this article is to clarify potential rationale for the trustworthiness of CEP models by reviewing evidence that has been (or could be) presented to support their credibility. We specifically address the complexity and multi-scale nature of CEP models which makes traditional model evaluation difficult. In addition, we make explicit some of the credibility justification that we believe is implicitly embedded in the CEP modeling literature. Overall, we provide a fresh perspective to CEP model credibility, and build a depiction and categorisation of the wide-ranging body of credibility evidence for CEP models. This paper also represents a step toward the extension of model evaluation methodologies that are currently being developed by the medical device community, to physiological models. PMID:29497385

Job Embeddedness Demonstrates Incremental Validity When Predicting Turnover Intentions for Australian University Employees

PubMed Central

Heritage, Brody; Gilbert, Jessica M.; Roberts, Lynne D.

2016-01-01

Job embeddedness is a construct that describes the manner in which employees can be enmeshed in their jobs, reducing their turnover intentions. Recent questions regarding the properties of quantitative job embeddedness measures, and their predictive utility, have been raised. Our study compared two competing reflective measures of job embeddedness, examining their convergent, criterion, and incremental validity, as a means of addressing these questions. Cross-sectional quantitative data from 246 Australian university employees (146 academic; 100 professional) was gathered. Our findings indicated that the two compared measures of job embeddedness were convergent when total scale scores were examined. Additionally, job embeddedness was capable of demonstrating criterion and incremental validity, predicting unique variance in turnover intention. However, this finding was not readily apparent with one of the compared job embeddedness measures, which demonstrated comparatively weaker evidence of validity. We discuss the theoretical and applied implications of these findings, noting that job embeddedness has a complementary place among established determinants of turnover intention. PMID:27199817

The Structured Assessment of Violence Risk in Adults with Intellectual Disability: A Systematic Review.

PubMed

Hounsome, J; Whittington, R; Brown, A; Greenhill, B; McGuire, J

2018-01-01

While structured professional judgement approaches to assessing and managing the risk of violence have been extensively examined in mental health/forensic settings, the application of the findings to people with an intellectual disability is less extensively researched and reviewed. This review aimed to assess whether risk assessment tools have adequate predictive validity for violence in adults with an intellectual disability. Standard systematic review methodology was used to identify and synthesize appropriate studies. A total of 14 studies were identified as meeting the inclusion criteria. These studies assessed the predictive validity of 18 different risk assessment tools, mainly in forensic settings. All studies concluded that the tools assessed were successful in predicting violence. Studies were generally of a high quality. There is good quality evidence that risk assessment tools are valid for people with intellectual disability who offend but further research is required to validate tools for use with people with intellectual disability who offend. © 2016 John Wiley & Sons Ltd.

Measuring striving for understanding and learning value of geometry: a validity study

NASA Astrophysics Data System (ADS)

Ubuz, Behiye; Aydınyer, Yurdagül

2017-11-01

The current study aimed to construct a questionnaire that measures students' personality traits related to striving for understanding and learning value of geometry and then examine its psychometric properties. Through the use of multiple methods on two independent samples of 402 and 521 middle school students, two studies were performed to address this issue to provide support for its validity. In Study 1, exploratory factor analysis indicated the two-factor model. In Study 2, confirmatory factor analysis indicated the better fit of two-factor model compared to one or three-factor model. Convergent and discriminant validity evidence provided insight into the distinctiveness of the two factors. Subgroup validity evidence revealed gender differences for striving for understanding geometry trait favouring girls and grade level differences for learning value of geometry trait favouring the sixth- and seventh-grade students. Predictive validity evidence demonstrated that the striving for understanding geometry trait but not learning value of geometry trait was significantly correlated with prior mathematics achievement. In both studies, each factor and the entire questionnaire showed satisfactory reliability. In conclusion, the questionnaire was psychometrically sound.

Evaluating the predictive accuracy and the clinical benefit of a nomogram aimed to predict survival in node-positive prostate cancer patients: External validation on a multi-institutional database.

PubMed

Bianchi, Lorenzo; Schiavina, Riccardo; Borghesi, Marco; Bianchi, Federico Mineo; Briganti, Alberto; Carini, Marco; Terrone, Carlo; Mottrie, Alex; Gacci, Mauro; Gontero, Paolo; Imbimbo, Ciro; Marchioro, Giansilvio; Milanese, Giulio; Mirone, Vincenzo; Montorsi, Francesco; Morgia, Giuseppe; Novara, Giacomo; Porreca, Angelo; Volpe, Alessandro; Brunocilla, Eugenio

2018-04-06

To assess the predictive accuracy and the clinical value of a recent nomogram predicting cancer-specific mortality-free survival after surgery in pN1 prostate cancer patients through an external validation. We evaluated 518 prostate cancer patients treated with radical prostatectomy and pelvic lymph node dissection with evidence of nodal metastases at final pathology, at 10 tertiary centers. External validation was carried out using regression coefficients of the previously published nomogram. The performance characteristics of the model were assessed by quantifying predictive accuracy, according to the area under the curve in the receiver operating characteristic curve and model calibration. Furthermore, we systematically analyzed the specificity, sensitivity, positive predictive value and negative predictive value for each nomogram-derived probability cut-off. Finally, we implemented decision curve analysis, in order to quantify the nomogram's clinical value in routine practice. External validation showed inferior predictive accuracy as referred to in the internal validation (65.8% vs 83.3%, respectively). The discrimination (area under the curve) of the multivariable model was 66.7% (95% CI 60.1-73.0%) by testing with receiver operating characteristic curve analysis. The calibration plot showed an overestimation throughout the range of predicted cancer-specific mortality-free survival rates probabilities. However, in decision curve analysis, the nomogram's use showed a net benefit when compared with the scenarios of treating all patients or none. In an external setting, the nomogram showed inferior predictive accuracy and suboptimal calibration characteristics as compared to that reported in the original population. However, decision curve analysis showed a clinical net benefit, suggesting a clinical implication to correctly manage pN1 prostate cancer patients after surgery. © 2018 The Japanese Urological Association.

Reconciled Rat and Human Metabolic Networks for Comparative Toxicogenomics and Biomarker Predictions

DTIC Science & Technology

2017-02-08

compared with the original human GPR rules (Supplementary Fig. 3). The consensus-based approach for filtering orthology annotations was designed to...ARTICLE Received 29 Jan 2016 | Accepted 13 Dec 2016 | Published 8 Feb 2017 Reconciled rat and human metabolic networks for comparative toxicogenomics...predictions in response to 76 drugs. We validate comparative predictions for xanthine derivatives with new experimental data and literature- based evidence

Assessing the Culture of Residency Using the C - Change Resident Survey: Validity Evidence in 34 U.S. Residency Programs.

PubMed

Pololi, Linda H; Evans, Arthur T; Civian, Janet T; Shea, Sandy; Brennan, Robert T

2017-07-01

A practical instrument is needed to reliably measure the clinical learning environment and professionalism for residents. To develop and present evidence of validity of an instrument to assess the culture of residency programs and the clinical learning environment. During 2014-2015, we surveyed residents using the C - Change Resident Survey to assess residents' perceptions of the culture in their programs. Residents in all years of training in 34 programs in internal medicine, pediatrics, and general surgery in 14 geographically diverse public and private academic health systems. The C - Change Resident Survey assessed residents' perceptions of 13 dimensions of the culture: Vitality, Self-Efficacy, Institutional Support, Relationships/Inclusion, Values Alignment, Ethical/Moral Distress, Respect, Mentoring, Work-Life Integration, Gender Equity, Racial/Ethnic Minority Equity, and self-assessed Competencies. We measured the internal reliability of each of the 13 dimensions and evaluated response process, content validity, and construct-related evidence validity by assessing relationships predicted by our conceptual model and prior research. We also assessed whether the measurements were sensitive to differences in specialty and across institutions. A total of 1708 residents completed the survey [internal medicine: n = 956, pediatrics: n = 411, general surgery: n = 311 (51% women; 16% underrepresented in medicine minority)], with a response rate of 70% (range across programs, 51-87%). Internal consistency of each dimension was high (Cronbach α: 0.73-0.90). The instrument was able to detect significant differences in the learning environment across programs and sites. Evidence of validity was supported by a good response process and the demonstration of several relationships predicted by our conceptual model. The C - Change Resident Survey assesses the clinical learning environment for residents, and we encourage further study of validity in different contexts. Results could be used to facilitate and monitor improvements in the clinical learning environment and resident well-being.

Using Nonsense Word Fluency to Predict Reading Proficiency in Kindergarten through Second Grade for English Learners and Native English Speakers

ERIC Educational Resources Information Center

Fien, Hank; Baker, Scott K.; Smolkowski, Keith; Smith, Jean L. Mercier; Kame'enui, Edward J.; Beck, Carrie Thomas

2008-01-01

This study examined the validity of Nonsense Word Fluency as an index of beginning reading proficiency for students in kindergarten through second grade. Validity evidence for Nonsense Word Fluency is addressed in the context of research-based instructional practices implemented on a large scale. Technical adequacy data are presented for all…

Validity of the Miller forensic assessment of symptoms test in psychiatric inpatients.

PubMed

Veazey, Connie H; Wagner, Alisha L; Hays, J Ray; Miller, Holly A

2005-06-01

This study investigated the validity of the Miller Forensic Assessment of Symptoms Test (M-FAST), a brief measure of malingering, in an inpatient psychiatric sample of 70. Among those patients who also completed the Personality Assessment Inventory (N=44), Total M-FAST score was related in the expected directions to the Personality Assessment Inventory validity scales and indexes, providing evidence for concurrent validity of the M-FAST. With the PAI malingering index used as a criterion, we examined the diagnostic efficiency of the M-FAST and found a cut score of 8 represented the best balance of sensitivity, specificity, positive predictive power, and negative predictive power. Based on this cut-score of 8, 16% of the population was classified as malingering. The M-FAST appears to be an excellent rapid screen for symptom exaggeration in this population and setting.

Conceptual and measurement issues in early parenting practices research: an epidemiologic perspective.

PubMed

Walker, Lorraine O; Kirby, Russell S

2010-11-01

Early parenting practices are significant to public health because of their linkages to child health outcomes. This paper focuses on the current state of the science regarding conceptual frameworks that incorporate early parenting practices in epidemiologic research and evidence supporting reliability and validity of self-report measures of such practices. Guided by a provisional definition of early parenting practices, literature searches were conducted using PubMed and Sociological Abstracts. Twenty-five published studies that included parent-report measures of early parenting practices met inclusion criteria. Findings on conceptual frameworks were analyzed qualitatively, whereas evidence of reliability and validity were organized into four domains (safety, feeding and oral health, development promotion, and discipline) and summarized in tabular form. Quantitative estimates of measures of reliability and validity were extracted, where available. We found two frameworks incorporating early parenting: one a program theory and the other a predictive model. We found no reported evidence of the reliability or validity of parent-report measures of safety or feeding and oral health practices. Evidence for reliability and validity were reported with greater frequency for development promotion and discipline practices, but report of the most pertinent type of reliability estimation, test-retest reliability, was rare. Failure to examine associations of early parenting practices with any child outcomes within most studies resulted in missed opportunities to indirectly estimate validity of parenting practice measures. Stronger evidence concerning specific measurement properties of early parenting practices is important to advancing maternal-child research, surveillance, and practice.

Reliability and validity of the parent efficacy for child healthy weight behaviour (PECHWB) scale.

PubMed

Palmer, F; Davis, M C

2014-05-01

Interventions for childhood overweight and obesity that target parents as the agents of change by increasing parent self-efficacy for facilitating their child's healthy weight behaviours require a reliable and valid tool to measure parent self-efficacy before and after interventions. Nelson and Davis developed the Parent Efficacy for Child Healthy Weight Behaviour (PECHWB) scale with good preliminary evidence of reliability and validity. The aim of this research was to provide further psychometric evidence from an independent Australian sample. Data were provided by a convenience sample of 261 primary caregivers of children aged 4-17 years via an online survey. PECHWB scores were correlated with scores on other self-report measures of parenting efficacy and 2- to 4-week test-retest reliability of the PECHWB was assessed. The results of the study confirmed the four-factor structure of the PECHWB (Fat and Sugar, Sedentary Behaviours, Physical Activity, and Fruit and Vegetables) and provided strong evidence of internal consistency and test-retest reliability, as well as good evidence of convergent validity. Future research should investigate the properties of the PECHWB in a sample of parents of overweight or obese children, including measures of child weight and actual child healthy weight behaviours to provide evidence of the concurrent and predictive validity of PECHWB scores. © 2013 John Wiley & Sons Ltd.

Selecting postoperative adjuvant systemic therapy for early stage breast cancer: A critical assessment of commercially available gene expression assays

PubMed Central

Schuur, Eric; Angel Aristizabal, Javier; Bargallo Rocha, Juan Enrique; Cabello, Cesar; Elizalde, Roberto; García‐Estévez, Laura; Gomez, Henry L.; Katz, Artur; Nuñez De Pierro, Aníbal

2017-01-01

Risk stratification of patients with early stage breast cancer may support adjuvant chemotherapy decision‐making. This review details the development and validation of six multi‐gene classifiers, each of which claims to provide useful prognostic and possibly predictive information for early stage breast cancer patients. A careful assessment is presented of each test's analytical validity, clinical validity, and clinical utility, as well as the quality of evidence supporting its use. PMID:28211064

The Teacher Sense of Efficacy Scale: Validation Evidence and Behavioral Prediction. WCER Working Paper No. 2006-7

ERIC Educational Resources Information Center

Heneman, Herbert G., III; Kimball, Steven; Milanowski, Anthony

2006-01-01

The present study contributes to knowledge of the construct validity of the short form of the Teacher Sense of Efficacy Scale (and by extension, given their similar content and psychometric properties, to the long form). The authors' research involves: (1) examining the psychometric properties of the TSES on a large sample of elementary, middle,…

Evidence for a relationship between trait gratitude and prosocial behaviour.

PubMed

Yost-Dubrow, Rachel; Dunham, Yarrow

2018-03-01

Prosocial behaviour towards unrelated others is communally beneficial but can be individually costly. The emotion of gratitude mitigates this cost by encouraging direct as well as "upstream" reciprocity, thereby facilitating cooperation. A widely used method for measuring trait gratitude is the Gratitude Questionnaire (GQ6) [McCullough, M., Emmons, R., & Tsang, J. (2002). The grateful disposition: A conceptual and empirical topography. Journal of Personality and Social Psychology, 82, 112-127. Retrieved from https://doi.org/10.1037/0022-3514.82.1.112 ]. Here we undertake an assessment of the external validity of the GQ6 by examining its relationship with two incentivized economic games that serve as face valid indices of generosity and reciprocity. In two studies (total N = 501) we find that trait gratitude as measured by the GQ6 predicts greater donations in a charity donation task as well as greater transfers and returns in an incentivized trust game. These results support the hypothesis that individuals with higher trait gratitude are more generous and trusting on average, and provide initial evidence as to the predictive validity of the GQ6.

Addressing criticisms of existing predictive bias research: cognitive ability test scores still overpredict African Americans' job performance.

PubMed

Berry, Christopher M; Zhao, Peng

2015-01-01

Predictive bias studies have generally suggested that cognitive ability test scores overpredict job performance of African Americans, meaning these tests are not predictively biased against African Americans. However, at least 2 issues call into question existing over-/underprediction evidence: (a) a bias identified by Aguinis, Culpepper, and Pierce (2010) in the intercept test typically used to assess over-/underprediction and (b) a focus on the level of observed validity instead of operational validity. The present study developed and utilized a method of assessing over-/underprediction that draws on the math of subgroup regression intercept differences, does not rely on the biased intercept test, allows for analysis at the level of operational validity, and can use meta-analytic estimates as input values. Therefore, existing meta-analytic estimates of key parameters, corrected for relevant statistical artifacts, were used to determine whether African American job performance remains overpredicted at the level of operational validity. African American job performance was typically overpredicted by cognitive ability tests across levels of job complexity and across conditions wherein African American and White regression slopes did and did not differ. Because the present study does not rely on the biased intercept test and because appropriate statistical artifact corrections were carried out, the present study's results are not affected by the 2 issues mentioned above. The present study represents strong evidence that cognitive ability tests generally overpredict job performance of African Americans. (c) 2015 APA, all rights reserved.

Predicting return to work after low back injury using the Psychosocial Risk for Occupational Disability Instrument: a validation study.

PubMed

Schultz, I Z; Crook, J; Berkowitz, J; Milner, R; Meloche, G R

2005-09-01

This paper reports on the predictive validity of a Psychosocial Risk for Occupational Disability Scale in the workers' compensation environment using a paper and pencil version of a previously validated multimethod instrument on a new, subacute sample of workers with low back pain. A cohort longitudinal study design with a randomly selected cohort off work for 4-6 weeks was applied. The questionnaire was completed by 111 eligible workers at 4-6 weeks following injury. Return to work status data at three months was obtained from 100 workers. Sixty-four workers had returned to work (RTW) and 36 had not (NRTW). Stepwise backward elimination resulted in a model with these predictors: Expectations of Recovery, SF-36 Vitality, SF-36 Mental Health, and Waddell Symptoms. The correct classification of RTW/NRTW was 79%, with sensitivity (NRTW) of 61% and specificity (RTW) of 89%. The area under the ROC curve was 84%. New evidence for predictive validity for the Psychosocial Risk-for-Disability Instrument was provided. The instrument can be useful and practical for prediction of return to work outcomes in the subacute stage after low back injury in the workers' compensation context.

A Stochastic Framework for Evaluating Seizure Prediction Algorithms Using Hidden Markov Models

PubMed Central

Wong, Stephen; Gardner, Andrew B.; Krieger, Abba M.; Litt, Brian

2007-01-01

Responsive, implantable stimulation devices to treat epilepsy are now in clinical trials. New evidence suggests that these devices may be more effective when they deliver therapy before seizure onset. Despite years of effort, prospective seizure prediction, which could improve device performance, remains elusive. In large part, this is explained by lack of agreement on a statistical framework for modeling seizure generation and a method for validating algorithm performance. We present a novel stochastic framework based on a three-state hidden Markov model (HMM) (representing interictal, preictal, and seizure states) with the feature that periods of increased seizure probability can transition back to the interictal state. This notion reflects clinical experience and may enhance interpretation of published seizure prediction studies. Our model accommodates clipped EEG segments and formalizes intuitive notions regarding statistical validation. We derive equations for type I and type II errors as a function of the number of seizures, duration of interictal data, and prediction horizon length and we demonstrate the model’s utility with a novel seizure detection algorithm that appeared to predicted seizure onset. We propose this framework as a vital tool for designing and validating prediction algorithms and for facilitating collaborative research in this area. PMID:17021032

Therapeutic Misconception in Research Subjects: Development and Validation of a Measure

PubMed Central

Appelbaum, Paul S.; Anatchkova, Milena; Albert, Karen; Dunn, Laura B.; Lidz, Charles W.

2013-01-01

Background Therapeutic misconception (TM), which occurs when research subjects fail to appreciate the distinction between the imperatives of clinical research and ordinary treatment, may undercut the process of obtaining meaningful consent to clinical research participation. Previous studies have found TM is widespread, but progress in addressing TM has been stymied by the absence of a validated method for assessing its presence. Purpose The goal of this study was to develop and validate a theoretically grounded measure of TM, assess its diagnostic accuracy, and test previous findings regarding its prevalence. Methods 220 participants were recruited from clinical trials at 4 academic medical centers in the U.S. Participants completed a 28-item Likert-type questionnaire to assess the presence of beliefs associated with TM, and a semi-structured TM interview designed to elicit their perceptions of the nature of the clinical trial in which they were participating. Data from the questionnaires were subjected to factor analysis and items with poor factor loadings were excluded. This resulted in a 10-item scale, with 3 strongly correlated factors and excellent internal consistency; the fit indices of the model across 10 training sets were consistent with the original results, suggesting a stable factor solution. Results The scale was validated against the TM interview, with significantly higher scores among subjects coded as displaying evidence of TM. ROC analysis based on a 10-fold internal cross-validation yielded AUC=.682 for any evidence of TM. When sensitivity (0.72) and specificity (0.61) were both optimized, Positive Predictive Value was 0.65 and Negative Predictive Value was 0.68, with a Positive Likelihood Ratio of 1.89, and a Negative Likelihood Ratio of 0.47. 50.5% (n=101) of participants manifested evidence of TM on the TM interview, a somewhat lower rate than in most previous studies. Limitations The predictive value of the scale compared with the “gold standard” clinical interview is modest, although similar to other instruments based on self-report assessing states of mind rather than discrete symptoms. Thus, although the scale can offer evidence of which subjects are at risk for distortions in their decisions and to what degree, it will not allow researchers to conclude definitively that TM is present in a given subject. Conclusions The development of a reliable and valid TM scale, even with modest predictive power, should permit investigators in clinical trials to identify subjects with tendencies to misinterpret the nature of the situation and to provide additional information to them. It should also stimulate research on how best to decrease TM and facilitate meaningful informed consent to clinical research. PMID:22942217

[Modeling in value-based medicine].

PubMed

Neubauer, A S; Hirneiss, C; Kampik, A

2010-03-01

Modeling plays an important role in value-based medicine (VBM). It allows decision support by predicting potential clinical and economic consequences, frequently combining different sources of evidence. Based on relevant publications and examples focusing on ophthalmology the key economic modeling methods are explained and definitions are given. The most frequently applied model types are decision trees, Markov models, and discrete event simulation (DES) models. Model validation includes besides verifying internal validity comparison with other models (external validity) and ideally validation of its predictive properties. The existing uncertainty with any modeling should be clearly stated. This is true for economic modeling in VBM as well as when using disease risk models to support clinical decisions. In economic modeling uni- and multivariate sensitivity analyses are usually applied; the key concepts here are tornado plots and cost-effectiveness acceptability curves. Given the existing uncertainty, modeling helps to make better informed decisions than without this additional information.

The Dirty Dozen Scale: Validation of a Polish Version and Extension of the Nomological Net.

PubMed

Czarna, Anna Z; Jonason, Peter K; Dufner, Michael; Kossowska, Małgorzata

2016-01-01

In five studies (total N = 1300) we developed and validated a Polish version of the Dirty Dozen measure (DTDD-P) that measures the three traits of the Dark Triad, Machiavellianism, psychopathy, and narcissism. We detail the presence and stability of a bifactor structure of the 12 items and present evidence for good internal consistency and test-retest reliability. We examine the nomological network surrounding the Dark Triad and show that both the Dark Triad total score and the subscales have acceptable validity. We also present evidence on the Dark Triad and moral behavior. Dark Triad predicts utilitarian moral choice (e.g., approval for sacrificing somebody's life for the sake of saving others) and this link is mediated by low empathic concern. In total, our results suggest that the Polish Dirty Dozen-Parszywa Dwunastka-is valid, stable, and useful for the study of lingering puzzles in the literature.

Demonstrating the validity of three general scores of PET in predicting higher education achievement in Israel.

PubMed

Oren, Carmel; Kennet-Cohen, Tamar; Turvall, Elliot; Allalouf, Avi

2014-01-01

The Psychometric Entrance Test (PET), used for admission to higher education in Israel together with the Matriculation (Bagrut), had in the past one general (total) score in which the weights for its domains: Verbal, Quantitative and English, were 2:2:1, respectively. In 2011, two additional total scores were introduced, with different weights for the Verbal and the Quantitative domains. This study compares the predictive validity of the three general scores of PET, and demonstrates validity in terms of utility. 100,863 freshmen students of all Israeli universities over the classes of 2005-2009. Regression weights and correlations of the predictors with FYGPA were computed. Simulations based on these results supplied the utility estimates. On average, PET is slightly more predictive than the Bagrut; using them both yields a better tool than either of them alone. Assigning differential weights to the components in the respective schools further improves the validity. The introduction of the new general scores of PET is validated by gathering and analyzing evidence based on relations of test scores to other variables. The utility of using the test can be demonstrated in ways different from correlations.

Quantitative and Systems Pharmacology. 1. In Silico Prediction of Drug-Target Interactions of Natural Products Enables New Targeted Cancer Therapy.

PubMed

Fang, Jiansong; Wu, Zengrui; Cai, Chuipu; Wang, Qi; Tang, Yun; Cheng, Feixiong

2017-11-27

Natural products with diverse chemical scaffolds have been recognized as an invaluable source of compounds in drug discovery and development. However, systematic identification of drug targets for natural products at the human proteome level via various experimental assays is highly expensive and time-consuming. In this study, we proposed a systems pharmacology infrastructure to predict new drug targets and anticancer indications of natural products. Specifically, we reconstructed a global drug-target network with 7,314 interactions connecting 751 targets and 2,388 natural products and built predictive network models via a balanced substructure-drug-target network-based inference approach. A high area under receiver operating characteristic curve of 0.96 was yielded for predicting new targets of natural products during cross-validation. The newly predicted targets of natural products (e.g., resveratrol, genistein, and kaempferol) with high scores were validated by various literature studies. We further built the statistical network models for identification of new anticancer indications of natural products through integration of both experimentally validated and computationally predicted drug-target interactions of natural products with known cancer proteins. We showed that the significantly predicted anticancer indications of multiple natural products (e.g., naringenin, disulfiram, and metformin) with new mechanism-of-action were validated by various published experimental evidence. In summary, this study offers powerful computational systems pharmacology approaches and tools for the development of novel targeted cancer therapies by exploiting the polypharmacology of natural products.

Comparison of Parent versus Child-Report of Child Impulsivity Traits and Prediction of Outcome Variables

PubMed Central

Zapolski, Tamika C. B.; Smith, Gregory T.

2013-01-01

Five personality traits that dispose individuals to rash or ill-advised action (i.e., sensation seeking, negative urgency, positive urgency, lack of planning, and lack of perseverance), can be reliably and validly assessed in children. This paper reports on the first test of parental reports of these traits. In a sample of 94 children (ages 7–13, mean age 10.6), the authors found the following. First, parental reports of the five traits in their children appeared to be reliable. Second, there was moderate convergent validity: parent and child reports of the same traits had a median correlation of r = .30. Third, there was adequate discriminant validity: within-parent reports on different traits had a median correlation of r = .11. Fourth, concurrent prediction of child behavior from parental reports generally was inconsistent with prior findings. Fifth, discrepancies between the two reporters did predict dysfunctional child behavior. There are advantages to securing both child self-report and parental report of personality dispositions to rash action, although there is limited evidence for the concurrent validity of parental reports. PMID:24039341

Validation of prognostic scores for clinical outcomes in cirrhotic patients with acute variceal bleeding.

PubMed

Motola-Kuba, Miguel; Escobedo-Arzate, Angélica; Tellez-Avila, Félix; Altamirano, José; Aguilar-Olivos, Nancy; González-Angulo, Alberto; Zamarripa-Dorsey, Felipe; Uribe, Misael; Chávez-Tapia, Norberto C

Background. The Rockall, Glasgow-Blatchford, and AIMS65 are useful and validated scoring systems for predicting the outcomes of patients with nonvariceal gastrointestinal bleeding. However, there are no validated evidence for using them to predict outcomes on variceal bleeding. The aim of this study was to evaluate and compare the prognostic accuracy of different nonvariceal bleeding scores with other liver-specific scoring systems in cirrhotic patients. A retrospective multicenter study that included 160 cirrhotic patients with acute variceal bleeding. The AUROC's to predict in-hospital mortality, and rebleeding, were analyzed for each scoring system. Overall in-hospital mortality occurred in 13% and in-hospital rebleeding in 12% of patients. The systems with the best AUROC value for predicting mortality were MELD (0.828; 95% CI 0.748-0.909), and AIMS65 (0.817; 95% CI 0.724-0.909). The best score systems for predicting rebleeding were Glasgow-Blatchford (0.756; 95% CI 0.640- 0.827), and Rockall (0.691; 95% CI 0.580-0.802). In addition to liver-specific scores, the AIMS65 score is accurate for predicting in-hospital mortality in cirrhotic patients with acute variceal bleeding. Other scoring systems might be useful for predicting significant clinical outcomes in these patients.

Reliability and validity of the Fear of Intimacy Scale in China.

PubMed

Ingersoll, Travis S; Norvilitis, Jill M; Zhang, Jie; Jia, Shuhua; Tetewsky, Sheldon

2008-05-01

Participants in China (n = 343) and the United States (n = 283) completed measures to assess the reliability and validity of the Fear of Intimacy Scale (Descutner & Thelen, 1991) with a Chinese population. Internal consistency was strong in both cultures, and the factor structure was also similar between cultures, with confirmatory factor analysis (CFA) identifying three-factor models in both samples. As evidence of convergent validity, the scale was positively correlated with depression and negatively correlated with social support and self-esteem. There were gender differences between cultures, but low levels of femininity were predictive of fear of intimacy in both cultures. The influence of individualism and collectivism varied, with high levels of individualism more predictive of a fear of intimacy in China than in the United States.

Emergency department triage scales and their components: a systematic review of the scientific evidence.

PubMed

Farrohknia, Nasim; Castrén, Maaret; Ehrenberg, Anna; Lind, Lars; Oredsson, Sven; Jonsson, Håkan; Asplund, Kjell; Göransson, Katarina E

2011-06-30

Emergency department (ED) triage is used to identify patients' level of urgency and treat them based on their triage level. The global advancement of triage scales in the past two decades has generated considerable research on the validity and reliability of these scales. This systematic review aims to investigate the scientific evidence for published ED triage scales. The following questions are addressed: 1. Does assessment of individual vital signs or chief complaints affect mortality during the hospital stay or within 30 days after arrival at the ED?2. What is the level of agreement between clinicians' triage decisions compared to each other or to a gold standard for each scale (reliability)? 3. How valid is each triage scale in predicting hospitalization and hospital mortality? A systematic search of the international literature published from 1966 through March 31, 2009 explored the British Nursing Index, Business Source Premier, CINAHL, Cochrane Library, EMBASE, and PubMed. Inclusion was limited to controlled studies of adult patients (≥ 15 years) visiting EDs for somatic reasons. Outcome variables were death in ED or hospital and need for hospitalization (validity). Methodological quality and clinical relevance of each study were rated as high, medium, or low. The results from the studies that met the inclusion criteria and quality standards were synthesized applying the internationally developed GRADE system. Each conclusion was then assessed as having strong, moderately strong, limited, or insufficient scientific evidence. If studies were not available, this was also noted.We found ED triage scales to be supported, at best, by limited and often insufficient evidence.The ability of the individual vital signs included in the different scales to predict outcome is seldom, if at all, studied in the ED setting. The scientific evidence to assess interrater agreement (reliability) was limited for one triage scale and insufficient or lacking for all other scales. Two of the scales yielded limited scientific evidence, and one scale yielded insufficient evidence, on which to assess the risk of early death or hospitalization in patients assigned to the two lowest triage levels on a 5-level scale (validity).

Emergency Department Triage Scales and Their Components: A Systematic Review of the Scientific Evidence

PubMed Central

2011-01-01

Emergency department (ED) triage is used to identify patients' level of urgency and treat them based on their triage level. The global advancement of triage scales in the past two decades has generated considerable research on the validity and reliability of these scales. This systematic review aims to investigate the scientific evidence for published ED triage scales. The following questions are addressed: 1. Does assessment of individual vital signs or chief complaints affect mortality during the hospital stay or within 30 days after arrival at the ED? 2. What is the level of agreement between clinicians' triage decisions compared to each other or to a gold standard for each scale (reliability)? 3. How valid is each triage scale in predicting hospitalization and hospital mortality? A systematic search of the international literature published from 1966 through March 31, 2009 explored the British Nursing Index, Business Source Premier, CINAHL, Cochrane Library, EMBASE, and PubMed. Inclusion was limited to controlled studies of adult patients (≥15 years) visiting EDs for somatic reasons. Outcome variables were death in ED or hospital and need for hospitalization (validity). Methodological quality and clinical relevance of each study were rated as high, medium, or low. The results from the studies that met the inclusion criteria and quality standards were synthesized applying the internationally developed GRADE system. Each conclusion was then assessed as having strong, moderately strong, limited, or insufficient scientific evidence. If studies were not available, this was also noted. We found ED triage scales to be supported, at best, by limited and often insufficient evidence. The ability of the individual vital signs included in the different scales to predict outcome is seldom, if at all, studied in the ED setting. The scientific evidence to assess interrater agreement (reliability) was limited for one triage scale and insufficient or lacking for all other scales. Two of the scales yielded limited scientific evidence, and one scale yielded insufficient evidence, on which to assess the risk of early death or hospitalization in patients assigned to the two lowest triage levels on a 5-level scale (validity). PMID:21718476

Beware of external validation! - A Comparative Study of Several Validation Techniques used in QSAR Modelling.

PubMed

Majumdar, Subhabrata; Basak, Subhash C

2018-04-26

Proper validation is an important aspect of QSAR modelling. External validation is one of the widely used validation methods in QSAR where the model is built on a subset of the data and validated on the rest of the samples. However, its effectiveness for datasets with a small number of samples but large number of predictors remains suspect. Calculating hundreds or thousands of molecular descriptors using currently available software has become the norm in QSAR research, owing to computational advances in the past few decades. Thus, for n chemical compounds and p descriptors calculated for each molecule, the typical chemometric dataset today has high value of p but small n (i.e. n < p). Motivated by the evidence of inadequacies of external validation in estimating the true predictive capability of a statistical model in recent literature, this paper performs an extensive and comparative study of this method with several other validation techniques. We compared four validation methods: leave-one-out, K-fold, external and multi-split validation, using statistical models built using the LASSO regression, which simultaneously performs variable selection and modelling. We used 300 simulated datasets and one real dataset of 95 congeneric amine mutagens for this evaluation. External validation metrics have high variation among different random splits of the data, hence are not recommended for predictive QSAR models. LOO has the overall best performance among all validation methods applied in our scenario. Results from external validation are too unstable for the datasets we analyzed. Based on our findings, we recommend using the LOO procedure for validating QSAR predictive models built on high-dimensional small-sample data. Copyright© Bentham Science Publishers; For any queries, please email at epub@benthamscience.org.

Predicting College Math Success: Do High School Performance and Gender Matter? Evidence from Sultan Qaboos University in Oman

ERIC Educational Resources Information Center

Islam, M. Mazharul; Al-Ghassani, Asma

2015-01-01

The objective of this study was to evaluate the performance of students of college of Science of Sultan Qaboos University (SQU) in Calculus I course, and examine the predictive validity of student's high school performance and gender for Calculus I success. The data for the study was extracted from students' database maintained by the Deanship of…

Evaluation of the DAVROS (Development And Validation of Risk-adjusted Outcomes for Systems of emergency care) risk-adjustment model as a quality indicator for healthcare

PubMed Central

Wilson, Richard; Goodacre, Steve W; Klingbajl, Marcin; Kelly, Anne-Maree; Rainer, Tim; Coats, Tim; Holloway, Vikki; Townend, Will; Crane, Steve

2014-01-01

Background and objective Risk-adjusted mortality rates can be used as a quality indicator if it is assumed that the discrepancy between predicted and actual mortality can be attributed to the quality of healthcare (ie, the model has attributional validity). The Development And Validation of Risk-adjusted Outcomes for Systems of emergency care (DAVROS) model predicts 7-day mortality in emergency medical admissions. We aimed to test this assumption by evaluating the attributional validity of the DAVROS risk-adjustment model. Methods We selected cases that had the greatest discrepancy between observed mortality and predicted probability of mortality from seven hospitals involved in validation of the DAVROS risk-adjustment model. Reviewers at each hospital assessed hospital records to determine whether the discrepancy between predicted and actual mortality could be explained by the healthcare provided. Results We received 232/280 (83%) completed review forms relating to 179 unexpected deaths and 53 unexpected survivors. The healthcare system was judged to have potentially contributed to 10/179 (8%) of the unexpected deaths and 26/53 (49%) of the unexpected survivors. Failure of the model to appropriately predict risk was judged to be responsible for 135/179 (75%) of the unexpected deaths and 2/53 (4%) of the unexpected survivors. Some 10/53 (19%) of the unexpected survivors died within a few months of the 7-day period of model prediction. Conclusions We found little evidence that deaths occurring in patients with a low predicted mortality from risk-adjustment could be attributed to the quality of healthcare provided. PMID:23605036

Rational selection of training and test sets for the development of validated QSAR models

NASA Astrophysics Data System (ADS)

Golbraikh, Alexander; Shen, Min; Xiao, Zhiyan; Xiao, Yun-De; Lee, Kuo-Hsiung; Tropsha, Alexander

2003-02-01

Quantitative Structure-Activity Relationship (QSAR) models are used increasingly to screen chemical databases and/or virtual chemical libraries for potentially bioactive molecules. These developments emphasize the importance of rigorous model validation to ensure that the models have acceptable predictive power. Using k nearest neighbors ( kNN) variable selection QSAR method for the analysis of several datasets, we have demonstrated recently that the widely accepted leave-one-out (LOO) cross-validated R2 (q2) is an inadequate characteristic to assess the predictive ability of the models [Golbraikh, A., Tropsha, A. Beware of q2! J. Mol. Graphics Mod. 20, 269-276, (2002)]. Herein, we provide additional evidence that there exists no correlation between the values of q 2 for the training set and accuracy of prediction ( R 2) for the test set and argue that this observation is a general property of any QSAR model developed with LOO cross-validation. We suggest that external validation using rationally selected training and test sets provides a means to establish a reliable QSAR model. We propose several approaches to the division of experimental datasets into training and test sets and apply them in QSAR studies of 48 functionalized amino acid anticonvulsants and a series of 157 epipodophyllotoxin derivatives with antitumor activity. We formulate a set of general criteria for the evaluation of predictive power of QSAR models.

The development and validation of the Incivility from Customers Scale.

PubMed

Wilson, Nicole L; Holmvall, Camilla M

2013-07-01

Scant research has examined customers as sources of workplace incivility, despite evidence suggesting that mistreatment is more common from organizational outsiders, including customers, than from organizational members (Grandey, Kern, & Frone, 2007; Schat & Kelloway, 2005). As an important step in extending the literature on customer incivility, we conducted two studies to develop and validate a measure of this construct. Study 1 used focus groups of retail and restaurant employees (n = 30) to elicit a list of uncivil customer behaviors, based on which we wrote initial scale items. Study 2 used a correlational survey design (n = 439) to pare down the number of scale items to 10 and to garner reliability and validity evidence for the scale. Exploratory and confirmatory factor analyses show that the scale is unidimensional and distinguishable from measures of the related, but distinct, constructs of interpersonal justice and psychological aggression from customers. Reliability analyses show that the scale is internally consistent. Significant correlations between the scale and individuals' job satisfaction, turnover intentions, and general and job-specific psychological strain provide evidence of criterion-related validity. Hierarchical regression analyses show that the scale significantly predicts three of four organizational and personal strain outcomes over and above a workplace incivility measure adapted for customer incivility, providing some evidence of incremental validity. Limitations and future research directions are discussed. PsycINFO Database Record (c) 2013 APA, all rights reserved.

Descriptive and predictive validity of somatic attributions in patients with somatoform disorders: a systematic review of quantitative research.

PubMed

Douzenis, Athanassios; Seretis, Dionysis

2013-09-01

Research on hypochondriasis and other somatoform disorders (SFD) has provided evidence that patients with SFD tend to attribute their symptoms to organic dysfunctions or disease. However, recent studies appear to discredit this. There is no systematic evidence on whether patients with SFD predominantly rely on somatic attributions, despite calls to include somatic attributions as a positive criterion of somatic symptom disorder (SSD) in the upcoming Diagnostic and Statistical Manual of Mental Disorders (DSM-5). This study is a systematic review of quantitative studies which assess the descriptive and predictive validity of somatic attribution in SFD. The literature search was restricted to studies with patients who met the DSM-IV criteria for SFD. Somatic attribution style in SFD has acceptable descriptive but insufficient predictive validity. This confirms that the overlap between somatic and psychological attributions is often substantial. Attribution style can discriminate between SFD patients with and without comorbidity. A somatic attribution style does not qualify as a positive criterion in SSD. However, there is an urgent need for further research on causal illness perceptions in the full spectrum of medically unexplained symptoms in order to confirm this result. Given its high prevalence, research on psychological attribution style is warranted. Re-attribution does not provide a framework sophisticated enough to address the needs of patients in primary care. Copyright © 2013 Elsevier Inc. All rights reserved.

The social perception of emotional abilities: expanding what we know about observer ratings of emotional intelligence.

PubMed

Elfenbein, Hillary Anger; Barsade, Sigal G; Eisenkraft, Noah

2015-02-01

We examine the social perception of emotional intelligence (EI) through the use of observer ratings. Individuals frequently judge others' emotional abilities in real-world settings, yet we know little about the properties of such ratings. This article examines the social perception of EI and expands the evidence to evaluate its reliability and cross-judge agreement, as well as its convergent, divergent, and predictive validity. Three studies use real-world colleagues as observers and data from 2,521 participants. Results indicate significant consensus across observers about targets' EI, moderate but significant self-observer agreement, and modest but relatively consistent discriminant validity across the components of EI. Observer ratings significantly predicted interdependent task performance, even after controlling for numerous factors. Notably, predictive validity was greater for observer-rated than for self-rated or ability-tested EI. We discuss the minimal associations of observer ratings with ability-tested EI, study limitations, future directions, and practical implications. PsycINFO Database Record (c) 2015 APA, all rights reserved.

The Reliability and Predictive Validity of the Stalking Risk Profile.

PubMed

McEwan, Troy E; Shea, Daniel E; Daffern, Michael; MacKenzie, Rachel D; Ogloff, James R P; Mullen, Paul E

2018-03-01

This study assessed the reliability and validity of the Stalking Risk Profile (SRP), a structured measure for assessing stalking risks. The SRP was administered at the point of assessment or retrospectively from file review for 241 adult stalkers (91% male) referred to a community-based forensic mental health service. Interrater reliability was high for stalker type, and moderate-to-substantial for risk judgments and domain scores. Evidence for predictive validity and discrimination between stalking recidivists and nonrecidivists for risk judgments depended on follow-up duration. Discrimination was moderate (area under the curve = 0.66-0.68) and positive and negative predictive values good over the full follow-up period ( Mdn = 170.43 weeks). At 6 months, discrimination was better than chance only for judgments related to stalking of new victims (area under the curve = 0.75); however, high-risk stalkers still reoffended against their original victim(s) 2 to 4 times as often as low-risk stalkers. Implications for the clinical utility and refinement of the SRP are discussed.

Attitudes without Objects: Evidence for a Dispositional Attitude, its Measurement, and its Consequences

PubMed Central

Hepler, Justin; Albarracin, Dolores

2013-01-01

We hypothesized that individuals may differ in the dispositional tendency to have positive versus negative attitudes, a trait termed the Dispositional Attitude. Across four studies, we developed a 16-item Dispositional Attitude Measure (DAM) and investigated its internal consistency, test-retest reliability, factor structure, convergent validity, discriminant validity, and predictive validity. DAM scores were (a) positively correlated with positive affect traits, curiosity-related traits, and individual pre-existing attitudes, (b) negatively correlated with negative affect traits, and (c) uncorrelated with theoretically unrelated traits. Dispositional attitudes also significantly predicted the valence of novel attitudes while controlling for theoretically relevant traits (such as the big-five and optimism). The dispositional attitude construct represents a new perspective in which attitudes are not simply a function of the properties of the stimuli under consideration, but are also a function of the properties of the evaluator. We discuss the intriguing implications of dispositional attitudes for many areas of research, including attitude formation, persuasion, and behavior prediction. PMID:23586409

Attitudes without objects: evidence for a dispositional attitude, its measurement, and its consequences.

PubMed

Hepler, Justin; Albarracín, Dolores

2013-06-01

We hypothesized that individuals may differ in the dispositional tendency to have positive vs. negative attitudes, a trait termed the dispositional attitude. Across 4 studies, we developed a 16-item Dispositional Attitude Measure (DAM) and investigated its internal consistency, test-retest reliability, factor structure, convergent validity, discriminant validity, and predictive validity. DAM scores were (a) positively correlated with positive affect traits, curiosity-related traits, and individual preexisting attitudes; (b) negatively correlated with negative affect traits; and (c) uncorrelated with theoretically unrelated traits. Dispositional attitudes also significantly predicted the valence of novel attitudes while controlling for theoretically relevant traits (such as the Big 5 and optimism). The dispositional attitude construct represents a new perspective in which attitudes are not simply a function of the properties of the stimuli under consideration, but are also a function of the properties of the evaluator. We discuss the intriguing implications of dispositional attitudes for many areas of research, including attitude formation, persuasion, and behavior prediction. PsycINFO Database Record (c) 2013 APA, all rights reserved.

Strong claims and weak evidence: reassessing the predictive validity of the IAT.

PubMed

Blanton, Hart; Jaccard, James; Klick, Jonathan; Mellers, Barbara; Mitchell, Gregory; Tetlock, Philip E

2009-05-01

The authors reanalyzed data from 2 influential studies-A. R. McConnell and J. M. Leibold and J. C. Ziegert and P. J. Hanges-that explore links between implicit bias and discriminatory behavior and that have been invoked to support strong claims about the predictive validity of the Implicit Association Test. In both of these studies, the inclusion of race Implicit Association Test scores in regression models reduced prediction errors by only tiny amounts, and Implicit Association Test scores did not permit prediction of individual-level behaviors. Furthermore, the results were not robust when the impact of rater reliability, statistical specifications, and/or outliers were taken into account, and reanalysis of A. R. McConnell & J. M. Leibold (2001) revealed a pattern of behavior consistent with a pro-Black behavioral bias, rather than the anti-Black bias suggested in the original study. (c) 2009 APA, all rights reserved.

Clinical Models and Algorithms for the Prediction of Retinopathy of Prematurity: A Report by the American Academy of Ophthalmology.

PubMed

Hutchinson, Amy K; Melia, Michele; Yang, Michael B; VanderVeen, Deborah K; Wilson, Lorri B; Lambert, Scott R

2016-04-01

To assess the accuracy with which available retinopathy of prematurity (ROP) predictive models detect clinically significant ROP and to what extent and at what risk these models allow for the reduction of screening examinations for ROP. A literature search of the PubMed and Cochrane Library databases was conducted last on May 1, 2015, and yielded 305 citations. After screening the abstracts of all 305 citations and reviewing the full text of 30 potentially eligible articles, the panel members determined that 22 met the inclusion criteria. One article included 2 studies, for a total of 23 studies reviewed. The panel extracted information about study design, study population, the screening algorithm tested, interventions, outcomes, and study quality. The methodologist divided the studies into 2 categories-model development and model validation-and assigned a level of evidence rating to each study. One study was rated level I evidence, 3 studies were rated level II evidence, and 19 studies were rated level III evidence. In some cohorts, some models would have allowed reductions in the number of infants screened for ROP without failing to identify infants requiring treatment. However, the small sample size and limited generalizability of the ROP predictive models included in this review preclude their widespread use to make all-or-none decisions about whether to screen individual infants for ROP. As an alternative, some studies proposed approaches to apply the models to reduce the number of examinations performed in low-risk infants. Additional research is needed to optimize ROP predictive model development, validation, and application before such models can be used widely to reduce the burdensome number of ROP screening examinations. Copyright © 2016 American Academy of Ophthalmology. Published by Elsevier Inc. All rights reserved.

External validation of the NUn score for predicting anastomotic leakage after oesophageal resection.

PubMed

Paireder, Matthias; Jomrich, Gerd; Asari, Reza; Kristo, Ivan; Gleiss, Andreas; Preusser, Matthias; Schoppmann, Sebastian F

2017-08-29

Early detection of anastomotic leakage (AL) after oesophageal resection for malignancy is crucial. This retrospective study validates a risk score, predicting AL, which includes C-reactive protein, albumin and white cell count in patients undergoing oesophageal resection between 2003 and 2014. For validation of the NUn score a receiver operating characteristic (ROC) curve is estimated. Area under the ROC curve (AUC) is reported with 95% confidence interval (CI). Among 258 patients (79.5% male) 32 patients showed signs of anastomotic leakage (12.4%). NUn score in our data has a median of 9.3 (range 6.2-17.6). The odds ratio for AL was 1.31 (CI 1.03-1.67; p = 0.028). AUC for AL was 0.59 (CI 0.47-0.72). Using the original cutoff value of 10, the sensitivity was 45.2% an the specificity was 73.8%. This results in a positive predictive value of 19.4% and a negative predictive value of 90.6%. The proportion of variation in AL occurrence, which is explained by the NUn score, was 2.5% (PEV = 0.025). This study provides evidence for an external validation of a simple risk score for AL after oesophageal resection. In this cohort, the NUn score is not useful due to its poor discrimination.

Supervisor Health and Safety Support: Scale Development and Validation

PubMed Central

Butts, Marcus M.; Hurst, Carrie S.; Eby, Lillian T.

2013-01-01

Executive Summary Two studies were conducted to develop a psychometrically sound measure of supervisor health and safety support (SHSS). We identified three dimensions of supervisor support (physical health, psychological health, safety) and used Study 1 to develop items and establish content validity. Study 2 was used to establish the dimensionality of the new measure and provide criterion-related and discriminant validity evidence of the measure using supervisor and subordinate data. The measure had incremental validity in predicting employee performance and psychological strain outcomes above and beyond general work support variables. Implications of these findings and for workplace support theory and practice are discussed. PMID:24771991

Prediction of the effect of formulation on the toxicity of chemicals.

PubMed

Mistry, Pritesh; Neagu, Daniel; Sanchez-Ruiz, Antonio; Trundle, Paul R; Vessey, Jonathan D; Gosling, John Paul

2017-01-01

Two approaches for the prediction of which of two vehicles will result in lower toxicity for anticancer agents are presented. Machine-learning models are developed using decision tree, random forest and partial least squares methodologies and statistical evidence is presented to demonstrate that they represent valid models. Separately, a clustering method is presented that allows the ordering of vehicles by the toxicity they show for chemically-related compounds.

Predictive Accuracy of the Liverpool Lung Project Risk Model for Stratifying Patients for Computed Tomography Screening for Lung Cancer

PubMed Central

Raji, Olaide Y.; Duffy, Stephen W.; Agbaje, Olorunshola F.; Baker, Stuart G.; Christiani, David C.; Cassidy, Adrian; Field, John K.

2013-01-01

Background External validation of existing lung cancer risk prediction models is limited. Using such models in clinical practice to guide the referral of patients for computed tomography (CT) screening for lung cancer depends on external validation and evidence of predicted clinical benefit. Objective To evaluate the discrimination of the Liverpool Lung Project (LLP) risk model and demonstrate its predicted benefit for stratifying patients for CT screening by using data from 3 independent studies from Europe and North America. Design Case–control and prospective cohort study. Setting Europe and North America. Patients Participants in the European Early Lung Cancer (EUELC) and Harvard case–control studies and the LLP population-based prospective cohort (LLPC) study. Measurements 5-year absolute risks for lung cancer predicted by the LLP model. Results The LLP risk model had good discrimination in both the Harvard (area under the receiver-operating characteristic curve [AUC], 0.76 [95% CI, 0.75 to 0.78]) and the LLPC (AUC, 0.82 [CI, 0.80 to 0.85]) studies and modest discrimination in the EUELC (AUC, 0.67 [CI, 0.64 to 0.69]) study. The decision utility analysis, which incorporates the harms and benefit of using a risk model to make clinical decisions, indicates that the LLP risk model performed better than smoking duration or family history alone in stratifying high-risk patients for lung cancer CT screening. Limitations The model cannot assess whether including other risk factors, such as lung function or genetic markers, would improve accuracy. Lack of information on asbestos exposure in the LLPC limited the ability to validate the complete LLP risk model. Conclusion Validation of the LLP risk model in 3 independent external data sets demonstrated good discrimination and evidence of predicted benefits for stratifying patients for lung cancer CT screening. Further studies are needed to prospectively evaluate model performance and evaluate the optimal population risk thresholds for initiating lung cancer screening. Primary Funding Source Roy Castle Lung Cancer Foundation. PMID:22910935

Validation Evidence of the Motivation for Teaching Scale in Secondary Education.

PubMed

Abós, Ángel; Sevil, Javier; Martín-Albo, José; Aibar, Alberto; García-González, Luis

2018-04-10

Grounded in self-determination theory, the aim of this study was to develop a scale with adequate psychometric properties to assess motivation for teaching and to explain some outcomes of secondary education teachers at work. The sample comprised 584 secondary education teachers. Analyses supported the five-factor model (intrinsic motivation, identified regulation, introjected regulation, external regulation and amotivation) and indicated the presence of a continuum of self-determination. Evidence of reliability was provided by Cronbach's alpha, composite reliability and average variance extracted. Multigroup confirmatory factor analyses supported the partial invariance (configural and metric) of the scale in different sub-samples, in terms of gender and type of school. Concurrent validity was analyzed by a structural equation modeling that explained 71% of the work dedication variance and 69% of the boredom at work variance. Work dedication was positively predicted by intrinsic motivation (ß = .56, p < .001) and external regulation (ß = .29, p < .001) and negatively predicted by introjected regulation (ß = -.22, p < .001) and amotivation (ß = -.49, p < .001). Boredom at work was negatively predicted by intrinsic motivation (ß = -.28, p < .005) and positively predicted by amotivation (ß = .68, p < .001). The Motivation for Teaching Scale in Secondary Education (Spanish acronym EME-ES, Escala de Motivación por la Enseñanza en Educación Secundaria) is discussed as a valid and reliable instrument. This is the first specific scale in the work context of secondary teachers that has integrated the five-factor structure together with their dedication and boredom at work.

Ecological validity of the screening module and the Daily Living tests of the Neuropsychological Assessment Battery using the Mayo-Portland Adaptability Inventory-4 in postacute brain injury rehabilitation.

PubMed

Zgaljardic, Dennis J; Yancy, Sybil; Temple, Richard O; Watford, Monica F; Miller, Rebekah

2011-11-01

The assessment of ecological validity of neuropsychological measures is an area of growing interest, particularly in the postacute brain injury rehabilitation (PABIR) setting, as there is an increasing demand for clinicians to address functional and real-world outcomes. In the current study, we assessed the predictive value of the Screening module and the Daily Living tests of the Neuropsychological Assessment Battery (NAB) using clinician ratings from the Mayo-Portland Adaptability Inventory-4 (MPAI-4) in patients with moderate to severe traumatic brain injury. Forty-seven individuals were each administered the NAB Screening module (NAB-SM) and the NAB Daily Living (NAB-DL) tests following admission to a residential PABIR program. MPAI-4 ratings were also obtained at admission. Linear regression analysis was used to examine the association between these functional and neuropsychological assessment measures. We replicated prior work (Temple at al., 2009) and expanded evidence for the ecological validity of the NAB-SM. Furthermore, our findings support the ecological validity of the NAB-DL Bill Payment, Judgment, and Map Reading tests with regards to functional skills and real-world activities. The current study supports prior work from our lab assessing the predictive value of the NAB-SM, as well as provides evidence for the ecological validity for select NAB-DL tests in patients with moderate to severe traumatic brain injury admitted to a residential PABIR program.

Extending the validity of the Feeding Practices and Structure Questionnaire.

PubMed

Jansen, Elena; Mallan, Kimberley M; Daniels, Lynne A

2015-06-30

Feeding practices are commonly examined as potentially modifiable determinants of children's eating behaviours and weight status. Although a variety of questionnaires exist to assess different feeding aspects, many lack thorough reliability and validity testing. The Feeding Practices and Structure Questionnaire (FPSQ) is a tool designed to measure early feeding practices related to non-responsive feeding and structure of the meal environment. Face validity, factorial validity, internal reliability and cross-sectional correlations with children's eating behaviours have been established in mothers with 2-year-old children. The aim of the present study was to further extend the validity of the FPSQ by examining factorial, construct and predictive validity, and stability. Participants were from the NOURISH randomised controlled trial which evaluated an intervention with first-time mothers designed to promote protective feeding practices. Maternal feeding practices (FP) and child eating behaviours were assessed when children were aged 2 years and 3.7 years (n = 388). Confirmatory Factor analysis, group differences, predictive relationships, and stability were tested. The original 9-factor structure was confirmed when children were aged 3.7 ± 0.3 years. Cronbach's alpha was above the recommended 0.70 cut-off for all factors except Structured Meal Timing, Over Restriction and Distrust in Appetite which were 0.58, 0.67 and 0.66 respectively. Allocated group differences reflected behaviour consistent with intervention content and all feeding practices were stable across both time points (range of r = 0.45-0.70). There was some evidence for the predictive validity of factors with 2 FP showing expected relationships, 2 FP showing expected and unexpected relationships and 5 FP showing no relationship. Reliability and validity was demonstrated for most subscales of the FPSQ. Future validation is warranted with culturally diverse samples and with fathers and other caregivers. The use of additional outcomes to further explore predictive validity is recommended as well as testing test-retest reliability of the questionnaire.

Choice Inconsistencies among the Elderly: Evidence from Plan Choice in the Medicare Part D Program: Comment.

PubMed

Ketcham, Jonathan D; Kuminoff, Nicolai V; Powers, Christopher A

2016-12-01

Consumers' enrollment decisions in Medicare Part D can be explained by Abaluck and Gruber’s (2011) model of utility maximization with psychological biases or by a neoclassical version of their model that precludes such biases. We evaluate these competing hypotheses by applying nonparametric tests of utility maximization and model validation tests to administrative data. We find that 79 percent of enrollment decisions from 2006 to 2010 satisfied basic axioms of consumer theory under the assumption of full information. The validation tests provide evidence against widespread psychological biases. In particular, we find that precluding psychological biases improves the structural model's out-of-sample predictions for consumer behavior.

External Validation and Evaluation of Reliability and Validity of the Modified Seoul National University Renal Stone Complexity Scoring System to Predict Stone-Free Status After Retrograde Intrarenal Surgery.

PubMed

Park, Juhyun; Kang, Minyong; Jeong, Chang Wook; Oh, Sohee; Lee, Jeong Woo; Lee, Seung Bae; Son, Hwancheol; Jeong, Hyeon; Cho, Sung Yong

2015-08-01

The modified Seoul National University Renal Stone Complexity scoring system (S-ReSC-R) for retrograde intrarenal surgery (RIRS) was developed as a tool to predict stone-free rate (SFR) after RIRS. We externally validated the S-ReSC-R. We retrospectively reviewed 159 patients who underwent RIRS. The S-ReSC-R was assigned from 1 to 12 according to the location and number of sites involved. The stone-free status was defined as no evidence of a stone or with clinically insignificant residual fragment stones less than 2 mm. Interobserver and test-retest reliabilities were evaluated. Statistical performance of the prediction model was assessed by its predictive accuracy, predictive probability, and clinical usefulness. Overall SFR was 73.0%. The SFRs were 86.7%, 70.2%, and 48.6% in low-score (1-2), intermediate-score (3-4), and high-score (5-12) groups, respectively (p<0.001). External validation of S-ReSC-R revealed an area under the curve (AUC) of 0.731 (95% CI 0.650-0.813). The AUC of the three-titered S-ReSC-R was 0.701 (95% CI 0.609-0.794). The calibration plot showed that the predicted probability of SFR had a concordance comparable to that of observed frequency. The Hosmer-Lemeshow goodness of fit test revealed a p-value of 0.01 for the S-ReSC-R and 0.90 for the three-titered S-ReSC-R. Interobserver and test-retest reliabilities revealed an almost perfect level of agreement. The present study proved the predictive value of S-ReSC-R to predict SFR following RIRS in an independent cohort. Interobserver and test-retest reliabilities confirmed that S-ReSC-R was reliable and valid.

Criterion and incremental validity of the emotion regulation questionnaire

PubMed Central

Ioannidis, Christos A.; Siegling, A. B.

2015-01-01

Although research on emotion regulation (ER) is developing, little attention has been paid to the predictive power of ER strategies beyond established constructs. The present study examined the incremental validity of the Emotion Regulation Questionnaire (ERQ; Gross and John, 2003), which measures cognitive reappraisal and expressive suppression, over and above the Big Five personality factors. It also extended the evidence for the measure's criterion validity to yet unexamined criteria. A university student sample (N = 203) completed the ERQ, a measure of the Big Five, and relevant cognitive and emotion-laden criteria. Cognitive reappraisal predicted positive affect beyond personality, as well as experiential flexibility and constructive self-assertion beyond personality and affect. Expressive suppression explained incremental variance in negative affect beyond personality and in experiential flexibility beyond personality and general affect. No incremental effects were found for worry, social anxiety, rumination, reflection, and preventing negative emotions. Implications for the construct validity and utility of the ERQ are discussed. PMID:25814967

Validating Machine Learning Algorithms for Twitter Data Against Established Measures of Suicidality.

PubMed

Braithwaite, Scott R; Giraud-Carrier, Christophe; West, Josh; Barnes, Michael D; Hanson, Carl Lee

2016-05-16

One of the leading causes of death in the United States (US) is suicide and new methods of assessment are needed to track its risk in real time. Our objective is to validate the use of machine learning algorithms for Twitter data against empirically validated measures of suicidality in the US population. Using a machine learning algorithm, the Twitter feeds of 135 Mechanical Turk (MTurk) participants were compared with validated, self-report measures of suicide risk. Our findings show that people who are at high suicidal risk can be easily differentiated from those who are not by machine learning algorithms, which accurately identify the clinically significant suicidal rate in 92% of cases (sensitivity: 53%, specificity: 97%, positive predictive value: 75%, negative predictive value: 93%). Machine learning algorithms are efficient in differentiating people who are at a suicidal risk from those who are not. Evidence for suicidality can be measured in nonclinical populations using social media data.

Validating Machine Learning Algorithms for Twitter Data Against Established Measures of Suicidality

PubMed Central

2016-01-01

Background One of the leading causes of death in the United States (US) is suicide and new methods of assessment are needed to track its risk in real time. Objective Our objective is to validate the use of machine learning algorithms for Twitter data against empirically validated measures of suicidality in the US population. Methods Using a machine learning algorithm, the Twitter feeds of 135 Mechanical Turk (MTurk) participants were compared with validated, self-report measures of suicide risk. Results Our findings show that people who are at high suicidal risk can be easily differentiated from those who are not by machine learning algorithms, which accurately identify the clinically significant suicidal rate in 92% of cases (sensitivity: 53%, specificity: 97%, positive predictive value: 75%, negative predictive value: 93%). Conclusions Machine learning algorithms are efficient in differentiating people who are at a suicidal risk from those who are not. Evidence for suicidality can be measured in nonclinical populations using social media data. PMID:27185366

Idiopathic Pulmonary Fibrosis: Clinically Meaningful Primary Endpoints in Phase 3 Clinical Trials

PubMed Central

Collard, Harold R.; Anstrom, Kevin J.; Flaherty, Kevin R.; Fleming, Thomas R.; King, Talmadge E.; Martinez, Fernando J.; Brown, Kevin K.

2012-01-01

Definitive evidence of clinical efficacy in a Phase 3 trial is best shown by a beneficial impact on a clinically meaningful endpoint—that is, an endpoint that directly measures how a patient feels (symptoms), functions (the ability to perform activities in daily life), or survives. In idiopathic pulmonary fibrosis (IPF), we believe the endpoints that best meet these criteria are all-cause mortality and all-cause nonelective hospitalization. There are no validated measures of symptoms or broader constructs such as health status or funtional status in IPF. A surrogate endpoint is defined as an indirect measure that is intended to substitute for a clinically meaningful endpoint. Surrogate endpoints can be appropriate outcome measures if validated. However, validation requires substantial evidence that the effect of an intervention on a clinically meaningful endpoint is reliably predicted by the effect of an intervention on the surrogate endpoint. For patients with IPF, there are currently no validated surrogate endpoints. PMID:22505745

The evolving role of physiotherapists in pre-employment screening for workplace injury prevention: are functional capacity evaluations the answer?

PubMed

Legge, Jennifer

2013-10-01

Musculoskeletal injuries account for the largest proportion of workplace injuries. In an attempt to predict, and subsequently manage, the risk of sprains and strains in the workplace, employers are turning to pre-employment screening. Functional capacity evaluations (FCEs) are increasing in popularity as a tool for pre-employment screening despite limited published evidence for their validity in healthy working populations. This narrative review will present an overview of the state of the evidence for pre-employment functional testing, propose a framework for decision-making to determine the suitability of assessment tools, and discuss the role and potential ethical challenges for physiotherapists conducting pre-employment functional testing. Much of the evidence surrounding the validity of functional testing is in the context of the injured worker and prediction of return to work. In healthy populations, FCE components, such as aerobic fitness and manual handling activities, have demonstrated predictability of workplace injury in a small number of studies. This predictability improves when workers' performance is compared with the job demands. This job-specific approach is also required to meet anti-discrimination requirements. There are a number of practical limitations to functional testing, although these are not limited to the pre-employment domain. Physiotherapists need to have a clear understanding of the legal requirements and potential ethical challenges that they may face when conducting pre-employment functional assessments (PEFAs). Further research is needed into the efficacy of pre-employment testing for workplace injury prevention. Physiotherapists and PEFAs are just one part of a holistic approach to workplace injury prevention.

Measurement of fatigue: Comparison of the reliability and validity of single-item and short measures to a comprehensive measure.

PubMed

Kim, Hee-Ju; Abraham, Ivo

2017-01-01

Evidence is needed on the clinicometric properties of single-item or short measures as alternatives to comprehensive measures. We examined whether two single-item fatigue measures (i.e., Likert scale, numeric rating scale) or a short fatigue measure were comparable to a comprehensive measure in reliability (i.e., internal consistency and test-retest reliability) and validity (i.e., convergent, concurrent, and predictive validity) in Korean young adults. For this quantitative study, we selected the Functional Assessment of Chronic Illness Therapy-Fatigue for the comprehensive measure and the Profile of Mood States-Brief, Fatigue subscale for the short measure; and constructed two single-item measures. A total of 368 students from four nursing colleges in South Korea participated. We used Cronbach's alpha and item-total correlation for internal consistency reliability and intraclass correlation coefficient for test-retest reliability. We assessed Pearson's correlation with a comprehensive measure for convergent validity, with perceived stress level and sleep quality for concurrent validity and the receiver operating characteristic curve for predictive validity. The short measure was comparable to the comprehensive measure in internal consistency reliability (Cronbach's alpha=0.81 vs. 0.88); test-retest reliability (intraclass correlation coefficient=0.66 vs. 0.61); convergent validity (r with comprehensive measure=0.79); concurrent validity (r with perceived stress=0.55, r with sleep quality=0.39) and predictive validity (area under curve=0.88). Single-item measures were not comparable to the comprehensive measure. A short fatigue measure exhibited similar levels of reliability and validity to the comprehensive measure in Korean young adults. Copyright © 2016 Elsevier Ltd. All rights reserved.

How Economists Use Literature and Drama.

ERIC Educational Resources Information Center

Watts, Michael

2002-01-01

Reviews how economists use passages, plots, characters, themes, and ideas from literature and drama in their professional writings. Explains that literary passages describe human behavior and motivations; serve as evidence of the characteristics of a particular time and place; validate economists predictions and understanding of rational behavior;…

ASME V\\&V challenge problem: Surrogate-based V&V

DOE Office of Scientific and Technical Information (OSTI.GOV)

Beghini, Lauren L.; Hough, Patricia D.

2015-12-18

The process of verification and validation can be resource intensive. From the computational model perspective, the resource demand typically arises from long simulation run times on multiple cores coupled with the need to characterize and propagate uncertainties. In addition, predictive computations performed for safety and reliability analyses have similar resource requirements. For this reason, there is a tradeoff between the time required to complete the requisite studies and the fidelity or accuracy of the results that can be obtained. At a high level, our approach is cast within a validation hierarchy that provides a framework in which we perform sensitivitymore » analysis, model calibration, model validation, and prediction. The evidence gathered as part of these activities is mapped into the Predictive Capability Maturity Model to assess credibility of the model used for the reliability predictions. With regard to specific technical aspects of our analysis, we employ surrogate-based methods, primarily based on polynomial chaos expansions and Gaussian processes, for model calibration, sensitivity analysis, and uncertainty quantification in order to reduce the number of simulations that must be done. The goal is to tip the tradeoff balance to improving accuracy without increasing the computational demands.« less

Active Inference and Learning in the Cerebellum.

PubMed

Friston, Karl; Herreros, Ivan

2016-09-01

This letter offers a computational account of Pavlovian conditioning in the cerebellum based on active inference and predictive coding. Using eyeblink conditioning as a canonical paradigm, we formulate a minimal generative model that can account for spontaneous blinking, startle responses, and (delay or trace) conditioning. We then establish the face validity of the model using simulated responses to unconditioned and conditioned stimuli to reproduce the sorts of behavior that are observed empirically. The scheme's anatomical validity is then addressed by associating variables in the predictive coding scheme with nuclei and neuronal populations to match the (extrinsic and intrinsic) connectivity of the cerebellar (eyeblink conditioning) system. Finally, we try to establish predictive validity by reproducing selective failures of delay conditioning, trace conditioning, and extinction using (simulated and reversible) focal lesions. Although rather metaphorical, the ensuing scheme can account for a remarkable range of anatomical and neurophysiological aspects of cerebellar circuitry-and the specificity of lesion-deficit mappings that have been established experimentally. From a computational perspective, this work shows how conditioning or learning can be formulated in terms of minimizing variational free energy (or maximizing Bayesian model evidence) using exactly the same principles that underlie predictive coding in perception.

ERP evidence for selective drop in attentional costs in uncertain environments: challenging a purely premotor account of covert orienting of attention.

PubMed

Lasaponara, Stefano; Chica, Ana B; Lecce, Francesca; Lupianez, Juan; Doricchi, Fabrizio

2011-07-01

Several studies have proved that the reliability of endogenous spatial cues linearly modulates the reaction time advantage in the processing of targets at validly cued vs. invalidly cued locations, i.e. the "validity effect". This would imply that with non-predictive cues, no "validity effect" should be observed. However, contrary to this prediction, one could hypothesize that attentional benefits by valid cuing (i.e. the RT advantage for validly vs. neutrally cued targets) can still be maintained with non-predictive cues, if the brain were endowed with mechanisms allowing the selective reduction in costs of reorienting from invalidly cued locations (i.e. the reduction of the RT disadvantage for invalidly vs. neutrally cued targets). This separated modulation of attentional benefits and costs would be adaptive in uncertain contexts where cues predict at chance level the location of targets. Through the joint recording of manual reaction times and event-related cerebral potentials (ERPs), we have found that this is the case and that relying on non-predictive endogenous cues results in abatement of attentional costs and the difference in the amplitude of the P1 brain responses evoked by invalidly vs. neutrally cued targets. In contrast, the use of non-predictive cues leaves unaffected attentional benefits and the difference in the amplitude of the N1 responses evoked by validly vs. neutrally cued targets. At the individual level, the drop in costs with non-predictive cues was matched with equivalent lateral biases in RTs to neutrally and invalidly cued targets presented in the left and right visual field. During the cue period, the drop in costs with non-predictive cues was preceded by reduction of the Early Directing Attention Negativity (EDAN) on posterior occipital sites and by enhancement of the frontal Anterior Directing Attention Negativity (ADAN) correlated to preparatory voluntary orienting. These findings demonstrate, for the first time, that the segregation of mechanisms regulating attentional benefits and costs helps efficiency of orienting in "uncertain" visual spatial contexts characterized by poor probabilistic association between cues and targets. Copyright © 2011 Elsevier Ltd. All rights reserved.

Development and validation of the Pediatric Anesthesia Behavior score--an objective measure of behavior during induction of anesthesia.

PubMed

Beringer, Richard M; Greenwood, Rosemary; Kilpatrick, Nicky

2014-02-01

Measuring perioperative behavior changes requires validated objective rating scales. We developed a simple score for children's behavior during induction of anesthesia (Pediatric Anesthesia Behavior score) and assessed its reliability, concurrent validity, and predictive validity. Data were collected as part of a wider observational study of perioperative behavior changes in children undergoing general anesthesia for elective dental extractions. One-hundred and two healthy children aged 2-12 were recruited. Previously validated behavioral scales were used as follows: the modified Yale Preoperative Anxiety Scale (m-YPAS); the induction compliance checklist (ICC); the Pediatric Anesthesia Emergence Delirium scale (PAED); and the Post-Hospitalization Behavior Questionnaire (PHBQ). Pediatric Anesthesia Behavior (PAB) score was independently measured by two investigators, to allow assessment of interobserver reliability. Concurrent validity was assessed by examining the correlation between the PAB score, the m-YPAS, and the ICC. Predictive validity was assessed by examining the association between the PAB score, the PAED scale, and the PHBQ. The PAB score correlated strongly with both the m-YPAS (P < 0.001) and the ICC (P < 0.001). PAB score was significantly associated with the PAED score (P = 0.031) and with the PHBQ (P = 0.034). Two independent investigators recorded identical PAB scores for 94% of children and overall, there was close agreement between scores (Kappa coefficient of 0.886 [P < 0.001]). The PAB score is simple to use and may predict which children are at increased risk of developing postoperative behavioral disturbance. This study provides evidence for its reliability and validity. © 2013 John Wiley & Sons Ltd.

Review of methods used by chiropractors to determine the site for applying manipulation

PubMed Central

2013-01-01

Background With the development of increasing evidence for the use of manipulation in the management of musculoskeletal conditions, there is growing interest in identifying the appropriate indications for care. Recently, attempts have been made to develop clinical prediction rules, however the validity of these clinical prediction rules remains unclear and their impact on care delivery has yet to be established. The current study was designed to evaluate the literature on the validity and reliability of the more common methods used by doctors of chiropractic to inform the choice of the site at which to apply spinal manipulation. Methods Structured searches were conducted in Medline, PubMed, CINAHL and ICL, supported by hand searches of archives, to identify studies of the diagnostic reliability and validity of common methods used to identify the site of treatment application. To be included, studies were to present original data from studies of human subjects and be designed to address the region or location of care delivery. Only English language manuscripts from peer-reviewed journals were included. The quality of evidence was ranked using QUADAS for validity and QAREL for reliability, as appropriate. Data were extracted and synthesized, and were evaluated in terms of strength of evidence and the degree to which the evidence was favourable for clinical use of the method under investigation. Results A total of 2594 titles were screened from which 201 articles met all inclusion criteria. The spectrum of manuscript quality was quite broad, as was the degree to which the evidence favoured clinical application of the diagnostic methods reviewed. The most convincing favourable evidence was for methods which confirmed or provoked pain at a specific spinal segmental level or region. There was also high quality evidence supporting the use, with limitations, of static and motion palpation, and measures of leg length inequality. Evidence of mixed quality supported the use, with limitations, of postural evaluation. The evidence was unclear on the applicability of measures of stiffness and the use of spinal x-rays. The evidence was of mixed quality, but unfavourable for the use of manual muscle testing, skin conductance, surface electromyography and skin temperature measurement. Conclusions A considerable range of methods is in use for determining where in the spine to administer spinal manipulation. The currently published evidence falls across a spectrum ranging from strongly favourable to strongly unfavourable in regard to using these methods. In general, the stronger and more favourable evidence is for those procedures which take a direct measure of the presumptive site of care– methods involving pain provocation upon palpation or localized tissue examination. Procedures which involve some indirect assessment for identifying the manipulable lesion of the spine–such as skin conductance or thermography–tend not to be supported by the available evidence. PMID:24499598

Word Memory Test Predicts Recovery in Claimants With Work-Related Head Injury.

PubMed

Colangelo, Annette; Abada, Abigail; Haws, Calvin; Park, Joanne; Niemeläinen, Riikka; Gross, Douglas P

2016-05-01

To investigate the predictive validity of the Word Memory Test (WMT), a verbal memory neuropsychological test developed as a performance validity measure to assess memory, effort, and performance consistency. Cohort study with 1-year follow-up. Workers' compensation rehabilitation facility. Participants included workers' compensation claimants with work-related head injury (N=188; mean age, 44y; 161 men [85.6%]). Not applicable. Outcome measures for determining predictive validity included days to suspension of wage replacement benefits during the 1-year follow-up and work status at discharge in claimants undergoing rehabilitation. Analysis included multivariable Cox and logistic regression. Better WMT performance was significantly but weakly correlated with younger age (r=-.30), documented brain abnormality (r=.28), and loss of consciousness at the time of injury (r=.25). Claimants with documented brain abnormalities on diagnostic imaging scans performed better (∼9%) on the WMT than those without brain abnormalities. The WMT predicted days receiving benefits (adjusted hazard ratio, 1.13; 95% confidence interval, 1.04-1.24) and work status outcome at program discharge (adjusted odds ratio, 1.62; 95% confidence interval, 1.13-2.34). Our results provide evidence for the predictive validity of the WMT in workers' compensation claimants. Younger claimants and those with more severe brain injuries performed better on the WMT. It may be that financial incentives or other factors related to the compensation claim affected the performance. Copyright © 2016 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.

Validation of the Spanish Version of the Mammography-Specific Self-Efficacy Scale.

PubMed

Jerome-D'Emilia, Bonnie; Suplee, Patricia; Akincigil, Ayse

2015-05-01

To consider psychometric estimates of the validity and reliability of the Spanish translation of a mammography-specific self-efficacy scale. A cross-sectional study. Three primarily Hispanic churches and a Hispanic community center in a low-income urban area of New Jersey. 153 low-income Hispanic women aged 40-85 years. The translated scale was administered to participants during a six-month period. Internal consistency, reliability, and construct and predictive validity were assessed. Demographic variables included income and insurance status. Outcome variables included total mammography-specific self-efficacy and having had a mammogram within the past two years. Preliminary evidence of reliability and validity were found, and predictive validity was demonstrated. The health needs of specific populations can be addressed only when research instruments have been appropriately validated and all relevant factors are considered. Diverse groups of low-income women face similar challenges and barriers in their efforts to get screened. Nurses are in an ideal position to help women with preventive care decision making (e.g., screening for breast cancer). Understanding how a woman's level of self-efficacy affects her decision making should be considered when counseling a client.

Reconsidering vocational interests for personnel selection: the validity of an interest-based selection test in relation to job knowledge, job performance, and continuance intentions.

PubMed

Van Iddekinge, Chad H; Putka, Dan J; Campbell, John P

2011-01-01

Although vocational interests have a long history in vocational psychology, they have received extremely limited attention within the recent personnel selection literature. We reconsider some widely held beliefs concerning the (low) validity of interests for predicting criteria important to selection researchers, and we review theory and empirical evidence that challenge such beliefs. We then describe the development and validation of an interests-based selection measure. Results of a large validation study (N = 418) reveal that interests predicted a diverse set of criteria—including measures of job knowledge, job performance, and continuance intentions—with corrected, cross-validated Rs that ranged from .25 to .46 across the criteria (mean R = .31). Interests also provided incremental validity beyond measures of general cognitive aptitude and facets of the Big Five personality dimensions in relation to each criterion. Furthermore, with a couple exceptions, the interest scales were associated with small to medium subgroup differences, which in most cases favored women and racial minorities. Taken as a whole, these results appear to call into question the prevailing thought that vocational interests have limited usefulness for selection.

Machine learning classification with confidence: application of transductive conformal predictors to MRI-based diagnostic and prognostic markers in depression.

PubMed

Nouretdinov, Ilia; Costafreda, Sergi G; Gammerman, Alexander; Chervonenkis, Alexey; Vovk, Vladimir; Vapnik, Vladimir; Fu, Cynthia H Y

2011-05-15

There is rapidly accumulating evidence that the application of machine learning classification to neuroimaging measurements may be valuable for the development of diagnostic and prognostic prediction tools in psychiatry. However, current methods do not produce a measure of the reliability of the predictions. Knowing the risk of the error associated with a given prediction is essential for the development of neuroimaging-based clinical tools. We propose a general probabilistic classification method to produce measures of confidence for magnetic resonance imaging (MRI) data. We describe the application of transductive conformal predictor (TCP) to MRI images. TCP generates the most likely prediction and a valid measure of confidence, as well as the set of all possible predictions for a given confidence level. We present the theoretical motivation for TCP, and we have applied TCP to structural and functional MRI data in patients and healthy controls to investigate diagnostic and prognostic prediction in depression. We verify that TCP predictions are as accurate as those obtained with more standard machine learning methods, such as support vector machine, while providing the additional benefit of a valid measure of confidence for each prediction. Copyright © 2010 Elsevier Inc. All rights reserved.

Reliability and Construct Validity of the Portuguese Version of the Psychological Capital Questionnaire.

PubMed

Antunes, Ana Cristina; Caetano, António; Pina E Cunha, Miguel

2017-06-01

The Psychological Capital Questionnaire (PCQ) is the most commonly used measure for assessing psychological capital in work settings. Although several studies confirmed its factorial validity, most validation studies only examined the four-factor structure preconized by Luthans, Youssef, and Avolio, not attending to empirical evidence on alternative factorial structures. The present study aimed to test the psychometric properties of the Portuguese version of the PCQ, by using two independent samples (NS1 = 542; NS2 = 115) of Portuguese employees. We conducted a series of confirmatory factor analyses and found that, unlike previous findings, a five-factor solution of the PCQ best fitted the data. The evidence obtained also supported the existence of a second-order factor, psychological capital. The coefficients of internal consistency, as measured by Cronbach's alpha, were adequate and test-retest reliability suggested that the PCQ presented a lower stability than personality factors. Convergent validity, assessed with average variance extracted, revealed problems in the optimism subscale. The discriminant validity of the PCQ was confirmed by its correlations with Positive and Negative Affect and Big Five personality factors. Hierarchical regression analyses showed that this measure has incremental validity over personality and affect when predicting job performance.

Phonological and Non-Phonological Language Skills as Predictors of Early Reading Performance

ERIC Educational Resources Information Center

Batson-Magnuson, LuAnn

2010-01-01

Accurate prediction of early childhood reading performance could help identify at-risk students, aid in the development of evidence-based intervention strategies, and further our theoretical understanding of reading development. This study assessed the validity of the Developmental Indicator for the Assessment of Learning (DIAL) language-based…

How Students Create Motivationally Supportive Learning Environments for Themselves: The Concept of Agentic Engagement

ERIC Educational Resources Information Center

Reeve, Johnmarshall

2013-01-01

The present study introduced "agentic engagement" as a newly proposed student-initiated pathway to greater achievement and greater motivational support. Study 1 developed the brief, construct-congruent, and psychometrically strong Agentic Engagement Scale. Study 2 provided evidence for the scale's construct and predictive validity, as…

Design and Validation of Assessment Tests for Young Children in Zambia

ERIC Educational Resources Information Center

Matafwali, Beatrice; Serpell, Robert

2014-01-01

Early childhood education has received unprecedented attention among African policymakers in recent years, recognizing that the early years form an important foundation upon which later development is anchored and noting evidence that various Early Childhood Development (ECD) indicators are predictive of future academic success. Central to the…

Measures of Hindu Pathways: Development and Preliminary Evidence of Reliability and Validity.

ERIC Educational Resources Information Center

Tarakeshwar, Nalini; Pargament, Kenneth I.; Mahoney, Annette

2003-01-01

Examines religious practices of Hindus in the United States and develops measures of their religious pathways. Four religious pathways were identified: devotion, ethical action, knowledge, and physical restraint/yoga. Results indicate that the measures of the religious pathways possessed adequate psychometric properties and were predictive of…

Cognitive Integrity Predicts Transitive Inference Performance Bias and Success

ERIC Educational Resources Information Center

Moses, Sandra N.; Villate, Christina; Binns, Malcolm A.; Davidson, Patrick S. R.; Ryan, Jennifer D.

2008-01-01

Transitive inference has traditionally been regarded as a relational proposition-based reasoning task, however, recent investigations question the validity of this assumption. Although some results support the use of a relational proposition-based approach, other studies find evidence for the use of associative learning. We examined whether…

Prognostic Biomarkers Used for Localised Prostate Cancer Management: A Systematic Review.

PubMed

Lamy, Pierre-Jean; Allory, Yves; Gauchez, Anne-Sophie; Asselain, Bernard; Beuzeboc, Philippe; de Cremoux, Patricia; Fontugne, Jacqueline; Georges, Agnès; Hennequin, Christophe; Lehmann-Che, Jacqueline; Massard, Christophe; Millet, Ingrid; Murez, Thibaut; Schlageter, Marie-Hélène; Rouvière, Olivier; Kassab-Chahmi, Diana; Rozet, François; Descotes, Jean-Luc; Rébillard, Xavier

2017-03-07

Prostate cancer stratification is based on tumour size, pretreatment PSA level, and Gleason score, but it remains imperfect. Current research focuses on the discovery and validation of novel prognostic biomarkers to improve the identification of patients at risk of aggressive cancer or of tumour relapse. This systematic review by the Intergroupe Coopérateur Francophone de Recherche en Onco-urologie (ICFuro) analysed new evidence on the analytical validity and clinical validity and utility of six prognostic biomarkers (PHI, 4Kscore, MiPS, GPS, Prolaris, Decipher). All available data for the six biomarkers published between January 2002 and April 2015 were systematically searched and reviewed. The main endpoints were aggressive prostate cancer prediction, additional value compared to classical prognostic parameters, and clinical benefit for patients with localised prostate cancer. The preanalytical and analytical validations were heterogeneous for all tests and often not adequate for the molecular signatures. Each biomarker was studied for specific indications (candidates for a first or second biopsy, and potential candidates for active surveillance, radical prostatectomy, or adjuvant treatment) for which the level of evidence (LOE) was variable. PHI and 4Kscore were the biomarkers with the highest LOE for discriminating aggressive and indolent tumours in different indications. Blood biomarkers (PHI and 4Kscore) have the highest LOE for the prediction of more aggressive prostate cancer and could help clinicians to manage patients with localised prostate cancer. The other biomarkers show a potential prognostic value; however, they should be evaluated in additional studies to confirm their clinical validity. We reviewed studies assessing the value of six prognostic biomarkers for prostate cancer. On the basis of the available evidence, some biomarkers could help in discriminating between aggressive and non-aggressive tumours with an additional value compared to the prognostic parameters currently used by clinicians. Copyright © 2017 European Association of Urology. Published by Elsevier B.V. All rights reserved.

Predicting Persistent Back Symptoms by Psychosocial Risk Factors: Validity Criteria for the ÖMPSQ and the HKF-R 10 in Germany.

PubMed

Riewe, E; Neubauer, E; Pfeifer, A C; Schiltenwolf, M

2016-01-01

10% of all individuals in Germany develop persistent symptoms due to nonspecific back pain (NSBP) causing up to 90% of direct and indirect expenses for health care systems. Evidence indicates a strong relationship between chronic nonspecific back pain and psychosocial risk factors. The Örebro Musculoskeletal Pain Screening Questionnaire (ÖMPSQ) and the German Heidelberger Kurzfragebogen Rückenschmerz (HKF-R 10) are deemed valid in prediction of persistent pain, functional loss or amount of sick leave. This study provides and discusses validity criteria for these questionnaires using ROC-curve analyses. Quality measurements included sensitivity and specificity, likelihood-ratio related test-efficiencies and clinical utility in regard to predictive values. 265 patients recruited from primary and secondary care units completed both questionnaires during the same timeframe. From the total, 133 patients returned a 6-month follow-up questionnaire to assess the validity criteria for outcomes of pain, function and sick leave. Based on heterogeneous cut-offs for the ÖMPSQ, sensitivity and specificity were moderate for outcome of pain (72%/75%). Very high sensitivity was observed for function (97%/57%) and high specificity for sick leave (63%/85%). The latter also applied to the HKF-R 10 (pain 50%/84%). Proportions between sensitivity and specificity were unbalanced except for the ÖMPSQ outcome of pain. Likelihood-ratios and positive predictive values ranged from low to moderate. Although the ÖMPSQ may be considered useful in identification of long-term functional loss or pain, over- and underestimation of patients at risk of chronic noncspecific back pain led to limited test-efficiencies and clinical utility for both questionnaires. Further studies are required to quantify the predictive validity of both questionnaires in Germany.

Are loss of control while eating and overeating valid constructs? A critical review of the literature

PubMed Central

Goldschmidt, Andrea B.

2017-01-01

Background Binge eating is a marker of weight gain and obesity, and a hallmark feature of eating disorders. Yet, its component constructs—overeating and loss of control (LOC) while eating—are poorly understood and difficult to measure. Objective To critically review the human literature concerning the validity of LOC and overeating across the age and weight spectrum. Data sources English-language articles addressing the face, convergent, discriminant, and predictive validity of LOC and overeating were included. Results LOC and overeating appear to have adequate face validity. Emerging evidence supports the convergent and predictive validity of the LOC construct, given its unique cross-sectional and prospective associations with numerous anthropometric, psychosocial, and eating behavior-related factors. Overeating may be best conceptualized as a marker of excess weight status. Limitations Binge eating constructs, particularly in the context of subjectively large episodes, are challenging to measure reliably. Few studies addressed overeating in the absence of LOC, thereby limiting conclusions about the validity of the overeating construct independent of LOC. Additional studies addressing the discriminant validity of both constructs are warranted. Discussion Suggestions for future weight-related research and for appropriately defining binge eating in the eating disorders diagnostic scheme are presented. PMID:28165655

Evaluation of polygenic risk scores for predicting breast and prostate cancer risk.

PubMed

Machiela, Mitchell J; Chen, Chia-Yen; Chen, Constance; Chanock, Stephen J; Hunter, David J; Kraft, Peter

2011-09-01

Recently, polygenic risk scores (PRS) have been shown to be associated with certain complex diseases. The approach has been based on the contribution of counting multiple alleles associated with disease across independent loci, without requiring compelling evidence that every locus had already achieved definitive genome-wide statistical significance. Whether PRS assist in the prediction of risk of common cancers is unknown. We built PRS from lists of genetic markers prioritized by their association with breast cancer (BCa) or prostate cancer (PCa) in a training data set and evaluated whether these scores could improve current genetic prediction of these specific cancers in independent test samples. We used genome-wide association data on 1,145 BCa cases and 1,142 controls from the Nurses' Health Study and 1,164 PCa cases and 1,113 controls from the Prostate Lung Colorectal and Ovarian Cancer Screening Trial. Ten-fold cross validation was used to build and evaluate PRS with 10 to 60,000 independent single nucleotide polymorphisms (SNPs). For both BCa and PCa, the models that included only published risk alleles maximized the cross-validation estimate of the area under the ROC curve (0.53 for breast and 0.57 for prostate). We found no significant evidence that PRS using common variants improved risk prediction for BCa and PCa over replicated SNP scores. © 2011 Wiley-Liss, Inc.

Predictive Validity of Delay Discounting Behavior in Adolescence: A Longitudinal Twin Study

PubMed Central

Isen, Joshua D.; Sparks, Jordan C.; Iacono, William G.

2014-01-01

A standard assumption in the delay discounting literature is that individuals who exhibit steeper discounting of hypothetical rewards also experience greater difficulty deferring gratification to real-world rewards. There is ample cross-sectional evidence that delay discounting paradigms reflect a variety of maladaptive psychosocial outcomes, including substance use pathology. We sought to determine whether a computerized assessment of hypothetical delay discounting (HDD) taps into behavioral impulsivity in a community sample of adolescent twins (N = 675). Using a longitudinal design, we hypothesized that greater HDD at age 14–15 predicts real-world impulsive choices and risk for substance use disorders in late adolescence. We also examined the genetic and environmental structure of HDD performance. Individual differences in HDD behavior showed moderate heritability, and were prospectively associated with real-world temporal discounting at age 17–18. Contrary to expectations, HDD was not consistently related to substance use or trait impulsivity. Although a significant association between HDD behavior and past substance use emerged in males, this effect was mediated by cognitive ability. In both sexes, HDD failed to predict a comprehensive index of substance use problems and behavioral disinhibition in late adolescence. In sum, we present some of the first evidence that HDD performance is heritable and predictive of real-world temporal discounting of rewards. Nevertheless, HDD might not serve as a valid marker of substance use disorder risk in younger adolescents, particularly females. PMID:24999868

Musical Preferences Predict Personality: Evidence From Active Listening and Facebook Likes.

PubMed

Nave, Gideon; Minxha, Juri; Greenberg, David M; Kosinski, Michal; Stillwell, David; Rentfrow, Jason

2018-03-01

Research over the past decade has shown that various personality traits are communicated through musical preferences. One limitation of that research is external validity, as most studies have assessed individual differences in musical preferences using self-reports of music-genre preferences. Are personality traits communicated through behavioral manifestations of musical preferences? We addressed this question in two large-scale online studies with demographically diverse populations. Study 1 ( N = 22,252) shows that reactions to unfamiliar musical excerpts predicted individual differences in personality-most notably, openness and extraversion-above and beyond demographic characteristics. Moreover, these personality traits were differentially associated with particular music-preference dimensions. The results from Study 2 ( N = 21,929) replicated and extended these findings by showing that an active measure of naturally occurring behavior, Facebook Likes for musical artists, also predicted individual differences in personality. In general, our findings establish the robustness and external validity of the links between musical preferences and personality.

Psychometric properties and the predictive validity of the insomnia daytime worry scale: a pilot study.

PubMed

Kallestad, Håvard; Hansen, Bjarne; Langsrud, Knut; Hjemdal, Odin; Stiles, Tore C

2010-01-01

The relationship between presleep worry and insomnia has been investigated in previous studies, but less attention has been given to the role of daytime worry and symptoms of insomnia. The aims of the current study were (a) to assess the psychometric properties of a novel scale measuring insomnia-specific worry during daytime and (b) to examine whether levels of daytime worry predict severity of insomnia symptoms. Participants (N = 353) completed the Insomnia Daytime Worry Scale (IDWS) and the Insomnia Severity Index. An explorative principal-axis factor analysis extracted two factors from the IDWS, accounting for 70.5% of the variance. The IDWS demonstrated good reliability. The total score of IDWS and both factors predicted levels of insomnia severity in two separate hierarchical regression analyses. This preliminary evidence suggests that the IDWS is a valid and reliable scale to measure daytime worry in insomnia.

Using Dynamic Assessment to Evaluate the Expressive Syntax of Children who use Augmentative and Alternative Communication

PubMed Central

King, Marika R.; Binger, Cathy; Kent-Walsh, Jennifer

2015-01-01

The developmental readiness of four 5-year-old children to produce basic sentences using graphic symbols on an augmentative and alternative communication (AAC) device during a dynamic assessment (DA) task was examined. Additionally, the ability of the DA task to predict performance on a subsequent experimental task was evaluated. A graduated prompting framework was used during DA. Measures included amount of support required to produce the targets, modifiability (change in participant performance) within a DA session, and predictive validity of DA. Participants accurately produced target structures with varying amounts of support. Modifiability within DA sessions was evident for some participants, and partial support was provided for the measures of predictive validity. These initial results indicate that DA may be a viable way to measure young children’s developmental readiness to learn how to sequence simple, rule-based messages via aided AAC. PMID:25621928

An evidence-based decision assistance model for predicting training outcome in juvenile guide dogs.

PubMed

Harvey, Naomi D; Craigon, Peter J; Blythe, Simon A; England, Gary C W; Asher, Lucy

2017-01-01

Working dog organisations, such as Guide Dogs, need to regularly assess the behaviour of the dogs they train. In this study we developed a questionnaire-style behaviour assessment completed by training supervisors of juvenile guide dogs aged 5, 8 and 12 months old (n = 1,401), and evaluated aspects of its reliability and validity. Specifically, internal reliability, temporal consistency, construct validity, predictive criterion validity (comparing against later training outcome) and concurrent criterion validity (comparing against a standardised behaviour test) were evaluated. Thirty-nine questions were sourced either from previously published literature or created to meet requirements identified via Guide Dogs staff surveys and staff feedback. Internal reliability analyses revealed seven reliable and interpretable trait scales named according to the questions within them as: Adaptability; Body Sensitivity; Distractibility; Excitability; General Anxiety; Trainability and Stair Anxiety. Intra-individual temporal consistency of the scale scores between 5-8, 8-12 and 5-12 months was high. All scales excepting Body Sensitivity showed some degree of concurrent criterion validity. Predictive criterion validity was supported for all seven scales, since associations were found with training outcome, at at-least one age. Thresholds of z-scores on the scales were identified that were able to distinguish later training outcome by identifying 8.4% of all dogs withdrawn for behaviour and 8.5% of all qualified dogs, with 84% and 85% specificity. The questionnaire assessment was reliable and could detect traits that are consistent within individuals over time, despite juvenile dogs undergoing development during the study period. By applying thresholds to scores produced from the questionnaire this assessment could prove to be a highly valuable decision-making tool for Guide Dogs. This is the first questionnaire-style assessment of juvenile dogs that has shown value in predicting the training outcome of individual working dogs.

Effect of response format on cognitive reflection: Validating a two- and four-option multiple choice question version of the Cognitive Reflection Test.

PubMed

Sirota, Miroslav; Juanchich, Marie

2018-03-27

The Cognitive Reflection Test, measuring intuition inhibition and cognitive reflection, has become extremely popular because it reliably predicts reasoning performance, decision-making, and beliefs. Across studies, the response format of CRT items sometimes differs, based on the assumed construct equivalence of tests with open-ended versus multiple-choice items (the equivalence hypothesis). Evidence and theoretical reasons, however, suggest that the cognitive processes measured by these response formats and their associated performances might differ (the nonequivalence hypothesis). We tested the two hypotheses experimentally by assessing the performance in tests with different response formats and by comparing their predictive and construct validity. In a between-subjects experiment (n = 452), participants answered stem-equivalent CRT items in an open-ended, a two-option, or a four-option response format and then completed tasks on belief bias, denominator neglect, and paranormal beliefs (benchmark indicators of predictive validity), as well as on actively open-minded thinking and numeracy (benchmark indicators of construct validity). We found no significant differences between the three response formats in the numbers of correct responses, the numbers of intuitive responses (with the exception of the two-option version, which had a higher number than the other tests), and the correlational patterns of the indicators of predictive and construct validity. All three test versions were similarly reliable, but the multiple-choice formats were completed more quickly. We speculate that the specific nature of the CRT items helps build construct equivalence among the different response formats. We recommend using the validated multiple-choice version of the CRT presented here, particularly the four-option CRT, for practical and methodological reasons. Supplementary materials and data are available at https://osf.io/mzhyc/ .

Translation and validation of the Canadian diabetes risk assessment questionnaire in China.

PubMed

Guo, Jia; Shi, Zhengkun; Chen, Jyu-Lin; Dixon, Jane K; Wiley, James; Parry, Monica

2018-01-01

To adapt the Canadian Diabetes Risk Assessment Questionnaire for the Chinese population and to evaluate its psychometric properties. A cross-sectional study was conducted with a convenience sample of 194 individuals aged 35-74 years from October 2014 to April 2015. The Canadian Diabetes Risk Assessment Questionnaire was adapted and translated for the Chinese population. Test-retest reliability was conducted to measure stability. Criterion and convergent validity of the adapted questionnaire were assessed using 2-hr 75 g oral glucose tolerance tests and the Finnish Diabetes Risk Scores, respectively. Sensitivity and specificity were evaluated to establish its predictive validity. The test-retest reliability was 0.988. Adequate validity of the adapted questionnaire was demonstrated by positive correlations found between the scores and 2-hr 75 g oral glucose tolerance tests (r = .343, p < .001) and with the Finnish Diabetes Risk Scores (r = .738, p < .001). The area under receiver operating characteristic curve was 0.705 (95% CI .632, .778), demonstrating moderate diagnostic value at a cutoff score of 30. The sensitivity was 73%, with a positive predictive value of 57% and negative predictive value of 78%. Our results provided evidence supporting the translation consistency, content validity, convergent validity, criterion validity, sensitivity, and specificity of the translated Canadian Diabetes Risk Assessment Questionnaire with minor modifications. This paper provides clinical, practical, and methodological information on how to adapt a diabetes risk calculator between cultures for public health nurses. © 2017 Wiley Periodicals, Inc.

Critical validation studies of neurofeedback.

PubMed

Gruzelier, John; Egner, Tobias

2005-01-01

The field of neurofeedback training has proceeded largely without validation. In this article the authors review studies directed at validating sensory motor rhythm, beta and alpha-theta protocols for improving attention, memory, and music performance in healthy participants. Importantly, benefits were demonstrable with cognitive and neurophysiologic measures that were predicted on the basis of regression models of learning to enhance sensory motor rhythm and beta activity. The first evidence of operant control over the alpha-theta ratio is provided, together with remarkable improvements in artistic aspects of music performance equivalent to two class grades in conservatory students. These are initial steps in providing a much needed scientific basis to neurofeedback.

miRWalk--database: prediction of possible miRNA binding sites by "walking" the genes of three genomes.

PubMed

Dweep, Harsh; Sticht, Carsten; Pandey, Priyanka; Gretz, Norbert

2011-10-01

MicroRNAs are small, non-coding RNA molecules that can complementarily bind to the mRNA 3'-UTR region to regulate the gene expression by transcriptional repression or induction of mRNA degradation. Increasing evidence suggests a new mechanism by which miRNAs may regulate target gene expression by binding in promoter and amino acid coding regions. Most of the existing databases on miRNAs are restricted to mRNA 3'-UTR region. To address this issue, we present miRWalk, a comprehensive database on miRNAs, which hosts predicted as well as validated miRNA binding sites, information on all known genes of human, mouse and rat. All mRNAs, mitochondrial genes and 10 kb upstream flanking regions of all known genes of human, mouse and rat were analyzed by using a newly developed algorithm named 'miRWalk' as well as with eight already established programs for putative miRNA binding sites. An automated and extensive text-mining search was performed on PubMed database to extract validated information on miRNAs. Combined information was put into a MySQL database. miRWalk presents predicted and validated information on miRNA-target interaction. Such a resource enables researchers to validate new targets of miRNA not only on 3'-UTR, but also on the other regions of all known genes. The 'Validated Target module' is updated every month and the 'Predicted Target module' is updated every 6 months. miRWalk is freely available at http://mirwalk.uni-hd.de/. Copyright © 2011 Elsevier Inc. All rights reserved.

Relapse Risk Assessment for Schizophrenia Patients (RASP): A New Self-Report Screening Tool.

PubMed

Velligan, Dawn; Carpenter, William; Waters, Heidi C; Gerlanc, Nicole M; Legacy, Susan N; Ruetsch, Charles

2018-01-01

The Relapse Assessment for Schizophrenia Patients (RASP) was developed as a six-question self-report screener that measures indicators of Increased Anxiety and Social Isolation to assess patient stability and predict imminent relapse. This paper describes the development and psychometric characteristics of the RASP. The RASP and Positive and Negative Syndrome Scale (PANSS) were administered to patients with schizophrenia (n=166) three separate times. Chart data were collected on a subsample of patients (n=81). Psychometric analyses of RASP included tests of reliability, construct validity, and concurrent validity of items. Factors from RASP were correlated with subscales from PANSS (sensitivity to change and criterion validity [agreement between RASP and evidence of relapse]). Test-retest reliability returned modest to strong agreement at the item level and strong agreement at the questionnaire level. RASP showed good item response curves and internal consistency for the total instrument and within each of the two subscales (Increased Anxiety and Social Isolation). RASP Total Score and subscales showed good concurrent validity when correlated with PANSS Total Score, Positive, Excitement, and Anxiety subscales. RASP correctly predicted relapse in 67% of cases, with good specificity and negative predictive power and acceptable positive predictive power and sensitivity. The reliability and validity data presented support the use of RASP in settings where addition of a brief self-report assessment of relapse risk among patients with schizophrenia may be of benefit. Ease of use and scoring, and the ability to administer without clinical supervision allows for routine administration and assessment of relapse risk.

Clinical evidence supporting pharmacogenomic biomarker testing provided in US Food and Drug Administration drug labels.

PubMed

Wang, Bo; Canestaro, William J; Choudhry, Niteesh K

2014-12-01

Genetic biomarkers that predict a drug's efficacy or likelihood of toxicity are assuming increasingly important roles in the personalization of pharmacotherapy, but concern exists that evidence that links use of some biomarkers to clinical benefit is insufficient. Nevertheless, information about the use of biomarkers appears in the labels of many prescription drugs, which may add confusion to the clinical decision-making process. To evaluate the evidence that supports pharmacogenomic biomarker testing in drug labels and how frequently testing is recommended. Publicly available US Food and Drug Administration databases. We identified drug labels that described the use of a biomarker and evaluated whether the label contained or referenced convincing evidence of its clinical validity (ie, the ability to predict phenotype) and clinical utility (ie, the ability to improve clinical outcomes) using guidelines published by the Evaluation of Genomic Applications in Practice and Prevention Working Group. We graded the completeness of the citation of supporting studies and determined whether the label recommended incorporation of biomarker test results in therapeutic decision making. Of the 119 drug-biomarker combinations, only 43 (36.1%) had labels that provided convincing clinical validity evidence, whereas 18 (15.1%) provided convincing evidence of clinical utility. Sixty-one labels (51.3%) made recommendations about how clinical decisions should be based on the results of a biomarker test; 36 (30.3%) of these contained convincing clinical utility data. A full description of supporting studies was included in 13 labels (10.9%). Fewer than one-sixth of drug labels contained or referenced convincing evidence of clinical utility of biomarker testing, whereas more than half made recommendations based on biomarker test results. It may be premature to include biomarker testing recommendations in drug labels when convincing data that link testing to patient outcomes do not exist.

The evolving role of physiotherapists in pre-employment screening for workplace injury prevention: are functional capacity evaluations the answer?

PubMed Central

Legge, Jennifer

2013-01-01

Background Musculoskeletal injuries account for the largest proportion of workplace injuries. In an attempt to predict, and subsequently manage, the risk of sprains and strains in the workplace, employers are turning to pre-employment screening. Functional capacity evaluations (FCEs) are increasing in popularity as a tool for pre-employment screening despite limited published evidence for their validity in healthy working populations. Objectives This narrative review will present an overview of the state of the evidence for pre-employment functional testing, propose a framework for decision-making to determine the suitability of assessment tools, and discuss the role and potential ethical challenges for physiotherapists conducting pre-employment functional testing. Major Findings Much of the evidence surrounding the validity of functional testing is in the context of the injured worker and prediction of return to work. In healthy populations, FCE components, such as aerobic fitness and manual handling activities, have demonstrated predictability of workplace injury in a small number of studies. This predictability improves when workers' performance is compared with the job demands. This job-specific approach is also required to meet anti-discrimination requirements. There are a number of practical limitations to functional testing, although these are not limited to the pre-employment domain. Physiotherapists need to have a clear understanding of the legal requirements and potential ethical challenges that they may face when conducting pre-employment functional assessments (PEFAs). Conclusions Further research is needed into the efficacy of pre-employment testing for workplace injury prevention. Physiotherapists and PEFAs are just one part of a holistic approach to workplace injury prevention. PMID:24124346

Predictive validity of cannabis consumption measures: Results from a national longitudinal study.

PubMed

Buu, Anne; Hu, Yi-Han; Pampati, Sanjana; Arterberry, Brooke J; Lin, Hsien-Chang

2017-10-01

Validating the utility of cannabis consumption measures for predicting later cannabis related symptomatology or progression to cannabis use disorder (CUD) is crucial for prevention and intervention work that may use consumption measures for quick screening. This study examined whether cannabis use quantity and frequency predicted CUD symptom counts, progression to onset of CUD, and persistence of CUD. Data from the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) at Wave 1 (2001-2002) and Wave 2 (2004-2005) were used to identify three risk samples: (1) current cannabis users at Wave 1 who were at risk for having CUD symptoms at Wave 2; (2) current users without lifetime CUD who were at risk for incident CUD; and (3) current users with past-year CUD who were at risk for persistent CUD. Logistic regression and zero-inflated Poisson models were used to examine the longitudinal effect of cannabis consumption on CUD outcomes. Higher frequency of cannabis use predicted lower likelihood of being symptom-free but it did not predict the severity of CUD symptomatology. Higher frequency of cannabis use also predicted higher likelihood of progression to onset of CUD and persistence of CUD. Cannabis use quantity, however, did not predict any of the developmental stages of CUD symptomatology examined in this study. This study has provided a new piece of evidence to support the predictive validity of cannabis use frequency based on national longitudinal data. The result supports the common practice of including frequency items in cannabis screening tools. Copyright © 2017 Elsevier Ltd. All rights reserved.

LRSSLMDA: Laplacian Regularized Sparse Subspace Learning for MiRNA-Disease Association prediction

PubMed Central

Huang, Li

2017-01-01

Predicting novel microRNA (miRNA)-disease associations is clinically significant due to miRNAs’ potential roles of diagnostic biomarkers and therapeutic targets for various human diseases. Previous studies have demonstrated the viability of utilizing different types of biological data to computationally infer new disease-related miRNAs. Yet researchers face the challenge of how to effectively integrate diverse datasets and make reliable predictions. In this study, we presented a computational model named Laplacian Regularized Sparse Subspace Learning for MiRNA-Disease Association prediction (LRSSLMDA), which projected miRNAs/diseases’ statistical feature profile and graph theoretical feature profile to a common subspace. It used Laplacian regularization to preserve the local structures of the training data and a L1-norm constraint to select important miRNA/disease features for prediction. The strength of dimensionality reduction enabled the model to be easily extended to much higher dimensional datasets than those exploited in this study. Experimental results showed that LRSSLMDA outperformed ten previous models: the AUC of 0.9178 in global leave-one-out cross validation (LOOCV) and the AUC of 0.8418 in local LOOCV indicated the model’s superior prediction accuracy; and the average AUC of 0.9181+/-0.0004 in 5-fold cross validation justified its accuracy and stability. In addition, three types of case studies further demonstrated its predictive power. Potential miRNAs related to Colon Neoplasms, Lymphoma, Kidney Neoplasms, Esophageal Neoplasms and Breast Neoplasms were predicted by LRSSLMDA. Respectively, 98%, 88%, 96%, 98% and 98% out of the top 50 predictions were validated by experimental evidences. Therefore, we conclude that LRSSLMDA would be a valuable computational tool for miRNA-disease association prediction. PMID:29253885

The Development of English and Mathematics Self-Efficacy: A Latent Growth Curve Analysis

ERIC Educational Resources Information Center

Phan, Huy P.

2012-01-01

Empirical research has provided evidence supporting the validation and prediction of 4 major sources of self-efficacy: enactive performance accomplishments, vicarious experiences, verbal persuasion, and emotional states. Other research studies have also attested to the importance and potency of self-efficacy in academic learning and achievement.…

Screen Twice, Cut Once: Assessing the Predictive Validity of Applicant Selection Tools

ERIC Educational Resources Information Center

Goldhaber, Dan; Grout, Cyrus; Huntington-Klein, Nick

2017-01-01

Despite their widespread use, there is little academic evidence on whether applicant selection instruments can improve teacher hiring. We examine the relationship between two screening instruments used by Spokane Public Schools to select classroom teachers and three teacher outcomes: value added, absences, and attrition. We observe all applicants…

Examining Readability Estimates' Predictions of Students' Oral Reading Rate: Spache, Lexile, and Forcast

ERIC Educational Resources Information Center

Ardoin, Scott P.; Williams, Jessica C.; Christ, Theodore J.; Klubnik, Cynthia; Wellborn, Claire

2010-01-01

Beyond reliability and validity, measures used to model student growth must consist of multiple probes that are equivalent in level of difficulty to establish consistent measurement conditions across time. Although existing evidence supports the reliability of curriculum-based measurement in reading (CBMR), few studies have empirically evaluated…

The Engaged Learning Index: Implications for Faculty Development

ERIC Educational Resources Information Center

Schreiner, L. A.; Louis, M. C.

2011-01-01

The Engaged Learning Index was designed by the authors to assess students' emotional, intellectual, and behavioral engagement in learning in a brief format that can be used in single courses or across multiple courses and institutions. In this study, evidence is presented for the reliability and predictive validity of the instrument based on a…

Does Response Variability Predict Distractibility among Adults with Attention-Deficit/Hyperactivity Disorder?

ERIC Educational Resources Information Center

Adams, Zachary W.; Roberts, Walter M.; Milich, Richard; Fillmore, Mark T.

2011-01-01

Increased intraindividual variability in response time (RTSD) has been observed reliably in attention-deficit/hyperactivity disorder (ADHD) and has often been used as a measure of inattention. RTSD is assumed to reflect attentional lapses and distractibility, though evidence for the validity of this connection is lacking. We assessed whether RTSD…

Social Support, Network Structure, and the Inventory of Socially Supportive Behaviors.

ERIC Educational Resources Information Center

Stokes, Joseph P.; Wilson, Diane Grimard

The Inventory of Socially Supportive Behaviors (ISSB) appears to be a satisfactory measure of social support with good reliability and some evidence of validity. To investigate the dimensionality of the ISSB through factor analytic procedures and to predict social support from social network variables, 179 college students (97 male, 82 female)…

Reliability and concurrent validity of the computer workstation checklist.

PubMed

Baker, Nancy A; Livengood, Heather; Jacobs, Karen

2013-01-01

Self-report checklists are used to assess computer workstation set up, typically by workers not trained in ergonomic assessment or checklist interpretation.Though many checklists exist, few have been evaluated for reliability and validity. This study examined reliability and validity of the Computer Workstation Checklist (CWC) to identify mismatches between workers' self-reported workstation problems. The CWC was completed at baseline and at 1 month to establish reliability. Validity was determined with CWC baseline data compared to an onsite workstation evaluation conducted by an expert in computer workstation assessment. Reliability ranged from fair to near perfect (prevalence-adjusted bias-adjusted kappa, 0.38-0.93); items with the strongest agreement were related to the input device, monitor, computer table, and document holder. The CWC had greater specificity (11 of 16 items) than sensitivity (3 of 16 items). The positive predictive value was greater than the negative predictive value for all questions. The CWC has strong reliability. Sensitivity and specificity suggested workers often indicated no problems with workstation setup when problems existed. The evidence suggests that while the CWC may not be valid when used alone, it may be a suitable adjunct to an ergonomic assessment completed by professionals.

Measuring attitudes towards suicide: Preliminary evaluation of an attitude towards suicide scale.

PubMed

Cwik, Jan Christopher; Till, Benedikt; Bieda, Angela; Blackwell, Simon E; Walter, Carolin; Teismann, Tobias

2017-01-01

Our study aimed to validate a previously published scale assessing attitudes towards suicide. Factor structure, convergent and discriminant validity, and predictive validity were investigated. Adult German participants (N=503; mean age=24.74years; age range=18-67years) anonymously completed a set of questionnaires. An exploratory factor analysis was conducted, and incongruous items were deleted. Subsequently, scale properties of the reduced scale and its construct validity were analyzed. A confirmatory factor analysis was then conducted in an independent sample (N=266; mean age=28.77years; age range=18-88years) to further confirm the factor structure of the questionnaire. Parallel analysis indicated a three-factor solution, which was also supported by confirmatory factor analysis: right to commit suicide, interpersonal gesture and resilience. The subscales demonstrated acceptable construct and discriminant validity. Cronbach's α for the subscales ranged from 0.67 to 0.83, explaining 49.70% of the total variance. Positive attitudes towards suicide proved to be predictive of suicide risk status, providing preliminary evidence for the utility of the scale. Future studies aiming to reproduce the factor structure in a more heterogeneous sample are warranted. Copyright © 2016 Elsevier Inc. All rights reserved.

Development of the Affordances in the Home Environment for Motor Development-Infant Scale.

PubMed

Caçola, Priscila; Gabbard, Carl; Santos, Denise C C; Batistela, Ana Carolina T

2011-12-01

The present study reports the development and application of the Affordances in the Home Environment for Motor Development-Infant Scale (AHEMD-IS), a parental self-report designed to assess the quantity and quality of affordances in the home environment that are conducive to motor development for infants aged 3-18 months. Steps in its development included use of expert feedback, establishment of construct validity, interrater and intrarater reliability, and predictive validity. With all phases of the project, 113 homes were involved. Intraclass correlation coefficients for interrater and intrarater reliability for the total score were 1 and 0.94, respectively. In addition, results indicate that the test has the characteristic of differentiating a wide range of scores. Regression analysis for the AHEMD-IS and motor development using the Alberta Infant Motor Scale supports preliminary evidence for predictive validity. Our findings suggest that the AHEMD-IS has sufficient reliability and validity as an instrument for assessing affordances in the home environment, with clinical and research applications. © 2011 The Authors. Pediatrics International © 2011 Japan Pediatric Society.

The preliminary analysis of the reliability and validity of the Chinese Edition of the CSBS DP.

PubMed

Lin, Chu-Sui; Chang, Shu-Hui; Cheng, Shu-Fen; Chao, Pen-Chiang; Chiu, Chun-Hao

2015-03-01

This study marked a preliminary attempt to standardize the Chinese Edition of the Communication and Symbolic Behavior Scales Developmental Profile (Wetherby & Prizant, 2002; CSBS DP) to assist in the early identification of young children with special needs in Taiwan. The study was conducted among 171 infants and toddlers aged 1-2. It also included a follow-up study one year after the initial test. Three domestically developed standardized child development inventories were used to measure the concurrent validity and predictive validity. The Chinese Edition of the CSBS DP demonstrated overall good test-retest and inter-rater reliability. It also showed good concurrent and predictive validity. The current study yields preliminary evidence that the Chinese Edition of the CSBS DP could be a valuable assessment tool worthy of wider distribution. Future research should employ random sampling to establish a true national norm. Additionally, the follow-up study needs to include atypical groups and to expand to children aged 6-12 months to strengthen the applicability of the instrument in Taiwan. Copyright © 2014 Elsevier Ltd. All rights reserved.

Predictive validity of the UK clinical aptitude test in the final years of medical school: a prospective cohort study.

PubMed

Husbands, Adrian; Mathieson, Alistair; Dowell, Jonathan; Cleland, Jennifer; MacKenzie, Rhoda

2014-04-23

The UK Clinical Aptitude Test (UKCAT) was designed to address issues identified with traditional methods of selection. This study aims to examine the predictive validity of the UKCAT and compare this to traditional selection methods in the senior years of medical school. This was a follow-up study of two cohorts of students from two medical schools who had previously taken part in a study examining the predictive validity of the UKCAT in first year. The sample consisted of 4th and 5th Year students who commenced their studies at the University of Aberdeen or University of Dundee medical schools in 2007. Data collected were: demographics (gender and age group), UKCAT scores; Universities and Colleges Admissions Service (UCAS) form scores; admission interview scores; Year 4 and 5 degree examination scores. Pearson's correlations were used to examine the relationships between admissions variables, examination scores, gender and age group, and to select variables for multiple linear regression analysis to predict examination scores. Ninety-nine and 89 students at Aberdeen medical school from Years 4 and 5 respectively, and 51 Year 4 students in Dundee, were included in the analysis. Neither UCAS form nor interview scores were statistically significant predictors of examination performance. Conversely, the UKCAT yielded statistically significant validity coefficients between .24 and .36 in four of five assessments investigated. Multiple regression analysis showed the UKCAT made a statistically significant unique contribution to variance in examination performance in the senior years. Results suggest the UKCAT appears to predict performance better in the later years of medical school compared to earlier years and provides modest supportive evidence for the UKCAT's role in student selection within these institutions. Further research is needed to assess the predictive validity of the UKCAT against professional and behavioural outcomes as the cohort commences working life.

Predictive validity of the UK clinical aptitude test in the final years of medical school: a prospective cohort study

PubMed Central

2014-01-01

Background The UK Clinical Aptitude Test (UKCAT) was designed to address issues identified with traditional methods of selection. This study aims to examine the predictive validity of the UKCAT and compare this to traditional selection methods in the senior years of medical school. This was a follow-up study of two cohorts of students from two medical schools who had previously taken part in a study examining the predictive validity of the UKCAT in first year. Methods The sample consisted of 4th and 5th Year students who commenced their studies at the University of Aberdeen or University of Dundee medical schools in 2007. Data collected were: demographics (gender and age group), UKCAT scores; Universities and Colleges Admissions Service (UCAS) form scores; admission interview scores; Year 4 and 5 degree examination scores. Pearson’s correlations were used to examine the relationships between admissions variables, examination scores, gender and age group, and to select variables for multiple linear regression analysis to predict examination scores. Results Ninety-nine and 89 students at Aberdeen medical school from Years 4 and 5 respectively, and 51 Year 4 students in Dundee, were included in the analysis. Neither UCAS form nor interview scores were statistically significant predictors of examination performance. Conversely, the UKCAT yielded statistically significant validity coefficients between .24 and .36 in four of five assessments investigated. Multiple regression analysis showed the UKCAT made a statistically significant unique contribution to variance in examination performance in the senior years. Conclusions Results suggest the UKCAT appears to predict performance better in the later years of medical school compared to earlier years and provides modest supportive evidence for the UKCAT’s role in student selection within these institutions. Further research is needed to assess the predictive validity of the UKCAT against professional and behavioural outcomes as the cohort commences working life. PMID:24762134

Assessing personal talent determinants in young racquet sport players: a systematic review.

PubMed

Faber, Irene R; Bustin, Paul M J; Oosterveld, Frits G J; Elferink-Gemser, Marije T; Nijhuis-Van der Sanden, Maria W G

2016-01-01

Since junior performances have little predictive value for future success, other solutions are sought to assess a young player's potential. The objectives of this systematic review are (1) to provide an overview of instruments measuring personal talent determinants of young players in racquet sports, and (2) to evaluate these instruments regarding their validity for talent development. Electronic searches were conducted in PubMed, PsychINFO, Web of Knowledge, ScienceDirect and SPORTDiscus (1990 to 31 March 2014). Search terms represented tennis, table tennis, badminton and squash, the concept of talent, methods of testing and children. Thirty articles with information regarding over 100 instruments were included. Validity evaluation showed that instruments focusing on intellectual and perceptual abilities, and coordinative skills discriminate elite from non-elite players and/or are related to current performance, but their predictive validity is not confirmed. There is moderate evidence that the assessments of mental and goal management skills predict future performance. Data on instruments measuring physical characteristics prohibit a conclusion due to conflicting findings. This systematic review yielded an ambiguous end point. The lack of longitudinal studies precludes verification of the instrument's capacity to forecast future performance. Future research should focus on instruments assessing multidimensional talent determinants and their predictive value in longitudinal designs.

Development of a wound healing index for patients with chronic wounds.

PubMed

Horn, Susan D; Fife, Caroline E; Smout, Randall J; Barrett, Ryan S; Thomson, Brett

2013-01-01

Randomized controlled trials in wound care generalize poorly because they exclude patients with significant comorbid conditions. Research using real-world wound care patients is hindered by lack of validated methods to stratify patients according to severity of underlying illnesses. We developed a comprehensive stratification system for patients with wounds that predicts healing likelihood. Complete medical record data on 50,967 wounds from the United States Wound Registry were assigned a clear outcome (healed, amputated, etc.). Factors known to be associated with healing were evaluated using logistic regression models. Significant variables (p < 0.05) were determined and subsequently tested on a holdout sample of data. A different model predicted healing for each wound type. Some variables predicted significantly in nearly all models: wound size, wound age, number of wounds, evidence of bioburden, tissue type exposed (Wagner grade or stage), being nonambulatory, and requiring hospitalization during the course of care. Variables significant in some models included renal failure, renal transplant, malnutrition, autoimmune disease, and cardiovascular disease. All models validated well when applied to the holdout sample. The "Wound Healing Index" can validly predict likelihood of wound healing among real-world patients and can facilitate comparative effectiveness research to identify patients needing advanced therapeutics. © 2013 by the Wound Healing Society.

Evidence-based occupational hearing screening II: validation of a screening methodology using measures of functional hearing ability.

PubMed

Soli, Sigfrid D; Amano-Kusumoto, Akiko; Clavier, Odile; Wilbur, Jed; Casto, Kristen; Freed, Daniel; Laroche, Chantal; Vaillancourt, Véronique; Giguère, Christian; Dreschler, Wouter A; Rhebergen, Koenraad S

2018-05-01

Validate use of the Extended Speech Intelligibility Index (ESII) for prediction of speech intelligibility in non-stationary real-world noise environments. Define a means of using these predictions for objective occupational hearing screening for hearing-critical public safety and law enforcement jobs. Analyses of predicted and measured speech intelligibility in recordings of real-world noise environments were performed in two studies using speech recognition thresholds (SRTs) and intelligibility measures. ESII analyses of the recordings were used to predict intelligibility. Noise recordings were made in prison environments and at US Army facilities for training ground and airborne forces. Speech materials included full bandwidth sentences and bandpass filtered sentences that simulated radio transmissions. A total of 22 adults with normal hearing (NH) and 15 with mild-moderate hearing impairment (HI) participated in the two studies. Average intelligibility predictions for individual NH and HI subjects were accurate in both studies (r 2  ≥ 0.94). Pooled predictions were slightly less accurate (0.78 ≤ r 2  ≤ 0.92). An individual's SRT and audiogram can accurately predict the likelihood of effective speech communication in noise environments with known ESII characteristics, where essential hearing-critical tasks are performed. These predictions provide an objective means of occupational hearing screening.

Combined expectancies: electrophysiological evidence for the adjustment of expectancy effects

PubMed Central

Mattler, Uwe; van der Lugt, Arie; Münte, Thomas F

2006-01-01

Background When subjects use cues to prepare for a likely stimulus or a likely response, reaction times are facilitated by valid cues but prolonged by invalid cues. In studies on combined expectancy effects, two cues can independently give information regarding two dimensions of the forthcoming task. In certain situations, cueing effects on one dimension are reduced when the cue on the other dimension is invalid. According to the Adjusted Expectancy Model, cues affect different processing levels and a mechanism is presumed which is sensitive to the validity of early level cues and leads to online adjustment of expectancy effects at later levels. To examine the predictions of this model cueing of stimulus modality was combined with response cueing. Results Behavioral measures showed the interaction of cueing effects. Electrophysiological measures of the lateralized readiness potential (LRP) and the N200 amplitude confirmed the predictions of the model. The LRP showed larger effects of response cues on response activation when modality cues were valid rather than invalid. N200 amplitude was largest with valid modality cues and invalid response cues, medium with invalid modality cues, and smallest with two valid cues. Conclusion Findings support the view that the validity of early level expectancies modulates the effects of late level expectancies, which included response activation and response conflict in the present study. PMID:16674805

Development and validation of a measure of display rule knowledge: the display rule assessment inventory.

PubMed

Matsumoto, David; Yoo, Seung Hee; Hirayama, Satoko; Petrova, Galina

2005-03-01

As one component of emotion regulation, display rules, which reflect the regulation of expressive behavior, have been the topic of many studies. Despite their theoretical and empirical importance, however, to date there is no measure of display rules that assesses a full range of behavioral responses that are theoretically possible when emotion is elicited. This article reports the development of a new measure of display rules that surveys 5 expressive modes: expression, deamplification, amplification, qualification, and masking. Two studies provide evidence for its internal and temporal reliability and for its content, convergent, discriminant, external, and concurrent predictive validity. Additionally, Study 1, involving American, Russian, and Japanese participants, demonstrated predictable cultural differences on each of the expressive modes. Copyright 2005 APA, all rights reserved.

Weighing the Evidence in Peters' Rule: Does Neuronal Morphology Predict Connectivity?

PubMed

Rees, Christopher L; Moradi, Keivan; Ascoli, Giorgio A

2017-02-01

Although the importance of network connectivity is increasingly recognized, identifying synapses remains challenging relative to the routine characterization of neuronal morphology. Thus, researchers frequently employ axon-dendrite colocations as proxies of potential connections. This putative equivalence, commonly referred to as Peters' rule, has been recently studied at multiple levels and scales, fueling passionate debates regarding its validity. Our critical literature review identifies three conceptually distinct but often confused applications: inferring neuron type circuitry, predicting synaptic contacts among individual cells, and estimating synapse numbers within neuron pairs. Paradoxically, at the originally proposed cell-type level, Peters' rule remains largely untested. Leveraging Hippocampome.org, we validate and refine the relationship between axonal-dendritic colocations and synaptic circuits, clarifying the interpretation of existing and forthcoming data. Copyright © 2016 Elsevier Ltd. All rights reserved.

Measurement of COPD Severity Using a Survey-Based Score

PubMed Central

Omachi, Theodore A.; Katz, Patricia P.; Yelin, Edward H.; Iribarren, Carlos; Blanc, Paul D.

2010-01-01

Background: A comprehensive survey-based COPD severity score has usefulness for epidemiologic and health outcomes research. We previously developed and validated the survey-based COPD Severity Score without using lung function or other physiologic measurements. In this study, we aimed to further validate the severity score in a different COPD cohort and using a combination of patient-reported and objective physiologic measurements. Methods: Using data from the Function, Living, Outcomes, and Work cohort study of COPD, we evaluated the concurrent and predictive validity of the COPD Severity Score among 1,202 subjects. The survey instrument is a 35-point score based on symptoms, medication and oxygen use, and prior hospitalization or intubation for COPD. Subjects were systemically assessed using structured telephone survey, spirometry, and 6-min walk testing. Results: We found evidence to support concurrent validity of the score. Higher COPD Severity Score values were associated with poorer FEV1 (r = −0.38), FEV1% predicted (r = −0.40), Body mass, Obstruction, Dyspnea, Exercise Index (r = 0.57), and distance walked in 6 min (r = −0.43) (P < .0001 in all cases). Greater COPD severity was also related to poorer generic physical health status (r = −0.49) and disease-specific health-related quality of life (r = 0.57) (P < .0001). The score also demonstrated predictive validity. It was also associated with a greater prospective risk of acute exacerbation of COPD defined as ED visits (hazard ratio [HR], 1.31; 95% CI, 1.24-1.39), hospitalizations (HR, 1.59; 95% CI, 1.44-1.75), and either measure of hospital-based care for COPD (HR, 1.34; 95% CI, 1.26-1.41) (P < .0001 in all cases). Conclusion: The COPD Severity Score is a valid survey-based measure of disease-specific severity, both in terms of concurrent and predictive validity. The score is a psychometrically sound instrument for use in epidemiologic and outcomes research in COPD. PMID:20040611

Integrated analysis of drug-induced gene expression profiles predicts novel hERG inhibitors.

PubMed

Babcock, Joseph J; Du, Fang; Xu, Kaiping; Wheelan, Sarah J; Li, Min

2013-01-01

Growing evidence suggests that drugs interact with diverse molecular targets mediating both therapeutic and toxic effects. Prediction of these complex interactions from chemical structures alone remains challenging, as compounds with different structures may possess similar toxicity profiles. In contrast, predictions based on systems-level measurements of drug effect may reveal pharmacologic similarities not evident from structure or known therapeutic indications. Here we utilized drug-induced transcriptional responses in the Connectivity Map (CMap) to discover such similarities among diverse antagonists of the human ether-à-go-go related (hERG) potassium channel, a common target of promiscuous inhibition by small molecules. Analysis of transcriptional profiles generated in three independent cell lines revealed clusters enriched for hERG inhibitors annotated using a database of experimental measurements (hERGcentral) and clinical indications. As a validation, we experimentally identified novel hERG inhibitors among the unannotated drugs in these enriched clusters, suggesting transcriptional responses may serve as predictive surrogates of cardiotoxicity complementing existing functional assays.

Integrated Analysis of Drug-Induced Gene Expression Profiles Predicts Novel hERG Inhibitors

PubMed Central

Babcock, Joseph J.; Du, Fang; Xu, Kaiping; Wheelan, Sarah J.; Li, Min

2013-01-01

Growing evidence suggests that drugs interact with diverse molecular targets mediating both therapeutic and toxic effects. Prediction of these complex interactions from chemical structures alone remains challenging, as compounds with different structures may possess similar toxicity profiles. In contrast, predictions based on systems-level measurements of drug effect may reveal pharmacologic similarities not evident from structure or known therapeutic indications. Here we utilized drug-induced transcriptional responses in the Connectivity Map (CMap) to discover such similarities among diverse antagonists of the human ether-à-go-go related (hERG) potassium channel, a common target of promiscuous inhibition by small molecules. Analysis of transcriptional profiles generated in three independent cell lines revealed clusters enriched for hERG inhibitors annotated using a database of experimental measurements (hERGcentral) and clinical indications. As a validation, we experimentally identified novel hERG inhibitors among the unannotated drugs in these enriched clusters, suggesting transcriptional responses may serve as predictive surrogates of cardiotoxicity complementing existing functional assays. PMID:23936032

Genomic Models of Short-Term Exposure Accurately Predict Long-Term Chemical Carcinogenicity and Identify Putative Mechanisms of Action

PubMed Central

Gusenleitner, Daniel; Auerbach, Scott S.; Melia, Tisha; Gómez, Harold F.; Sherr, David H.; Monti, Stefano

2014-01-01

Background Despite an overall decrease in incidence of and mortality from cancer, about 40% of Americans will be diagnosed with the disease in their lifetime, and around 20% will die of it. Current approaches to test carcinogenic chemicals adopt the 2-year rodent bioassay, which is costly and time-consuming. As a result, fewer than 2% of the chemicals on the market have actually been tested. However, evidence accumulated to date suggests that gene expression profiles from model organisms exposed to chemical compounds reflect underlying mechanisms of action, and that these toxicogenomic models could be used in the prediction of chemical carcinogenicity. Results In this study, we used a rat-based microarray dataset from the NTP DrugMatrix Database to test the ability of toxicogenomics to model carcinogenicity. We analyzed 1,221 gene-expression profiles obtained from rats treated with 127 well-characterized compounds, including genotoxic and non-genotoxic carcinogens. We built a classifier that predicts a chemical's carcinogenic potential with an AUC of 0.78, and validated it on an independent dataset from the Japanese Toxicogenomics Project consisting of 2,065 profiles from 72 compounds. Finally, we identified differentially expressed genes associated with chemical carcinogenesis, and developed novel data-driven approaches for the molecular characterization of the response to chemical stressors. Conclusion Here, we validate a toxicogenomic approach to predict carcinogenicity and provide strong evidence that, with a larger set of compounds, we should be able to improve the sensitivity and specificity of the predictions. We found that the prediction of carcinogenicity is tissue-dependent and that the results also confirm and expand upon previous studies implicating DNA damage, the peroxisome proliferator-activated receptor, the aryl hydrocarbon receptor, and regenerative pathology in the response to carcinogen exposure. PMID:25058030

Validity and reliability of clinical prediction rules used to screen for cervical spine injury in alert low-risk patients with blunt trauma to the neck: part 2. A systematic review from the Cervical Assessment and Diagnosis Research Evaluation (CADRE) Collaboration.

PubMed

Moser, N; Lemeunier, N; Southerst, D; Shearer, H; Murnaghan, K; Sutton, D; Côté, P

2018-06-01

To update findings of the 2000-2010 Bone and Joint Decade Task Force on Neck Pain and its Associated Disorders (Neck Pain Task Force) on the validity and reliability of clinical prediction rules used to screen for cervical spine injury in alert low-risk adult patients with blunt trauma to the neck. We searched four databases from 2005 to 2015. Pairs of independent reviewers critically appraised eligible studies using the modified QUADAS-2 and QAREL criteria. We synthesized low risk of bias studies following best evidence synthesis principles. We screened 679 citations; five had a low risk of bias and were included in our synthesis. The sensitivity of the Canadian C-spine rule ranged from 0.90 to 1.00 with negative predictive values ranging from 99 to 100%. Inter-rater reliability of the Canadian C-spine rule varied from k = 0.60 between nurses and physicians to k = 0.93 among paramedics. The inter-rater reliability of the Nexus Low-Risk Criteria was k = 0.53 between resident physicians and faculty physicians. Our review adds new evidence to the Neck Pain Task Force and supports the use of clinical prediction rules in emergency care settings to screen for cervical spine injury in alert low-risk adult patients with blunt trauma to the neck. The Canadian C-spine rule consistently demonstrated excellent sensitivity and negative predictive values. Our review, however, suggests that the reproducibility of the clinical predictions rules varies depending on the examiners level of training and experience.

Poor early response to methotrexate portends inadequate long-term outcomes in patients with moderate-to-severe psoriasis: Evidence from 2 phase 3 clinical trials.

PubMed

Gordon, Kenneth B; Betts, Keith A; Sundaram, Murali; Signorovitch, James E; Li, Junlong; Xie, Meng; Wu, Eric Q; Okun, Martin M

2017-12-01

Most methotrexate-treated psoriasis patients do not achieve a long-term PASI75 (75% reduction from baseline Psoriasis Area and Severity Index score) response. Indications of nonresponse can be apparent after only 4 weeks of treatment. To develop a prediction rule to identify patients unlikely to respond adequately to methotrexate. Patient-level data from CHAMPION (NCT00235820, N = 110) was used to construct a prediction model for week 16 PASI75 by using patient baseline characteristics and week 4 PASI25. A prediction rule was determined on the basis of the sensitivity and specificity and validated in terms of week 16 PASI75 response in an independent validation sample from trial M10-255 (NCT00679731, N = 163). PASI25 achievement at week 4 (odds ratio = 8.917) was highly predictive of response with methotrexate at week 16. Patients with a predicted response probability <30% were recommended to discontinue methotrexate. The rates of week 16 PASI75 response were 65.8% and 21.1% (P < .001) for patients recommended to continue and discontinue methotrexate, respectively. The CHAMPION trial excluded patients previously treated with biologics, and the M10-255 trial had no restrictions. A prediction rule was developed and validated to identify patients unlikely to respond adequately to methotrexate. The rule indicates that 4 weeks of methotrexate might be sufficient to predict long-term response with limited safety risk. Copyright © 2017 American Academy of Dermatology, Inc. Published by Elsevier Inc. All rights reserved.

Development of an Itemwise Efficiency Scoring Method: Concurrent, Convergent, Discriminant, and Neuroimaging-Based Predictive Validity Assessed in a Large Community Sample

PubMed Central

Moore, Tyler M.; Reise, Steven P.; Roalf, David R.; Satterthwaite, Theodore D.; Davatzikos, Christos; Bilker, Warren B.; Port, Allison M.; Jackson, Chad T.; Ruparel, Kosha; Savitt, Adam P.; Baron, Robert B.; Gur, Raquel E.; Gur, Ruben C.

2016-01-01

Traditional “paper-and-pencil” testing is imprecise in measuring speed and hence limited in assessing performance efficiency, but computerized testing permits precision in measuring itemwise response time. We present a method of scoring performance efficiency (combining information from accuracy and speed) at the item level. Using a community sample of 9,498 youths age 8-21, we calculated item-level efficiency scores on four neurocognitive tests, and compared the concurrent, convergent, discriminant, and predictive validity of these scores to simple averaging of standardized speed and accuracy-summed scores. Concurrent validity was measured by the scores' abilities to distinguish men from women and their correlations with age; convergent and discriminant validity were measured by correlations with other scores inside and outside of their neurocognitive domains; predictive validity was measured by correlations with brain volume in regions associated with the specific neurocognitive abilities. Results provide support for the ability of itemwise efficiency scoring to detect signals as strong as those detected by standard efficiency scoring methods. We find no evidence of superior validity of the itemwise scores over traditional scores, but point out several advantages of the former. The itemwise efficiency scoring method shows promise as an alternative to standard efficiency scoring methods, with overall moderate support from tests of four different types of validity. This method allows the use of existing item analysis methods and provides the convenient ability to adjust the overall emphasis of accuracy versus speed in the efficiency score, thus adjusting the scoring to the real-world demands the test is aiming to fulfill. PMID:26866796

Analytic Validation of Immunohistochemistry Assays: New Benchmark Data From a Survey of 1085 Laboratories.

PubMed

Stuart, Lauren N; Volmar, Keith E; Nowak, Jan A; Fatheree, Lisa A; Souers, Rhona J; Fitzgibbons, Patrick L; Goldsmith, Jeffrey D; Astles, J Rex; Nakhleh, Raouf E

2017-09-01

- A cooperative agreement between the College of American Pathologists (CAP) and the United States Centers for Disease Control and Prevention was undertaken to measure laboratories' awareness and implementation of an evidence-based laboratory practice guideline (LPG) on immunohistochemical (IHC) validation practices published in 2014. - To establish new benchmark data on IHC laboratory practices. - A 2015 survey on IHC assay validation practices was sent to laboratories subscribed to specific CAP proficiency testing programs and to additional nonsubscribing laboratories that perform IHC testing. Specific questions were designed to capture laboratory practices not addressed in a 2010 survey. - The analysis was based on responses from 1085 laboratories that perform IHC staining. Ninety-six percent (809 of 844) always documented validation of IHC assays. Sixty percent (648 of 1078) had separate procedures for predictive and nonpredictive markers, 42.7% (220 of 515) had procedures for laboratory-developed tests, 50% (349 of 697) had procedures for testing cytologic specimens, and 46.2% (363 of 785) had procedures for testing decalcified specimens. Minimum case numbers were specified by 85.9% (720 of 838) of laboratories for nonpredictive markers and 76% (584 of 768) for predictive markers. Median concordance requirements were 95% for both types. For initial validation, 75.4% (538 of 714) of laboratories adopted the 20-case minimum for nonpredictive markers and 45.9% (266 of 579) adopted the 40-case minimum for predictive markers as outlined in the 2014 LPG. The most common method for validation was correlation with morphology and expected results. Laboratories also reported which assay changes necessitated revalidation and their minimum case requirements. - Benchmark data on current IHC validation practices and procedures may help laboratories understand the issues and influence further refinement of LPG recommendations.

On Nomological Validity and Auxiliary Assumptions: The Importance of Simultaneously Testing Effects in Social Cognitive Theories Applied to Health Behavior and Some Guidelines

PubMed Central

Hagger, Martin S.; Gucciardi, Daniel F.; Chatzisarantis, Nikos L. D.

2017-01-01

Tests of social cognitive theories provide informative data on the factors that relate to health behavior, and the processes and mechanisms involved. In the present article, we contend that tests of social cognitive theories should adhere to the principles of nomological validity, defined as the degree to which predictions in a formal theoretical network are confirmed. We highlight the importance of nomological validity tests to ensure theory predictions can be disconfirmed through observation. We argue that researchers should be explicit on the conditions that lead to theory disconfirmation, and identify any auxiliary assumptions on which theory effects may be conditional. We contend that few researchers formally test the nomological validity of theories, or outline conditions that lead to model rejection and the auxiliary assumptions that may explain findings that run counter to hypotheses, raising potential for ‘falsification evasion.’ We present a brief analysis of studies (k = 122) testing four key social cognitive theories in health behavior to illustrate deficiencies in reporting theory tests and evaluations of nomological validity. Our analysis revealed that few articles report explicit statements suggesting that their findings support or reject the hypotheses of the theories tested, even when findings point to rejection. We illustrate the importance of explicit a priori specification of fundamental theory hypotheses and associated auxiliary assumptions, and identification of the conditions which would lead to rejection of theory predictions. We also demonstrate the value of confirmatory analytic techniques, meta-analytic structural equation modeling, and Bayesian analyses in providing robust converging evidence for nomological validity. We provide a set of guidelines for researchers on how to adopt and apply the nomological validity approach to testing health behavior models. PMID:29163307

Prediction models for the risk of spontaneous preterm birth based on maternal characteristics: a systematic review and independent external validation.

PubMed

Meertens, Linda J E; van Montfort, Pim; Scheepers, Hubertina C J; van Kuijk, Sander M J; Aardenburg, Robert; Langenveld, Josje; van Dooren, Ivo M A; Zwaan, Iris M; Spaanderman, Marc E A; Smits, Luc J M

2018-04-17

Prediction models may contribute to personalized risk-based management of women at high risk of spontaneous preterm delivery. Although prediction models are published frequently, often with promising results, external validation generally is lacking. We performed a systematic review of prediction models for the risk of spontaneous preterm birth based on routine clinical parameters. Additionally, we externally validated and evaluated the clinical potential of the models. Prediction models based on routinely collected maternal parameters obtainable during first 16 weeks of gestation were eligible for selection. Risk of bias was assessed according to the CHARMS guidelines. We validated the selected models in a Dutch multicenter prospective cohort study comprising 2614 unselected pregnant women. Information on predictors was obtained by a web-based questionnaire. Predictive performance of the models was quantified by the area under the receiver operating characteristic curve (AUC) and calibration plots for the outcomes spontaneous preterm birth <37 weeks and <34 weeks of gestation. Clinical value was evaluated by means of decision curve analysis and calculating classification accuracy for different risk thresholds. Four studies describing five prediction models fulfilled the eligibility criteria. Risk of bias assessment revealed a moderate to high risk of bias in three studies. The AUC of the models ranged from 0.54 to 0.67 and from 0.56 to 0.70 for the outcomes spontaneous preterm birth <37 weeks and <34 weeks of gestation, respectively. A subanalysis showed that the models discriminated poorly (AUC 0.51-0.56) for nulliparous women. Although we recalibrated the models, two models retained evidence of overfitting. The decision curve analysis showed low clinical benefit for the best performing models. This review revealed several reporting and methodological shortcomings of published prediction models for spontaneous preterm birth. Our external validation study indicated that none of the models had the ability to predict spontaneous preterm birth adequately in our population. Further improvement of prediction models, using recent knowledge about both model development and potential risk factors, is necessary to provide an added value in personalized risk assessment of spontaneous preterm birth. © 2018 The Authors Acta Obstetricia et Gynecologica Scandinavica published by John Wiley & Sons Ltd on behalf of Nordic Federation of Societies of Obstetrics and Gynecology (NFOG).

Observed Parenting Behavior with Teens: Measurement Invariance and Predictive Validity Across Race

PubMed Central

Skinner, Martie L.; MacKenzie, Elizabeth P.; Haggerty, Kevin P.; Hill, Karl G.; Roberson, Kendra C.

2011-01-01

Previous reports supporting measurement equality between European American and African American families have often focused on self-reported risk factors or observed parent behavior with young children. This study examines equality of measurement of observer ratings of parenting behavior with adolescents during structured tasks; mean levels of observed parenting; and predictive validity of teen self-reports of antisocial behaviors and beliefs using a sample of 163 African American and 168 European American families. Multiple-group confirmatory factor analyses supported measurement invariance across ethnic groups for 4 measures of observed parenting behavior: prosocial rewards, psychological costs, antisocial rewards, and problem solving. Some mean-level differences were found: African American parents exhibited lower levels of prosocial rewards, higher levels of psychological costs, and lower problem solving when compared to European Americans. No significant mean difference was found in rewards for antisocial behavior. Multigroup structural equation models suggested comparable relationships across race (predictive validity) between parenting constructs and youth antisocial constructs (i.e., drug initiation, positive drug attitudes, antisocial attitudes, problem behaviors) in all but one of the tested relationships. This study adds to existing evidence that family-based interventions targeting parenting behaviors can be generalized to African American families. PMID:21787057

Self-Reported Emotion Reactivity Among Early-Adolescent Girls: Evidence for Convergent and Discriminant Validity in an Urban Community Sample.

PubMed

Evans, Spencer C; Blossom, Jennifer B; Canter, Kimberly S; Poppert-Cordts, Katrina; Kanine, Rebecca; Garcia, Andrea; Roberts, Michael C

2016-05-01

Emotion reactivity, measured via the self-report Emotion Reactivity Scale (ERS), has shown unique associations with different forms of psychopathology and suicidal thoughts and behaviors; however, this limited body of research has been conducted among adults and older adolescents of predominantly White/European ethnic backgrounds. The present study investigated the validity of ERS scores for measuring emotion reactivity among an urban community sample of middle-school-age girls. Participants (N = 93, ages 11-15, 76% African-American, 18% Latina) completed the ERS and measures of emotion coping, internalizing problems, proactive and reactive aggression, negative life events, and lifetime suicidal ideation and substance use. As hypothesized, ERS scores were significantly associated with internalizing problems, poor emotion coping, negative life events, reactive aggression, and suicidal ideation (evidence for convergent validity), but showed little to no association with proactive aggression or lifetime substance use (evidence for discriminant validity). A series of logistic regressions were conducted to further explore the associations among internalizing problems, emotion reactivity, and suicidal ideation. With depressive symptoms included in the model, emotion reactivity was no longer uniquely predictive of lifetime suicidal ideation, nor did it serve as a moderator of other associations. In conjunction with previous research, these findings offer further support for the construct validity and research utility of the ERS as a self-report measure of emotion reactivity in adolescents. Copyright © 2016. Published by Elsevier Ltd.

Mapping the Moral Domain

PubMed Central

Graham, Jesse; Nosek, Brian A.; Haidt, Jonathan; Iyer, Ravi; Koleva, Spassena; Ditto, Peter H.

2010-01-01

The moral domain is broader than the empathy and justice concerns assessed by existing measures of moral competence, and it is not just a subset of the values assessed by value inventories. To fill the need for reliable and theoretically-grounded measurement of the full range of moral concerns, we developed the Moral Foundations Questionnaire (MFQ) based on a theoretical model of five universally available (but variably developed) sets of moral intuitions: Harm/care, Fairness/reciprocity, Ingroup/loyalty, Authority/respect, and Purity/sanctity. We present evidence for the internal and external validity of the scale and the model, and in doing so present new findings about morality: 1. Comparative model fitting of confirmatory factor analyses provides empirical justification for a five-factor structure of moral concerns. 2. Convergent/discriminant validity evidence suggests that moral concerns predict personality features and social group attitudes not previously considered morally relevant. 3. We establish pragmatic validity of the measure in providing new knowledge and research opportunities concerning demographic and cultural differences in moral intuitions. These analyses provide evidence for the usefulness of Moral Foundations Theory in simultaneously increasing the scope and sharpening the resolution of psychological views of morality. PMID:21244182

Validity and reliability testing of the Prenatal Psychosocial Profile.

PubMed

Curry, M A; Campbell, R A; Christian, M

1994-04-01

Two studies of low-income pregnant women (N = 179) were done to examine the validity and reliability of the Prenatal Psychosocial Profile (PPP). The PPP, a composite of the Rosenberg Self-Esteem Scale, the Support Behaviors Inventory, and a newly developed measure of stress, is a brief, comprehensive clinical assessment of psychosocial risk during pregnancy. Construct validity of the stress scale was supported by theoretically predicted negative correlations with self-esteem, partner support, and support from others (N = 91). Convergent validity of the stress scale was demonstrated by a correlation of .71 with the Difficult Life Circumstances Scale. Adequate levels of internal consistency were found. Interrelationships between the four subscales were consistent with the underlying conceptualization, and there was beginning evidence of the factorial independence of the subscales.

Can a Two-Question Test Be Reliable and Valid for Predicting Academic Outcomes?

ERIC Educational Resources Information Center

Bridgeman, Brent

2016-01-01

Scores on essay-based assessments that are part of standardized admissions tests are typically given relatively little weight in admissions decisions compared to the weight given to scores from multiple-choice assessments. Evidence is presented to suggest that more weight should be given to these assessments. The reliability of the writing scores…

Screen Twice, Cut Once: Assessing the Predictive Validity of Teacher Selection Tools

ERIC Educational Resources Information Center

Goldhaber, Dan; Grout, Cyrus; Huntington-Klein, Nick

2015-01-01

It is well documented that teachers can have profound effects on student outcomes. Empirical estimates find that a one standard deviation increase in teacher quality raises student test achievement by 10 to 25 percent of a standard deviation. More recent evidence shows that the effectiveness of teachers can affect long-term student outcomes, such…

Construct and Predictive Validity Evidence for Curriculum-Based Measures of Early Literacy and Numeracy Skills in Kindergarten

ERIC Educational Resources Information Center

Betts, Joseph; Pickart, Mary; Heistad, Dave

2009-01-01

The assessment of early literacy and numeracy skills can provide useful and important information in pursuance of the goal to increase student academic achievement. At present, there have been promising results using curriculum-based measurement (CBM) for evaluating early literacy and early numeracy. There has been little research investigating…

Prognostic Indicators: Predicting Degree of Change from Interventions for Adolescents at Risk for Language Weakness

ERIC Educational Resources Information Center

Poll, Gerard H.; Burke, Lisa; Miller, Carol A.; Fiene, Judy

2017-01-01

Prognostic statements are a standard component of assessments for adolescents at risk for language-learning disabilities, but there is limited evidence on the validity of prognostic indicators. In two studies, we collected measures of language ability and candidate prognostic indicators from adolescents age 12 to 13. We conducted an expository…

Concordance between actual and pharmacogenetic predicted desvenlafaxine dose needed to achieve remission in major depressive disorder: a 10-week open-label study

PubMed Central

Müller, Daniel J.; Ng, Chee H.; Byron, Keith; Berk, Michael; Singh, Ajeet B.

2017-01-01

Background Pharmacogenetic-based dosing support tools have been developed to personalize antidepressant-prescribing practice. However, the clinical validity of these tools has not been adequately tested, particularly for specific antidepressants. Objective To examine the concordance between the actual dose and a polygene pharmacogenetic predicted dose of desvenlafaxine needed to achieve symptom remission. Materials and methods A 10-week, open-label, prospective trial of desvenlafaxine among Caucasian adults with major depressive disorder (n=119) was conducted. Dose was clinically adjusted and at the completion of the trial, the clinical dose needed to achieve remission was compared with the predicted dose needed to achieve remission. Results Among remitters (n=95), there was a strong concordance (Kendall’s τ-b=0.84, P=0.0001; Cohen’s κ=0.82, P=0.0001) between the actual and the predicted dose need to achieve symptom remission, showing high sensitivity (≥85%), specificity (≥86%), and accuracy (≥89%) of the tool. Conclusion Findings provide initial evidence for the clinical validity of a polygene pharmacogenetic-based tool for desvenlafaxine dosing. PMID:27779571

Concordance between actual and pharmacogenetic predicted desvenlafaxine dose needed to achieve remission in major depressive disorder: a 10-week open-label study.

PubMed

Bousman, Chad A; Müller, Daniel J; Ng, Chee H; Byron, Keith; Berk, Michael; Singh, Ajeet B

2017-01-01

Pharmacogenetic-based dosing support tools have been developed to personalize antidepressant-prescribing practice. However, the clinical validity of these tools has not been adequately tested, particularly for specific antidepressants. To examine the concordance between the actual dose and a polygene pharmacogenetic predicted dose of desvenlafaxine needed to achieve symptom remission. A 10-week, open-label, prospective trial of desvenlafaxine among Caucasian adults with major depressive disorder (n=119) was conducted. Dose was clinically adjusted and at the completion of the trial, the clinical dose needed to achieve remission was compared with the predicted dose needed to achieve remission. Among remitters (n=95), there was a strong concordance (Kendall's τ-b=0.84, P=0.0001; Cohen's κ=0.82, P=0.0001) between the actual and the predicted dose need to achieve symptom remission, showing high sensitivity (≥85%), specificity (≥86%), and accuracy (≥89%) of the tool. Findings provide initial evidence for the clinical validity of a polygene pharmacogenetic-based tool for desvenlafaxine dosing.

Validation of Models Used to Inform Colorectal Cancer Screening Guidelines: Accuracy and Implications.

PubMed

Rutter, Carolyn M; Knudsen, Amy B; Marsh, Tracey L; Doria-Rose, V Paul; Johnson, Eric; Pabiniak, Chester; Kuntz, Karen M; van Ballegooijen, Marjolein; Zauber, Ann G; Lansdorp-Vogelaar, Iris

2016-07-01

Microsimulation models synthesize evidence about disease processes and interventions, providing a method for predicting long-term benefits and harms of prevention, screening, and treatment strategies. Because models often require assumptions about unobservable processes, assessing a model's predictive accuracy is important. We validated 3 colorectal cancer (CRC) microsimulation models against outcomes from the United Kingdom Flexible Sigmoidoscopy Screening (UKFSS) Trial, a randomized controlled trial that examined the effectiveness of one-time flexible sigmoidoscopy screening to reduce CRC mortality. The models incorporate different assumptions about the time from adenoma initiation to development of preclinical and symptomatic CRC. Analyses compare model predictions to study estimates across a range of outcomes to provide insight into the accuracy of model assumptions. All 3 models accurately predicted the relative reduction in CRC mortality 10 years after screening (predicted hazard ratios, with 95% percentile intervals: 0.56 [0.44, 0.71], 0.63 [0.51, 0.75], 0.68 [0.53, 0.83]; estimated with 95% confidence interval: 0.56 [0.45, 0.69]). Two models with longer average preclinical duration accurately predicted the relative reduction in 10-year CRC incidence. Two models with longer mean sojourn time accurately predicted the number of screen-detected cancers. All 3 models predicted too many proximal adenomas among patients referred to colonoscopy. Model accuracy can only be established through external validation. Analyses such as these are therefore essential for any decision model. Results supported the assumptions that the average time from adenoma initiation to development of preclinical cancer is long (up to 25 years), and mean sojourn time is close to 4 years, suggesting the window for early detection and intervention by screening is relatively long. Variation in dwell time remains uncertain and could have important clinical and policy implications. © The Author(s) 2016.

Developing symptom-based predictive models of endometriosis as a clinical screening tool: results from a multicenter study

PubMed Central

Nnoaham, Kelechi E.; Hummelshoj, Lone; Kennedy, Stephen H.; Jenkinson, Crispin; Zondervan, Krina T.

2012-01-01

Objective To generate and validate symptom-based models to predict endometriosis among symptomatic women prior to undergoing their first laparoscopy. Design Prospective, observational, two-phase study, in which women completed a 25-item questionnaire prior to surgery. Setting Nineteen hospitals in 13 countries. Patient(s) Symptomatic women (n = 1,396) scheduled for laparoscopy without a previous surgical diagnosis of endometriosis. Intervention(s) None. Main Outcome Measure(s) Sensitivity and specificity of endometriosis diagnosis predicted by symptoms and patient characteristics from optimal models developed using multiple logistic regression analyses in one data set (phase I), and independently validated in a second data set (phase II) by receiver operating characteristic (ROC) curve analysis. Result(s) Three hundred sixty (46.7%) women in phase I and 364 (58.2%) in phase II were diagnosed with endometriosis at laparoscopy. Menstrual dyschezia (pain on opening bowels) and a history of benign ovarian cysts most strongly predicted both any and stage III and IV endometriosis in both phases. Prediction of any-stage endometriosis, although improved by ultrasound scan evidence of cyst/nodules, was relatively poor (area under the curve [AUC] = 68.3). Stage III and IV disease was predicted with good accuracy (AUC = 84.9, sensitivity of 82.3% and specificity 75.8% at an optimal cut-off of 0.24). Conclusion(s) Our symptom-based models predict any-stage endometriosis relatively poorly and stage III and IV disease with good accuracy. Predictive tools based on such models could help to prioritize women for surgical investigation in clinical practice and thus contribute to reducing time to diagnosis. We invite other researchers to validate the key models in additional populations. PMID:22657249

Renal Tumor Anatomic Complexity: Clinical Implications for Urologists.

PubMed

Joshi, Shreyas S; Uzzo, Robert G

2017-05-01

Anatomic tumor complexity can be objectively measured and reported using nephrometry. Various scoring systems have been developed in an attempt to correlate tumor complexity with intraoperative and postoperative outcomes. Nephrometry may also predict tumor biology in a noninvasive, reproducible manner. Other scoring systems can help predict surgical complexity and the likelihood of complications, independent of tumor characteristics. The accumulated data in this new field provide provocative evidence that objectifying anatomic complexity can consolidate reporting mechanisms and improve metrics of comparisons. Further prospective validation is needed to understand the full descriptive and predictive ability of the various nephrometry scores. Copyright © 2017 Elsevier Inc. All rights reserved.

Meta-analytic estimates predict the effectiveness of emotion regulation strategies in the "real world": reply to Augustine and Hemenover (2013).

PubMed

Miles, Eleanor; Sheeran, Paschal; Webb, Thomas L

2013-05-01

Augustine and Hemenover (2013) were right to state that meta-analyses should be accurate and generalizable. However, we disagree that our meta-analysis of emotion regulation strategies (Webb, Miles, & Sheeran, 2012) fell short in these respects. Augustine and Hemenover's concerns appear to have accrued from misunderstandings of our inclusion criteria or from disagreements with methodological decisions that are crucial to the validity of meta-analysis. This response clarifies the bases of these decisions and discusses implications for the accuracy and validity of meta-analyses. Furthermore, we show that our findings are consistent with theoretical predictions and previous reviews, and we present new evidence that the effect sizes that we obtained are generalizable. In particular, we demonstrate that our estimates of the effectiveness of emotion regulation strategies reveal how well these strategies predict important emotional outcomes over 1 year. PsycINFO Database Record (c) 2013 APA, all rights reserved.

Building a computer program to support children, parents, and distraction during healthcare procedures.

PubMed

Hanrahan, Kirsten; McCarthy, Ann Marie; Kleiber, Charmaine; Ataman, Kaan; Street, W Nick; Zimmerman, M Bridget; Ersig, Anne L

2012-10-01

This secondary data analysis used data mining methods to develop predictive models of child risk for distress during a healthcare procedure. Data used came from a study that predicted factors associated with children's responses to an intravenous catheter insertion while parents provided distraction coaching. From the 255 items used in the primary study, 44 predictive items were identified through automatic feature selection and used to build support vector machine regression models. Models were validated using multiple cross-validation tests and by comparing variables identified as explanatory in the traditional versus support vector machine regression. Rule-based approaches were applied to the model outputs to identify overall risk for distress. A decision tree was then applied to evidence-based instructions for tailoring distraction to characteristics and preferences of the parent and child. The resulting decision support computer application, titled Children, Parents and Distraction, is being used in research. Future use will support practitioners in deciding the level and type of distraction intervention needed by a child undergoing a healthcare procedure.

[Validity evidence of the Health-Related Quality of Life for Drug Abusers Test based on the Biaxial Model of Addiction].

PubMed

Lozano, Oscar M; Rojas, Antonio J; Pérez, Cristino; González-Sáiz, Francisco; Ballesta, Rosario; Izaskun, Bilbao

2008-05-01

The aim of this work is to show evidence of the validity of the Health-Related Quality of Life for Drug Abusers Test (HRQoLDA Test). This test was developed to measure specific HRQoL for drugs abusers, within the theoretical addiction framework of the biaxial model. The sample comprised 138 patients diagnosed with opiate drug dependence. In this study, the following constructs and variables of the biaxial model were measured: severity of dependence, physical health status, psychological adjustment and substance consumption. Results indicate that the HRQoLDA Test scores are related to dependency and consumption-related problems. Multiple regression analysis reveals that HRQoL can be predicted from drug dependence, physical health status and psychological adjustment. These results contribute empirical evidence of the theoretical relationships established between HRQoL and the biaxial model, and they support the interpretation of the HRQoLDA Test to measure HRQoL in drug abusers, thus providing a test to measure this specific construct in this population.

Ethical leadership: meta-analytic evidence of criterion-related and incremental validity.

PubMed

Ng, Thomas W H; Feldman, Daniel C

2015-05-01

This study examines the criterion-related and incremental validity of ethical leadership (EL) with meta-analytic data. Across 101 samples published over the last 15 years (N = 29,620), we observed that EL demonstrated acceptable criterion-related validity with variables that tap followers' job attitudes, job performance, and evaluations of their leaders. Further, followers' trust in the leader mediated the relationships of EL with job attitudes and performance. In terms of incremental validity, we found that EL significantly, albeit weakly in some cases, predicted task performance, citizenship behavior, and counterproductive work behavior-even after controlling for the effects of such variables as transformational leadership, use of contingent rewards, management by exception, interactional fairness, and destructive leadership. The article concludes with a discussion of ways to strengthen the incremental validity of EL. (PsycINFO Database Record (c) 2015 APA, all rights reserved).

Development and validation of the coping with terror scale.

PubMed

Stein, Nathan R; Schorr, Yonit; Litz, Brett T; King, Lynda A; King, Daniel W; Solomon, Zahava; Horesh, Danny

2013-10-01

Terrorism creates lingering anxiety about future attacks. In prior terror research, the conceptualization and measurement of coping behaviors were constrained by the use of existing coping scales that index reactions to daily hassles and demands. The authors created and validated the Coping with Terror Scale to fill the measurement gap. The authors emphasized content validity, leveraging the knowledge of terror experts and groups of Israelis. A multistep approach involved construct definition and item generation, trimming and refining the measure, exploring the factor structure underlying item responses, and garnering evidence for reliability and validity. The final scale comprised six factors that were generally consistent with the authors' original construct specifications. Scores on items linked to these factors demonstrate good reliability and validity. Future studies using the Coping with Terror Scale with other populations facing terrorist threats are needed to test its ability to predict resilience, functional impairment, and psychological distress.

Role of test motivation in intelligence testing.

PubMed

Duckworth, Angela Lee; Quinn, Patrick D; Lynam, Donald R; Loeber, Rolf; Stouthamer-Loeber, Magda

2011-05-10

Intelligence tests are widely assumed to measure maximal intellectual performance, and predictive associations between intelligence quotient (IQ) scores and later-life outcomes are typically interpreted as unbiased estimates of the effect of intellectual ability on academic, professional, and social life outcomes. The current investigation critically examines these assumptions and finds evidence against both. First, we examined whether motivation is less than maximal on intelligence tests administered in the context of low-stakes research situations. Specifically, we completed a meta-analysis of random-assignment experiments testing the effects of material incentives on intelligence-test performance on a collective 2,008 participants. Incentives increased IQ scores by an average of 0.64 SD, with larger effects for individuals with lower baseline IQ scores. Second, we tested whether individual differences in motivation during IQ testing can spuriously inflate the predictive validity of intelligence for life outcomes. Trained observers rated test motivation among 251 adolescent boys completing intelligence tests using a 15-min "thin-slice" video sample. IQ score predicted life outcomes, including academic performance in adolescence and criminal convictions, employment, and years of education in early adulthood. After adjusting for the influence of test motivation, however, the predictive validity of intelligence for life outcomes was significantly diminished, particularly for nonacademic outcomes. Collectively, our findings suggest that, under low-stakes research conditions, some individuals try harder than others, and, in this context, test motivation can act as a third-variable confound that inflates estimates of the predictive validity of intelligence for life outcomes.

Role of test motivation in intelligence testing

PubMed Central

Duckworth, Angela Lee; Quinn, Patrick D.; Lynam, Donald R.; Loeber, Rolf; Stouthamer-Loeber, Magda

2011-01-01

Intelligence tests are widely assumed to measure maximal intellectual performance, and predictive associations between intelligence quotient (IQ) scores and later-life outcomes are typically interpreted as unbiased estimates of the effect of intellectual ability on academic, professional, and social life outcomes. The current investigation critically examines these assumptions and finds evidence against both. First, we examined whether motivation is less than maximal on intelligence tests administered in the context of low-stakes research situations. Specifically, we completed a meta-analysis of random-assignment experiments testing the effects of material incentives on intelligence-test performance on a collective 2,008 participants. Incentives increased IQ scores by an average of 0.64 SD, with larger effects for individuals with lower baseline IQ scores. Second, we tested whether individual differences in motivation during IQ testing can spuriously inflate the predictive validity of intelligence for life outcomes. Trained observers rated test motivation among 251 adolescent boys completing intelligence tests using a 15-min “thin-slice” video sample. IQ score predicted life outcomes, including academic performance in adolescence and criminal convictions, employment, and years of education in early adulthood. After adjusting for the influence of test motivation, however, the predictive validity of intelligence for life outcomes was significantly diminished, particularly for nonacademic outcomes. Collectively, our findings suggest that, under low-stakes research conditions, some individuals try harder than others, and, in this context, test motivation can act as a third-variable confound that inflates estimates of the predictive validity of intelligence for life outcomes. PMID:21518867

Systematically Differentiating Functions for Alternatively Spliced Isoforms through Integrating RNA-seq Data

PubMed Central

Menon, Rajasree; Wen, Yuchen; Omenn, Gilbert S.; Kretzler, Matthias; Guan, Yuanfang

2013-01-01

Integrating large-scale functional genomic data has significantly accelerated our understanding of gene functions. However, no algorithm has been developed to differentiate functions for isoforms of the same gene using high-throughput genomic data. This is because standard supervised learning requires ‘ground-truth’ functional annotations, which are lacking at the isoform level. To address this challenge, we developed a generic framework that interrogates public RNA-seq data at the transcript level to differentiate functions for alternatively spliced isoforms. For a specific function, our algorithm identifies the ‘responsible’ isoform(s) of a gene and generates classifying models at the isoform level instead of at the gene level. Through cross-validation, we demonstrated that our algorithm is effective in assigning functions to genes, especially the ones with multiple isoforms, and robust to gene expression levels and removal of homologous gene pairs. We identified genes in the mouse whose isoforms are predicted to have disparate functionalities and experimentally validated the ‘responsible’ isoforms using data from mammary tissue. With protein structure modeling and experimental evidence, we further validated the predicted isoform functional differences for the genes Cdkn2a and Anxa6. Our generic framework is the first to predict and differentiate functions for alternatively spliced isoforms, instead of genes, using genomic data. It is extendable to any base machine learner and other species with alternatively spliced isoforms, and shifts the current gene-centered function prediction to isoform-level predictions. PMID:24244129

The construct validity of HPAT-Ireland for the selection of medical students: unresolved issues and future research implications.

PubMed

Kelly, Maureen E; O'Flynn, Siun

2017-05-01

Aptitude tests are widely used in selection. However, despite certain advantages their use remains controversial. This paper aims to critically appraise five sources of evidence for the construct validity of the Health Professions Admission Test (HPAT)-Ireland, an aptitude test used for selecting undergraduate medical students. The objectives are to identify gaps in the evidence, draw comparisons with other aptitude tests and outline future research directions. Our appraisal of the literature found that stakeholder feedback indicates that there is reasonable evidence for test content validity for two of the three sections of HPAT-Ireland. By contrast the Non-Verbal Reasoning section is widely criticised as having limited relevance to medical school performance and future clinical practice. In terms of concurrent validity there is a significant small to medium, negative correlation with school exit examinations, but not consistently so across all studies (r = -0.18, -0.28, 0.017). Likewise predictive validity studies vary, from negative to moderate strength correlations with examination performance during early years at medical school. Five studies indicate that HPAT-Ireland is supported in principle by the majority of stakeholders. While one consequence of its introduction is that successful applicants are now coming from more diverse academic backgrounds, there is no evidence that the socio-economic background of medical school entrants has been altered significantly. Negative perceptions of unfairness relating to gender, coaching and socio-economics remain. The evidence to date suggests that while there are slight gender differences, initially favouring males, these vary year on year. In conclusion, the attitudes towards, and performance of, HPAT-Ireland is not unlike that of other aptitude tests widely used internationally. The main justifications for its introduction have been achieved, in that Ireland no longer relies exclusively on a single measure of academic record for selection to medical school. However a number of areas require further research and exploration.

Validation of Nurse Practitioner Primary Care Organizational Climate Questionnaire: A New Tool to Study Nurse Practitioner Practice Settings.

PubMed

Poghosyan, Lusine; Chaplin, William F; Shaffer, Jonathan A

2017-04-01

Favorable organizational climate in primary care settings is necessary to expand the nurse practitioner (NP) workforce and promote their practice. Only one NP-specific tool, the Nurse Practitioner Primary Care Organizational Climate Questionnaire (NP-PCOCQ), measures NP organizational climate. We confirmed NP-PCOCQ's factor structure and established its predictive validity. A crosssectional survey design was used to collect data from 314 NPs in Massachusetts in 2012. Confirmatory factor analysis and regression models were used. The 4-factor model characterized NP-PCOCQ. The NP-PCOCQ score predicted job satisfaction (beta = .36; p < .001) and intent to leave job (odds ratio = .28; p = .011). NP-PCOCQ can be used by researchers to produce new evidence and by administrators to assess organizational climate in their clinics. Further testing of NP-PCOCQ is needed.

Prediction of citation counts for clinical articles at two years using data available within three weeks of publication: retrospective cohort study

PubMed Central

2008-01-01

Objective To determine if citation counts at two years could be predicted for clinical articles that pass basic criteria for critical appraisal using data within three weeks of publication from external sources and an online article rating service. Design Retrospective cohort study. Setting Online rating service, Canada. Participants 1274 articles from 105 journals published from January to June 2005, randomly divided into a 60:40 split to provide derivation and validation datasets. Main outcome measures 20 article and journal features, including ratings of clinical relevance and newsworthiness, routinely collected by the McMaster online rating of evidence system, compared with citation counts at two years. Results The derivation analysis showed that the regression equation accounted for 60% of the variation (R2=0.60, 95% confidence interval 0.538 to 0.629). This model applied to the validation dataset gave a similar prediction (R2=0.56, 0.476 to 0.596, shrinkage 0.04; shrinkage measures how well the derived equation matches data from the validation dataset). Cited articles in the top half and top third were predicted with 83% and 61% sensitivity and 72% and 82% specificity. Higher citations were predicted by indexing in numerous databases; number of authors; abstraction in synoptic journals; clinical relevance scores; number of cited references; and original, multicentred, and therapy articles from journals with a greater proportion of articles abstracted. Conclusion Citation counts can be reliably predicted at two years using data within three weeks of publication. PMID:18292132

In silico modeling to predict drug-induced phospholipidosis

DOE Office of Scientific and Technical Information (OSTI.GOV)

Choi, Sydney S.; Kim, Jae S.; Valerio, Luis G., E-mail: luis.valerio@fda.hhs.gov

2013-06-01

Drug-induced phospholipidosis (DIPL) is a preclinical finding during pharmaceutical drug development that has implications on the course of drug development and regulatory safety review. A principal characteristic of drugs inducing DIPL is known to be a cationic amphiphilic structure. This provides evidence for a structure-based explanation and opportunity to analyze properties and structures of drugs with the histopathologic findings for DIPL. In previous work from the FDA, in silico quantitative structure–activity relationship (QSAR) modeling using machine learning approaches has shown promise with a large dataset of drugs but included unconfirmed data as well. In this study, we report the constructionmore » and validation of a battery of complementary in silico QSAR models using the FDA's updated database on phospholipidosis, new algorithms and predictive technologies, and in particular, we address high performance with a high-confidence dataset. The results of our modeling for DIPL include rigorous external validation tests showing 80–81% concordance. Furthermore, the predictive performance characteristics include models with high sensitivity and specificity, in most cases above ≥ 80% leading to desired high negative and positive predictivity. These models are intended to be utilized for regulatory toxicology applied science needs in screening new drugs for DIPL. - Highlights: • New in silico models for predicting drug-induced phospholipidosis (DIPL) are described. • The training set data in the models is derived from the FDA's phospholipidosis database. • We find excellent predictivity values of the models based on external validation. • The models can support drug screening and regulatory decision-making on DIPL.« less

Validity and reliability of a questionnaire to assess social skills in traumatic brain injury: A preliminary study.

PubMed

Francis, Heather M; Osborne-Crowley, Katherine; McDonald, Skye

2017-01-01

To describe the reliability and validity of a new measure, the Social Skills Questionnaire for Traumatic Brain Injury (SSQ-TBI). Fifty-one adults with severe TBI completed the SSQ-TBI questionnaire. Scores were compared to informant- and self-report on questionnaires addressing frontal lobe mediated behaviour, as well as performance on an objective measure of social cognition and neuropsychological tasks, in order to provide evidence of concurrent, divergent and predictive validity. Internal consistency was excellent at α = 0.90. Convergent validity was good, with informant ratings on the SSQ-TBI significantly correlated with Neuropsychiatric Inventory Disinhibition sub-scales (r = 0.50-63), the Current Behaviour Scale (r = 0.39-0.48) and Frontal Systems Behaviour Scale (r = 0.60-0.83). However, no relationship was seen with an objective measure of social skills or neuropsychological tasks of disinhibition. There was a significant relationship with real-world psychosocial outcomes on the Sydney Psychosocial Reintegration Scale-2 (r = -0.38--0.69) Conclusions: This study provides preliminary findings of good internal consistency and convergent and predictive validity of a social skills questionnaire adapted to be appropriate for individuals with TBI. Further assessment of psychometric properties such as test-re-test reliability and factor structure is warranted.

Evaluating Validity Evidence for USMLE Step 2 Clinical Skills Data Gathering and Data Interpretation Scores: Does Performance Predict History-Taking and Physical Examination Ratings for First-Year Internal Medicine Residents?

PubMed

Cuddy, Monica M; Winward, Marcia L; Johnston, Mary M; Lipner, Rebecca S; Clauser, Brian E

2016-01-01

To add to the small body of validity research addressing whether scores from performance assessments of clinical skills are related to performance in supervised patient settings, the authors examined relationships between United States Medical Licensing Examination (USMLE) Step 2 Clinical Skills (CS) data gathering and data interpretation scores and subsequent performance in history taking and physical examination in internal medicine residency training. The sample included 6,306 examinees from 238 internal medicine residency programs who completed Step 2 CS for the first time in 2005 and whose performance ratings from their first year of residency training were available. Hierarchical linear modeling techniques were used to examine the relationships among Step 2 CS data gathering and data interpretation scores and history-taking and physical examination ratings. Step 2 CS data interpretation scores were positively related to both history-taking and physical examination ratings. Step 2 CS data gathering scores were not related to either history-taking or physical examination ratings after other USMLE scores were taken into account. Step 2 CS data interpretation scores provide useful information for predicting subsequent performance in history taking and physical examination in supervised practice and thus provide validity evidence for their intended use as an indication of readiness to enter supervised practice. The results show that there is less evidence to support the usefulness of Step 2 CS data gathering scores. This study provides important information for practitioners interested in Step 2 CS specifically or in performance assessments of medical students' clinical skills more generally.

The Potential Role of Sensory Testing, Skin Biopsy, and Functional Brain Imaging as Biomarkers in Chronic Pain Clinical Trials: IMMPACT Considerations.

PubMed

Smith, Shannon M; Dworkin, Robert H; Turk, Dennis C; Baron, Ralf; Polydefkis, Michael; Tracey, Irene; Borsook, David; Edwards, Robert R; Harris, Richard E; Wager, Tor D; Arendt-Nielsen, Lars; Burke, Laurie B; Carr, Daniel B; Chappell, Amy; Farrar, John T; Freeman, Roy; Gilron, Ian; Goli, Veeraindar; Haeussler, Juergen; Jensen, Troels; Katz, Nathaniel P; Kent, Jeffrey; Kopecky, Ernest A; Lee, David A; Maixner, William; Markman, John D; McArthur, Justin C; McDermott, Michael P; Parvathenani, Lav; Raja, Srinivasa N; Rappaport, Bob A; Rice, Andrew S C; Rowbotham, Michael C; Tobias, Jeffrey K; Wasan, Ajay D; Witter, James

2017-07-01

Valid and reliable biomarkers can play an important role in clinical trials as indicators of biological or pathogenic processes or as a signal of treatment response. Currently, there are no biomarkers for pain qualified by the U.S. Food and Drug Administration or the European Medicines Agency for use in clinical trials. This article summarizes an Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials meeting in which 3 potential biomarkers were discussed for use in the development of analgesic treatments: 1) sensory testing, 2) skin punch biopsy, and 3) brain imaging. The empirical evidence supporting the use of these tests is described within the context of the 4 categories of biomarkers: 1) diagnostic, 2) prognostic, 3) predictive, and 4) pharmacodynamic. Although sensory testing, skin punch biopsy, and brain imaging are promising tools for pain in clinical trials, additional evidence is needed to further support and standardize these tests for use as biomarkers in pain clinical trials. The applicability of sensory testing, skin biopsy, and brain imaging as diagnostic, prognostic, predictive, and pharmacodynamic biomarkers for use in analgesic treatment trials is considered. Evidence in support of their use and outlining problems is presented, as well as a call for further standardization and demonstrations of validity and reliability. Copyright © 2017 American Pain Society. All rights reserved.

An evidence-based decision assistance model for predicting training outcome in juvenile guide dogs

PubMed Central

Craigon, Peter J.; Blythe, Simon A.; England, Gary C. W.; Asher, Lucy

2017-01-01

Working dog organisations, such as Guide Dogs, need to regularly assess the behaviour of the dogs they train. In this study we developed a questionnaire-style behaviour assessment completed by training supervisors of juvenile guide dogs aged 5, 8 and 12 months old (n = 1,401), and evaluated aspects of its reliability and validity. Specifically, internal reliability, temporal consistency, construct validity, predictive criterion validity (comparing against later training outcome) and concurrent criterion validity (comparing against a standardised behaviour test) were evaluated. Thirty-nine questions were sourced either from previously published literature or created to meet requirements identified via Guide Dogs staff surveys and staff feedback. Internal reliability analyses revealed seven reliable and interpretable trait scales named according to the questions within them as: Adaptability; Body Sensitivity; Distractibility; Excitability; General Anxiety; Trainability and Stair Anxiety. Intra-individual temporal consistency of the scale scores between 5–8, 8–12 and 5–12 months was high. All scales excepting Body Sensitivity showed some degree of concurrent criterion validity. Predictive criterion validity was supported for all seven scales, since associations were found with training outcome, at at-least one age. Thresholds of z-scores on the scales were identified that were able to distinguish later training outcome by identifying 8.4% of all dogs withdrawn for behaviour and 8.5% of all qualified dogs, with 84% and 85% specificity. The questionnaire assessment was reliable and could detect traits that are consistent within individuals over time, despite juvenile dogs undergoing development during the study period. By applying thresholds to scores produced from the questionnaire this assessment could prove to be a highly valuable decision-making tool for Guide Dogs. This is the first questionnaire-style assessment of juvenile dogs that has shown value in predicting the training outcome of individual working dogs. PMID:28614347

Pulmonary function tests as outcomes for systemic sclerosis interstitial lung disease.

PubMed

Caron, Melissa; Hoa, Sabrina; Hudson, Marie; Schwartzman, Kevin; Steele, Russell

2018-06-30

Interstitial lung disease (ILD) is the leading cause of morbidity and mortality in systemic sclerosis (SSc). We performed a systematic review to characterise the use and validation of pulmonary function tests (PFTs) as surrogate markers for systemic sclerosis-associated interstitial lung disease (SSc-ILD) progression.Five electronic databases were searched to identify all relevant studies. Included studies either used at least one PFT measure as a longitudinal outcome for SSc-ILD progression ( i.e. outcome studies) and/or reported at least one classical measure of validity for the PFTs in SSc-ILD ( i.e. validation studies).This systematic review included 169 outcome studies and 50 validation studies. Diffusing capacity of the lung for carbon monoxide ( D LCO ) was cumulatively the most commonly used outcome until 2010 when it was surpassed by forced vital capacity (FVC). FVC (% predicted) was the primary endpoint in 70.4% of studies, compared to 11.3% for % predicted D LCO Only five studies specifically aimed to validate the PFTs: two concluded that D LCO was the best measure of SSc-ILD extent, while the others did not favour any PFT. These studies also showed respectable validity measures for total lung capacity (TLC).Despite the current preference for FVC, available evidence suggests that D LCO and TLC should not yet be discounted as potential surrogate markers for SSc-ILD progression. Copyright ©ERS 2018.

Development and validation of the Dimensional Anhedonia Rating Scale (DARS) in a community sample and individuals with major depression.

PubMed

Rizvi, Sakina J; Quilty, Lena C; Sproule, Beth A; Cyriac, Anna; Michael Bagby, R; Kennedy, Sidney H

2015-09-30

Anhedonia, a core symptom of Major Depressive Disorder (MDD), is predictive of antidepressant non-response. In contrast to the definition of anhedonia as a "loss of pleasure", neuropsychological studies provide evidence for multiple facets of hedonic function. The aim of the current study was to develop and validate the Dimensional Anhedonia Rating Scale (DARS), a dynamic scale that measures desire, motivation, effort and consummatory pleasure across hedonic domains. Following item selection procedures and reliability testing using data from community participants (N=229) (Study 1), the 17-item scale was validated in an online study with community participants (N=150) (Study 2). The DARS was also validated in unipolar or bipolar depressed patients (n=52) and controls (n=50) (Study 3). Principal components analysis of the 17-item DARS revealed a 4-component structure mapping onto the domains of anhedonia: hobbies, food/drink, social activities, and sensory experience. Reliability of the DARS subscales was high across studies (Cronbach's α=0.75-0.92). The DARS also demonstrated good convergent and divergent validity. Hierarchical regression analysis revealed the DARS showed additional utility over the Snaith-Hamilton Pleasure Scale (SHAPS) in predicting reward function and distinguishing MDD subgroups. These studies provide support for the reliability and validity of the DARS. Copyright © 2015 Elsevier Ireland Ltd. All rights reserved.

Questioning the Validity of the 4+/5+ Binge or Heavy Drinking Criterion in College and Clinical Populations

PubMed Central

Pearson, Matthew R.; Kirouac, Megan; Witkiewitz, Katie

2015-01-01

Background and Aims The terms “binge drinking” and “heavy drinking” are both typically operationalized as 4+/5+ standard drinks per occasion for women/men and are commonly used as a proxy for non-problematic (<4/<5) versus problematic (4+/5+) drinking in multiple research contexts. The Food and Drug Administration in the United States (US) recently proposed the 4+/5+ criterion as a primary efficacy endpoint in their guidance for trials examining new medications for alcohol use disorders (AUDs). Internationally, similar cut-offs have been proposed, with the European Medicines Agency having identified reductions in the number of heavy drinking days (defined as 40/60g pure alcohol in women/men) as a primary endpoint for efficacy trials with a harm reduction goal. Analysis and Evidence We question the validity of the 4+/5+ cutoff (and other similar cutoffs) on multiple accounts. The 4+/5+ cutoff has not been shown to have unique predictive validity or clinical utility. The cutoff has been created based on retrospective self-reports and its use demonstrates ecological bias. Given strong evidence that the relationship between alcohol consumption and problems related to drinking is at least monotonic, if not linear, there is little existing evidence to support the 4+/5+ cutoff as a valid marker of problematic alcohol use. Conclusions There is little empirical evidence for the 4+/5+ units per occasion threshold for “binge” or “heavy” drinking in indexing treatment efficacy. Further consideration of an appropriate threshold seems to be warranted. PMID:27605077

Toward Supersonic Retropropulsion CFD Validation

NASA Technical Reports Server (NTRS)

Kleb, Bil; Schauerhamer, D. Guy; Trumble, Kerry; Sozer, Emre; Barnhardt, Michael; Carlson, Jan-Renee; Edquist, Karl

2011-01-01

This paper begins the process of verifying and validating computational fluid dynamics (CFD) codes for supersonic retropropulsive flows. Four CFD codes (DPLR, FUN3D, OVERFLOW, and US3D) are used to perform various numerical and physical modeling studies toward the goal of comparing predictions with a wind tunnel experiment specifically designed to support CFD validation. Numerical studies run the gamut in rigor from code-to-code comparisons to observed order-of-accuracy tests. Results indicate that this complex flowfield, involving time-dependent shocks and vortex shedding, design order of accuracy is not clearly evident. Also explored is the extent of physical modeling necessary to predict the salient flowfield features found in high-speed Schlieren images and surface pressure measurements taken during the validation experiment. Physical modeling studies include geometric items such as wind tunnel wall and sting mount interference, as well as turbulence modeling that ranges from a RANS (Reynolds-Averaged Navier-Stokes) 2-equation model to DES (Detached Eddy Simulation) models. These studies indicate that tunnel wall interference is minimal for the cases investigated; model mounting hardware effects are confined to the aft end of the model; and sparse grid resolution and turbulence modeling can damp or entirely dissipate the unsteadiness of this self-excited flow.

General Cognitive Ability Predicts Job Performance

DTIC Science & Technology

1992-01-01

of criteria has been and remains the center of controversy. Early intelligence test developers such as Binet and Simon were proponents of g but...Vocational Behavior ( Gottfredson , 1986) documen --,d the renewed interest as did the evidence emerging from validity generalization studies (Hunter, 1983...Among the jobs with highest average estimated intelligence were accounting, engineering, and medicine. Jobs with middling average estimated intelligence

Predictive Validity of Established Cut Points for Risk and Protective Factor Scales from the Communities That Care Youth Survey

ERIC Educational Resources Information Center

Briney, John S.; Brown, Eric C.; Hawkins, J. David; Arthur, Michael W.

2012-01-01

Community coalitions are a popular strategy to coordinate activities and resources to prevent adolescent substance use and delinquent behavior. Despite early evidence of their lack of effectiveness, a new generation of community coalitions has shown positive results in preventing youth substance use and delinquency. This success can be attributed…

Screen Twice, Cut Once: Assessing the Predictive Validity of Teacher Selection Tools. Working Paper 120

ERIC Educational Resources Information Center

Goldhaber, Dan; Grout, Cyrus; Huntington-Klein, Nick

2014-01-01

Evidence suggests that teacher hiring in public schools is ad hoc and often fails to result in good selection among applicants. Some districts use structured selection instruments in the hiring process, but we know little about the efficacy of such tools. In this paper, we evaluate the ability of applicant selection tools used by the Spokane…

Dispositional greed.

PubMed

Seuntjens, Terri G; Zeelenberg, Marcel; van de Ven, Niels; Breugelmans, Seger M

2015-06-01

Greed is an important motive: it is seen as both productive (a source of ambition; the motor of the economy) and destructive (undermining social relationships; the cause of the late 2000s financial crisis). However, relatively little is known about what greed is and does. This article reports on 5 studies that develop and test the 7-item Dispositional Greed Scale (DGS). Study 1 (including 4 separate samples from 2 different countries, total N = 6092) provides evidence for the construct and discriminant validity of the DGS in terms of positive correlations with maximization, self-interest, envy, materialism, and impulsiveness, and negative correlations with self-control and life satisfaction. Study 2 (N = 290) presents further evidence for discriminant validity, finding that the DGS predicts greedy behavioral tendencies over and above materialism. Furthermore, the DGS predicts economic behavior: greedy people allocate more money to themselves in dictator games (Study 3, N = 300) and ultimatum games (Study 4, N = 603), and take more in a resource dilemma (Study 5, N = 305). These findings shed light on what greed is and does, how people differ in greed, and how greed can be measured. In addition, they show the importance of greed in economic behavior and provide directions for future studies. (c) 2015 APA, all rights reserved).

Comment on ``Ratchet universality in the presence of thermal noise''

NASA Astrophysics Data System (ADS)

Quintero, Niurka R.; Alvarez-Nodarse, Renato; Cuesta, José A.

2013-12-01

A recent paper [P. J. Martínez and R. Chacón, Phys. Rev. EPLEEE81539-375510.1103/PhysRevE.87.062114 87, 062114 (2013)] presents numerical simulations on a system exhibiting directed ratchet transport of a driven overdamped Brownian particle subjected to a spatially periodic, symmetric potential. The authors claim that their simulations prove the existence of a universal waveform of the external force that optimally enhances directed transport, hence confirming the validity of a previous conjecture put forth by one of them in the limit of vanishing noise intensity. With minor corrections due to noise, the conjecture holds even in the presence of noise, according to the authors. On the basis of their results the authors claim that all previous theories, which predict a different optimal force waveform, are incorrect. In this Comment we provide sufficient numerical evidence showing that there is no such universal force waveform and that the evidence obtained by the authors otherwise is due to their particular choice of parameters. Our simulations also suggest that previous theories correctly predict the shape of the optimal waveform within their validity regime, namely, when the forcing is weak. On the contrary, the aforementioned conjecture does not hold.

Comment on "Ratchet universality in the presence of thermal noise".

PubMed

Quintero, Niurka R; Alvarez-Nodarse, Renato; Cuesta, José A

2013-12-01

A recent paper [P. J. Martínez and R. Chacón, Phys. Rev. E 87, 062114 (2013)] presents numerical simulations on a system exhibiting directed ratchet transport of a driven overdamped Brownian particle subjected to a spatially periodic, symmetric potential. The authors claim that their simulations prove the existence of a universal waveform of the external force that optimally enhances directed transport, hence confirming the validity of a previous conjecture put forth by one of them in the limit of vanishing noise intensity. With minor corrections due to noise, the conjecture holds even in the presence of noise, according to the authors. On the basis of their results the authors claim that all previous theories, which predict a different optimal force waveform, are incorrect. In this Comment we provide sufficient numerical evidence showing that there is no such universal force waveform and that the evidence obtained by the authors otherwise is due to their particular choice of parameters. Our simulations also suggest that previous theories correctly predict the shape of the optimal waveform within their validity regime, namely, when the forcing is weak. On the contrary, the aforementioned conjecture does not hold.

Validity in work-based assessment: expanding our horizons.

PubMed

Govaerts, Marjan; van der Vleuten, Cees P M

2013-12-01

Although work-based assessments (WBA) may come closest to assessing habitual performance, their use for summative purposes is not undisputed. Most criticism of WBA stems from approaches to validity consistent with the quantitative psychometric framework. However, there is increasing research evidence that indicates that the assumptions underlying the predictive, deterministic framework of psychometrics may no longer hold. In this discussion paper we argue that meaningfulness and appropriateness of current validity evidence can be called into question and that we need alternative strategies to assessment and validity inquiry that build on current theories of learning and performance in complex and dynamic workplace settings. Drawing from research in various professional fields we outline key issues within the mechanisms of learning, competence and performance in the context of complex social environments and illustrate their relevance to WBA. In reviewing recent socio-cultural learning theory and research on performance and performance interpretations in work settings, we demonstrate that learning, competence (as inferred from performance) as well as performance interpretations are to be seen as inherently contextualised, and can only be under-stood 'in situ'. Assessment in the context of work settings may, therefore, be more usefully viewed as a socially situated interpretive act. We propose constructivist-interpretivist approaches towards WBA in order to capture and understand contextualised learning and performance in work settings. Theoretical assumptions underlying interpretivist assessment approaches call for a validity theory that provides the theoretical framework and conceptual tools to guide the validation process in the qualitative assessment inquiry. Basic principles of rigour specific to qualitative research have been established, and they can and should be used to determine validity in interpretivist assessment approaches. If used properly, these strategies generate trustworthy evidence that is needed to develop the validity argument in WBA, allowing for in-depth and meaningful information about professional competence. © 2013 John Wiley & Sons Ltd.

Exploring discrepancies between quantitative validation results and the geomorphic plausibility of statistical landslide susceptibility maps

NASA Astrophysics Data System (ADS)

Steger, Stefan; Brenning, Alexander; Bell, Rainer; Petschko, Helene; Glade, Thomas

2016-06-01

Empirical models are frequently applied to produce landslide susceptibility maps for large areas. Subsequent quantitative validation results are routinely used as the primary criteria to infer the validity and applicability of the final maps or to select one of several models. This study hypothesizes that such direct deductions can be misleading. The main objective was to explore discrepancies between the predictive performance of a landslide susceptibility model and the geomorphic plausibility of subsequent landslide susceptibility maps while a particular emphasis was placed on the influence of incomplete landslide inventories on modelling and validation results. The study was conducted within the Flysch Zone of Lower Austria (1,354 km2) which is known to be highly susceptible to landslides of the slide-type movement. Sixteen susceptibility models were generated by applying two statistical classifiers (logistic regression and generalized additive model) and two machine learning techniques (random forest and support vector machine) separately for two landslide inventories of differing completeness and two predictor sets. The results were validated quantitatively by estimating the area under the receiver operating characteristic curve (AUROC) with single holdout and spatial cross-validation technique. The heuristic evaluation of the geomorphic plausibility of the final results was supported by findings of an exploratory data analysis, an estimation of odds ratios and an evaluation of the spatial structure of the final maps. The results showed that maps generated by different inventories, classifiers and predictors appeared differently while holdout validation revealed similar high predictive performances. Spatial cross-validation proved useful to expose spatially varying inconsistencies of the modelling results while additionally providing evidence for slightly overfitted machine learning-based models. However, the highest predictive performances were obtained for maps that explicitly expressed geomorphically implausible relationships indicating that the predictive performance of a model might be misleading in the case a predictor systematically relates to a spatially consistent bias of the inventory. Furthermore, we observed that random forest-based maps displayed spatial artifacts. The most plausible susceptibility map of the study area showed smooth prediction surfaces while the underlying model revealed a high predictive capability and was generated with an accurate landslide inventory and predictors that did not directly describe a bias. However, none of the presented models was found to be completely unbiased. This study showed that high predictive performances cannot be equated with a high plausibility and applicability of subsequent landslide susceptibility maps. We suggest that greater emphasis should be placed on identifying confounding factors and biases in landslide inventories. A joint discussion between modelers and decision makers of the spatial pattern of the final susceptibility maps in the field might increase their acceptance and applicability.

Performance validity testing in neuropsychology: a clinical guide, critical review, and update on a rapidly evolving literature.

PubMed

Lippa, Sara M

2018-04-01

Over the past two decades, there has been much research on measures of response bias and myriad measures have been validated in a variety of clinical and research samples. This critical review aims to guide clinicians through the use of performance validity tests (PVTs) from test selection and administration through test interpretation and feedback. Recommended cutoffs and relevant test operating characteristics are presented. Other important issues to consider during test selection, administration, interpretation, and feedback are discussed including order effects, coaching, impact on test data, and methods to combine measures and improve predictive power. When interpreting performance validity measures, neuropsychologists must use particular caution in cases of dementia, low intelligence, English as a second language/minority cultures, or low education. PVTs provide valuable information regarding response bias and, under the right circumstances, can provide excellent evidence of response bias. Only after consideration of the entire clinical picture, including validity test performance, can concrete determinations regarding the validity of test data be made.

Using task performance to inform treatment planning for youth with ADHD: A systematic review.

PubMed

Molitor, Stephen J; Langberg, Joshua M

2017-12-01

The role that neuropsychological task performance plays in the assessment of Attention-Deficit/Hyperactivity Disorder (ADHD) is currently ambiguous, and findings are mixed regarding whether tasks have validity for diagnosing the disorder. Irrespective of their validity for diagnosing ADHD, neuropsychological tasks could provide valuable information to mental health professionals if they can inform recommendations for treatment targets and modalities. Therefore, this review sought to synthesize the available evidence related to the use of neuropsychological task performance as a tool for informing treatment planning for youth with ADHD. Reviewed studies focused on examinations of associations between task performance and academic, social, and health outcomes, as well as response to treatment. Twenty-five relevant studies using samples of youth diagnosed with ADHD in clinical, community, and school settings were identified. Review of the evidence suggests that task performance may be useful in identifying individuals with ADHD at risk for academic impairment. However, the evidence is less compelling for identifying youth at risk for impaired social functioning or poor health outcomes. The review also found that task performance is likely useful for predicting response to treatment with methylphenidate. Across studies, evidence indicated that interpreting task performance in an integrated manner, such as a factor score or mean score, was more consistently useful for predicting outcomes of interest than interpreting performance from a single task. Implications for the use of tasks in ADHD assessments are discussed, and future directions are outlined for further examining the clinical utility of task performance. Copyright © 2017 Elsevier Ltd. All rights reserved.

Prior probability and feature predictability interactively bias perceptual decisions

PubMed Central

Dunovan, Kyle E.; Tremel, Joshua J.; Wheeler, Mark E.

2014-01-01

Anticipating a forthcoming sensory experience facilitates perception for expected stimuli but also hinders perception for less likely alternatives. Recent neuroimaging studies suggest that expectation biases arise from feature-level predictions that enhance early sensory representations and facilitate evidence accumulation for contextually probable stimuli while suppressing alternatives. Reasonably then, the extent to which prior knowledge biases subsequent sensory processing should depend on the precision of expectations at the feature level as well as the degree to which expected features match those of an observed stimulus. In the present study we investigated how these two sources of uncertainty modulated pre- and post-stimulus bias mechanisms in the drift-diffusion model during a probabilistic face/house discrimination task. We tested several plausible models of choice bias, concluding that predictive cues led to a bias in both the starting-point and rate of evidence accumulation favoring the more probable stimulus category. We further tested the hypotheses that prior bias in the starting-point was conditional on the feature-level uncertainty of category expectations and that dynamic bias in the drift-rate was modulated by the match between expected and observed stimulus features. Starting-point estimates suggested that subjects formed a constant prior bias in favor of the face category, which exhibits less feature-level variability, that was strengthened or weakened by trial-wise predictive cues. Furthermore, we found that the gain on face/house evidence was increased for stimuli with less ambiguous features and that this relationship was enhanced by valid category expectations. These findings offer new evidence that bridges psychological models of decision-making with recent predictive coding theories of perception. PMID:24978303

20 CFR 404.727 - Evidence of a deemed valid marriage.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 20 Employees' Benefits 2 2010-04-01 2010-04-01 false Evidence of a deemed valid marriage. 404.727... DISABILITY INSURANCE (1950- ) Evidence Evidence of Age, Marriage, and Death § 404.727 Evidence of a deemed valid marriage. (a) General. A deemed valid marriage is a ceremonial marriage we consider valid even...

A Clinical Tool for the Prediction of Venous Thromboembolism in Pediatric Trauma Patients.

PubMed

Connelly, Christopher R; Laird, Amy; Barton, Jeffrey S; Fischer, Peter E; Krishnaswami, Sanjay; Schreiber, Martin A; Zonies, David H; Watters, Jennifer M

2016-01-01

Although rare, the incidence of venous thromboembolism (VTE) in pediatric trauma patients is increasing, and the consequences of VTE in children are significant. Studies have demonstrated increasing VTE risk in older pediatric trauma patients and improved VTE rates with institutional interventions. While national evidence-based guidelines for VTE screening and prevention are in place for adults, none exist for pediatric patients, to our knowledge. To develop a risk prediction calculator for VTE in children admitted to the hospital after traumatic injury to assist efforts in developing screening and prophylaxis guidelines for this population. Retrospective review of 536,423 pediatric patients 0 to 17 years old using the National Trauma Data Bank from January 1, 2007, to December 31, 2012. Five mixed-effects logistic regression models of varying complexity were fit on a training data set. Model validity was determined by comparison of the area under the receiver operating characteristic curve (AUROC) for the training and validation data sets from the original model fit. A clinical tool to predict the risk of VTE based on individual patient clinical characteristics was developed from the optimal model. Diagnosis of VTE during hospital admission. Venous thromboembolism was diagnosed in 1141 of 536,423 children (overall rate, 0.2%). The AUROCs in the training data set were high (range, 0.873-0.946) for each model, with minimal AUROC attenuation in the validation data set. A prediction tool was developed from a model that achieved a balance of high performance (AUROCs, 0.945 and 0.932 in the training and validation data sets, respectively; P = .048) and parsimony. Points are assigned to each variable considered (Glasgow Coma Scale score, age, sex, intensive care unit admission, intubation, transfusion of blood products, central venous catheter placement, presence of pelvic or lower extremity fractures, and major surgery), and the points total is converted to a VTE risk score. The predicted risk of VTE ranged from 0.0% to 14.4%. We developed a simple clinical tool to predict the risk of developing VTE in pediatric trauma patients. It is based on a model created using a large national database and was internally validated. The clinical tool requires external validation but provides an initial step toward the development of the specific VTE protocols for pediatric trauma patients.

Testing the Relations Between Impulsivity-Related Traits, Suicidality, and Nonsuicidal Self-Injury: A Test of the Incremental Validity of the UPPS Model

PubMed Central

Lynam, Donald R.; Miller, Joshua D.; Miller, Drew J.; Bornovalova, Marina A.; Lejuez, C. W.

2011-01-01

Borderline personality disorder (BPD) has received significant attention as a predictor of suicidal behavior (SB) and nonsuicidal self-injury (NSSI). Despite significant promise, trait impulsivity has received less attention. Understanding the relations between impulsivity and SB and NSSI is confounded, unfortunately, by the heterogeneous nature of impulsivity. This study examined the relations among 4 personality pathways to impulsive behavior studied via the UPPS model of impulsivity and SB and NSSI in a residential sample of drug abusers (N = 76). In this study, we tested whether these 4 impulsivity-related traits (i.e., Negative Urgency, Sensation Seeking, Lack of Premeditation, and Lack of Perseverance) provide incremental validity in the statistical prediction of SB and NSSI above and beyond BPD; they do. We also tested whether BPD symptoms provide incremental validity in the prediction of SB and NSSI above and beyond these impulsivity-related traits; they do not. In addition to the main effects of Lack of Premeditation and Negative Urgency, we found evidence of a robust interaction between these 2 personality traits. The current results argue strongly for the consideration of these 2 impulsivity-related domains—alone and in interaction—when attempting to understand and predict SB and NSSI. PMID:21833346

Testing the relations between impulsivity-related traits, suicidality, and nonsuicidal self-injury: a test of the incremental validity of the UPPS model.

PubMed

Lynam, Donald R; Miller, Joshua D; Miller, Drew J; Bornovalova, Marina A; Lejuez, C W

2011-04-01

Borderline personality disorder (BPD) has received significant attention as a predictor of suicidal behavior (SB) and nonsuicidal self-injury (NSSI). Despite significant promise, trait impulsivity has received less attention. Understanding the relations between impulsivity and SB and NSSI is confounded, unfortunately, by the heterogeneous nature of impulsivity. This study examined the relations among 4 personality pathways to impulsive behavior studied via the UPPS model of impulsivity and SB and NSSI in a residential sample of drug abusers (N = 76). In this study, we tested whether these 4 impulsivity-related traits (i.e., Negative Urgency, Sensation Seeking, Lack of Premeditation, and Lack of Perseverance) provide incremental validity in the statistical prediction of SB and NSSI above and beyond BPD; they do. We also tested whether BPD symptoms provide incremental validity in the prediction of SB and NSSI above and beyond these impulsivity-related traits; they do not. In addition to the main effects of Lack of Premeditation and Negative Urgency, we found evidence of a robust interaction between these 2 personality traits. The current results argue strongly for the consideration of these 2 impulsivity-related domains--alone and in interaction--when attempting to understand and predict SB and NSSI.

Tinnitus Screener: Results From the First 100 Participants in an Epidemiology Study.

PubMed

Henry, James A; Griest, Susan; Austin, Don; Helt, Wendy; Gordon, Jane; Thielman, Emily; Theodoroff, Sarah M; Lewis, M Samantha; Blankenship, Cody; Zaugg, Tara L; Carlson, Kathleen

2016-06-01

In the Noise Outcomes in Servicemembers Epidemiology Study, Veterans recently separated from the military undergo comprehensive assessments to initiate long-term monitoring of their auditory function. We developed the Tinnitus Screener, a four-item algorithmic instrument that determines whether tinnitus is present and, if so, whether it is constant or intermittent, or whether only temporary tinnitus has been experienced. Predictive validity data are presented for the first 100 Noise Outcomes in Servicemembers Epidemiology Study participants. The Tinnitus Screener was administered to participants by telephone. In lieu of a gold standard for determining tinnitus presence, the predictive validity of the tinnitus category assigned to participants on the basis of the Screener results was assessed when the participants attended audiologic testing. Of the 100 participants, 67 screened positive for intermittent or constant tinnitus. Three were categorized as "temporary" tinnitus only, and 30 were categorized as "no tinnitus." Tinnitus categorization was predictively valid with 96 of the 100 participants. These results provide preliminary evidence that the Screener may be suitable for quickly determining essential parameters of reported tinnitus. We have since revised the instrument to differentiate acute from chronic tinnitus and to identify occasional tinnitus. We are also obtaining measures that will enable assessment of its test-retest reliability.

Validation of biomarkers to predict response to immunotherapy in cancer: Volume I - pre-analytical and analytical validation.

PubMed

Masucci, Giuseppe V; Cesano, Alessandra; Hawtin, Rachael; Janetzki, Sylvia; Zhang, Jenny; Kirsch, Ilan; Dobbin, Kevin K; Alvarez, John; Robbins, Paul B; Selvan, Senthamil R; Streicher, Howard Z; Butterfield, Lisa H; Thurin, Magdalena

2016-01-01

Immunotherapies have emerged as one of the most promising approaches to treat patients with cancer. Recently, there have been many clinical successes using checkpoint receptor blockade, including T cell inhibitory receptors such as cytotoxic T-lymphocyte-associated antigen 4 (CTLA-4) and programmed cell death-1 (PD-1). Despite demonstrated successes in a variety of malignancies, responses only typically occur in a minority of patients in any given histology. Additionally, treatment is associated with inflammatory toxicity and high cost. Therefore, determining which patients would derive clinical benefit from immunotherapy is a compelling clinical question. Although numerous candidate biomarkers have been described, there are currently three FDA-approved assays based on PD-1 ligand expression (PD-L1) that have been clinically validated to identify patients who are more likely to benefit from a single-agent anti-PD-1/PD-L1 therapy. Because of the complexity of the immune response and tumor biology, it is unlikely that a single biomarker will be sufficient to predict clinical outcomes in response to immune-targeted therapy. Rather, the integration of multiple tumor and immune response parameters, such as protein expression, genomics, and transcriptomics, may be necessary for accurate prediction of clinical benefit. Before a candidate biomarker and/or new technology can be used in a clinical setting, several steps are necessary to demonstrate its clinical validity. Although regulatory guidelines provide general roadmaps for the validation process, their applicability to biomarkers in the cancer immunotherapy field is somewhat limited. Thus, Working Group 1 (WG1) of the Society for Immunotherapy of Cancer (SITC) Immune Biomarkers Task Force convened to address this need. In this two volume series, we discuss pre-analytical and analytical (Volume I) as well as clinical and regulatory (Volume II) aspects of the validation process as applied to predictive biomarkers for cancer immunotherapy. To illustrate the requirements for validation, we discuss examples of biomarker assays that have shown preliminary evidence of an association with clinical benefit from immunotherapeutic interventions. The scope includes only those assays and technologies that have established a certain level of validation for clinical use (fit-for-purpose). Recommendations to meet challenges and strategies to guide the choice of analytical and clinical validation design for specific assays are also provided.

Interpretation of Ambiguity in Individuals with Obsessive-Compulsive Symptoms

PubMed Central

Kuckertz, Jennie M.; Amir, Nader; Tobin, Anastacia C.; Najmi, Sadia

2013-01-01

In two experiments we examined the psychometric properties of a new measure of interpretation bias in individuals with obsessive-compulsive symptoms (OCs). In Experiment 1, 38 individuals high in OC symptoms, 34 individuals high in anxiety and dysphoric symptoms, and 31 asymptomatic individuals completed the measure. Results revealed that the Word Sentence Association Test for OCD (WSAO) can differentiate those with OC symptoms from both a matched anxious/dysphoric group and a non-anxious/non-dysphoric group. In a second experiment, we tested the predictive validity of the WSAO using a performance-based behavioral approach test of contamination fears, and found that the WSAO was a better predictor of avoidance than an established measure of OC washing symptoms (Obsessive Compulsive Inventory-Revised, washing subscale). Our results provide preliminary evidence for the reliability and validity of the WSAO as well as its usefulness in predicting response to behavioral challenge above and beyond OC symptoms, depression, and anxiety. PMID:24179287

Development and Validation of the Narrative Quality Assessment Tool.

PubMed

Kim, Wonsun Sunny; Shin, Cha-Nam; Kathryn Larkey, Linda; Roe, Denise J

2017-04-01

The use of storytelling in health promotion has grown over the past 2 decades, showing promise for moving people to initiate healthy behavior change. Given the increasingly prevalent role of storytelling in health promotion research and the need to more clearly identify what storytelling elements and mediators may better predict behavior change, there is a need to develop measures to specifically assess these factors in a cultural community context. The purpose of this study is to develop and preliminarily validate a narrative quality assessment tool for measuring elements of storytelling that are predicted to affect attitude and behavior change (i.e., narrative characteristics, identification, and transportation) within a cultural community setting using a culture-centric model. Reliability and validity of these scales were assessed with repeated administrations among 74 Latino men and women with a mean age of 39.6 years (SD = 11.47 years). The confirmatory factor analysis in addition to internal consistency tests revealed preliminary evidence for reliability and validity of the narrative characteristics, identification, and transportation scales. Cronbach's alpha ranged from .92 to .94. Items revealed adequate factor loadings (.85-.98) and good model fit. The new scales provide the first step in moving the assessment of narrative quality into a culturally relevant context for evaluation of story use in health promotion. The results present valuable information for nurse researchers to guide the development and testing of culturally grounded storytelling interventions' potential to predict attitude and behavior change for patients.

In Silico Knockout Screening of Plasmodium falciparum Reactions and Prediction of Novel Essential Reactions by Analysing the Metabolic Network

PubMed Central

Isewon, Itunuoluwa; Aromolaran, Olufemi; Oladipupo, Olufunke

2018-01-01

Malaria is an infectious disease that affects close to half a million individuals every year and Plasmodium falciparum is a major cause of malaria. The treatment of this disease could be done effectively if the essential enzymes of this parasite are specifically targeted. Nevertheless, the development of the parasite in resisting existing drugs now makes discovering new drugs a core responsibility. In this study, a novel computational model that makes the prediction of new and validated antimalarial drug target cheaper, easier, and faster has been developed. We have identified new essential reactions as potential targets for drugs in the metabolic network of the parasite. Among the top seven (7) predicted essential reactions, four (4) have been previously identified in earlier studies with biological evidence and one (1) has been with computational evidence. The results from our study were compared with an extensive list of seventy-seven (77) essential reactions with biological evidence from a previous study. We present a list of thirty-one (31) potential candidates for drug targets in Plasmodium falciparum which includes twenty-four (24) new potential candidates for drug targets. PMID:29789805

Advances in the treatment of polyarticular juvenile idiopathic arthritis

PubMed Central

Webb, Kate; Wedderburn, Lucy R.

2015-01-01

Purpose of review To review recent advances in the management strategies of polyarticular course juvenile idiopathic arthritis (JIA) and identify unanswered questions and avenues for further research. Recent findings There is evidence for an early, aggressive, treat-to-target approach for polyarticular JIA. Clinical disease activity criteria have been recently defined and validated, including criteria for inactive disease and the juvenile arthritis disease activity score (JADAS). There is a need for evidence-based, defined disease targets and biomarkers for prediction of response, including targets for remission induction, and guidelines on drug withdrawal. Recent treatment consensus plans and guidelines are discussed and compared, including the 2015 NHS England clinical policy statement, the 2014 Childhood Arthritis and Rheumatology Research Alliance (CARRA) treatment plans and the 2011 American College of Rheumatology (ACR) guidelines. Evidence for new agents such as tocilizumab, rituximab, golimumab, ustekinumab, certolizumab and tofacitinib is promising: the recent clinical trials are summarized here. Stratification of individual patient treatment remains a goal, and predictive biomarkers have been shown to predict success in the withdrawal of methotrexate therapy. Summary There are promising advances in the treatment approaches, disease activity criteria, clinical guidelines, pharmaceutical choices and individually stratified therapy choices for polyarticular JIA. PMID:26147756

In silico prediction of novel therapeutic targets using gene-disease association data.

PubMed

Ferrero, Enrico; Dunham, Ian; Sanseau, Philippe

2017-08-29

Target identification and validation is a pressing challenge in the pharmaceutical industry, with many of the programmes that fail for efficacy reasons showing poor association between the drug target and the disease. Computational prediction of successful targets could have a considerable impact on attrition rates in the drug discovery pipeline by significantly reducing the initial search space. Here, we explore whether gene-disease association data from the Open Targets platform is sufficient to predict therapeutic targets that are actively being pursued by pharmaceutical companies or are already on the market. To test our hypothesis, we train four different classifiers (a random forest, a support vector machine, a neural network and a gradient boosting machine) on partially labelled data and evaluate their performance using nested cross-validation and testing on an independent set. We then select the best performing model and use it to make predictions on more than 15,000 genes. Finally, we validate our predictions by mining the scientific literature for proposed therapeutic targets. We observe that the data types with the best predictive power are animal models showing a disease-relevant phenotype, differential expression in diseased tissue and genetic association with the disease under investigation. On a test set, the neural network classifier achieves over 71% accuracy with an AUC of 0.76 when predicting therapeutic targets in a semi-supervised learning setting. We use this model to gain insights into current and failed programmes and to predict 1431 novel targets, of which a highly significant proportion has been independently proposed in the literature. Our in silico approach shows that data linking genes and diseases is sufficient to predict novel therapeutic targets effectively and confirms that this type of evidence is essential for formulating or strengthening hypotheses in the target discovery process. Ultimately, more rapid and automated target prioritisation holds the potential to reduce both the costs and the development times associated with bringing new medicines to patients.

Validation of High Frequency (HF) Propagation Prediction Models in the Arctic region

NASA Astrophysics Data System (ADS)

Athieno, R.; Jayachandran, P. T.

2014-12-01

Despite the emergence of modern techniques for long distance communication, Ionospheric communication in the high frequency (HF) band (3-30 MHz) remains significant to both civilian and military users. However, the efficient use of the ever-varying ionosphere as a propagation medium is dependent on the reliability of ionospheric and HF propagation prediction models. Most available models are empirical implying that data collection has to be sufficiently large to provide good intended results. The models we present were developed with little data from the high latitudes which necessitates their validation. This paper presents the validation of three long term High Frequency (HF) propagation prediction models over a path within the Arctic region. Measurements of the Maximum Usable Frequency for a 3000 km range (MUF (3000) F2) for Resolute, Canada (74.75° N, 265.00° E), are obtained from hand-scaled ionograms generated by the Canadian Advanced Digital Ionosonde (CADI). The observations have been compared with predictions obtained from the Ionospheric Communication Enhanced Profile Analysis Program (ICEPAC), Voice of America Coverage Analysis Program (VOACAP) and International Telecommunication Union Recommendation 533 (ITU-REC533) for 2009, 2011, 2012 and 2013. A statistical analysis shows that the monthly predictions seem to reproduce the general features of the observations throughout the year though it is more evident in the winter and equinox months. Both predictions and observations show a diurnal and seasonal variation. The analysed models did not show large differences in their performances. However, there are noticeable differences across seasons for the entire period analysed: REC533 gives a better performance in winter months while VOACAP has a better performance for both equinox and summer months. VOACAP gives a better performance in the daily predictions compared to ICEPAC though, in general, the monthly predictions seem to agree more with the observations compared to the daily predictions.

Clinical utility of the Neurobehavioral Symptom Inventory validity scales to screen for symptom exaggeration following traumatic brain injury.

PubMed

Lange, Rael T; Brickell, Tracey A; Lippa, Sara M; French, Louis M

2015-01-01

The purpose of this study was to examine the clinical utility of three recently developed validity scales (Validity-10, NIM5, and LOW6) designed to screen for symptom exaggeration using the Neurobehavioral Symptom Inventory (NSI). Participants were 272 U.S. military service members who sustained a mild, moderate, severe, or penetrating traumatic brain injury (TBI) and who were evaluated by the neuropsychology service at Walter Reed Army Medical Center within 199 weeks post injury. Participants were divided into two groups based on the Negative Impression Management scale of the Personality Assessment Inventory: (a) those who failed symptom validity testing (SVT-fail; n = 27) and (b) those who passed symptom validity testing (SVT-pass; n = 245). Participants in the SVT-fail group had significantly higher scores (p<.001) on the Validity-10, NIM5, LOW6, NSI total, and Personality Assessment Inventory (PAI) clinical scales (range: d = 0.76 to 2.34). Similarly high sensitivity, specificity, positive predictive power (PPP), and negative predictive (NPP) values were found when using all three validity scales to differentiate SVT-fail versus SVT-pass groups. However, the Validity-10 scale consistently had the highest overall values. The optimal cutoff score for the Validity-10 scale to identify possible symptom exaggeration was ≥19 (sensitivity = .59, specificity = .89, PPP = .74, NPP = .80). For the majority of people, these findings provide support for the use of the Validity-10 scale as a screening tool for possible symptom exaggeration. When scores on the Validity-10 exceed the cutoff score, it is recommended that (a) researchers and clinicians do not interpret responses on the NSI, and (b) clinicians follow up with a more detailed evaluation, using well-validated symptom validity measures (e.g., Minnesota Multiphasic Personality Inventory-2 Restructured Form, MMPI-2-RF, validity scales), to seek confirmatory evidence to support an hypothesis of symptom exaggeration.

The Redesigned SAT® Pilot Predictive Validity Study: A First Look. Research Report 2016-1

ERIC Educational Resources Information Center

Shaw, Emily J.; Marini, Jessica P.; Beard, Jonathan; Shmueli, Doron; Young, Linda; Ng, Helen

2016-01-01

In February of 2013, the College Board announced it would undertake a redesign of the SAT® in order to develop an assessment that better reflects the work that students will do in college, focusing on the core knowledge and skills that evidence has shown to be critical in preparation for college and career. The redesigned test will be introduced…

Examining an Executive Function Rating Scale as a Predictor of Achievement in Children at Risk for Behavior Problems

ERIC Educational Resources Information Center

Sadeh, Shanna S.; Burns, Matthew K.; Sullivan, Amanda L.

2012-01-01

Evidence suggests that executive function (EF) may be a potent and malleable predictor of academic achievement in children. Schools may be able to use this predictive power if researchers develop EF measures that not only have ecological and construct validity, but also are also efficient and affordable. To this end, Garcia-Barrera and colleagues…

External validation of anti-Müllerian hormone based prediction of live birth in assisted conception

PubMed Central

2013-01-01

Background Chronological age and oocyte yield are independent determinants of live birth in assisted conception. Anti-Müllerian hormone (AMH) is strongly associated with oocyte yield after controlled ovarian stimulation. We have previously assessed the ability of AMH and age to independently predict live birth in an Italian assisted conception cohort. Herein we report the external validation of the nomogram in 822 UK first in vitro fertilization (IVF) cycles. Methods Retrospective cohort consisting of 822 patients undergoing their first IVF treatment cycle at Glasgow Centre for Reproductive Medicine. Analyses were restricted to women aged between 25 and 42 years of age. All women had an AMH measured prior to commencing their first IVF cycle. The performance of the model was assessed; discrimination by the area under the receiver operator curve (ROCAUC) and model calibration by the predicted probability versus observed probability. Results Live births occurred in 29.4% of the cohort. The observed and predicted outcomes showed no evidence of miscalibration (p = 0.188). The ROCAUC was 0.64 (95% CI: 0.60, 0.68), suggesting moderate and similar discrimination to the original model. The ROCAUC for a continuous model of age and AMH was 0.65 (95% CI 0.61, 0.69), suggesting that the original categories of AMH were appropriate. Conclusions We confirm by external validation that AMH and age are independent predictors of live birth. Although the confidence intervals for each category are wide, our results support the assessment of AMH in larger cohorts with detailed baseline phenotyping for live birth prediction. PMID:23294733

Predictive validity of callous-unemotional traits measured in early adolescence with respect to multiple antisocial outcomes.

PubMed

McMahon, Robert J; Witkiewitz, Katie; Kotler, Julie S

2010-11-01

This study investigated the predictive validity of youth callous-unemotional (CU) traits, as measured in early adolescence (Grade 7) by the Antisocial Process Screening Device (APSD; Frick & Hare, 2001), in a longitudinal sample (N = 754). Antisocial outcomes, assessed in adolescence and early adulthood, included self-reported general delinquency from 7th grade through 2 years post-high school, self-reported serious crimes through 2 years post-high school, juvenile and adult arrest records through 1 year post-high school, and antisocial personality disorder symptoms and diagnosis at 2 years post-high school. CU traits measured in 7th grade were highly predictive of 5 of the 6 antisocial outcomes-general delinquency, juvenile and adult arrests, and early adult antisocial personality disorder criterion count and diagnosis-over and above prior and concurrent conduct problem behavior (i.e., criterion counts of oppositional defiant disorder and conduct disorder) and attention-deficit/hyperactivity disorder (criterion count). Incorporating a CU traits specifier for those with a diagnosis of conduct disorder improved the positive prediction of antisocial outcomes, with a very low false-positive rate. There was minimal evidence of moderation by sex, race, or urban/rural status. Urban/rural status moderated one finding, with being from an urban area associated with stronger relations between CU traits and adult arrests. Findings clearly support the inclusion of CU traits as a specifier for the diagnosis of conduct disorder, at least with respect to predictive validity. PsycINFO Database Record (c) 2010 APA, all rights reserved

CASPer, an online pre-interview screen for personal/professional characteristics: prediction of national licensure scores.

PubMed

Dore, Kelly L; Reiter, Harold I; Kreuger, Sharyn; Norman, Geoffrey R

2017-05-01

Typically, only a minority of applicants to health professional training are invited to interview. However, pre-interview measures of cognitive skills predict for national licensure scores (Gauer et al. in Med Educ Online 21 2016) and subsequently licensure scores predict for performance in practice (Tamblyn et al. in JAMA 288(23): 3019-3026, 2002; Tamblyn et al. in JAMA 298(9):993-1001, 2007). Assessment of personal and professional characteristics, with the same psychometric rigour of measures of cognitive abilities, are needed upstream in the selection to health profession training programs. To fill that need, Computer-based Assessment for Sampling Personal characteristics (CASPer)-an on-line, video-based screening test-was created. In this paper, we examine the correlation between CASPer and Canadian national licensure examination outcomes in 109 doctors who took CASPer at the time of selection to medical school. Specifically, CASPer scores were correlated against performance on cognitive and 'non-cognitive' subsections of both the Medical Council of Canada Qualifying Examination (MCCQE) Parts I (end of medical school) and Part II (18 months into specialty training). Unlike most national licensure exams, MCCQE has specific subcomponents examining personal/professional qualities, providing a unique opportunity for comparison. The results demonstrated moderate predictive validity of CASPer to national licensure outcomes of personal/professional characteristics three to six years after admission to medical school. These types of disattenuated correlations (r = 0.3-0.5) are not otherwise predicted by traditional screening measures. These data support the ability of a computer-based strategy to screen applicants in a feasible, reliable test, which has now demonstrated predictive validity, lending evidence of its validation for medical school applicant selection.

Validation of Alternative In Vitro Methods to Animal Testing: Concepts, Challenges, Processes and Tools.

PubMed

Griesinger, Claudius; Desprez, Bertrand; Coecke, Sandra; Casey, Warren; Zuang, Valérie

This chapter explores the concepts, processes, tools and challenges relating to the validation of alternative methods for toxicity and safety testing. In general terms, validation is the process of assessing the appropriateness and usefulness of a tool for its intended purpose. Validation is routinely used in various contexts in science, technology, the manufacturing and services sectors. It serves to assess the fitness-for-purpose of devices, systems, software up to entire methodologies. In the area of toxicity testing, validation plays an indispensable role: "alternative approaches" are increasingly replacing animal models as predictive tools and it needs to be demonstrated that these novel methods are fit for purpose. Alternative approaches include in vitro test methods, non-testing approaches such as predictive computer models up to entire testing and assessment strategies composed of method suites, data sources and decision-aiding tools. Data generated with alternative approaches are ultimately used for decision-making on public health and the protection of the environment. It is therefore essential that the underlying methods and methodologies are thoroughly characterised, assessed and transparently documented through validation studies involving impartial actors. Importantly, validation serves as a filter to ensure that only test methods able to produce data that help to address legislative requirements (e.g. EU's REACH legislation) are accepted as official testing tools and, owing to the globalisation of markets, recognised on international level (e.g. through inclusion in OECD test guidelines). Since validation creates a credible and transparent evidence base on test methods, it provides a quality stamp, supporting companies developing and marketing alternative methods and creating considerable business opportunities. Validation of alternative methods is conducted through scientific studies assessing two key hypotheses, reliability and relevance of the test method for a given purpose. Relevance encapsulates the scientific basis of the test method, its capacity to predict adverse effects in the "target system" (i.e. human health or the environment) as well as its applicability for the intended purpose. In this chapter we focus on the validation of non-animal in vitro alternative testing methods and review the concepts, challenges, processes and tools fundamental to the validation of in vitro methods intended for hazard testing of chemicals. We explore major challenges and peculiarities of validation in this area. Based on the notion that validation per se is a scientific endeavour that needs to adhere to key scientific principles, namely objectivity and appropriate choice of methodology, we examine basic aspects of study design and management, and provide illustrations of statistical approaches to describe predictive performance of validated test methods as well as their reliability.

Playing prosocial video games increases the accessibility of prosocial thoughts.

PubMed

Greitemeyer, Tobias; Osswald, Silvia

2011-01-01

Past research has provided abundant evidence that playing violent video games increases aggressive tendencies. In contrast, evidence on possible positive effects of video game exposure on prosocial tendencies has been relatively sparse. The present research tested and found support for the hypothesis that exposure to prosocial video games increases the accessibility of prosocial thoughts. These results provide support to the predictive validity of the General Learning Model (Buckley & Anderson, 2006) for the effects of exposure to prosocial media on social tendencies. Thus, depending on the content of the video game, playing video games can harm but may also benefit social relations.

Evidence regarding the utility of multiple mini-interview (MMI) for selection to undergraduate health programs: A BEME systematic review: BEME Guide No. 37.

PubMed

Rees, Eliot L; Hawarden, Ashley W; Dent, Gordon; Hays, Richard; Bates, Joanna; Hassell, Andrew B

2016-05-01

In the 11 years since its development at McMaster University Medical School, the multiple mini-interview (MMI) has become a popular selection tool. We aimed to systematically explore, analyze and synthesize the evidence regarding MMIs for selection to undergraduate health programs. The review protocol was peer-reviewed and prospectively registered with the Best Evidence Medical Education (BEME) collaboration. Thirteen databases were searched through 34 terms and their Boolean combinations. Seven key journals were hand-searched since 2004. The reference sections of all included studies were screened. Studies meeting the inclusion criteria were coded independently by two reviewers using a modified BEME coding sheet. Extracted data were synthesized through narrative synthesis. A total of 4338 citations were identified and screened, resulting in 41 papers that met inclusion criteria. Thirty-two studies report data for selection to medicine, six for dentistry, three for veterinary medicine, one for pharmacy, one for nursing, one for rehabilitation, and one for health science. Five studies investigated selection to more than one profession. MMIs used for selection to undergraduate health programs appear to have reasonable feasibility, acceptability, validity, and reliability. Reliability is optimized by including 7-12 stations, each with one examiner. The evidence is stronger for face validity, with more research needed to explore content validity and predictive validity. In published studies, MMIs do not appear biased against applicants on the basis of age, gender, or socio-economic status. However, applicants of certain ethnic and social backgrounds did less well in a very small number of published studies. Performance on MMIs does not correlate strongly with other measures of noncognitive attributes, such as personality inventories and measures of emotional intelligence. MMI does not automatically mean a more reliable selection process but it can do, if carefully designed. Effective MMIs require careful identification of the noncognitive attributes sought by the program and institution. Attention needs to be given to the number of stations, the blueprint and examiner training. More work is required on MMIs as they may disadvantage groups of certain ethnic or social backgrounds. There is a compelling argument for multi-institutional studies to investigate areas such as the relationship of MMI content to curriculum domains, graduate outcomes, and social missions; relationships of applicants' performance on different MMIs; bias in selecting applicants of minority groups; and the long-term outcomes appropriate for studies of predictive validity.

Why don't you exercise? Development of the Amotivation Toward Exercise Scale among older inactive individuals.

PubMed

Vlachopoulos, Symeon P; Gigoudi, Maria A

2008-07-01

This article reports on the development and initial validation of the Amotivation Toward Exercise Scale (ATES), which reflects a taxonomy of older adults' reasons to refrain from exercise. Drawing on work by Pelletier, Dion, Tuson, and Green-Demers (1999) and Legault, Green-Demers, and Pelletier (2006), these dimensions were the outcome beliefs, capacity beliefs, effort beliefs, and value amotivation beliefs toward exercise. The results supported a 4-factor correlated model that fit the data better than either a unidimensional model or a 4-factor uncorrelated model or a hierarchical model with strong internal reliability for all the subscales. Evidence also emerged for the discriminant validity of the subscale scores. Furthermore, the predictive validity of the subscale scores was supported, and satisfactory measurement invariance was demonstrated across the calibration and validation samples, supporting the generalizability of the scale's measurement properties.

Does IQ Really Predict Job Performance?

PubMed Central

Richardson, Ken; Norgate, Sarah H.

2015-01-01

IQ has played a prominent part in developmental and adult psychology for decades. In the absence of a clear theoretical model of internal cognitive functions, however, construct validity for IQ tests has always been difficult to establish. Test validity, therefore, has always been indirect, by correlating individual differences in test scores with what are assumed to be other criteria of intelligence. Job performance has, for several reasons, been one such criterion. Correlations of around 0.5 have been regularly cited as evidence of test validity, and as justification for the use of the tests in developmental studies, in educational and occupational selection and in research programs on sources of individual differences. Here, those correlations are examined together with the quality of the original data and the many corrections needed to arrive at them. It is concluded that considerable caution needs to be exercised in citing such correlations for test validation purposes. PMID:26405429

The influence of work personality on job satisfaction: incremental validity and mediation effects.

PubMed

Heller, Daniel; Ferris, D Lance; Brown, Douglas; Watson, David

2009-08-01

Drawing from recent developments regarding the contextual nature of personality (e.g., D. Wood & B. W. Roberts, 2006), we conducted 2 studies (1 cross-sectional and 1 longitudinal over 1 year) to examine the validity of work personality in predicting job satisfaction and its mediation of the effect of global personality on job satisfaction. Study 1 showed that (a) individuals vary systematically in their personality between roles- they were significantly more conscientious and open to experience and less extraverted at work compared to at home; (b) work personality was a better predictor of job satisfaction than both global personality and home personality; and (c) work personality demonstrated incremental validity above and beyond the other two personality measures. Study 2 further showed that each of the work personality dimensions fully mediated the association between its corresponding global personality trait and job satisfaction. Evidence for the discriminant validity of the findings is also presented.

Development and validation of a preference based measure derived from the Cambridge Pulmonary Hypertension Outcome Review (CAMPHOR) for use in cost utility analyses.

PubMed

McKenna, Stephen P; Ratcliffe, Julie; Meads, David M; Brazier, John E

2008-08-21

Pulmonary Hypertension is a severe and incurable disease with poor prognosis. A suite of new disease-specific measures--the Cambridge Pulmonary Hypertension Outcome Review (CAMPHOR) - was recently developed for use in this condition. The purpose of this study was to develop and validate a preference based measure from the CAMPHOR that could be used in cost-utility analyses. Items were selected that covered major issues covered by the CAMPHOR QoL scale (activities, travelling, dependence and communication). These were used to create 36 health states that were valued by 249 people representative of the UK adult population, using the time trade-off (TTO) technique. Data from the TTO interviews were analysed using both aggregate and individual level modelling. Finally, the original CAMPHOR validation data were used to validate the new preference based model. The predicted health state values ranged from 0.962 to 0.136. The mean level model selected for analyzing the data had good explanatory power (0.936), did not systematically over- or underestimate the observed mean health state values and showed no evidence of auto correlation in the prediction errors. The value of less than 1 reflects a background level of ill health in state 1111, as judged by the respondents. Scores derived from the new measure had excellent test-retest reliability (0.85) and construct validity. The CAMPHOR utility score appears better able to distinguish between WHO functional classes (II and III) than the EQ-5D and SF-6D. The tariff derived in this study can be used to classify an individual into a health state based on their responses to the CAMPHOR. The results of this study widen the evidence base for conducting economic evaluations of interventions designed to improve QoL for patients with PH.

Measuring disability: a systematic review of the validity and reliability of the Global Activity Limitations Indicator (GALI).

PubMed

Van Oyen, Herman; Bogaert, Petronille; Yokota, Renata T C; Berger, Nicolas

2018-01-01

GALI or Global Activity Limitation Indicator is a global survey instrument measuring participation restriction. GALI is the measure underlying the European indicator Healthy Life Years (HLY). Gali has a substantial policy use within the EU and its Member States. The objective of current paper is to bring together what is known from published manuscripts on the validity and the reliability of GALI. Following the PRISMA guidelines, two search strategies (PUBMED, Google Scholar) were combined to identify manuscripts published in English with publication date 2000 or beyond. Articles were classified as reliability studies, concurrent or predictive validity studies, in national or international populations. Four cross-sectional studies (of which 2 international) studied how GALI relates to other health measures (concurrent validity). A dose-response effect by GALI severity level on the association with the other health status measures was observed in the national studies. The 2 international studies (SHARE, EHIS) concluded that the odds of reporting participation restriction was higher in subjects with self-reported or observed functional limitations. In SHARE, the size of the Odds Ratio's (ORs) in the different countries was homogeneous, while in EHIS the size of the ORs varied more strongly. For the predictive validity, subjects were followed over time (4 studies of which one international). GALI proved, both in national and international data, to be a consistent predictor of future health outcomes both in terms of mortality and health care expenditure. As predictors of mortality, the two distinct health concepts, self-rated health and GALI, acted independently and complementary of each other. The one reliability study identified reported a sufficient reliability of GALI. GALI as inclusive one question instrument fits all conceptual characteristics specified for a global measure on participation restriction. In none of the studies, included in the review, there was evidence of a failing validity. The review shows that GALI has a good and sufficient concurrent and predictive validity, and reliability.

Development and validation of a clinical prediction rule to identify suspected breast cancer: a prospective cohort study.

PubMed

Galvin, Rose; Joyce, Doireann; Downey, Eithne; Boland, Fiona; Fahey, Tom; Hill, Arnold K

2014-10-03

The number of primary care referrals of women with breast symptoms to symptomatic breast units (SBUs) has increased exponentially in the past decade in Ireland. The aim of this study is to develop and validate a clinical prediction rule (CPR) to identify women with breast cancer so that a more evidence based approach to referral from primary care to these SBUs can be developed. We analysed routine data from a prospective cohort of consecutive women reviewed at a SBU with breast symptoms. The dataset was split into a derivation and validation cohort. Regression analysis was used to derive a CPR from the patient's history and clinical findings. Validation of the CPR consisted of estimating the number of breast cancers predicted to occur compared with the actual number of observed breast cancers across deciles of risk. A total of 6,590 patients were included in the derivation study and 4.9% were diagnosed with breast cancer. Independent clinical predictors for breast cancer were: increasing age by year (adjusted odds ratio 1.08, 95% CI 1.07-1.09); presence of a lump (5.63, 95% CI 4.2-7.56); nipple change (2.77, 95% CI 1.68-4.58) and nipple discharge (2.09, 95% CI 1.1-3.97). Validation of the rule (n = 911) demonstrated that the probability of breast cancer was higher with an increasing number of these independent variables. The Hosmer-Lemeshow goodness of fit showed no overall significant difference between the expected and the observed numbers of breast cancer (χ(2)HL: 6.74, p-value: 0.56). This study derived and validated a CPR for breast cancer in women attending an Irish national SBU. We found that increasing age, presence of a lump, nipple discharge and nipple change are all associated with increased risk of breast cancer. Further validation of the rule is necessary as well as an assessment of its impact on referral practice.

Applying Risk Prediction Models to Optimize Lung Cancer Screening: Current Knowledge, Challenges, and Future Directions.

PubMed

Sakoda, Lori C; Henderson, Louise M; Caverly, Tanner J; Wernli, Karen J; Katki, Hormuzd A

2017-12-01

Risk prediction models may be useful for facilitating effective and high-quality decision-making at critical steps in the lung cancer screening process. This review provides a current overview of published lung cancer risk prediction models and their applications to lung cancer screening and highlights both challenges and strategies for improving their predictive performance and use in clinical practice. Since the 2011 publication of the National Lung Screening Trial results, numerous prediction models have been proposed to estimate the probability of developing or dying from lung cancer or the probability that a pulmonary nodule is malignant. Respective models appear to exhibit high discriminatory accuracy in identifying individuals at highest risk of lung cancer or differentiating malignant from benign pulmonary nodules. However, validation and critical comparison of the performance of these models in independent populations are limited. Little is also known about the extent to which risk prediction models are being applied in clinical practice and influencing decision-making processes and outcomes related to lung cancer screening. Current evidence is insufficient to determine which lung cancer risk prediction models are most clinically useful and how to best implement their use to optimize screening effectiveness and quality. To address these knowledge gaps, future research should be directed toward validating and enhancing existing risk prediction models for lung cancer and evaluating the application of model-based risk calculators and its corresponding impact on screening processes and outcomes.

Biomarkers and surrogate endpoints in glaucoma clinical trials

PubMed Central

Medeiros, Felipe A

2015-01-01

Surrogate endpoints are often used as replacements for true clinically relevant endpoints in several areas of medicine, as they enable faster and less expensive clinical trials. However, without proper validation, the use of surrogates may lead to incorrect conclusions about the efficacy and safety of treatments. This article reviews the general requirements for validating surrogate endpoints and provides a critical assessment of the use of intraocular pressure (IOP), visual fields, and structural measurements of the optic nerve as surrogate endpoints in glaucoma clinical trials. A valid surrogate endpoint must be able to predict the clinically relevant endpoint and fully capture the effect of an intervention on that endpoint. Despite its widespread use in clinical trials, no proper validation of IOP as a surrogate endpoint has ever been conducted for any class of IOP-lowering treatments. Evidence has accumulated with regard to the role of imaging measurements of optic nerve damage as surrogate endpoints in glaucoma. These measurements are predictive of functional losses in the disease and may explain, at least in part, treatment effects on clinically relevant endpoints. The use of composite endpoints in glaucoma trials may overcome weaknesses of the use of structural or functional endpoints in isolation. Unless research is dedicated to fully develop and validate suitable endpoints that can be used in glaucoma clinical trials, we run the risk of inappropriate judgments about the value of new therapies. PMID:25034049

Multiattribute health utility scoring for the computerized adaptive measure CAT-5D-QOL was developed and validated.

PubMed

Kopec, Jacek A; Sayre, Eric C; Rogers, Pamela; Davis, Aileen M; Badley, Elizabeth M; Anis, Aslam H; Abrahamowicz, Michal; Russell, Lara; Rahman, Md Mushfiqur; Esdaile, John M

2015-10-01

The CAT-5D-QOL is a previously reported item response theory (IRT)-based computerized adaptive tool to measure five domains (attributes) of health-related quality of life. The objective of this study was to develop and validate a multiattribute health utility (MAHU) scoring method for this instrument. The MAHU scoring system was developed in two stages. In phase I, we obtained standard gamble (SG) utilities for 75 hypothetical health states in which only one domain varied (15 states per domain). In phase II, we obtained SG utilities for 256 multiattribute states. We fit a multiplicative regression model to predict SG utilities from the five IRT domain scores. The prediction model was constrained using data from phase I. We validated MAHU scores by comparing them with the Health Utilities Index Mark 3 (HUI3) and directly measured utilities and by assessing between-group discrimination. MAHU scores have a theoretical range from -0.842 to 1. In the validation study, the scores were, on average, higher than HUI3 utilities and lower than directly measured SG utilities. MAHU scores correlated strongly with the HUI3 (Spearman ρ = 0.78) and discriminated well between groups expected to differ in health status. Results reported here provide initial evidence supporting the validity of the MAHU scoring system for the CAT-5D-QOL. Copyright © 2015 Elsevier Inc. All rights reserved.

The Alcohol Sensitivity Questionnaire: Evidence for Construct Validity

PubMed Central

Fleming, Kimberly A.; Bartholow, Bruce D.; Hilgard, Joseph B.; McCarthy, Denis M.; O’Neill, Susan E.; Steinley, Douglas; Sher, Kenneth J.

2016-01-01

Background Variability in sensitivity to the acute effects of alcohol is an important risk factor for the development of alcohol use disorder (AUD). The most commonly used retrospective self-report measure of sensitivity, the Self-Rating of the Effects of Alcohol form (SRE), queries a limited number of alcohol effects and relies on respondents’ ability to recall experiences that might have occurred in the distant past. Here, we investigated the construct validity of an alternative measure that queries a larger number of alcohol effects, the Alcohol Sensitivity Questionnaire (ASQ), and compared it to the SRE in predicting momentary subjective responses to an acute dose of alcohol. Method Healthy young adults (N = 423) completed the SRE and the ASQ and then were randomly assigned to consume either alcohol or a placebo beverage (between-subjects manipulation). Stimulation and sedation (Biphasic Alcohol Effects Scale) and subjective intoxication were measured multiple times after drinking. Results Hierarchical linear models showed that the ASQ reliably predicted each of these outcomes following alcohol but not placebo consumption, provided unique prediction beyond that associated with differences in recent alcohol involvement, and was preferred over the SRE (in terms of model fit) in direct model comparisons of stimulation and sedation. Conclusions The ASQ compared favorably with the better-known SRE in predicting increased stimulation and reduced sedation following an acute alcohol challenge. The ASQ appears to be a valid self-report measure of alcohol sensitivity and therefore holds promise for identifying individuals at-risk for AUD and related problems. PMID:27012527

Derivation and Evaluation of a Risk-Scoring Tool to Predict Participant Attrition in a Lifestyle Intervention Project.

PubMed

Jiang, Luohua; Yang, Jing; Huang, Haixiao; Johnson, Ann; Dill, Edward J; Beals, Janette; Manson, Spero M; Roubideaux, Yvette

2016-05-01

Participant attrition in clinical trials and community-based interventions is a serious, common, and costly problem. In order to develop a simple predictive scoring system that can quantify the risk of participant attrition in a lifestyle intervention project, we analyzed data from the Special Diabetes Program for Indians Diabetes Prevention Program (SDPI-DP), an evidence-based lifestyle intervention to prevent diabetes in 36 American Indian and Alaska Native communities. SDPI-DP participants were randomly divided into a derivation cohort (n = 1600) and a validation cohort (n = 801). Logistic regressions were used to develop a scoring system from the derivation cohort. The discriminatory power and calibration properties of the system were assessed using the validation cohort. Seven independent factors predicted program attrition: gender, age, household income, comorbidity, chronic pain, site's user population size, and average age of site staff. Six factors predicted long-term attrition: gender, age, marital status, chronic pain, site's user population size, and average age of site staff. Each model exhibited moderate to fair discriminatory power (C statistic in the validation set: 0.70 for program attrition, and 0.66 for long-term attrition) and excellent calibration. The resulting scoring system offers a low-technology approach to identify participants at elevated risk for attrition in future similar behavioral modification intervention projects, which may inform appropriate allocation of retention resources. This approach also serves as a model for other efforts to prevent participant attrition.

Biomarkers and Surrogate Endpoints: Lessons Learned From Glaucoma

PubMed Central

Medeiros, Felipe A.

2017-01-01

With the recent progress in imaging technologies for assessment of structural damage in glaucoma, a debate has emerged on whether these measurements can be used as valid surrogate endpoints in clinical trials evaluating new therapies for the disease. A discussion of surrogates should be grounded on knowledge acquired from their use in other areas of medicine as well as regulatory requirements. This article reviews the conditions for valid surrogacy in the context of glaucoma clinical trials and critically evaluates the role of biomarkers such as IOP and imaging measurements as potential surrogates for clinically relevant outcomes. Valid surrogate endpoints must be able to predict a clinically relevant endpoint, such as loss of vision or decrease in quality of life. In addition, the effect of a proposed treatment on the surrogate must capture the effect of the treatment on the clinically relevant endpoint. Despite its widespread use in clinical trials, no proper validation of IOP as a surrogate endpoint has yet been conducted for any class of IOP-lowering treatments. Although strong evidence has accumulated about imaging measurements as predictors of relevant functional outcomes in glaucoma, there is still insufficient evidence to support their use as valid surrogate endpoints. However, imaging biomarkers could potentially be used as part of composite endpoints in glaucoma trials, overcoming weaknesses of the use of structural or functional endpoints in isolation. Efforts should be taken to properly design and conduct studies that can provide proper validation of potential biomarkers in glaucoma clinical trials. PMID:28475699

Biomarkers and Surrogate Endpoints: Lessons Learned From Glaucoma.

PubMed

Medeiros, Felipe A

2017-05-01

With the recent progress in imaging technologies for assessment of structural damage in glaucoma, a debate has emerged on whether these measurements can be used as valid surrogate endpoints in clinical trials evaluating new therapies for the disease. A discussion of surrogates should be grounded on knowledge acquired from their use in other areas of medicine as well as regulatory requirements. This article reviews the conditions for valid surrogacy in the context of glaucoma clinical trials and critically evaluates the role of biomarkers such as IOP and imaging measurements as potential surrogates for clinically relevant outcomes. Valid surrogate endpoints must be able to predict a clinically relevant endpoint, such as loss of vision or decrease in quality of life. In addition, the effect of a proposed treatment on the surrogate must capture the effect of the treatment on the clinically relevant endpoint. Despite its widespread use in clinical trials, no proper validation of IOP as a surrogate endpoint has yet been conducted for any class of IOP-lowering treatments. Although strong evidence has accumulated about imaging measurements as predictors of relevant functional outcomes in glaucoma, there is still insufficient evidence to support their use as valid surrogate endpoints. However, imaging biomarkers could potentially be used as part of composite endpoints in glaucoma trials, overcoming weaknesses of the use of structural or functional endpoints in isolation. Efforts should be taken to properly design and conduct studies that can provide proper validation of potential biomarkers in glaucoma clinical trials.

Development and validation of a multi-dimensional measure of intellectual humility

PubMed Central

Alfano, Mark; Iurino, Kathryn; Stey, Paul; Robinson, Brian; Christen, Markus; Yu, Feng; Lapsley, Daniel

2017-01-01

This paper presents five studies on the development and validation of a scale of intellectual humility. This scale captures cognitive, affective, behavioral, and motivational components of the construct that have been identified by various philosophers in their conceptual analyses of intellectual humility. We find that intellectual humility has four core dimensions: Open-mindedness (versus Arrogance), Intellectual Modesty (versus Vanity), Corrigibility (versus Fragility), and Engagement (versus Boredom). These dimensions display adequate self-informant agreement, and adequate convergent, divergent, and discriminant validity. In particular, Open-mindedness adds predictive power beyond the Big Six for an objective behavioral measure of intellectual humility, and Intellectual Modesty is uniquely related to Narcissism. We find that a similar factor structure emerges in Germanophone participants, giving initial evidence for the model’s cross-cultural generalizability. PMID:28813478

The factor structure and psychometric properties of the Spanish version of the Mayer-Salovey-Caruso Emotional Intelligence Test.

PubMed

Sanchez-Garcia, Manuel; Extremera, Natalio; Fernandez-Berrocal, Pablo

2016-11-01

This research examined evidence regarding the reliability and validity of scores on the Spanish version of the Mayer-Salovey-Caruso Emotional Intelligence Test, Version 2.0 (MSCEIT; Mayer, Salovey, & Caruso, 2002). In Study 1, we found a close convergence of the Spanish consensus scores and the general and expert consensus scores determined with Mayer, Salovey, Caruso, and Sitarenios (2003) data. The MSCEIT also demonstrated adequate evidence of reliability of test scores as estimated by internal consistency and test-retest correlation after 12 weeks. Confirmatory factor analysis supported a 3-level higher factor model with 8 manifest variables (task scores), 4 first-level factors (corresponding to the 4-branch model of Mayer & Salovey [1997], with 2 tasks for each branch), 2 second-level factors (experiential and strategic areas, with 2 branches for each area), and 1 third-level factor (overall emotional intelligence [EI]), and multigroup analyses supported MSCEIT cross-gender invariance. Study 2 found evidence for the discriminant validity of scores on the MSCEIT subscales, which were differentially related to personality and self-reported EI. Study 3 provided evidence of the incremental validity of scores on the MSCEIT, which added significant variance to the prospective prediction of psychological well-being after controlling for personality traits. The psychometric properties of the Spanish MSCEIT are similar to those of the original English version, supporting its use for assessing emotional abilities in the Spanish population. (PsycINFO Database Record (c) 2016 APA, all rights reserved).

Instrumental and statistical methods for the comparison of class evidence

NASA Astrophysics Data System (ADS)

Liszewski, Elisa Anne

Trace evidence is a major field within forensic science. Association of trace evidence samples can be problematic due to sample heterogeneity and a lack of quantitative criteria for comparing spectra or chromatograms. The aim of this study is to evaluate different types of instrumentation for their ability to discriminate among samples of various types of trace evidence. Chemometric analysis, including techniques such as Agglomerative Hierarchical Clustering, Principal Components Analysis, and Discriminant Analysis, was employed to evaluate instrumental data. First, automotive clear coats were analyzed by using microspectrophotometry to collect UV absorption data. In total, 71 samples were analyzed with classification accuracy of 91.61%. An external validation was performed, resulting in a prediction accuracy of 81.11%. Next, fiber dyes were analyzed using UV-Visible microspectrophotometry. While several physical characteristics of cotton fiber can be identified and compared, fiber color is considered to be an excellent source of variation, and thus was examined in this study. Twelve dyes were employed, some being visually indistinguishable. Several different analyses and comparisons were done, including an inter-laboratory comparison and external validations. Lastly, common plastic samples and other polymers were analyzed using pyrolysis-gas chromatography/mass spectrometry, and their pyrolysis products were then analyzed using multivariate statistics. The classification accuracy varied dependent upon the number of classes chosen, but the plastics were grouped based on composition. The polymers were used as an external validation and misclassifications occurred with chlorinated samples all being placed into the category containing PVC.

Building a Computer Program to Support Children, Parents, and Distraction during Healthcare Procedures

PubMed Central

McCarthy, Ann Marie; Kleiber, Charmaine; Ataman, Kaan; Street, W. Nick; Zimmerman, M. Bridget; Ersig, Anne L.

2012-01-01

This secondary data analysis used data mining methods to develop predictive models of child risk for distress during a healthcare procedure. Data used came from a study that predicted factors associated with children’s responses to an intravenous catheter insertion while parents provided distraction coaching. From the 255 items used in the primary study, 44 predictive items were identified through automatic feature selection and used to build support vector machine regression models. Models were validated using multiple cross-validation tests and by comparing variables identified as explanatory in the traditional versus support vector machine regression. Rule-based approaches were applied to the model outputs to identify overall risk for distress. A decision tree was then applied to evidence-based instructions for tailoring distraction to characteristics and preferences of the parent and child. The resulting decision support computer application, the Children, Parents and Distraction (CPaD), is being used in research. Future use will support practitioners in deciding the level and type of distraction intervention needed by a child undergoing a healthcare procedure. PMID:22805121

A review of the revised Functional Capacity Index as a predictor of 12 month outcomes following injury.

PubMed

Palmer, Cameron S; Cameron, Peter A; Gabbe, Belinda J

2017-03-01

The measurement of functional outcomes following severe trauma has been widely recognised as a priority for countries with developed trauma systems. In this respect, the Functional Capacity Index (FCI), a multi-attribute index which has been incorporated into the most recent Abbreviated Injury Scale (AIS) dictionary, is potentially attractive as it offers 12-month functional outcome predictions for patients captured by existing AIS-coded datasets. This review paper outlines the development, construction and validation of the predictive form of the FCI (termed the pFCI), the modifications made which produced the currently available 'revised' pFCI, and the extent to which the revised pFCI has been validated and used. The original pFCI performed poorly in validation studies. The revised pFCI does not address many of the identified limitations of the original version, and despite the ready availability of a truncated version in the AIS dictionary, it has only been used in a handful of studies since its introduction several years ago. Additionally, there is little evidence for its validity. It is suggested that the pFCI should be better validated, whether in the narrow population group of young, healthy individuals for which it was developed, or in the wider population of severely injured patients. Methods for accounting for the presence of multiple injures (of which two have currently been used) should also be evaluated. Many factors other than anatomical injury are known to affect functional outcomes following trauma. However, it is intuitive that any model which attempts to predict the ongoing morbidity burden in a trauma population should consider the effects of the injuries sustained. Although the revised pFCI potentially offers a low-cost assessment of likely functional limitations resulting from anatomical injury, it must be more rigorously evaluated before more comprehensive predictive tools can be developed from it. Copyright © 2017 Elsevier Ltd. All rights reserved.

Validity of Sluggish Cognitive Tempo in South America: An Initial Examination Using Mother and Teacher Ratings of Chilean Children.

PubMed

Belmar, Marta; Servera, Mateu; Becker, Stephen P; Burns, G Leonard

2017-06-01

To examine the validity of sluggish cognitive tempo (SCT) and ADHD-inattention (ADHD-IN) symptoms in children from Chile. Mothers and teachers rated SCT, ADHD-IN, ADHD-hyperactivity/impulsivity (ADHD-HI), oppositional defiant disorder (ODD), anxiety, depression, academic impairment, social impairment, and peer rejection (teachers only) in 652 Chilean children (55% boys) aged 6 to 14 years. For both mother and teacher ratings, the eight SCT symptoms and nine ADHD-IN symptoms showed substantial loadings on their respective factors (convergent validity) along with loadings close to zero on the alternative factor (discriminant validity). ADHD-IN showed a uniquely stronger relationship than SCT with ADHD-HI and ODD whereas SCT showed a uniquely stronger relationship than ADHD-IN with anxiety and depression. Although ADHD-IN uniquely predicted academic impairment and social difficulties, SCT did not. This study provides the first evidence for the validity of SCT among children outside of North America or Western Europe.

Gene expression profiling and expanded immunohistochemistry tests to guide the use of adjuvant chemotherapy in breast cancer management: a systematic review and cost-effectiveness analysis.

PubMed

Ward, S; Scope, A; Rafia, R; Pandor, A; Harnan, S; Evans, P; Wyld, L

2013-10-01

Gene expression profiling (GEP) and expanded immunohistochemistry (IHC) tests aim to improve decision-making relating to adjuvant chemotherapy for women with early breast cancer. The aim of this report is to assess the clinical effectiveness and cost-effectiveness of nine GEP and expanded IHC tests compared with current prognostic tools in guiding the use of adjuvant chemotherapy in patients with early breast cancer in England and Wales. The nine tests are BluePrint, Breast Cancer Index (BCI), IHC4, MammaPrint, Mammostrat, NPI plus (NPI+), OncotypeDX, PAM50 and Randox Breast Cancer Array. Databases searched included MEDLINE, MEDLINE In-Process & Other Non-Indexed Citations, EMBASE and The Cochrane Library. Databases were searched from January 2009 to May 2011 for the OncotypeDX and MammaPrint tests and from January 2002 to May 2011 for the other tests. A systematic review of the evidence on clinical effectiveness (analytical validity, clinical validity and clinical utility) and cost-effectiveness was conducted. An economic model was developed to evaluate the cost-effectiveness of adjuvant chemotherapy treatment guided by four of the nine test (OncotypeDX, IHC4, MammaPrint and Mammostrat) compared with current clinical practice in England and Wales, using clinicopathological parameters, in women with oestrogen receptor-positive (ER+), lymph node-negative (LN-), human epidermal growth factor receptor type 2-negative (HER2-) early breast cancer. The literature searches for clinical effectiveness identified 5993 citations, of which 32 full-text papers or abstracts (30 studies) satisfied the criteria for the effectiveness review. A narrative synthesis was performed. Evidence for OncotypeDX supported the prognostic capability of the test. There was some evidence on the impact of the test on decision-making and to support the case that OncotypeDX predicts chemotherapy benefit; however, few studies were UK based and limitations in relation to study design were identified. Evidence for MammaPrint demonstrated that the test score was a strong independent prognostic factor, but the evidence is non-UK based and is based on small sample sizes. Evidence on the Mammostrat test showed that the test was an independent prognostic tool for women with ER+, tamoxifen-treated breast cancer. The three studies appeared to be of reasonable quality and provided data from a UK setting (one study). One large study reported on clinical validity of the IHC4 test, with IHC4 score a highly significant predictor of distant recurrence. This study included data from a UK setting and appeared to be of reasonable quality. Evidence for the remaining five tests (PAM50, NPI+, BCI, BluePrint and Randox) was limited. The economic analysis suggests that treatment guided using IHC4 has the greatest potential to be cost-effective at a £20,000 threshold, given the low cost of the test; however, further research is needed on the analytical validity and clinical utility of IHC4, and the exact cost of the test needs to be confirmed. Current limitations in the evidence base produce significant uncertainty in the results. OncotypeDX has a more robust evidence base, but further evidence on its impact on decision-making in the UK and the predictive ability of the test in an ER+, LN-, HER- population receiving current drug regimens is needed. For MammaPrint and Mammostrat there were significant gaps in the available evidence and the estimates of cost-effectiveness produced were not considered to be robust by the External Assessment Group. Methodological weaknesses in the clinical evidence base relate to heterogeneity of patient cohorts and issues arising from the retrospective nature of the evidence. Further evidence is required on the clinical utility of all of the tests and on UK-based populations. A key area of uncertainty relates to whether the tests provide prognostic or predictive ability. The clinical evidence base for OncotypeDX is considered to be the most robust. The economic analysis suggested that treatment guided using IHC4 has the most potential to be cost-effective at a threshold of £20,000; however, the evidence base to support IHC4 needs significant further research. PROSPERO 2011:CRD42011001361, available from www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42011001361.

From Vivaldi to Beatles and back: predicting lateralized brain responses to music.

PubMed

Alluri, Vinoo; Toiviainen, Petri; Lund, Torben E; Wallentin, Mikkel; Vuust, Peter; Nandi, Asoke K; Ristaniemi, Tapani; Brattico, Elvira

2013-12-01

We aimed at predicting the temporal evolution of brain activity in naturalistic music listening conditions using a combination of neuroimaging and acoustic feature extraction. Participants were scanned using functional Magnetic Resonance Imaging (fMRI) while listening to two musical medleys, including pieces from various genres with and without lyrics. Regression models were built to predict voxel-wise brain activations which were then tested in a cross-validation setting in order to evaluate the robustness of the hence created models across stimuli. To further assess the generalizability of the models we extended the cross-validation procedure by including another dataset, which comprised continuous fMRI responses of musically trained participants to an Argentinean tango. Individual models for the two musical medleys revealed that activations in several areas in the brain belonging to the auditory, limbic, and motor regions could be predicted. Notably, activations in the medial orbitofrontal region and the anterior cingulate cortex, relevant for self-referential appraisal and aesthetic judgments, could be predicted successfully. Cross-validation across musical stimuli and participant pools helped identify a region of the right superior temporal gyrus, encompassing the planum polare and the Heschl's gyrus, as the core structure that processed complex acoustic features of musical pieces from various genres, with or without lyrics. Models based on purely instrumental music were able to predict activation in the bilateral auditory cortices, parietal, somatosensory, and left hemispheric primary and supplementary motor areas. The presence of lyrics on the other hand weakened the prediction of activations in the left superior temporal gyrus. Our results suggest spontaneous emotion-related processing during naturalistic listening to music and provide supportive evidence for the hemispheric specialization for categorical sounds with realistic stimuli. We herewith introduce a powerful means to predict brain responses to music, speech, or soundscapes across a large variety of contexts. © 2013.

A novel measure of compulsive food restriction in anorexia nervosa: validation of the Self-Starvation Scale (SS).

PubMed

Godier, Lauren R; Park, Rebecca J

2015-04-01

The characteristic relentless self-starvation behaviour seen in Anorexia Nervosa (AN) has been described as evidence of compulsivity, with increasing suggestion of transdiagnostic parallels with addictive behaviour. There is a paucity of standardised self-report measures of compulsive behaviour in eating disorders (EDs). Measures that index the concept of compulsive self-starvation in AN are needed to explore the suggested parallels with addictions. With this aim a novel measure of self-starvation was developed (the Self-Starvation Scale, SS). 126 healthy participants, and 78 individuals with experience of AN, completed the new measure along with existing measures of eating disorder symptoms, anxiety and depression. Initial validation in the healthy sample indicated good reliability and construct validity, and incremental validity in predicting eating disorder symptoms. The psychometric properties of the SS scale were replicated in the AN sample. The ability of this scale to predict ED symptoms was particularly strong in individuals currently suffering from AN. These results suggest the SS may be a useful index of compulsive food restriction in AN. The concept of 'starvation dependence' in those with eating disorders, as a parallel with addiction, may be of clinical and theoretical importance. Copyright © 2014 Elsevier Ltd. All rights reserved.

Validating a measure to assess factors that affect assistive technology use by students with disabilities in elementary and secondary education.

PubMed

Zapf, Susan A; Scherer, Marcia J; Baxter, Mary F; H Rintala, Diana

2016-01-01

The purpose of this study was to measure the predictive validity, internal consistency and clinical utility of the Matching Assistive Technology to Child & Augmentative Communication Evaluation Simplified (MATCH-ACES) assessment. Twenty-three assistive technology team evaluators assessed 35 children using the MATCH-ACES assessment. This quasi-experimental study examined the internal consistency, predictive validity and clinical utility of the MATCH-ACES assessment. The MATCH-ACES assessment predisposition scales had good internal consistency across all three scales. A significant relationship was found between (a) high student perseverance and need for assistive technology and (b) high teacher comfort and interest in technology use (p = (0).002). Study results indicate that the MATCH-ACES assessment has good internal consistency and validity. Predisposition characteristics of student and teacher combined can influence the level of assistive technology use; therefore, assistive technology teams should assess predisposition factors of the user when recommending assistive technology. Implications for Rehabilitation Educational and medical professionals should be educated on evidence-based assistive technology assessments. Personal experience and psychosocial factors can influence the outcome use of assistive technology. Assistive technology assessments must include an intervention plan for assistive technology service delivery to measure effective outcome use.

Psychometric properties of the Late-Life Function and Disability Instrument: a systematic review

PubMed Central

2014-01-01

Background The choice of measure for use as a primary outcome in geriatric research is contingent upon the construct of interest and evidence for its psychometric properties. The Late-Life Function and Disability Instrument (LLFDI) has been widely used to assess functional limitations and disability in studies with older adults. The primary aim of this systematic review was to evaluate the current available evidence for the psychometric properties of the LLFDI. Methods Published studies of any design reporting results based on administration of the original version of the LLFDI in community-dwelling older adults were identified after searches of 9 electronic databases. Data related to construct validity (convergent/divergent and known-groups validity), test-retest reliability and sensitivity to change were extracted. Effect sizes were calculated for within-group changes and summarized graphically. Results Seventy-one studies including 17,301 older adults met inclusion criteria. Data supporting the convergent/divergent and known-groups validity for both the Function and Disability components were extracted from 30 and 18 studies, respectively. High test-retest reliability was found for the Function component, while results for the Disability component were more variable. Sensitivity to change of the LLFDI was confirmed based on findings from 25 studies. The basic lower extremity subscale and overall summary score of the Function component and limitation dimension of the Disability component were associated with the strongest relative effect sizes. Conclusions There is extensive evidence to support the construct validity and sensitivity to change of the LLFDI among various clinical populations of community-dwelling older adults. Further work is needed on predictive validity and values for clinically important change. Findings from this review can be used to guide the selection of the most appropriate LLFDI subscale for use an outcome measure in geriatric research and practice. PMID:24476510

20 CFR 404.725 - Evidence of a valid ceremonial marriage.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 20 Employees' Benefits 2 2010-04-01 2010-04-01 false Evidence of a valid ceremonial marriage. 404... DISABILITY INSURANCE (1950- ) Evidence Evidence of Age, Marriage, and Death § 404.725 Evidence of a valid ceremonial marriage. (a) General. A valid ceremonial marriage is one that follows procedures set by law in...

A review and psychometric evaluation of pregnancy-specific stress measures.

PubMed

Alderdice, Fiona; Lynn, Fiona; Lobel, Marci

2012-06-01

Considerable evidence has accumulated on the association between pregnancy-specific stress and adverse birth outcomes with an increasing number of measures of pregnancy-specific stress being developed internationally. However, the introduction of these measures has not always been theoretically or psychometrically grounded, resulting in questions about the quality and direction of such research. This review summarizes evidence on the reliability and validity of pregnancy-specific stress measures identified between 1980 and October 2010. Fifteen pregnancy-specific stress measures were identified. Cronbach's alpha coefficient ranged from 0.51-0.96 and predictive validity data on preterm birth were reported for five measures. Convergent validity data suggest that pregnancy-specific stress is related to, but distinct from, global stress. Findings from this review consolidate current knowledge on pregnancy-specific stress as a consistent predictor of premature birth. This review also advances awareness of the range of measures of pregnancy-specific stress and documents their strengths and limitations based on published reliability and validity data. Careful consideration needs to be given as to which measures to use in future research to maximize the development of stress theory in pregnancy and appropriate interventions for women who experience stress in pregnancy. An international, strategic collaboration is recommended to advance knowledge in this area of study.

Trait-specific dependence in romantic relationships.

PubMed

Ellis, Bruce J; Simpson, Jeffry A; Campbell, Lorne

2002-10-01

Informed by three theoretical frameworks--trait psychology, evolutionary psychology, and interdependence theory--we report four investigations designed to develop and test the reliability and validity of a new construct and accompanying multiscale inventory, the Trait-Specific Dependence Inventory (TSDI). The TSDI assesses comparisons between present and alternative romantic partners on major dimensions of mate value. In Study 1, principal components analyses revealed that the provisional pool of theory-generated TSDI items were represented by six factors: Agreeable/Committed, Resource Accruing Potential, Physical Prowess, Emotional Stability, Surgency, and Physical Attractiveness. In Study 2, confirmatory factor analysis replicated these results on a different sample and tested how well different structural models fit the data. Study 3 provided evidence for the convergent and discriminant validity of the six TSDI scales by correlating each one with a matched personality trait scale that did not explicitly incorporate comparisons between partners. Study 4 provided further validation evidence, revealing that the six TSDI scales successfully predicted three relationship outcome measures--love, time investment, and anger/upset--above and beyond matched sets of traditional personality trait measures. These results suggest that the TSDI is a reliable, valid, and unique construct that represents a new trait-specific method of assessing dependence in romantic relationships. The construct of trait-specific dependence is introduced and linked with other theories of mate value.

Overview of Heat Addition and Efficiency Predictions for an Advanced Stirling Convertor

NASA Technical Reports Server (NTRS)

Wilson, Scott D.; Reid, Terry; Schifer, Nicholas; Briggs, Maxwell

2011-01-01

Past methods of predicting net heat input needed to be validated. Validation effort pursued with several paths including improving model inputs, using test hardware to provide validation data, and validating high fidelity models. Validation test hardware provided direct measurement of net heat input for comparison to predicted values. Predicted value of net heat input was 1.7 percent less than measured value and initial calculations of measurement uncertainty were 2.1 percent (under review). Lessons learned during validation effort were incorporated into convertor modeling approach which improved predictions of convertor efficiency.

Inductive matrix completion for predicting gene-disease associations.

PubMed

Natarajan, Nagarajan; Dhillon, Inderjit S

2014-06-15

Most existing methods for predicting causal disease genes rely on specific type of evidence, and are therefore limited in terms of applicability. More often than not, the type of evidence available for diseases varies-for example, we may know linked genes, keywords associated with the disease obtained by mining text, or co-occurrence of disease symptoms in patients. Similarly, the type of evidence available for genes varies-for example, specific microarray probes convey information only for certain sets of genes. In this article, we apply a novel matrix-completion method called Inductive Matrix Completion to the problem of predicting gene-disease associations; it combines multiple types of evidence (features) for diseases and genes to learn latent factors that explain the observed gene-disease associations. We construct features from different biological sources such as microarray expression data and disease-related textual data. A crucial advantage of the method is that it is inductive; it can be applied to diseases not seen at training time, unlike traditional matrix-completion approaches and network-based inference methods that are transductive. Comparison with state-of-the-art methods on diseases from the Online Mendelian Inheritance in Man (OMIM) database shows that the proposed approach is substantially better-it has close to one-in-four chance of recovering a true association in the top 100 predictions, compared to the recently proposed Catapult method (second best) that has <15% chance. We demonstrate that the inductive method is particularly effective for a query disease with no previously known gene associations, and for predicting novel genes, i.e. genes that are previously not linked to diseases. Thus the method is capable of predicting novel genes even for well-characterized diseases. We also validate the novelty of predictions by evaluating the method on recently reported OMIM associations and on associations recently reported in the literature. Source code and datasets can be downloaded from http://bigdata.ices.utexas.edu/project/gene-disease. © The Author 2014. Published by Oxford University Press.

Proposed Modifications to the Conceptual Model of Coaching Efficacy and Additional Validity Evidence for the Coaching Efficacy Scale II-High School Teams

ERIC Educational Resources Information Center

Myers, Nicholas; Feltz, Deborah; Chase, Melissa

2011-01-01

The purpose of this study was to determine whether theoretically relevant sources of coaching efficacy could predict the measures derived from the Coaching Efficacy Scale II-High School Teams (CES II-HST). Data were collected from head coaches of high school teams in the United States (N = 799). The analytic framework was a multiple-group…

The Association between Elementary Teacher Licensure Test Scores and Student Growth in Mathematics: An Analysis of Massachusetts MTEL and MCAS Tests

ERIC Educational Resources Information Center

LeGeros, Life

2013-01-01

This quasi-experimental value-added study provided evidence for the predictive validity of the Massachusetts MTEL General Curriculum Mathematics Subtest by finding an association between the licensure test results of 130 teachers and the growth of their 2640 grade 4 and 5 students. The study took advantage of a natural experiment that arose due to…

It's Selective, but Is It Effective? Exploring the Predictive Validity of Teacher Selection Tools. CEDR Brief. Policy Brief No. 2014-­9

ERIC Educational Resources Information Center

Goldhaber, Dan; Grout, Cyrus; Huntington-Klein, Nick

2014-01-01

Evidence suggests teacher hiring in public schools is ad-hoc and often does not result in good selection amongst applicants. Some districts use structured selection instruments in the hiring process, but we know little about the efficacy of such tools. In this paper we evaluate the ability of applicant selection tools used by the Spokane Public…

When fast logic meets slow belief: Evidence for a parallel-processing model of belief bias.

PubMed

Trippas, Dries; Thompson, Valerie A; Handley, Simon J

2017-05-01

Two experiments pitted the default-interventionist account of belief bias against a parallel-processing model. According to the former, belief bias occurs because a fast, belief-based evaluation of the conclusion pre-empts a working-memory demanding logical analysis. In contrast, according to the latter both belief-based and logic-based responding occur in parallel. Participants were given deductive reasoning problems of variable complexity and instructed to decide whether the conclusion was valid on half the trials or to decide whether the conclusion was believable on the other half. When belief and logic conflict, the default-interventionist view predicts that it should take less time to respond on the basis of belief than logic, and that the believability of a conclusion should interfere with judgments of validity, but not the reverse. The parallel-processing view predicts that beliefs should interfere with logic judgments only if the processing required to evaluate the logical structure exceeds that required to evaluate the knowledge necessary to make a belief-based judgment, and vice versa otherwise. Consistent with this latter view, for the simplest reasoning problems (modus ponens), judgments of belief resulted in lower accuracy than judgments of validity, and believability interfered more with judgments of validity than the converse. For problems of moderate complexity (modus tollens and single-model syllogisms), the interference was symmetrical, in that validity interfered with belief judgments to the same degree that believability interfered with validity judgments. For the most complex (three-term multiple-model syllogisms), conclusion believability interfered more with judgments of validity than vice versa, in spite of the significant interference from conclusion validity on judgments of belief.

Impulsive versus premeditated aggression in the prediction of violent criminal recidivism.

PubMed

Swogger, Marc T; Walsh, Zach; Christie, Michael; Priddy, Brittany M; Conner, Kenneth R

2015-01-01

Past aggression is a potent predictor of future aggression and informs the prediction of violent criminal recidivism. However, aggression is a heterogeneous construct and different types of aggression may confer different levels of risk for future violence. In this prospective study of 91 adults in a pretrial diversion program, we examined (a) premeditated versus impulsive aggression in the prediction of violent recidivism during a one-year follow-up period, and (b) whether either type of aggression would have incremental validity in the prediction of violent recidivism after taking into account frequency of past general aggression. Findings indicate that premeditated, but not impulsive, aggression predicts violent recidivism. Moreover, premeditated aggression remained a predictor of recidivism even with general aggression frequency in the model. Results provide preliminary evidence that the assessment of premeditated aggression provides relevant information for the management of violent offenders. © 2014 Wiley Periodicals, Inc.

Impulsive versus Premeditated Aggression in the Prediction of Violent Criminal Recidivism

PubMed Central

Swogger, Marc T.; Walsh, Zach; Christie, Michael; Priddy, Brittany M.; Conner, Kenneth R.

2015-01-01

Past aggression is a potent predictor of future aggression and informs the prediction of violent criminal recidivism. However, aggression is a heterogeneous construct and different types of aggression may confer different levels of risk for future violence. In this prospective study of 91 adults in a pretrial diversion program, we examined a) premeditated versus impulsive aggression in the prediction of violent recidivism during a one-year follow-up period, and b) whether either type of aggression would have incremental validity in the prediction of violent recidivism after taking into account frequency of past general aggression. Findings indicate that premeditated, but not impulsive, aggression predicts violent recidivism. Moreover, premeditated aggression remained a predictor of recidivism even with general aggression frequency in the model. Results provide preliminary evidence that the assessment of premeditated aggression provides relevant information for the management of violent offenders. PMID:25043811

The application of SHERPA (Systematic Human Error Reduction and Prediction Approach) in the development of compensatory cognitive rehabilitation strategies for stroke patients with left and right brain damage.

PubMed

Hughes, Charmayne M L; Baber, Chris; Bienkiewicz, Marta; Worthington, Andrew; Hazell, Alexa; Hermsdörfer, Joachim

2015-01-01

Approximately 33% of stroke patients have difficulty performing activities of daily living, often committing errors during the planning and execution of such activities. The objective of this study was to evaluate the ability of the human error identification (HEI) technique SHERPA (Systematic Human Error Reduction and Prediction Approach) to predict errors during the performance of daily activities in stroke patients with left and right hemisphere lesions. Using SHERPA we successfully predicted 36 of the 38 observed errors, with analysis indicating that the proportion of predicted and observed errors was similar for all sub-tasks and severity levels. HEI results were used to develop compensatory cognitive strategies that clinicians could employ to reduce or prevent errors from occurring. This study provides evidence for the reliability and validity of SHERPA in the design of cognitive rehabilitation strategies in stroke populations.

Yaruss, Coleman, and Hammer (2006): an exemplar of non-evidence-based practice in stuttering treatment.

PubMed

Ingham, Roger J

2007-07-01

This letter is a response to a recent report by J. S. Yaruss, C. Coleman, and D. Hammer (2006) that described a treatment program for preschool children who stutter. Problems with the Yaruss et al. study fall into four domains: (a) failure to provide clinicians with replicable procedures, (b) failure to collect valid and reliable speech performance data, (c) failure to control for predictable improvement in children who have been stuttering for less than 15 months, and (d) the advocacy of procedures for which there is no credible research evidence. The claims made for the efficacy of this treatment are problematic and essentially violate the principles of evidence-based practice as recommended by the American Speech-Language-Hearing Association (ASHA).

Design Characteristics Influence Performance of Clinical Prediction Rules in Validation: A Meta-Epidemiological Study

PubMed Central

Ban, Jong-Wook; Emparanza, José Ignacio; Urreta, Iratxe; Burls, Amanda

2016-01-01

Background Many new clinical prediction rules are derived and validated. But the design and reporting quality of clinical prediction research has been less than optimal. We aimed to assess whether design characteristics of validation studies were associated with the overestimation of clinical prediction rules’ performance. We also aimed to evaluate whether validation studies clearly reported important methodological characteristics. Methods Electronic databases were searched for systematic reviews of clinical prediction rule studies published between 2006 and 2010. Data were extracted from the eligible validation studies included in the systematic reviews. A meta-analytic meta-epidemiological approach was used to assess the influence of design characteristics on predictive performance. From each validation study, it was assessed whether 7 design and 7 reporting characteristics were properly described. Results A total of 287 validation studies of clinical prediction rule were collected from 15 systematic reviews (31 meta-analyses). Validation studies using case-control design produced a summary diagnostic odds ratio (DOR) 2.2 times (95% CI: 1.2–4.3) larger than validation studies using cohort design and unclear design. When differential verification was used, the summary DOR was overestimated by twofold (95% CI: 1.2 -3.1) compared to complete, partial and unclear verification. The summary RDOR of validation studies with inadequate sample size was 1.9 (95% CI: 1.2 -3.1) compared to studies with adequate sample size. Study site, reliability, and clinical prediction rule was adequately described in 10.1%, 9.4%, and 7.0% of validation studies respectively. Conclusion Validation studies with design shortcomings may overestimate the performance of clinical prediction rules. The quality of reporting among studies validating clinical prediction rules needs to be improved. PMID:26730980

Design Characteristics Influence Performance of Clinical Prediction Rules in Validation: A Meta-Epidemiological Study.

PubMed

Ban, Jong-Wook; Emparanza, José Ignacio; Urreta, Iratxe; Burls, Amanda

2016-01-01

Many new clinical prediction rules are derived and validated. But the design and reporting quality of clinical prediction research has been less than optimal. We aimed to assess whether design characteristics of validation studies were associated with the overestimation of clinical prediction rules' performance. We also aimed to evaluate whether validation studies clearly reported important methodological characteristics. Electronic databases were searched for systematic reviews of clinical prediction rule studies published between 2006 and 2010. Data were extracted from the eligible validation studies included in the systematic reviews. A meta-analytic meta-epidemiological approach was used to assess the influence of design characteristics on predictive performance. From each validation study, it was assessed whether 7 design and 7 reporting characteristics were properly described. A total of 287 validation studies of clinical prediction rule were collected from 15 systematic reviews (31 meta-analyses). Validation studies using case-control design produced a summary diagnostic odds ratio (DOR) 2.2 times (95% CI: 1.2-4.3) larger than validation studies using cohort design and unclear design. When differential verification was used, the summary DOR was overestimated by twofold (95% CI: 1.2 -3.1) compared to complete, partial and unclear verification. The summary RDOR of validation studies with inadequate sample size was 1.9 (95% CI: 1.2 -3.1) compared to studies with adequate sample size. Study site, reliability, and clinical prediction rule was adequately described in 10.1%, 9.4%, and 7.0% of validation studies respectively. Validation studies with design shortcomings may overestimate the performance of clinical prediction rules. The quality of reporting among studies validating clinical prediction rules needs to be improved.

Fluorescence quenching and the "ring-mode" to "red-mode" transition in alkali inductively coupled plasmas

NASA Astrophysics Data System (ADS)

Huang, M.; Bazurto, R.; Camparo, J.

2018-01-01

The ring-mode to red-mode transition in alkali metal inductively coupled plasmas (ICPs) (i.e., rf-discharge lamps) is perhaps the most important physical phenomenon affecting these devices as optical pumping light sources for atomic clocks and magnetometers. It sets the limit on useful ICP operating temperature, thereby setting a limit on ICP light output for atomic-clock/magnetometer signal generation, and it is a temperature region of ICP operation associated with discharge instability. Previous work has suggested that the mechanism driving the ring-mode to red-mode transition is associated with radiation trapping, but definitive experimental evidence validating that hypothesis has been lacking. Based on that hypothesis, one would predict that the introduction of an alkali-fluorescence quenching gas (i.e., N2) into the ICP would increase the ring-mode to red-mode transition temperature. Here, we test that prediction, finding direct evidence supporting the radiation-trapping hypothesis.

Falls screening and assessment tools used in acute mental health settings: a review of policies in England and Wales

PubMed Central

Narayanan, V.; Dickinson, A.; Victor, C.; Griffiths, C.; Humphrey, D.

2016-01-01

Objectives There is an urgent need to improve the care of older people at risk of falls or who experience falls in mental health settings. The aims of this study were to evaluate the individual falls risk assessment tools adopted by National Health Service (NHS) mental health trusts in England and healthcare boards in Wales, to evaluate the comprehensiveness of these tools and to review their predictive validity. Methods All NHS mental health trusts in England (n = 56) and healthcare boards in Wales (n = 6) were invited to supply their falls policies and other relevant documentation (e.g. local falls audits). In order to check the comprehensiveness of tools listed in policy documents, the risk variables of the tools adopted by the mental health trusts’ policies were compared with the 2004 National Institute for Health and Care Excellence (NICE) falls prevention guidelines. A comprehensive analytical literature review was undertaken to evaluate the predictive validity of the tools used in these settings. Results Falls policies were obtained from 46 mental health trusts. Thirty-five policies met the study inclusion criteria and were included in the analysis. The main falls assessment tools used were the St. Thomas’ Risk Assessment Tool in Falling Elderly Inpatients (STRATIFY), Falls Risk Assessment Scale for the Elderly, Morse Falls Scale (MFS) and Falls Risk Assessment Tool (FRAT). On detailed examination, a number of different versions of the FRAT were evident; validated tools had inconsistent predictive validity and none of them had been validated in mental health settings. Conclusions Falls risk assessment is the most commonly used component of risk prevention strategies, but most policies included unvalidated tools and even well validated tool such as the STRATIFY and the MFS that are reported to have inconsistent predictive accuracy. This raises questions about operational usefulness, as none of these tools have been tested in acute mental health settings. The falls risk assessment tools from only four mental health trusts met all the recommendations of the NICE falls guidelines on multifactorial assessment for prevention of falls. The recent NICE (2013) guidance states that tools predicting risk using numeric scales should no longer be used; however, multifactorial risk assessment and interventions tailored to patient needs is recommended. Trusts will need to update their policies in response to this guidance. PMID:26395210

The re-labelling of dysthymic disorder to persistent depressive disorder in DSM-5: old wine in new bottles?

PubMed

Rhebergen, Didi; Graham, Rebecca

2014-01-01

Dysthymic disorder and other chronic depressive disorders have recently been merged in DSM-5 into a 'persistent depressive disorder' category. As its introduction in DSM-III, the validity of dysthymic disorder has long been challenged, posing concerns regarding the validity of its successor--persistent depressive disorder. This review aims to present recent findings regarding the validity and utility of dysthymic disorder. Several recent studies raise questions regarding the validity of dysthymic disorder, namely, results indicating a significant overlap between dysthymic disorder and other mood and/or anxiety disorders, failure of such a diagnosis to predict illness outcome and the lack of any validation strategy identifying that it is a depressive entity or subtype. Research findings indicate that dysthymic disorder is a heterogeneous diagnosis encompassing many different depressive (and anxiety or personality weighted) conditions, and without clear evidence of its validity as a diagnostic entity. As dysthymic disorder is a key component of DSM-defined persistent depressive disorder--the latter is at similar risk of providing a heterogeneous domain diagnosis, and thus limiting identification of specific causative factors and preferential treatment modality.

The role of tissue damage in whiplash associated disorders: Discussion paper 1

PubMed Central

Bogduk, Nikolai; Ivancic, Paul C.; McLean, Samuel A.; Siegmund, Gunter P.; Winkelstein, Beth

2011-01-01

STUDY DESIGN Non-systematic review of cervical spine lesions in whiplash-associated disorders (WAD). OBJECTIVE To describe whiplash injury models in terms of basic and clinical science, to summarize what can and cannot be explained by injury models, and to highlight future research areas to better understand the role of tissue damage in WAD. SUMMARY OF BACKGROUND DATA The frequent lack of detectable tissue damage has raised questions about whether tissue damage is necessary for WAD and what role it plays in the clinical context of WAD. METHODS Non-systematic review. RESULTS Lesions of various tissues have been documented by numerous investigations conducted in animals, cadavers, healthy volunteers and patients. Most lesions are undetected by imaging techniques. For zygapophysial (facet) joints, lesions have been predicted by bioengineering studies and validated through animal studies; for zygapophysial joint pain, a valid diagnostic test and a proven treatment are available. Lesions of dorsal root ganglia, discs, ligaments, muscles and vertebral artery have been documented in biomechanical and autopsy studies, but no valid diagnostic test is available to assess their clinical relevance. The proportion of WAD patients in whom a persistent lesion is the major determinant of ongoing symptoms is unknown. Psychosocial factors, stress reactions and generalized hyperalgesia have also been shown to predict WAD outcomes. CONCLUSION There is evidence supporting a lesion-based model in WAD. Lack of macroscopically identifiable tissue damage does not rule out the presence of painful lesions. The best available evidence concerns zygapophysial joint pain. The clinical relevance of other lesions needs to be addressed by future research. PMID:22020601

Why Do I Feel More Confident? Bandura's Sources Predict Preservice Teachers' Latent Changes in Teacher Self-Efficacy

PubMed Central

Pfitzner-Eden, Franziska

2016-01-01

Teacher self-efficacy (TSE) is associated with a multitude of positive outcomes for teachers and students. However, the development of TSE is an under-researched area. Bandura (1997) proposed four sources of self-efficacy: mastery experiences, vicarious experiences, verbal persuasion, and physiological and affective states. This study introduces a first instrument to assess the four sources for TSE in line with Bandura's conception. Gathering evidence of convergent validity, the contribution that each source made to the development of TSE during a practicum at a school was explored for two samples of German preservice teachers. The first sample (N = 359) were beginning preservice teachers who completed an observation practicum. The second sample (N = 395) were advanced preservice teachers who completed a teaching practicum. The source measure showed good reliability, construct validity, and convergent validity. Latent true change modeling was applied to explore how the sources predicted changes in TSE. Three different models were compared. As expected, results showed that TSE changes in both groups were significantly predicted by mastery experiences, with a stronger relationship in the advanced group. Further, the results indicated that mastery experiences were largely informed by the other three sources to varying degrees depending on the type of practicum. Implications for the practice of teacher education are discussed in light of the results. PMID:27807422

Why Do I Feel More Confident? Bandura's Sources Predict Preservice Teachers' Latent Changes in Teacher Self-Efficacy.

PubMed

Pfitzner-Eden, Franziska

2016-01-01

Teacher self-efficacy (TSE) is associated with a multitude of positive outcomes for teachers and students. However, the development of TSE is an under-researched area. Bandura (1997) proposed four sources of self-efficacy: mastery experiences, vicarious experiences, verbal persuasion, and physiological and affective states. This study introduces a first instrument to assess the four sources for TSE in line with Bandura's conception. Gathering evidence of convergent validity, the contribution that each source made to the development of TSE during a practicum at a school was explored for two samples of German preservice teachers. The first sample ( N = 359) were beginning preservice teachers who completed an observation practicum. The second sample ( N = 395) were advanced preservice teachers who completed a teaching practicum. The source measure showed good reliability, construct validity, and convergent validity. Latent true change modeling was applied to explore how the sources predicted changes in TSE. Three different models were compared. As expected, results showed that TSE changes in both groups were significantly predicted by mastery experiences, with a stronger relationship in the advanced group. Further, the results indicated that mastery experiences were largely informed by the other three sources to varying degrees depending on the type of practicum. Implications for the practice of teacher education are discussed in light of the results.

Efficacy of an evidence-based clinical decision support in primary care practices: a randomized clinical trial.

PubMed

McGinn, Thomas G; McCullagh, Lauren; Kannry, Joseph; Knaus, Megan; Sofianou, Anastasia; Wisnivesky, Juan P; Mann, Devin M

2013-09-23

There is consensus that incorporating clinical decision support into electronic health records will improve quality of care, contain costs, and reduce overtreatment, but this potential has yet to be demonstrated in clinical trials. To assess the influence of a customized evidence-based clinical decision support tool on the management of respiratory tract infections and on the effectiveness of integrating evidence at the point of care. In a randomized clinical trial, we implemented 2 well-validated integrated clinical prediction rules, namely, the Walsh rule for streptococcal pharyngitis and the Heckerling rule for pneumonia. INTERVENTIONS AND MAIN OUTCOMES AND MEASURES: The intervention group had access to the integrated clinical prediction rule tool and chose whether to complete risk score calculators, order medications, and generate progress notes to assist with complex decision making at the point of care. The intervention group completed the integrated clinical prediction rule tool in 57.5% of visits. Providers in the intervention group were significantly less likely to order antibiotics than the control group (age-adjusted relative risk, 0.74; 95% CI, 0.60-0.92). The absolute risk of the intervention was 9.2%, and the number needed to treat was 10.8. The intervention group was significantly less likely to order rapid streptococcal tests compared with the control group (relative risk, 0.75; 95% CI, 0.58-0.97; P= .03). The integrated clinical prediction rule process for integrating complex evidence-based clinical decision report tools is of relevant importance for national initiatives, such as Meaningful Use. TRIAL REGISTRATION clinicaltrials.gov Identifier: NCT01386047.

Strong expectations cancel locality effects: evidence from Hindi.

PubMed

Husain, Samar; Vasishth, Shravan; Srinivasan, Narayanan

2014-01-01

Expectation-driven facilitation (Hale, 2001; Levy, 2008) and locality-driven retrieval difficulty (Gibson, 1998, 2000; Lewis & Vasishth, 2005) are widely recognized to be two critical factors in incremental sentence processing; there is accumulating evidence that both can influence processing difficulty. However, it is unclear whether and how expectations and memory interact. We first confirm a key prediction of the expectation account: a Hindi self-paced reading study shows that when an expectation for an upcoming part of speech is dashed, building a rarer structure consumes more processing time than building a less rare structure. This is a strong validation of the expectation-based account. In a second study, we show that when expectation is strong, i.e., when a particular verb is predicted, strong facilitation effects are seen when the appearance of the verb is delayed; however, when expectation is weak, i.e., when only the part of speech "verb" is predicted but a particular verb is not predicted, the facilitation disappears and a tendency towards a locality effect is seen. The interaction seen between expectation strength and distance shows that strong expectations cancel locality effects, and that weak expectations allow locality effects to emerge.

Strong Expectations Cancel Locality Effects: Evidence from Hindi

PubMed Central

Husain, Samar; Vasishth, Shravan; Srinivasan, Narayanan

2014-01-01

Expectation-driven facilitation (Hale, 2001; Levy, 2008) and locality-driven retrieval difficulty (Gibson, 1998, 2000; Lewis & Vasishth, 2005) are widely recognized to be two critical factors in incremental sentence processing; there is accumulating evidence that both can influence processing difficulty. However, it is unclear whether and how expectations and memory interact. We first confirm a key prediction of the expectation account: a Hindi self-paced reading study shows that when an expectation for an upcoming part of speech is dashed, building a rarer structure consumes more processing time than building a less rare structure. This is a strong validation of the expectation-based account. In a second study, we show that when expectation is strong, i.e., when a particular verb is predicted, strong facilitation effects are seen when the appearance of the verb is delayed; however, when expectation is weak, i.e., when only the part of speech “verb” is predicted but a particular verb is not predicted, the facilitation disappears and a tendency towards a locality effect is seen. The interaction seen between expectation strength and distance shows that strong expectations cancel locality effects, and that weak expectations allow locality effects to emerge. PMID:25010700

Observational evidence of European summer weather patterns predictable from spring

NASA Astrophysics Data System (ADS)

Ossó, Albert; Sutton, Rowan; Shaffrey, Len; Dong, Buwen

2018-01-01

Forecasts of summer weather patterns months in advance would be of great value for a wide range of applications. However, seasonal dynamical model forecasts for European summers have very little skill, particularly for rainfall. It has not been clear whether this low skill reflects inherent unpredictability of summer weather or, alternatively, is a consequence of weaknesses in current forecast systems. Here we analyze atmosphere and ocean observations and identify evidence that a specific pattern of summertime atmospheric circulation––the summer East Atlantic (SEA) pattern––is predictable from the previous spring. An index of North Atlantic sea-surface temperatures in March–April can predict the SEA pattern in July–August with a cross-validated correlation skill above 0.6. Our analyses show that the sea-surface temperatures influence atmospheric circulation and the position of the jet stream over the North Atlantic. The SEA pattern has a particularly strong influence on rainfall in the British Isles, which we find can also be predicted months ahead with a significant skill of 0.56. Our results have immediate application to empirical forecasts of summer rainfall for the United Kingdom, Ireland, and northern France and also suggest that current dynamical model forecast systems have large potential for improvement.

Alternative Interpretation of Low-Energy Nuclear Reaction Processes with Deuterated Metals Based on the Bose-Einstein Condensation Mechanism

NASA Astrophysics Data System (ADS)

Kim, Yeong E.; Passell, Thomas O.

2006-02-01

Recently, a generalization of the Bose-Einstein condensation (BEC) mechanism has been made to a ground-state mixture of two different species of positively charged bosons in harmonic traps. The theory has been used to describe (D + Li) reactions in the low energy nuclear reaction (LENR) processes in condensed matter and predicts that the (D + Li) reaction rates can be larger than (D + D) reaction rates by as much as a factor of ~50, implying that (D + Li) reactions may be occuring in addition to the (D + D) reactions. A survey of the existing data from LENR experiments is carried out to check the validity of the theoretical prediction. We conclude that there is compelling experimental evidence which support the theoretical prediction. New experimental tests of the theoretical prediction are suggested.

Construct, Concurrent and Predictive Validity of the URICA: Data from Two Multi-site Clinical Trials

PubMed Central

Field, Craig A.; Adinoff, Bryon; Harris, T. Robert; Ball, Samuel A.; Carroll, Kathleen M.

2011-01-01

Background A better understanding of how to measure motivation to change and how it relates to behavior change in patients with drug and alcohol dependence would broaden our understanding of the role of motivation in addiction treatment. Methods Two multi-site, randomized clinical trials comparing brief motivational interventions with standard care were conducted in the National Institute on Drug Abuse Clinical Trials Network. Patients with primary drug dependence and alcohol dependence entering outpatient treatment participated in a study of either Motivational Enhancement Therapy (n=431) or Motivational Interviewing (n=423). The construct, concurrent, and predictive validity of two composite measures of motivation to change derived from the University of Rhode Island Change Assessment (URICA): Readiness to Change (RTC) and Committed Action (CA) were evaluated. Results Confirmatory factor analysis confirmed the a priori factor structure of the URICA. RTC was significantly associated with measures of addiction severity at baseline (r=.12-.52, p<.05). Although statistically significant (p<.01), the correlations between treatment outcomes and RTC were low (r=-.15 and -18). Additional analyses did not support a moderating or mediating effect of motivation on treatment retention or substance use. Conclusions The construct validity of the URICA was confirmed separately in a large sample of drug- and alcohol-dependent patients. However, evidence for the predictive validity of composite scores was very limited and there were no moderating or mediating effects of either measure on treatment outcome. Thus, increased motivation to change, as measured by the composite scores of motivation derived from the URICA, does not appear to influence treatment outcome. PMID:19157723

Rapid learning in practice: A lung cancer survival decision support system in routine patient care data

PubMed Central

Dekker, Andre; Vinod, Shalini; Holloway, Lois; Oberije, Cary; George, Armia; Goozee, Gary; Delaney, Geoff P.; Lambin, Philippe; Thwaites, David

2016-01-01

Background and purpose A rapid learning approach has been proposed to extract and apply knowledge from routine care data rather than solely relying on clinical trial evidence. To validate this in practice we deployed a previously developed decision support system (DSS) in a typical, busy clinic for non-small cell lung cancer (NSCLC) patients. Material and methods Gender, age, performance status, lung function, lymph node status, tumor volume and survival were extracted without review from clinical data sources for lung cancer patients. With these data the DSS was tested to predict overall survival. Results 3919 lung cancer patients were identified with 159 eligible for inclusion, due to ineligible histology or stage, non-radical dose, missing tumor volume or survival. The DSS successfully identified a good prognosis group and a medium/poor prognosis group (2 year OS 69% vs. 27/30%, p < 0.001). Stage was less discriminatory (2 year OS 47% for stage I–II vs. 36% for stage IIIA–IIIB, p = 0.12) with most good prognosis patients having higher stage disease. The DSS predicted a large absolute overall survival benefit (~40%) for a radical dose compared to a non-radical dose in patients with a good prognosis, while no survival benefit of radical radiotherapy was predicted for patients with a poor prognosis. Conclusions A rapid learning environment is possible with the quality of clinical data sufficient to validate a DSS. It uses patient and tumor features to identify prognostic groups in whom therapy can be individualized based on predicted outcomes. Especially the survival benefit of a radical versus non-radical dose predicted by the DSS for various prognostic groups has clinical relevance, but needs to be prospectively validated. PMID:25241994

Validation of educational assessments: a primer for simulation and beyond.

PubMed

Cook, David A; Hatala, Rose

2016-01-01

Simulation plays a vital role in health professions assessment. This review provides a primer on assessment validation for educators and education researchers. We focus on simulation-based assessment of health professionals, but the principles apply broadly to other assessment approaches and topics. Validation refers to the process of collecting validity evidence to evaluate the appropriateness of the interpretations, uses, and decisions based on assessment results. Contemporary frameworks view validity as a hypothesis, and validity evidence is collected to support or refute the validity hypothesis (i.e., that the proposed interpretations and decisions are defensible). In validation, the educator or researcher defines the proposed interpretations and decisions, identifies and prioritizes the most questionable assumptions in making these interpretations and decisions (the "interpretation-use argument"), empirically tests those assumptions using existing or newly-collected evidence, and then summarizes the evidence as a coherent "validity argument." A framework proposed by Messick identifies potential evidence sources: content, response process, internal structure, relationships with other variables, and consequences. Another framework proposed by Kane identifies key inferences in generating useful interpretations: scoring, generalization, extrapolation, and implications/decision. We propose an eight-step approach to validation that applies to either framework: Define the construct and proposed interpretation, make explicit the intended decision(s), define the interpretation-use argument and prioritize needed validity evidence, identify candidate instruments and/or create/adapt a new instrument, appraise existing evidence and collect new evidence as needed, keep track of practical issues, formulate the validity argument, and make a judgment: does the evidence support the intended use? Rigorous validation first prioritizes and then empirically evaluates key assumptions in the interpretation and use of assessment scores. Validation science would be improved by more explicit articulation and prioritization of the interpretation-use argument, greater use of formal validation frameworks, and more evidence informing the consequences and implications of assessment.

Validity of Teacher Ratings in Selecting Influential Aggressive Adolescents for a Targeted Preventive Intervention

PubMed Central

Henry, David B.; Miller-Johnson, Shari; Simon, Thomas R.; Schoeny, Michael E.

2009-01-01

This study describes a method for using teacher nominations and ratings to identify socially influential, aggressive middle school students for participation in a targeted violence prevention intervention. The teacher nomination method is compared with peer nominations of aggression and influence to obtain validity evidence. Participants were urban, predominantly African American and Latino sixth-grade students who were involved in a pilot study for a large multi-site violence prevention project. Convergent validity was suggested by the high correlation of teacher ratings of peer influence and peer nominations of social influence. The teacher ratings of influence demonstrated acceptable sensitivity and specificity when predicting peer nominations of influence among the most aggressive children. Results are discussed m terms of the application of teacher nominations and ratings in large trials and full implementation of targeted prevention programs. PMID:16378226

Capability beliefs regarding evidence-based practice are associated with application of EBP and research use: validation of a new measure.

PubMed

Wallin, Lars; Boström, Anne-Marie; Gustavsson, J Petter

2012-08-01

Beliefs about capabilities, or self-efficacy, is a construct originating in social cognitive psychology. Capability beliefs have been found to be positively associated with intention and healthcare practice behaviour. A measure of an individual's beliefs about his/her capability to apply the components of evidence-based practice (EBP) has potential to be useful in implementation research. To evaluate the concurrent validity and internal structure of a new scale measuring nurses' capability beliefs regarding EBP. Data were taken from a prospective longitudinal study in Sweden (the Longitudinal Analyses of Nursing Education and Entry in Worklife [LANE]). A cohort of nursing students who graduated in the autumn of 2004 that was followed up 2 years after their graduation was used (n= 1,256). Concurrent validity was tested relating different levels of capability beliefs to extent of research use and application of EBP. An item-response approach was applied in the evaluation of internal structure of the proposed scale (six items). The psychometric analyses indicated that the six items could be summed to reflect a one-dimensional scale. Nurses with the highest level of capability beliefs reported that they used research findings in clinical practice more than twice as often as those with lower levels of capability beliefs. They also participated in the implementation of evidence seven times more often. There is a need for further studies of the construct and predictive validity of the scale. It should also be validated in other groups of health professionals. Learning including mastery experiences, role modelling, social persuasion, and manageable stress could be used in undergraduate education as well as practice development to increase beliefs about capabilities which might open the way to increased application of EBP in healthcare practice. This new measure is well grounded in social cognitive theory, functions as a one-dimensional scale and possesses promising properties of concurrent validity. ©2012 Sigma Theta Tau International.

Assessing dependency using self-report and indirect measures: examining the significance of discrepancies.

PubMed

Cogswell, Alex; Alloy, Lauren B; Karpinski, Andrew; Grant, David A

2010-07-01

The present study addressed convergence between self-report and indirect approaches to assessing dependency. We were moderately successful in validating an implicit measure, which was found to be reliable, orthogonal to 2 self-report instruments, and predictive of external criteria. This study also examined discrepancies between scores on self-report and implicit measures, and has implications for their significance. The possibility that discrepancies themselves are pathological was not supported, although discrepancies were associated with particular personality profiles. Finally, this study offered additional evidence for the relation between dependency and depressive symptomatology and identified implicit dependency as contributing unique variance in predicting past major depression.

Become the PPUPET Master: Mastering Pressure Ulcer Risk Assessment With the Pediatric Pressure Ulcer Prediction and Evaluation Tool (PPUPET).

PubMed

Sterken, David J; Mooney, JoAnn; Ropele, Diana; Kett, Alysha; Vander Laan, Karen J

2015-01-01

Hospital acquired pressure ulcers (HAPU) are serious, debilitating, and preventable complications in all inpatient populations. Despite evidence of the development of pressure ulcers in the pediatric population, minimal research has been done. Based on observations gathered during quarterly HAPU audits, bedside nursing staff recognized trends in pressure ulcer locations that were not captured using current pressure ulcer risk assessment tools. Together, bedside nurses and nursing leadership created and conducted multiple research studies to investigate the validity and reliability of the Pediatric Pressure Ulcer Prediction and Evaluation Tool (PPUPET). Copyright © 2015 Elsevier Inc. All rights reserved.

Application of Data-Driven Evidential Belief Functions to Prospectivity Mapping for Aquamarine-Bearing Pegmatites, Lundazi District, Zambia

DOE Office of Scientific and Technical Information (OSTI.GOV)

Carranza, E. J. M., E-mail: carranza@itc.nl; Woldai, T.; Chikambwe, E. M.

A case application of data-driven estimation of evidential belief functions (EBFs) is demonstrated to prospectivity mapping in Lundazi district (eastern Zambia). Spatial data used to represent recognition criteria of prospectivity for aquamarine-bearing pegmatites include mapped granites, mapped faults/fractures, mapped shear zones, and radioelement concentration ratios derived from gridded airborne radiometric data. Data-driven estimates EBFs take into account not only (a) spatial association between an evidential map layer and target deposits but also (b) spatial relationships between classes of evidences in an evidential map layer. Data-driven estimates of EBFs can indicate which spatial data provide positive or negative evidence of prospectivity.more » Data-driven estimates of EBFs of only spatial data providing positive evidence of prospectivity were integrated according to Dempster's rule of combination. Map of integrated degrees of belief was used to delineate zones of relative degress of prospectivity for aquamarine-bearing pegmatites. The predictive map has at least 85% prediction rate and at least 79% success rate of delineating training and validation deposits, respectively. The results illustrate usefulness of data-driven estimation of EBFs in GIS-based predictive mapping of mineral prospectivity. The results also show usefulness of EBFs in managing uncertainties associated with evidential maps.« less

The Mt. Hood challenge: cross-testing two diabetes simulation models.

PubMed

Brown, J B; Palmer, A J; Bisgaard, P; Chan, W; Pedula, K; Russell, A

2000-11-01

Starting from identical patients with type 2 diabetes, we compared the 20-year predictions of two computer simulation models, a 1998 version of the IMIB model and version 2.17 of the Global Diabetes Model (GDM). Primary measures of outcome were 20-year cumulative rates of: survival, first (incident) acute myocardial infarction (AMI), first stroke, proliferative diabetic retinopathy (PDR), macro-albuminuria (gross proteinuria, or GPR), and amputation. Standardized test patients were newly diagnosed males aged 45 or 75, with high and low levels of glycated hemoglobin (HbA(1c)), systolic blood pressure (SBP), and serum lipids. Both models generated realistic results and appropriate responses to changes in risk factors. Compared with the GDM, the IMIB model predicted much higher rates of mortality and AMI, and fewer strokes. These differences can be explained by differences in model architecture (Markov vs. microsimulation), different evidence bases for cardiovascular prediction (Framingham Heart Study cohort vs. Kaiser Permanente patients), and isolated versus interdependent prediction of cardiovascular events. Compared with IMIB, GDM predicted much higher lifetime costs, because of lower mortality and the use of a different costing method. It is feasible to cross-validate and explicate dissimilar diabetes simulation models using standardized patients. The wide differences in the model results that we observed demonstrate the need for cross-validation. We propose to hold a second 'Mt Hood Challenge' in 2001 and invite all diabetes modelers to attend.

Disentangling the Predictive Validity of High School Grades for Academic Success in University

ERIC Educational Resources Information Center

Vulperhorst, Jonne; Lutz, Christel; de Kleijn, Renske; van Tartwijk, Jan

2018-01-01

To refine selective admission models, we investigate which measure of prior achievement has the best predictive validity for academic success in university. We compare the predictive validity of three core high school subjects to the predictive validity of high school grade point average (GPA) for academic achievement in a liberal arts university…

Assessing Suicide Risk Among Callers to Crisis Hotlines: A Confirmatory Factor Analysis

PubMed Central

Witte, Tracy K.; Gould, Madelyn S.; Munfakh, Jimmie Lou Harris; Kleinman, Marjorie; Joiner, Thomas E.; Kalafat, John

2012-01-01

Our goal was to investigate the factor structure of a risk assessment tool utilized by suicide hotlines and to determine the predictive validity of the obtained factors in predicting subsequent suicidal behavior. 1,085 suicidal callers to crisis hotlines were divided into three sub-samples, which allowed us to conduct an independent Exploratory Factor Analysis (EFA), EFA in a Confirmatory Factor Analysis (EFA/CFA) framework, and CFA. Similar to previous factor analytic studies (Beck et al., 1997; Holden & DeLisle, 2005; Joiner, Rudd, & Rajab, 1997; Witte et al., 2006), we found consistent evidence for a two-factor solution, with one factor representing a more pernicious form of suicide risk (i.e., Resolved Plans and Preparations) and one factor representing more mild suicidal ideation (i.e., Suicidal Desire and Ideation). Using structural equation modeling techniques, we found preliminary evidence that the Resolved Plans and Preparations factor trended toward being more predictive of suicidal ideation than the Suicidal Desire and Ideation factor. This factor analytic study is the first longitudinal study of the obtained factors. PMID:20578186

Structural and Predictive Properties of the Emotional Quotient Inventory Youth Version-Short Form (EQ-i:YV[S]).

PubMed

Davis, Sarah K; Wigelsworth, Michael

2018-01-01

Emotional intelligence (EI) is a popular construct with concentrated areas of application in education and health contexts. There is a need for reliable and valid measurement of EI in young people, with brief yet sensitive measures of the construct preferable for use in time-limited settings. However, the proliferation of EI measures has often outpaced rigorous psychometric evaluation (Gignac, 2009 ). Using data from 849 adolescents (407 females, 422 males) aged 11 to 16 years (M age 13.4, SD = 1.2 years), this article systematically examines the structural and predictive properties of a frequently employed measure of adolescent trait EI-the Emotional Quotient Inventory Youth Version-Short Form (EQ-i:YV[S]); Bar-On & Parker, 2000 ). Although the intended multidimensional factor structure was recovered through confirmatory factor analysis, the statistical and conceptual coherency of the underlying model was inadequate. Using a multitrait-multimethod approach, the EQ-i:YV(S) was found to converge with other measures of EI; however, evidence for divergent validity (Big Five personality dimensions) was less robust. Predictive utility for adolescent mental health outcomes (depression, disruptive behavior) was also limited. Findings suggest that use of the EQ-i:YV(S) for predictive or evaluative purposes should be avoided until refinements to the scale are made.

Consistency between direct and indirect trial evidence: is direct evidence always more reliable?

PubMed

Madan, Jason; Stevenson, Matt D; Cooper, Katy L; Ades, A E; Whyte, Sophie; Akehurst, Ron

2011-01-01

To present a case study involving the reduction in incidence of febrile neutropenia (FN) after chemotherapy with granulocyte colony-stimulating factors (G-CSFs), illustrating difficulties that may arise when following the common preference for direct evidence over indirect evidence. Evidence of the efficacy of treatments was identified from two previous systematic reviews. We used Bayesian evidence synthesis to estimate relative treatment effects based on direct evidence, indirect evidence, and both pooled together. We checked for inconsistency between direct and indirect evidence and explored the role of one specific trial using cross-validation. A subsequent review identified further studies not available at the time of the original analysis. We repeated the analyses on the enlarged evidence base. We found substantial inconsistency in the original evidence base. The median odds ratio of FN for primary pegfilgrastim versus no primary G-CSF was 0.06 (95% credible interval: 0.02-0.19) based on direct evidence, but 0.27 (95% credible interval: 0.13-0.53) based on indirect evidence (P value for consistency hypothesis 0.027). The additional trials were consistent with the earlier indirect, rather than the direct, evidence, and there was no inconsistency between direct and indirect estimates in the updated evidence. The earlier inconsistency was due to one trial comparing primary pegfilgrastim with no primary G-CSF. Predictive cross-validation showed that this study was inconsistent with the evidence as a whole and with other trials making this comparison. Both the Cochrane Handbook and the NICE Methods Guide express a preference for direct evidence. A more robust strategy, which is in line with the accepted principles of evidence synthesis, would be to combine all relevant and appropriate information, whether direct or indirect. Copyright © 2011 International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. All rights reserved.

Relating lab to life: Decrements in attention over time predict math productivity among children with ADHD.

PubMed

Fosco, Whitney D; Hawk, Larry W

2017-02-01

A child's ability to sustain attention over time (AOT) is critical in attention-deficit/hyperactivity disorder (ADHD), yet no prior work has examined the extent to which a child's decrement in AOT on laboratory tasks relates to clinically-relevant behavior. The goal of this study is to provide initial evidence for the criterion validity of laboratory assessments of AOT. A total of 20 children with ADHD (7-12 years of age) who were enrolled in a summer treatment program completed two lab attention tasks (a continuous performance task and a self-paced choice discrimination task) and math seatwork. Analyses focused on relations between attention task parameters and math productivity. Individual differences in overall attention (OA) measures (averaged across time) accounted for 23% of the variance in math productivity, supporting the criterion validity of lab measures of attention. The criterion validity was enhanced by consideration of changes in AOT. Performance on all laboratory attention measures deteriorated as time-on-task increased, and individual differences in the decrement in AOT accounted for 40% of the variance in math productivity. The only variable to uniquely predict math productivity was from the self-paced choice discrimination task. This study suggests that attention tasks in the lab do predict a clinically-relevant target behavior in children with ADHD, supporting their use as a means to study attention processes in a controlled environment. Furthermore, this prediction is improved when attention is examined as a function of time-on-task and when the attentional demands are consistent between lab and life contexts.

Empirically Examining the Risk of Intimate Partner Violence: The Revised Domestic Violence Screening Instrument (DVSI-R)

PubMed Central

Grant, Stephen R

2006-01-01

SYNOPSIS Objective This study extends recent research on assessing the risk of intimate partner violence by determining the concurrent and predictive validity of a revised version of the Domestic Violence Screening Instrument (DVSI-R) and whether evidence of such validity is sustained independent of perpetrator demographic characteristics and forms of intimate violence. The analyses highlight violent incidents involving multiple victims as an indicator of “severe” violence. Previous research did not address these issues. Methods Data were analyzed on 14,970 assessments conducted in the State of Connecticut from September 1, 2004 through May 2, 2005. Hierarchical regression and receiver operating characteristic analyses were used to address the objectives of this research. Results The empirical findings support the concurrent and predictive validity of the DVSI-R and show that it is robust in its applicability. The findings further show that incidents involving multiple victims are highly associated with DVSI-R risk scores and recidivistic violence. Conclusion Validating and demonstrating the robustness of a risk assessment instrument is only a first step in preventing violence involving intimate partners or others in family or family-like relationships. The challenge is to train professionals responsible for addressing the problem of such violence to link valid risk assessments to well-crafted strategies of supervision and treatment so that the victimized or other potential victims are protected and perpetrators are held accountable for their actions. PMID:16827441

The development and validation of the client expectations of massage scale.

PubMed

Boulanger, Karen T; Campo, Shelly; Glanville, Jennifer L; Lowe, John B; Yang, Jingzhen

2012-01-01

Although there is evidence that client expectations influence client outcomes, a valid and reliable scale for measuring the range of client expectations for both massage therapy and the behaviors of their massage therapists does not exist. Understanding how client expectations influence client outcomes would provide insight into how massage achieves its reported effects. To develop and validate the Client Expectations of Massage Scale (CEMS), a measure of clients' clinical, educational, interpersonal, and outcome expectations. Offices of licensed massage therapists in Iowa. A practice-based research methodology was used to collect data from two samples of massage therapy clients. For Sample 1, 21 volunteer massage therapists collected data from their clients before the massage. Factor analysis was conducted to test construct validity and coefficient alpha was used to assess reliability. Correlational analyses with the CEMS, previous measures of client expectations, and the Life Orientation Test-Revised were examined to test the convergent and discriminant validity of the CEMS. For Sample 2, 24 massage therapists distributed study materials for clients to complete before and after a massage therapy session. Structural equation modeling was used to assess the construct, discriminant, and predictive validity of the CEMS. Sample 1 involved 320 and Sample 2 involved 321 adult massage clients. Standard care provided by licensed massage therapists. Numeric Rating Scale for pain and Positive and Negative Affect Schedule-Revised (including the Serenity subscale). The CEMS demonstrated good construct, convergent, discriminant and predictive validity, and adequate reliability. Client expectations were generally positive toward massage and their massage therapists. Positive outcome expectations had a positive effect on clients' changes in pain and serenity. High interpersonal expectations had a negative effect on clients' changes in serenity. Client expectations contribute to the nonspecific effects of massage therapy.

Quantifying prognosis with risk predictions.

PubMed

Pace, Nathan L; Eberhart, Leopold H J; Kranke, Peter R

2012-01-01

Prognosis is a forecast, based on present observations in a patient, of their probable outcome from disease, surgery and so on. Research methods for the development of risk probabilities may not be familiar to some anaesthesiologists. We briefly describe methods for identifying risk factors and risk scores. A probability prediction rule assigns a risk probability to a patient for the occurrence of a specific event. Probability reflects the continuum between absolute certainty (Pi = 1) and certified impossibility (Pi = 0). Biomarkers and clinical covariates that modify risk are known as risk factors. The Pi as modified by risk factors can be estimated by identifying the risk factors and their weighting; these are usually obtained by stepwise logistic regression. The accuracy of probabilistic predictors can be separated into the concepts of 'overall performance', 'discrimination' and 'calibration'. Overall performance is the mathematical distance between predictions and outcomes. Discrimination is the ability of the predictor to rank order observations with different outcomes. Calibration is the correctness of prediction probabilities on an absolute scale. Statistical methods include the Brier score, coefficient of determination (Nagelkerke R2), C-statistic and regression calibration. External validation is the comparison of the actual outcomes to the predicted outcomes in a new and independent patient sample. External validation uses the statistical methods of overall performance, discrimination and calibration and is uniformly recommended before acceptance of the prediction model. Evidence from randomised controlled clinical trials should be obtained to show the effectiveness of risk scores for altering patient management and patient outcomes.

“Connectedness to Nature Scale”: Validity and Reliability in the French Context

PubMed Central

Navarro, Oscar; Olivos, Pablo; Fleury-Bahi, Ghozlane

2017-01-01

Connectedness to nature represents the relationship of the self with the natural environment and has been operationalized using different scales. One of the most systematically studied in the Anglo-Saxon context is the Connectedness to Nature Scale (CNS). In an attempt to study the psychometric properties of this instrument in a French-speaking context, three studies (Study 1 n = 204, Study 2 n = 153, and Study 3 n = 322) were carried out in France to provide evidence of the internal consistency of the CNS, as well as its convergent, discriminant, and predictive validity. Moreover, as anticipated, positive correlations between the CNS and the environmental identity and environmental concerns scales were observed. Based on factorial analyses of maximum likelihood and reliability, an improvement in the psychometric properties was identified by eliminating three items. Through confirmatory factor analysis, the factorial structure and the psychometric properties of the CNS French version were confirmed, as well as their significate regression prediction on eudaimonic wellbeing. PMID:29312052

Educational testing validity and reliability in pharmacy and medical education literature.

PubMed

Hoover, Matthew J; Jung, Rose; Jacobs, David M; Peeters, Michael J

2013-12-16

To evaluate and compare the reliability and validity of educational testing reported in pharmacy education journals to medical education literature. Descriptions of validity evidence sources (content, construct, criterion, and reliability) were extracted from articles that reported educational testing of learners' knowledge, skills, and/or abilities. Using educational testing, the findings of 108 pharmacy education articles were compared to the findings of 198 medical education articles. For pharmacy educational testing, 14 articles (13%) reported more than 1 validity evidence source while 83 articles (77%) reported 1 validity evidence source and 11 articles (10%) did not have evidence. Among validity evidence sources, content validity was reported most frequently. Compared with pharmacy education literature, more medical education articles reported both validity and reliability (59%; p<0.001). While there were more scholarship of teaching and learning (SoTL) articles in pharmacy education compared to medical education, validity, and reliability reporting were limited in the pharmacy education literature.

Health impact assessment of Roma housing policies in Central and Eastern Europe: A comparative analysis

DOE Office of Scientific and Technical Information (OSTI.GOV)

Molnar, Agnes, E-mail: molnar.agnes@sph.unideb.hu; Adam, Balazs, E-mail: adam.balazs@sph.unideb.hu; Antova, Temenujka, E-mail: t.antova@ncphp.government.bg

2012-02-15

Marginalised Roma communities in European countries live in substandard housing conditions the improvement of which has been one of the major issues of the Decade of Roma Inclusion, the ongoing intergovernmental European Roma programme. The paper presents EU-funded health impact assessments of national Roma housing policies and programmes in 3 Central and Eastern European countries in light of the evaluation of a completed local project in a fourth CEE country so as to compare predicted effects to observed ones. Housing was predicted to have beneficial health effects by improving indoor and outdoor conditions, access to services, and socioeconomic conditions. Negativemore » impacts were predicted only in terms of maintenance expenses and housing tenure. However, observed impacts of the completed local project did not fully support predictions especially in terms of social networks, satisfaction with housing and neighbourhood, and inhabitant safety. In order to improve the predictive value of HIA, more evidence should be produced by the careful evaluation of locally implemented housing projects. In addition, current evidence is in favour of planning Roma housing projects at the local rather than at the national level in alignment with the principle of subsidiarity. - Highlights: Black-Right-Pointing-Pointer Predictive validity of HIA of national Roma housing policies - in light of current evidence - is low. Black-Right-Pointing-Pointer Implemented housing projects should be comprehensively evaluated to improve reliability of HIA. Black-Right-Pointing-Pointer Roma housing projects should be planned at the local rather than at the national level. Black-Right-Pointing-Pointer HIA should be used to plan Roma housing projects at the local level.« less

Assessing the stability of human locomotion: a review of current measures

PubMed Central

Bruijn, S. M.; Meijer, O. G.; Beek, P. J.; van Dieën, J. H.

2013-01-01

Falling poses a major threat to the steadily growing population of the elderly in modern-day society. A major challenge in the prevention of falls is the identification of individuals who are at risk of falling owing to an unstable gait. At present, several methods are available for estimating gait stability, each with its own advantages and disadvantages. In this paper, we review the currently available measures: the maximum Lyapunov exponent (λS and λL), the maximum Floquet multiplier, variability measures, long-range correlations, extrapolated centre of mass, stabilizing and destabilizing forces, foot placement estimator, gait sensitivity norm and maximum allowable perturbation. We explain what these measures represent and how they are calculated, and we assess their validity, divided up into construct validity, predictive validity in simple models, convergent validity in experimental studies, and predictive validity in observational studies. We conclude that (i) the validity of variability measures and λS is best supported across all levels, (ii) the maximum Floquet multiplier and λL have good construct validity, but negative predictive validity in models, negative convergent validity and (for λL) negative predictive validity in observational studies, (iii) long-range correlations lack construct validity and predictive validity in models and have negative convergent validity, and (iv) measures derived from perturbation experiments have good construct validity, but data are lacking on convergent validity in experimental studies and predictive validity in observational studies. In closing, directions for future research on dynamic gait stability are discussed. PMID:23516062

Understanding and predicting suicidality using a combined genomic and clinical risk assessment approach

PubMed Central

Niculescu, A B; Levey, D F; Phalen, P L; Le-Niculescu, H; Dainton, H D; Jain, N; Belanger, E; James, A; George, S; Weber, H; Graham, D L; Schweitzer, R; Ladd, T B; Learman, R; Niculescu, E M; Vanipenta, N P; Khan, F N; Mullen, J; Shankar, G; Cook, S; Humbert, C; Ballew, A; Yard, M; Gelbart, T; Shekhar, A; Schork, N J; Kurian, S M; Sandusky, G E; Salomon, D R

2015-01-01

Worldwide, one person dies every 40 seconds by suicide, a potentially preventable tragedy. A limiting step in our ability to intervene is the lack of objective, reliable predictors. We have previously provided proof of principle for the use of blood gene expression biomarkers to predict future hospitalizations due to suicidality, in male bipolar disorder participants. We now generalize the discovery, prioritization, validation, and testing of such markers across major psychiatric disorders (bipolar disorder, major depressive disorder, schizoaffective disorder, and schizophrenia) in male participants, to understand commonalities and differences. We used a powerful within-participant discovery approach to identify genes that change in expression between no suicidal ideation and high suicidal ideation states (n=37 participants out of a cohort of 217 psychiatric participants followed longitudinally). We then used a convergent functional genomics (CFG) approach with existing prior evidence in the field to prioritize the candidate biomarkers identified in the discovery step. Next, we validated the top biomarkers from the prioritization step for relevance to suicidal behavior, in a demographically matched cohort of suicide completers from the coroner's office (n=26). The biomarkers for suicidal ideation only are enriched for genes involved in neuronal connectivity and schizophrenia, the biomarkers also validated for suicidal behavior are enriched for genes involved in neuronal activity and mood. The 76 biomarkers that survived Bonferroni correction after validation for suicidal behavior map to biological pathways involved in immune and inflammatory response, mTOR signaling and growth factor regulation. mTOR signaling is necessary for the effects of the rapid-acting antidepressant agent ketamine, providing a novel biological rationale for its possible use in treating acute suicidality. Similarly, MAOB, a target of antidepressant inhibitors, was one of the increased biomarkers for suicidality. We also identified other potential therapeutic targets or biomarkers for drugs known to mitigate suicidality, such as omega-3 fatty acids, lithium and clozapine. Overall, 14% of the top candidate biomarkers also had evidence for involvement in psychological stress response, and 19% for involvement in programmed cell death/cellular suicide (apoptosis). It may be that in the face of adversity (stress), death mechanisms are turned on at a cellular (apoptosis) and organismal level. Finally, we tested the top increased and decreased biomarkers from the discovery for suicidal ideation (CADM1, CLIP4, DTNA, KIF2C), prioritization with CFG for prior evidence (SAT1, SKA2, SLC4A4), and validation for behavior in suicide completers (IL6, MBP, JUN, KLHDC3) steps in a completely independent test cohort of psychiatric participants for prediction of suicidal ideation (n=108), and in a future follow-up cohort of psychiatric participants (n=157) for prediction of psychiatric hospitalizations due to suicidality. The best individual biomarker across psychiatric diagnoses for predicting suicidal ideation was SLC4A4, with a receiver operating characteristic (ROC) area under the curve (AUC) of 72%. For bipolar disorder in particular, SLC4A4 predicted suicidal ideation with an AUC of 93%, and future hospitalizations with an AUC of 70%. SLC4A4 is involved in brain extracellular space pH regulation. Brain pH has been implicated in the pathophysiology of acute panic attacks. We also describe two new clinical information apps, one for affective state (simplified affective state scale, SASS) and one for suicide risk factors (Convergent Functional Information for Suicide, CFI-S), and how well they predict suicidal ideation across psychiatric diagnoses (AUC of 85% for SASS, AUC of 89% for CFI-S). We hypothesized a priori, based on our previous work, that the integration of the top biomarkers and the clinical information into a universal predictive measure (UP-Suicide) would show broad-spectrum predictive ability across psychiatric diagnoses. Indeed, the UP-Suicide was able to predict suicidal ideation across psychiatric diagnoses with an AUC of 92%. For bipolar disorder, it predicted suicidal ideation with an AUC of 98%, and future hospitalizations with an AUC of 94%. Of note, both types of tests we developed (blood biomarkers and clinical information apps) do not require asking the individual assessed if they have thoughts of suicide, as individuals who are truly suicidal often do not share that information with clinicians. We propose that the widespread use of such risk prediction tests as part of routine or targeted healthcare assessments will lead to early disease interception followed by preventive lifestyle modifications and proactive treatment. PMID:26283638

Measurement of academic entitlement.

PubMed

Miller, Brian K

2013-10-01

Members of Generation Y, or Millennials, have been accused of being lazy, whiny, pampered, and entitled, particularly in the college classroom. Using an equity theory framework, eight items from a measure of work entitlement were adapted to measure academic entitlement in a university setting in three independent samples. In Study 1 (n = 229), confirmatory factor analyses indicated good model fit to a unidimensional structure for the data. In Study 2 (n = 200), the questionnaire predicted unique variance in university satisfaction beyond two more general measures of dispositional entitlement. In Study 3 (n = 161), the measure predicted unique variance in perceptions of grade fairness beyond that which was predicted by another measure of academic entitlement. This analysis provides evidence of discriminant, convergent, incremental, concurrent criterion-related, and construct validity for the Academic Equity Preference Questionnaire.

Assessing suicide risk among callers to crisis hotlines: a confirmatory factor analysis.

PubMed

Witte, Tracy K; Gould, Madelyn S; Munfakh, Jimmie Lou Harris; Kleinman, Marjorie; Joiner, Thomas E; Kalafat, John

2010-09-01

Our goal was to investigate the factor structure of a risk assessment tool utilized by suicide hotlines and to determine the predictive validity of the obtained factors in predicting subsequent suicidal behavior. We conducted an Exploratory Factor Analysis (EFA), an EFA in a Confirmatory Factor Analysis (EFA/CFA) framework, and a CFA on independent subsamples derived from a total sample of 1,085. Similar to previous studies, we found consistent evidence for a two-factor solution, with one factor representing a more pernicious form of suicide risk (i.e., Resolved Plans and Preparations; RPP) and one factor representing milder suicidal ideation (i.e., Suicidal Desire and Ideation; SDI). The RPP factor trended toward being more predictive of suicidal ideation at follow-up than the SDI factor. (c) 2010 Wiley Periodicals, Inc.

Sustained Implementation Support Scale: Validation of a Measure of Program Characteristics and Workplace Functioning for Sustained Program Implementation.

PubMed

Hodge, Lauren M; Turner, Karen M T; Sanders, Matthew R; Filus, Ania

2017-07-01

An evaluation measure of enablers and inhibitors to sustained evidence-based program (EBP) implementation may provide a useful tool to enhance organizations' capacity. This paper outlines preliminary validation of such a measure. An expert informant and consumer feedback approach was used to tailor constructs from two existing measures assessing key domains associated with sustained implementation. Validity and reliability were evaluated for an inventory composed of five subscales: Program benefits, Program burden, Workplace support, Workplace cohesion, and Leadership style. Exploratory and confirmatory factor analysis with a sample of 593 Triple P-Positive Parenting Program-practitioners led to a 28-item scale with good reliability and good convergent, discriminant, and predictive validity. Practitioners sustaining implementation at least 3 years post-training were more likely to have supervision/peer support, reported higher levels of program benefit, workplace support, and positive leadership style, and lower program burden compared to practitioners who were non-sustainers.

Bayesian assurance and sample size determination in the process validation life-cycle.

PubMed

Faya, Paul; Seaman, John W; Stamey, James D

2017-01-01

Validation of pharmaceutical manufacturing processes is a regulatory requirement and plays a key role in the assurance of drug quality, safety, and efficacy. The FDA guidance on process validation recommends a life-cycle approach which involves process design, qualification, and verification. The European Medicines Agency makes similar recommendations. The main purpose of process validation is to establish scientific evidence that a process is capable of consistently delivering a quality product. A major challenge faced by manufacturers is the determination of the number of batches to be used for the qualification stage. In this article, we present a Bayesian assurance and sample size determination approach where prior process knowledge and data are used to determine the number of batches. An example is presented in which potency uniformity data is evaluated using a process capability metric. By using the posterior predictive distribution, we simulate qualification data and make a decision on the number of batches required for a desired level of assurance.

Modelling by partial least squares the relationship between the HPLC mobile phases and analytes on phenyl column.

PubMed

Markopoulou, Catherine K; Kouskoura, Maria G; Koundourellis, John E

2011-06-01

Twenty-five descriptors and 61 structurally different analytes have been used on a partial least squares (PLS) to latent structure technique in order to study chromatographically their interaction mechanism on a phenyl column. According to the model, 240 different retention times of the analytes, expressed as Y variable (log k), at different % MeOH mobile-phase concentrations have been correlated with their theoretical most important structural or molecular descriptors. The goodness-of-fit was estimated by the coefficient of multiple determinations r(2) (0.919), and the root mean square error of estimation (RMSEE=0.1283) values with a predictive ability (Q(2)) of 0.901. The model was further validated using cross-validation (CV), validated by 20 response permutations r(2) (0.0, 0.0146), Q(2) (0.0, -0.136) and validated by external prediction. The contribution of certain mechanism interactions between the analytes, the mobile phase and the column, proportional or counterbalancing is also studied. Trying to evaluate the influence on Y of every variable in a PLS model, VIP (variables importance in the projection) plot provides evidence that lipophilicity (expressed as Log D, Log P), polarizability, refractivity and the eluting power of the mobile phase are dominant in the retention mechanism on a phenyl column. Copyright © 2011 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim.

Changing the approach to treatment choice in epilepsy using big data.

PubMed

Devinsky, Orrin; Dilley, Cynthia; Ozery-Flato, Michal; Aharonov, Ranit; Goldschmidt, Ya'ara; Rosen-Zvi, Michal; Clark, Chris; Fritz, Patty

2016-03-01

A UCB-IBM collaboration explored the application of machine learning to large claims databases to construct an algorithm for antiepileptic drug (AED) choice for individual patients. Claims data were collected between January 2006 and September 2011 for patients with epilepsy > 16 years of age. A subset of patient claims with a valid index date of AED treatment change (new, add, or switch) were used to train the AED prediction model by retrospectively evaluating an index date treatment for subsequent treatment change. Based on the trained model, a model-predicted AED regimen with the lowest likelihood of treatment change was assigned to each patient in the group of test claims, and outcomes were evaluated to test model validity. The model had 72% area under receiver operator characteristic curve, indicating good predictive power. Patients who were given the model-predicted AED regimen had significantly longer survival rates (time until a treatment change event) and lower expected health resource utilization on average than those who received another treatment. The actual prescribed AED regimen at the index date matched the model-predicted AED regimen in only 13% of cases; there were large discrepancies in the frequency of use of certain AEDs/combinations between model-predicted AED regimens and those actually prescribed. Chances of treatment success were improved if patients received the model-predicted treatment. Using the model's prediction system may enable personalized, evidence-based epilepsy care, accelerating the match between patients and their ideal therapy, thereby delivering significantly better health outcomes for patients and providing health-care savings by applying resources more efficiently. Our goal will be to strengthen the predictive power of the model by integrating diverse data sets and potentially moving to prospective data collection. Crown Copyright © 2015. Published by Elsevier Inc. All rights reserved.

A new test for the assessment of working memory in clinical settings: Validation and norming of a month ordering task.

PubMed

Buekenhout, Imke; Leitão, José; Gomes, Ana A

2018-05-24

Month ordering tasks have been used in experimental settings to obtain measures of working memory (WM) capacity in older/clinical groups based solely on their face validity. We sought to assess the appropriateness of using a month ordering task in other contexts, including clinical settings, as a psychometrically sound WM assessment. To this end, we constructed a month ordering task (ucMOT), studied its reliability (internal consistency and temporal stability), and gathered construct-related and criterion-related validity evidence for its use as a WM assessment. The ucMOT proved to be internally consistent and temporally stable, and analyses of the criterion-related validity evidence revealed that its scores predicted the efficiency of language comprehension processes known to depend crucially on WM resources, namely, processes involved in pronoun interpretation. Furthermore, all ucMOT items discriminated between younger and older age groups; the global scores were significantly correlated with scores on well-established WM tasks and presented lower correlations with instruments that evaluate different (although related) processes, namely, inhibition and processing speed. We conclude that the ucMOT possesses solid psychometric properties. Accordingly, we acquired normative data for the Portuguese population, which we present as a regression-based algorithm that yields z scores adjusted for age, gender, and years of formal education. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Independent external validation of predictive models for urinary dysfunction following external beam radiotherapy of the prostate: Issues in model development and reporting.

PubMed

Yahya, Noorazrul; Ebert, Martin A; Bulsara, Max; Kennedy, Angel; Joseph, David J; Denham, James W

2016-08-01

Most predictive models are not sufficiently validated for prospective use. We performed independent external validation of published predictive models for urinary dysfunctions following radiotherapy of the prostate. Multivariable models developed to predict atomised and generalised urinary symptoms, both acute and late, were considered for validation using a dataset representing 754 participants from the TROG 03.04-RADAR trial. Endpoints and features were harmonised to match the predictive models. The overall performance, calibration and discrimination were assessed. 14 models from four publications were validated. The discrimination of the predictive models in an independent external validation cohort, measured using the area under the receiver operating characteristic (ROC) curve, ranged from 0.473 to 0.695, generally lower than in internal validation. 4 models had ROC >0.6. Shrinkage was required for all predictive models' coefficients ranging from -0.309 (prediction probability was inverse to observed proportion) to 0.823. Predictive models which include baseline symptoms as a feature produced the highest discrimination. Two models produced a predicted probability of 0 and 1 for all patients. Predictive models vary in performance and transferability illustrating the need for improvements in model development and reporting. Several models showed reasonable potential but efforts should be increased to improve performance. Baseline symptoms should always be considered as potential features for predictive models. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

Development and Validation of Computational Fluid Dynamics Models for Prediction of Heat Transfer and Thermal Microenvironments of Corals

PubMed Central

Ong, Robert H.; King, Andrew J. C.; Mullins, Benjamin J.; Cooper, Timothy F.; Caley, M. Julian

2012-01-01

We present Computational Fluid Dynamics (CFD) models of the coupled dynamics of water flow, heat transfer and irradiance in and around corals to predict temperatures experienced by corals. These models were validated against controlled laboratory experiments, under constant and transient irradiance, for hemispherical and branching corals. Our CFD models agree very well with experimental studies. A linear relationship between irradiance and coral surface warming was evident in both the simulation and experimental result agreeing with heat transfer theory. However, CFD models for the steady state simulation produced a better fit to the linear relationship than the experimental data, likely due to experimental error in the empirical measurements. The consistency of our modelling results with experimental observations demonstrates the applicability of CFD simulations, such as the models developed here, to coral bleaching studies. A study of the influence of coral skeletal porosity and skeletal bulk density on surface warming was also undertaken, demonstrating boundary layer behaviour, and interstitial flow magnitude and temperature profiles in coral cross sections. Our models compliment recent studies showing systematic changes in these parameters in some coral colonies and have utility in the prediction of coral bleaching. PMID:22701582

User Guidelines and Best Practices for CASL VUQ Analysis Using Dakota

DOE Office of Scientific and Technical Information (OSTI.GOV)

Adams, Brian M.; Coleman, Kayla; Hooper, Russell W.

2016-10-04

In general, Dakota is the Consortium for Advanced Simulation of Light Water Reactors (CASL) delivery vehicle for verification, validation, and uncertainty quantification (VUQ) algorithms. It permits ready application of the VUQ methods described above to simulation codes by CASL researchers, code developers, and application engineers. More specifically, the CASL VUQ Strategy [33] prescribes the use of Predictive Capability Maturity Model (PCMM) assessments [37]. PCMM is an expert elicitation tool designed to characterize and communicate completeness of the approaches used for computational model definition, verification, validation, and uncertainty quantification associated with an intended application. Exercising a computational model with the methodsmore » in Dakota will yield, in part, evidence for a predictive capability maturity model (PCMM) assessment. Table 1.1 summarizes some key predictive maturity related activities (see details in [33]), with examples of how Dakota fits in. This manual offers CASL partners a guide to conducting Dakota-based VUQ studies for CASL problems. It motivates various classes of Dakota methods and includes examples of their use on representative application problems. On reading, a CASL analyst should understand why and how to apply Dakota to a simulation problem.« less

Validation of PICA Ablation and Thermal-Response Model at Low Heat Flux

NASA Technical Reports Server (NTRS)

Milos, Frank S.; Chen, Yih-Kanq

2009-01-01

Phenolic Impregnated Carbon Ablator (PICA) was the forebody heatshield material on the Stardust sample-return capsule and is also a primary candidate material for the Mars Science Lander (MSL), the Orion Crew Module, and the SpaceX Dragon vehicle. As part of the heatshield qualification for Orion, physical and thermal properties of virgin and charred PICA were measured, and an ablation and thermal response model was developed. We validated the model by comparing it with recession and temperature data from stagnation arcjet tests conducted over a wide range of stagnation heat flux of 107 to 1102 W/sq cm. The effect of orthotropic thermal conductivity was evident in the thermal response of the arcjet models. In general, model predictions compared well with the data; however, the uncertainty of the recession prediction was greatest for heat fluxes below 200 W/sq cm. More recent MSL testing focused on the low heat flux regime of 45 to 250 W/sq cm. The new results confirm the recession uncertainty, especially for pressures below 6 kPa. In this work we focus on improving the model predictions for MSL and Orion tests below 250 W/sq cm.

Predictive regulatory models in Drosophila melanogaster by integrative inference of transcriptional networks

PubMed Central

Marbach, Daniel; Roy, Sushmita; Ay, Ferhat; Meyer, Patrick E.; Candeias, Rogerio; Kahveci, Tamer; Bristow, Christopher A.; Kellis, Manolis

2012-01-01

Gaining insights on gene regulation from large-scale functional data sets is a grand challenge in systems biology. In this article, we develop and apply methods for transcriptional regulatory network inference from diverse functional genomics data sets and demonstrate their value for gene function and gene expression prediction. We formulate the network inference problem in a machine-learning framework and use both supervised and unsupervised methods to predict regulatory edges by integrating transcription factor (TF) binding, evolutionarily conserved sequence motifs, gene expression, and chromatin modification data sets as input features. Applying these methods to Drosophila melanogaster, we predict ∼300,000 regulatory edges in a network of ∼600 TFs and 12,000 target genes. We validate our predictions using known regulatory interactions, gene functional annotations, tissue-specific expression, protein–protein interactions, and three-dimensional maps of chromosome conformation. We use the inferred network to identify putative functions for hundreds of previously uncharacterized genes, including many in nervous system development, which are independently confirmed based on their tissue-specific expression patterns. Last, we use the regulatory network to predict target gene expression levels as a function of TF expression, and find significantly higher predictive power for integrative networks than for motif or ChIP-based networks. Our work reveals the complementarity between physical evidence of regulatory interactions (TF binding, motif conservation) and functional evidence (coordinated expression or chromatin patterns) and demonstrates the power of data integration for network inference and studies of gene regulation at the systems level. PMID:22456606

Assessment method for epithermal gold deposits in Northeast Washington State using weights-of-evidence GIS modeling

USGS Publications Warehouse

Boleneus, D.E.; Raines, G.L.; Causey, J.D.; Bookstrom, A.A.; Frost, T.P.; Hyndman, P.C.

2001-01-01

The weights-of-evidence analysis, a quantitative mineral resource mapping tool, is used to delineate favorable areas for epithermal gold deposits and to predict future exploration activity of the mineral industry for similar deposits in a four-county area (222 x 277 km), including the Okanogan and Colville National Forests of northeastern Washington. Modeling is applied in six steps: (1) building a spatial digital database, (2) extracting predictive evidence for a particular deposit, based on an exploration model, (3) calculating relative weights for each predictive map, (4) combining the geologic evidence maps to predict the location of undiscovered mineral resources and (5) measuring the intensity of recent exploration activity by use of mining claims on federal lands, and (6) combining mineral resource and exploration activity into an assessment model of future mining activity. The analysis is accomplished on a personal computer using ArcView GIS platform with Spatial Analyst and Weights-of-Evidence software. In accord with the descriptive model for epithermal gold deposits, digital geologic evidential themes assembled include lithologic map units, thrust faults, normal faults, and igneous dikes. Similarly, geochemical evidential themes include placer gold deposits and gold and silver analyses from stream sediment (silt) samples from National Forest lands. Fifty mines, prospects, or occurrences of epithermal gold deposits, the training set, define the appropriate a really-associated terrane. The areal (or spatial) correlation of each evidential theme with the training set yield predictor theme maps for lithology, placer sites and normal faults. The weights-of-evidence analysis disqualified the thrust fault, dike, and gold and silver silt analyses evidential themes because they lacked spatial correlation with the training set. The decision to accept or reject evidential themes as predictors is assisted by considering probabilistic data consisting of weights and contrast values calculated for themes according to areal correlation with the training sites. Predictor themes having acceptable weights and contrast values are combined into a preliminary model to predict the locations of undiscovered epithermal gold deposits. This model facilitates ranking of tracts as non-permissive, permissive or favorable categories based on exclusionary, passive, and active criteria through evaluation of probabilistic data provided by interaction of predictor themes. The method is very similar to the visual inspection method of drawing conclusions from anomalies on a manually overlain system of maps. This method serves as a model for future mineral assessment procedures because of its objective nature. To develop a model to predict future exploration activity, the locations of lode mining claims were summarized for 1980, 1985, 1990, and 1996. Land parcels containing historic claims were identified either as those with mining claims present in 1980 or valid claims present in 1985. Current claim parcels were identified as those containing valid lode claims in either 1990 or 1996. A consistent parcel contains both historic and current claims. The epithermal gold and mining claim activity models were combined into an assessment (or mineral resource-activity) model to assist in land use decisions by providing a prediction of mineral exploration activity on federal land in the next decade. Ranks in the assessment model are: (1) no activity, (2) low activity, (3) low to moderate activity, (4) moderate activity and (5) high activity.

Evaluation of a panel of 28 biomarkers for the non-invasive diagnosis of endometriosis.

PubMed

Vodolazkaia, A; El-Aalamat, Y; Popovic, D; Mihalyi, A; Bossuyt, X; Kyama, C M; Fassbender, A; Bokor, A; Schols, D; Huskens, D; Meuleman, C; Peeraer, K; Tomassetti, C; Gevaert, O; Waelkens, E; Kasran, A; De Moor, B; D'Hooghe, T M

2012-09-01

At present, the only way to conclusively diagnose endometriosis is laparoscopic inspection, preferably with histological confirmation. This contributes to the delay in the diagnosis of endometriosis which is 6-11 years. So far non-invasive diagnostic approaches such as ultrasound (US), MRI or blood tests do not have sufficient diagnostic power. Our aim was to develop and validate a non-invasive diagnostic test with a high sensitivity (80% or more) for symptomatic endometriosis patients, without US evidence of endometriosis, since this is the group most in need of a non-invasive test. A total of 28 inflammatory and non-inflammatory plasma biomarkers were measured in 353 EDTA plasma samples collected at surgery from 121 controls without endometriosis at laparoscopy and from 232 women with endometriosis (minimal-mild n = 148; moderate-severe n = 84), including 175 women without preoperative US evidence of endometriosis. Surgery was done during menstrual (n = 83), follicular (n = 135) and luteal (n = 135) phases of the menstrual cycle. For analysis, the data were randomly divided into an independent training (n = 235) and a test (n = 118) data set. Statistical analysis was done using univariate and multivariate (logistic regression and least squares support vector machines (LS-SVM) approaches in training- and test data set separately to validate our findings. In the training set, two models of four biomarkers (Model 1: annexin V, VEGF, CA-125 and glycodelin; Model 2: annexin V, VEGF, CA-125 and sICAM-1) analysed in plasma, obtained during the menstrual phase, could predict US-negative endometriosis with a high sensitivity (81-90%) and an acceptable specificity (68-81%). The same two models predicted US-negative endometriosis in the independent validation test set with a high sensitivity (82%) and an acceptable specificity (63-75%). In plasma samples obtained during menstruation, multivariate analysis of four biomarkers (annexin V, VEGF, CA-125 and sICAM-1/or glycodelin) enabled the diagnosis of endometriosis undetectable by US with a sensitivity of 81-90% and a specificity of 63-81% in independent training- and test data set. The next step is to apply these models for preoperative prediction of endometriosis in an independent set of patients with infertility and/or pain without US evidence of endometriosis, scheduled for laparoscopy.

Direct Validation of Differential Prediction.

ERIC Educational Resources Information Center

Lunneborg, Clifford E.

Using academic achievement data for 655 University students, direct validation of differential predictions based on a battery of aptitude/achievement measures selected for their differential prediction efficiency was attempted. In the cross-validation of the prediction of actual differences among five academic area GPA's, this set of differential…

Revisiting the Measurement of Anomie

PubMed Central

Teymoori, Ali; Jetten, Jolanda; Bastian, Brock; Ariyanto, Amarina; Autin, Frédérique; Ayub, Nadia; Badea, Constantina; Besta, Tomasz; Butera, Fabrizio; Costa-Lopes, Rui; Cui, Lijuan; Fantini, Carole; Finchilescu, Gillian; Gaertner, Lowell; Gollwitzer, Mario; Gómez, Ángel; González, Roberto; Hong, Ying Yi; Jensen, Dorthe Høj; Karasawa, Minoru; Kessler, Thomas; Klein, Olivier; Lima, Marcus; Mähönen, Tuuli Anna; Megevand, Laura; Morton, Thomas; Paladino, Paola; Polya, Tibor; Ruza, Aleksejs; Shahrazad, Wan; Sharma, Sushama; Torres, Ana Raquel; van der Bles, Anne Marthe; Wohl, Michael

2016-01-01

Sociologists coined the term “anomie” to describe societies that are characterized by disintegration and deregulation. Extending beyond conceptualizations of anomie that conflate the measurements of anomie as ‘a state of society’ and as a ‘state of mind’, we disentangle these conceptualizations and develop an analysis and measure of this phenomenon focusing on anomie as a perception of the ‘state of society’. We propose that anomie encompasses two dimensions: a perceived breakdown in social fabric (i.e., disintegration as lack of trust and erosion of moral standards) and a perceived breakdown in leadership (i.e., deregulation as lack of legitimacy and effectiveness of leadership). Across six studies we present evidence for the validity of the new measure, the Perception of Anomie Scale (PAS). Studies 1a and 1b provide evidence for the proposed factor structure and internal consistency of PAS. Studies 2a-c provide evidence of convergent and discriminant validity. Finally, assessing PAS in 28 countries, we show that PAS correlates with national indicators of societal functioning and that PAS predicts national identification and well-being (Studies 3a & 3b). The broader implications of the anomie construct for the study of group processes are discussed. PMID:27383133

Revisiting the Measurement of Anomie.

PubMed

Teymoori, Ali; Jetten, Jolanda; Bastian, Brock; Ariyanto, Amarina; Autin, Frédérique; Ayub, Nadia; Badea, Constantina; Besta, Tomasz; Butera, Fabrizio; Costa-Lopes, Rui; Cui, Lijuan; Fantini, Carole; Finchilescu, Gillian; Gaertner, Lowell; Gollwitzer, Mario; Gómez, Ángel; González, Roberto; Hong, Ying Yi; Jensen, Dorthe Høj; Karasawa, Minoru; Kessler, Thomas; Klein, Olivier; Lima, Marcus; Mähönen, Tuuli Anna; Megevand, Laura; Morton, Thomas; Paladino, Paola; Polya, Tibor; Ruza, Aleksejs; Shahrazad, Wan; Sharma, Sushama; Torres, Ana Raquel; van der Bles, Anne Marthe; Wohl, Michael

2016-01-01

Sociologists coined the term "anomie" to describe societies that are characterized by disintegration and deregulation. Extending beyond conceptualizations of anomie that conflate the measurements of anomie as 'a state of society' and as a 'state of mind', we disentangle these conceptualizations and develop an analysis and measure of this phenomenon focusing on anomie as a perception of the 'state of society'. We propose that anomie encompasses two dimensions: a perceived breakdown in social fabric (i.e., disintegration as lack of trust and erosion of moral standards) and a perceived breakdown in leadership (i.e., deregulation as lack of legitimacy and effectiveness of leadership). Across six studies we present evidence for the validity of the new measure, the Perception of Anomie Scale (PAS). Studies 1a and 1b provide evidence for the proposed factor structure and internal consistency of PAS. Studies 2a-c provide evidence of convergent and discriminant validity. Finally, assessing PAS in 28 countries, we show that PAS correlates with national indicators of societal functioning and that PAS predicts national identification and well-being (Studies 3a & 3b). The broader implications of the anomie construct for the study of group processes are discussed.

Indirect Measurement of Sexual Orientation: Comparison of the Implicit Relational Assessment Procedure, Viewing Time, and Choice Reaction Time Tasks.

PubMed

Rönspies, Jelena; Schmidt, Alexander F; Melnikova, Anna; Krumova, Rosina; Zolfagari, Asadeh; Banse, Rainer

2015-07-01

The present study was conducted to validate an adaptation of the Implicit Relational Assessment Procedure (IRAP) as an indirect latency-based measure of sexual orientation. Furthermore, reliability and criterion validity of the IRAP were compared to two established indirect measures of sexual orientation: a Choice Reaction Time task (CRT) and a Viewing Time (VT) task. A sample of 87 heterosexual and 35 gay men completed all three indirect measures in an online study. The IRAP and the VT predicted sexual orientation nearly perfectly. Both measures also showed a considerable amount of convergent validity. Reliabilities (internal consistencies) reached satisfactory levels. In contrast, the CRT did not tap into sexual orientation in the present study. In sum, the VT measure performed best, with the IRAP showing only slightly lower reliability and criterion validity, whereas the CRT did not yield any evidence of reliability or criterion validity in the present research. The results were discussed in the light of specific task properties of the indirect latency-based measures (task-relevance vs. task-irrelevance).

Development and validation of a reading-related assessment battery in Malay for the purpose of dyslexia assessment.

PubMed

Lee, Lay Wah

2008-06-01

Malay is an alphabetic language with transparent orthography. A Malay reading-related assessment battery which was conceptualised based on the International Dyslexia Association definition of dyslexia was developed and validated for the purpose of dyslexia assessment. The battery consisted of ten tests: Letter Naming, Word Reading, Non-word Reading, Spelling, Passage Reading, Reading Comprehension, Listening Comprehension, Elision, Rapid Letter Naming and Digit Span. Content validity was established by expert judgment. Concurrent validity was obtained using the schools' language tests as criterion. Evidence of predictive and construct validity was obtained through regression analyses and factor analyses. Phonological awareness was the most significant predictor of word-level literacy skills in Malay, with rapid naming making independent secondary contributions. Decoding and listening comprehension made separate contributions to reading comprehension, with decoding as the more prominent predictor. Factor analysis revealed four factors: phonological decoding, phonological naming, comprehension and verbal short-term memory. In conclusion, despite differences in orthography, there are striking similarities in the theoretical constructs of reading-related tasks in Malay and in English.

SCORE should be preferred to Framingham to predict cardiovascular death in French population.

PubMed

Marchant, Ivanny; Boissel, Jean-Pierre; Kassaï, Behrouz; Bejan, Theodora; Massol, Jacques; Vidal, Chrystelle; Amsallem, Emmanuel; Naudin, Florence; Galan, Pilar; Czernichow, Sébastien; Nony, Patrice; Gueyffier, François

2009-10-01

Numerous studies have examined the validity of available scores to predict the absolute cardiovascular risk. We developed a virtual population based on data representative of the French population and compared the performances of the two most popular risk equations to predict cardiovascular death: Framingham and SCORE. A population was built based on official French demographic statistics and summarized data from representative observational studies. The 10-year coronary and cardiovascular death risk and their ratio were computed for each individual by SCORE and Framingham equations. The resulting rates were compared with those derived from national vital statistics. Framingham overestimated French coronary deaths by 2.8 in men and 1.9 in women, and cardiovascular deaths by 1.5 in men and 1.3 in women. SCORE overestimated coronary death by 1.6 in men and 1.7 in women, and underestimated cardiovascular death by 0.94 in men and 0.85 in women. Our results revealed an exaggerated representation of coronary among cardiovascular death predicted by Framingham, with coronary death exceeding cardiovascular death in some individual profiles. Sensitivity analyses gave some insights to explain the internal inconsistency of the Framingham equations. Evidence is that SCORE should be preferred to Framingham to predict cardiovascular death risk in French population. This discrepancy between prediction scores is likely to be observed in other populations. To improve the validation of risk equations, specific guidelines should be issued to harmonize the outcomes definition across epidemiologic studies. Prediction models should be calibrated for risk differences in the space and time dimensions.

Sentinel node status prediction by four statistical models: results from a large bi-institutional series (n = 1132).

PubMed

Mocellin, Simone; Thompson, John F; Pasquali, Sandro; Montesco, Maria C; Pilati, Pierluigi; Nitti, Donato; Saw, Robyn P; Scolyer, Richard A; Stretch, Jonathan R; Rossi, Carlo R

2009-12-01

To improve selection for sentinel node (SN) biopsy (SNB) in patients with cutaneous melanoma using statistical models predicting SN status. About 80% of patients currently undergoing SNB are node negative. In the absence of conclusive evidence of a SNBassociated survival benefit, these patients may be over-treated. Here, we tested the efficiency of 4 different models in predicting SN status. The clinicopathologic data (age, gender, tumor thickness, Clark level, regression, ulceration, histologic subtype, and mitotic index) of 1132 melanoma patients who had undergone SNB at institutions in Italy and Australia were analyzed. Logistic regression, classification tree, random forest, and support vector machine models were fitted to the data. The predictive models were built with the aim of maximizing the negative predictive value (NPV) and reducing the rate of SNB procedures though minimizing the error rate. After cross-validation logistic regression, classification tree, random forest, and support vector machine predictive models obtained clinically relevant NPV (93.6%, 94.0%, 97.1%, and 93.0%, respectively), SNB reduction (27.5%, 29.8%, 18.2%, and 30.1%, respectively), and error rates (1.8%, 1.8%, 0.5%, and 2.1%, respectively). Using commonly available clinicopathologic variables, predictive models can preoperatively identify a proportion of patients ( approximately 25%) who might be spared SNB, with an acceptable (1%-2%) error. If validated in large prospective series, these models might be implemented in the clinical setting for improved patient selection, which ultimately would lead to better quality of life for patients and optimization of resource allocation for the health care system.

Analytical Modeling for Mechanical Strength Prediction with Raman Spectroscopy and Fractured Surface Morphology of Novel Coconut Shell Powder Reinforced: Epoxy Composites

NASA Astrophysics Data System (ADS)

Singh, Savita; Singh, Alok; Sharma, Sudhir Kumar

2017-06-01

In this paper, an analytical modeling and prediction of tensile and flexural strength of three dimensional micro-scaled novel coconut shell powder (CSP) reinforced epoxy polymer composites have been reported. The novel CSP has a specific mixing ratio of different coconut shell particle size. A comparison is made between obtained experimental strength and modified Guth model. The result shows a strong evidence for non-validation of modified Guth model for strength prediction. Consequently, a constitutive modeled equation named Singh model has been developed to predict the tensile and flexural strength of this novel CSP reinforced epoxy composite. Moreover, high resolution Raman spectrum shows that 40 % CSP reinforced epoxy composite has high dielectric constant to become an alternative material for capacitance whereas fractured surface morphology revealed that a strong bonding between novel CSP and epoxy polymer for the application as light weight composite materials in engineering.

Mammographic density, breast cancer risk and risk prediction

PubMed Central

Vachon, Celine M; van Gils, Carla H; Sellers, Thomas A; Ghosh, Karthik; Pruthi, Sandhya; Brandt, Kathleen R; Pankratz, V Shane

2007-01-01

In this review, we examine the evidence for mammographic density as an independent risk factor for breast cancer, describe the risk prediction models that have incorporated density, and discuss the current and future implications of using mammographic density in clinical practice. Mammographic density is a consistent and strong risk factor for breast cancer in several populations and across age at mammogram. Recently, this risk factor has been added to existing breast cancer risk prediction models, increasing the discriminatory accuracy with its inclusion, albeit slightly. With validation, these models may replace the existing Gail model for clinical risk assessment. However, absolute risk estimates resulting from these improved models are still limited in their ability to characterize an individual's probability of developing cancer. Promising new measures of mammographic density, including volumetric density, which can be standardized using full-field digital mammography, will likely result in a stronger risk factor and improve accuracy of risk prediction models. PMID:18190724

Validity evidence as a key marker of quality of technical skill assessment in OTL-HNS.

PubMed

Labbé, Mathilde; Young, Meredith; Nguyen, Lily H P

2018-01-13

Quality monitoring of assessment practices should be a priority in all residency programs. Validity evidence is one of the main hallmarks of assessment quality and should be collected to support the interpretation and use of assessment data. Our objective was to identify, synthesize, and present the validity evidence reported supporting different technical skill assessment tools in otolaryngology-head and neck surgery (OTL-HNS). We performed a secondary analysis of data generated through a systematic review of all published tools for assessing technical skills in OTL-HNS (n = 16). For each tool, we coded validity evidence according to the five types of evidence described by the American Educational Research Association's interpretation of Messick's validity framework. Descriptive statistical analyses were conducted. All 16 tools included in our analysis were supported by internal structure and relationship to variables validity evidence. Eleven articles presented evidence supporting content. Response process was discussed only in one article, and no study reported on evidence exploring consequences. We present the validity evidence reported for 16 rater-based tools that could be used for work-based assessment of OTL-HNS residents in the operating room. The articles included in our review were consistently deficient in evidence for response process and consequences. Rater-based assessment tools that support high-stakes decisions that impact the learner and programs should include several sources of validity evidence. Thus, use of any assessment should be done with careful consideration of the context-specific validity evidence supporting score interpretation, and we encourage deliberate continual assessment quality-monitoring. NA. Laryngoscope, 2018. © 2018 The American Laryngological, Rhinological and Otological Society, Inc.

Validity evidence for the Fundamentals of Laparoscopic Surgery (FLS) program as an assessment tool: a systematic review.

PubMed

Zendejas, Benjamin; Ruparel, Raaj K; Cook, David A

2016-02-01

The Fundamentals of Laparoscopic Surgery (FLS) program uses five simulation stations (peg transfer, precision cutting, loop ligation, and suturing with extracorporeal and intracorporeal knot tying) to teach and assess laparoscopic surgery skills. We sought to summarize evidence regarding the validity of scores from the FLS assessment. We systematically searched for studies evaluating the FLS as an assessment tool (last search update February 26, 2013). We classified validity evidence using the currently standard validity framework (content, response process, internal structure, relations with other variables, and consequences). From a pool of 11,628 studies, we identified 23 studies reporting validity evidence for FLS scores. Studies involved residents (n = 19), practicing physicians (n = 17), and medical students (n = 8), in specialties of general (n = 17), gynecologic (n = 4), urologic (n = 1), and veterinary (n = 1) surgery. Evidence was most common in the form of relations with other variables (n = 22, most often expert-novice differences). Only three studies reported internal structure evidence (inter-rater or inter-station reliability), two studies reported content evidence (i.e., derivation of assessment elements), and three studies reported consequences evidence (definition of pass/fail thresholds). Evidence nearly always supported the validity of FLS total scores. However, the loop ligation task lacks discriminatory ability. Validity evidence confirms expected relations with other variables and acceptable inter-rater reliability, but other validity evidence is sparse. Given the high-stakes use of this assessment (required for board eligibility), we suggest that more validity evidence is required, especially to support its content (selection of tasks and scoring rubric) and the consequences (favorable and unfavorable impact) of assessment.

Predicting cognitive function from clinical measures of physical function and health status in older adults.

PubMed

Bolandzadeh, Niousha; Kording, Konrad; Salowitz, Nicole; Davis, Jennifer C; Hsu, Liang; Chan, Alison; Sharma, Devika; Blohm, Gunnar; Liu-Ambrose, Teresa

2015-01-01

Current research suggests that the neuropathology of dementia-including brain changes leading to memory impairment and cognitive decline-is evident years before the onset of this disease. Older adults with cognitive decline have reduced functional independence and quality of life, and are at greater risk for developing dementia. Therefore, identifying biomarkers that can be easily assessed within the clinical setting and predict cognitive decline is important. Early recognition of cognitive decline could promote timely implementation of preventive strategies. We included 89 community-dwelling adults aged 70 years and older in our study, and collected 32 measures of physical function, health status and cognitive function at baseline. We utilized an L1-L2 regularized regression model (elastic net) to identify which of the 32 baseline measures were strongly predictive of cognitive function after one year. We built three linear regression models: 1) based on baseline cognitive function, 2) based on variables consistently selected in every cross-validation loop, and 3) a full model based on all the 32 variables. Each of these models was carefully tested with nested cross-validation. Our model with the six variables consistently selected in every cross-validation loop had a mean squared prediction error of 7.47. This number was smaller than that of the full model (115.33) and the model with baseline cognitive function (7.98). Our model explained 47% of the variance in cognitive function after one year. We built a parsimonious model based on a selected set of six physical function and health status measures strongly predictive of cognitive function after one year. In addition to reducing the complexity of the model without changing the model significantly, our model with the top variables improved the mean prediction error and R-squared. These six physical function and health status measures can be easily implemented in a clinical setting.

Stratospheric ozone - Fragile shield. [SST exhausts and Freons impact

NASA Technical Reports Server (NTRS)

Hoffert, M. I.; Stewart, R. W.

1975-01-01

Atmospheric models that have been used in major studies on the possible impact of SST exhausts and Freons on stratospheric ozone are discussed and compared. An overview is given of ozone-reduction estimates that they produce, together with an assessment of possible effects of atmospheric testing of thermonuclear bombs in an attempt to find direct observational evidence for ozone depletion resulting from human activities. It is concluded that clear validation of atmospheric-model predictions is lacking.

Predicting Relapse in Patients With Medulloblastoma by Integrating Evidence From Clinical and Genomic Features

PubMed Central

Tamayo, Pablo; Cho, Yoon-Jae; Tsherniak, Aviad; Greulich, Heidi; Ambrogio, Lauren; Schouten-van Meeteren, Netteke; Zhou, Tianni; Buxton, Allen; Kool, Marcel; Meyerson, Matthew; Pomeroy, Scott L.; Mesirov, Jill P.

2011-01-01

Purpose Despite significant progress in the molecular understanding of medulloblastoma, stratification of risk in patients remains a challenge. Focus has shifted from clinical parameters to molecular markers, such as expression of specific genes and selected genomic abnormalities, to improve accuracy of treatment outcome prediction. Here, we show how integration of high-level clinical and genomic features or risk factors, including disease subtype, can yield more comprehensive, accurate, and biologically interpretable prediction models for relapse versus no-relapse classification. We also introduce a novel Bayesian nomogram indicating the amount of evidence that each feature contributes on a patient-by-patient basis. Patients and Methods A Bayesian cumulative log-odds model of outcome was developed from a training cohort of 96 children treated for medulloblastoma, starting with the evidence provided by clinical features of metastasis and histology (model A) and incrementally adding the evidence from gene-expression–derived features representing disease subtype–independent (model B) and disease subtype–dependent (model C) pathways, and finally high-level copy-number genomic abnormalities (model D). The models were validated on an independent test cohort (n = 78). Results On an independent multi-institutional test data set, models A to D attain an area under receiver operating characteristic (au-ROC) curve of 0.73 (95% CI, 0.60 to 0.84), 0.75 (95% CI, 0.64 to 0.86), 0.80 (95% CI, 0.70 to 0.90), and 0.78 (95% CI, 0.68 to 0.88), respectively, for predicting relapse versus no relapse. Conclusion The proposed models C and D outperform the current clinical classification schema (au-ROC, 0.68), our previously published eight-gene outcome signature (au-ROC, 0.71), and several new schemas recently proposed in the literature for medulloblastoma risk stratification. PMID:21357789

The 2014 Sandia Verification and Validation Challenge: Problem statement

DOE PAGES

Hu, Kenneth; Orient, George

2016-01-18

This paper presents a case study in utilizing information from experiments, models, and verification and validation (V&V) to support a decision. It consists of a simple system with data and models provided, plus a safety requirement to assess. The goal is to pose a problem that is flexible enough to allow challengers to demonstrate a variety of approaches, but constrained enough to focus attention on a theme. This was accomplished by providing a good deal of background information in addition to the data, models, and code, but directing the participants' activities with specific deliverables. In this challenge, the theme ismore » how to gather and present evidence about the quality of model predictions, in order to support a decision. This case study formed the basis of the 2014 Sandia V&V Challenge Workshop and this resulting special edition of the ASME Journal of Verification, Validation, and Uncertainty Quantification.« less

Development and psychometric evaluation of a quantitative measure of "fat talk".

PubMed

MacDonald Clarke, Paige; Murnen, Sarah K; Smolak, Linda

2010-01-01

Based on her anthropological research, Nichter (2000) concluded that it is normative for many American girls to engage in body self-disparagement in the form of "fat talk." The purpose of the present two studies was to develop a quantitative measure of fat talk. A series of 17 scenarios were created in which "Naomi" is talking with a female friend(s) and there is an expression of fat talk. College women respondents rated the frequency with which they would behave in a similar way as the women in each scenario. A nine-item one-factor scale was determined through principal components analysis and its scores yielded evidence of internal consistency reliability, test-retest reliability over a five-week time period, construct validity, discriminant validity, and incremental validity in that it predicted unique variance in body shame and eating disorder symptoms above and beyond other measures of self-objectification. Copyright 2009 Elsevier Ltd. All rights reserved.

A Novel Model for Predicting Incident Moderate to Severe Anemia and Iron Deficiency in Patients with Newly Diagnosed Ulcerative Colitis.

PubMed

Khan, Nabeel; Patel, Dhruvan; Shah, Yash; Yang, Yu-Xiao

2017-05-01

Anemia and iron deficiency are common complications of ulcerative colitis (UC). We aimed to develop and internally validate a prediction model for the incidence of moderate to severe anemia and iron deficiency anemia (IDA) in newly diagnosed patients with UC. Multivariable logistic regression was performed among a nationwide cohort of patients who were newly diagnosed with UC in the VA health-care system. Model development was performed in a random two-third of the total cohort and then validated in the remaining one-third of the cohort. As candidate predictors, we examined routinely available data at the time of UC diagnosis including demographics, medications, laboratory results, and endoscopy findings. A total of 789 patients met the inclusion criteria. For the outcome of moderate to severe anemia, age, albumin level and mild anemia at UC diagnosis were predictors selected for the model. The AUC for this model was 0.69 (95% CI 0.64-0.74). For the outcome of moderate to severe anemia with evidence of iron deficiency, the predictors included African-American ethnicity, mild anemia, age, and albumin level at UC diagnosis. The AUC was 0.76, (95% CI 0.69-0.82). Calibration was consistently good in all models (Hosmer-Lemeshow goodness of fit p > 0.05). The models performed similarly in the internal validation cohort. We developed and internally validated a prognostic model for predicting the risk of moderate to severe anemia and IDA among newly diagnosed patients with UC. This will help identify patients at high risk of these complications, who could benefit from surveillance and preventive measures.

Prognostic indices for early mortality in ischaemic stroke - meta-analysis.

PubMed

Mattishent, K; Kwok, C S; Mahtani, A; Pelpola, K; Myint, P K; Loke, Y K

2016-01-01

Several models have been developed to predict mortality in ischaemic stroke. We aimed to evaluate systematically the performance of published stroke prognostic scores. We searched MEDLINE and EMBASE in February 2014 for prognostic models (published between 2003 and 2014) used in predicting early mortality (<6 months) after ischaemic stroke. We evaluated discriminant ability of the tools through meta-analysis of the area under the curve receiver operating characteristic curve (AUROC) or c-statistic. We evaluated the following components of study validity: collection of prognostic variables, neuroimaging, treatment pathways and missing data. We identified 18 articles (involving 163 240 patients) reporting on the performance of prognostic models for mortality in ischaemic stroke, with 15 articles providing AUC for meta-analysis. Most studies were either retrospective, or post hoc analyses of prospectively collected data; all but three reported validation data. The iSCORE had the largest number of validation cohorts (five) within our systematic review and showed good performance in four different countries, pooled AUC 0.84 (95% CI 0.82-0.87). We identified other potentially useful prognostic tools that have yet to be as extensively validated as iSCORE - these include SOAR (2 studies, pooled AUC 0.79, 95% CI 0.78-0.80), GWTG (2 studies, pooled AUC 0.72, 95% CI 0.72-0.72) and PLAN (1 study, pooled AUC 0.85, 95% CI 0.84-0.87). Our meta-analysis has identified and summarized the performance of several prognostic scores with modest to good predictive accuracy for early mortality in ischaemic stroke, with the iSCORE having the broadest evidence base. © 2015 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

Why does self-reported emotional intelligence predict job performance? A meta-analytic investigation of mixed EI.

PubMed

Joseph, Dana L; Jin, Jing; Newman, Daniel A; O'Boyle, Ernest H

2015-03-01

Recent empirical reviews have claimed a surprisingly strong relationship between job performance and self-reported emotional intelligence (also commonly called trait EI or mixed EI), suggesting self-reported/mixed EI is one of the best known predictors of job performance (e.g., ρ = .47; Joseph & Newman, 2010b). Results further suggest mixed EI can robustly predict job performance beyond cognitive ability and Big Five personality traits (Joseph & Newman, 2010b; O'Boyle, Humphrey, Pollack, Hawver, & Story, 2011). These criterion-related validity results are problematic, given the paucity of evidence and the questionable construct validity of mixed EI measures themselves. In the current research, we update and reevaluate existing evidence for mixed EI, in light of prior work regarding the content of mixed EI measures. Results of the current meta-analysis demonstrate that (a) the content of mixed EI measures strongly overlaps with a set of well-known psychological constructs (i.e., ability EI, self-efficacy, and self-rated performance, in addition to Conscientiousness, Emotional Stability, Extraversion, and general mental ability; multiple R = .79), (b) an updated estimate of the meta-analytic correlation between mixed EI and supervisor-rated job performance is ρ = .29, and (c) the mixed EI-job performance relationship becomes nil (β = -.02) after controlling for the set of covariates listed above. Findings help to establish the construct validity of mixed EI measures and further support an intuitive theoretical explanation for the uncommonly high association between mixed EI and job performance--mixed EI instruments assess a combination of ability EI and self-perceptions, in addition to personality and cognitive ability. PsycINFO Database Record (c) 2015 APA, all rights reserved.

Reliability and validity of the Haitian Creole PHQ-9.

PubMed

Marc, Linda G; Henderson, Whitney R; Desrosiers, Astrid; Testa, Marcia A; Jean, Samuel E; Akom, Eniko Edit

2014-12-01

There is limited information on depression in Haitians and this is partly attributable to the absence of culturally and linguistically adapted measures for depression. To perform a psychometric evaluation of the Haitian-Creole version of the PHQ-9 administered to men who have sex with men (MSM) in the Republic of Haiti. This study uses a cross-sectional design and data are from the Integrated Behavioral and Biological HIV Survey (IBBS) for MSM in Haiti. Inclusion criteria required that participants be male, ≥ 18 years, report sexual relations with a male partner in the last 12 months, and lived in Haiti during the past 3 months. Respondent Driven Sampling was used for participant recruitment. A structured questionnaire was verbally administered in Haitian-Creole capturing information on sociodemographics, sexual behaviors, human immunodeficiency virus (HIV) status and depressive symptomatology using the PHQ-9. Psychometric analyses of the translated PHQ-9 assessed unidimensionality, factor structure, reliability, construct validity, and differential item functioning (DIF) across subgroups (age, educational level, sexual orientation and HIV status). In a study population of 1,028 MSM, the Haitian-Creole version of the PHQ-9 is unidimensional, has moderately high internal consistency reliability (α = 0.78), and shows evidence of construct validity where HIV-positive subjects have greater depression (p = 0.002). There is no evidence of DIF across age, education, sexual orientation or HIV status. HIV-positive MSM are twice as likely to screen positive for moderately severe and severe depressive symptoms compared to their HIV-negative counterparts. There is strong evidence for the psychometric adequacy of the translated PHQ-9 screening tool as a measure of depression with MSM in Haiti. Future research is necessary to examine the predictive validity of depression for subsequent health behaviors or clinical outcomes among Haitian MSM.

Improving the residency admissions process by integrating a professionalism assessment: a validity and feasibility study.

PubMed

Bajwa, Nadia M; Yudkowsky, Rachel; Belli, Dominique; Vu, Nu Viet; Park, Yoon Soo

2017-03-01

The purpose of this study was to provide validity and feasibility evidence in measuring professionalism using the Professionalism Mini-Evaluation Exercise (P-MEX) scores as part of a residency admissions process. In 2012 and 2013, three standardized-patient-based P-MEX encounters were administered to applicants invited for an interview at the University of Geneva Pediatrics Residency Program. Validity evidence was gathered for P-MEX content (item analysis); response process (qualitative feedback); internal structure (inter-rater reliability with intraclass correlation and Generalizability); relations to other variables (correlations); and consequences (logistic regression to predict admission). To improve reliability, Kane's formula was used to create an applicant composite score using P-MEX, structured letter of recommendation (SLR), and structured interview (SI) scores. Applicant rank lists using composite scores versus faculty global ratings were compared using the Wilcoxon signed-rank test. Seventy applicants were assessed. Moderate associations were found between pairwise correlations of P-MEX scores and SLR (r = 0.25, P = .036), SI (r = 0.34, P = .004), and global ratings (r = 0.48, P < .001). Generalizability of the P-MEX using three cases was moderate (G-coefficient = 0.45). P-MEX scores had the greatest correlation with acceptance (r = 0.56, P < .001), were the strongest predictor of acceptance (OR 4.37, P < .001), and increased pseudo R-squared by 0.20 points. Including P-MEX scores increased composite score reliability from 0.51 to 0.74. Rank lists of applicants using composite score versus global rating differed significantly (z = 5.41, P < .001). Validity evidence supports the use of P-MEX scores to improve the reliability of the residency admissions process by improving applicant composite score reliability.

Evidence for the Continuous Latent Structure of Mania in the Epidemiologic Catchment Area from Multiple Latent Structure and Construct Validation Methodologies

PubMed Central

Prisciandaro, James J.; Roberts, John E.

2011-01-01

Background Although psychiatric diagnostic systems have conceptualized mania as a discrete phenomenon, appropriate latent structure investigations testing this conceptualization are lacking. In contrast to these diagnostic systems, several influential theories of mania have suggested a continuous conceptualization. The present study examined whether mania has a continuous or discrete latent structure using a comprehensive approach including taxometric, information-theoretic latent distribution modeling (ITLDM), and predictive validity methodologies in the Epidemiologic Catchment Area (ECA) study. Methods Eight dichotomous manic symptom items were submitted to a variety of latent structural analyses; including factor analyses, taxometric procedures, and ITLDM; in 10,105 ECA community participants. Additionally, a variety of continuous and discrete models of mania were compared in terms of their relative abilities to predict outcomes (i.e., health service utilization, internalizing and externalizing disorders, and suicidal behavior). Results Taxometric and ITLDM analyses consistently supported a continuous conceptualization of mania. In ITLDM analyses, a continuous model of mania demonstrated 6:52:1 odds over the best fitting latent class model of mania. Factor analyses suggested that the continuous structure of mania was best represented by a single latent factor. Predictive validity analyses demonstrated a consistent superior ability of continuous models of mania relative to discrete models. Conclusions The present study provided three independent lines of support for a continuous conceptualization of mania. The implications of a continuous model of mania are discussed. PMID:20507671

Evaluating Shortened Versions of the AUDIT as Screeners for Alcohol Use Problems in a General Population Study.

PubMed

Nayak, Madhabika B; Bond, Jason C; Greenfield, Thomas K

2015-01-01

Efficient alcohol screening measures are important to prevent or treat alcohol use disorders (AUDs). We studied different versions of the Alcohol Use Disorders Identification Test (AUDIT) comparing their performance to the full AUDIT and an AUD measure as screeners for alcohol use problems in Goa, India. Data from a general population study on 743 male drinkers aged 18-49 years are reported. Drinkers completed the AUDIT and an AUD measure. We created shorter versions of the AUDIT by (a) collapsing AUDIT item responses into three and two categories and (b) deleting two items with the lowest factor loadings. Each version was evaluated using factor, reliability and validity, and differential item functioning (DIF) analysis by age, education, standard of living index (SLI), and area of residence. A single factor solution was found for each version with lower factor loadings for items on guilt and concern. There were no significant differences among the different AUDIT versions in predicting AUD. No significant DIF was found by education, SLI or area of residence. DIF was observed for the alcohol frequency item by age. The AUDIT may be used with dichotomized response options without loss of predictive validity. A shortened eight-item dichotomized scale can adequately screen for AUDs in Goa when brevity is of paramount importance, although with lower predictive validity. Although the frequency item was endorsed more by older men, there is no evidence that the AUDIT items perform differently in other groups of male drinkers in Goa.

Evidence-based selection process to the Master of Public Health program at Medical University.

PubMed

Panczyk, Mariusz; Juszczyk, Grzegorz; Zarzeka, Aleksander; Samoliński, Łukasz; Belowska, Jarosława; Cieślak, Ilona; Gotlib, Joanna

2017-09-11

Evaluation of the predictive validity of selected sociodemographic factors and admission criteria for Master's studies in Public Health at the Faculty of Health Sciences, Medical University of Warsaw (MUW). For the evaluation purposes recruitment data and learning results of students enrolled between 2008 and 2012 were used (N = 605, average age 22.9 ± 3.01). The predictive analysis was performed using the multiple linear regression method. In the proposed regression model 12 predictors were selected, including: sex, age, professional degree (BA), the Bachelor's studies grade point average (GPA), total score of the preliminary examination broken down into five thematic areas. Depending on the tested model, one of two dependent variables was used: first-year GPA or cumulative GPA in the Master program. The regression model based on the result variable of Master's GPA program was better matched to data in comparison to the model based on the first year GPA (adjusted R 2 0.413 versus 0.476 respectively). The Bachelor's studies GPA and each of the five subtests comprising the test entrance exam were significant predictors of success achieved by a student both after the first year and at the end of the course of studies. Criteria of admissions with total score of MCQs exam and Bachelor's studies GPA can be successfully used for selection of the candidates for Master's degree studies in Public Health. The high predictive validity of the recruitment system confirms the validity of the adopted admission policy at MUW.

A comparison of the psychometric properties of the psychopathic personality inventory full-length and short-form versions.

PubMed

Kastner, Rebecca M; Sellbom, Martin; Lilienfeld, Scott O

2012-03-01

The Psychopathic Personality Inventory (PPI) has shown promising construct validity as a measure of psychopathy. Because of its relative efficiency, a short-form version of the PPI (PPI-SF) was developed and has proven useful in many psychopathy studies. The validity of the PPI-SF, however, has not been thoroughly examined, and no studies have directly compared the validity of the short form with that of the full-length version. The current study was designed to compare the psychometric properties of both PPI versions, with an emphasis on convergent and discriminant validity in predicting external criteria conceptually relevant to psychopathy. We used both prison (n = 558) and college samples (n = 322) for this investigation. PPI scale scores were more reliable and more strongly correlated with the conceptually relevant criterion measures compared with the PPI-SF, particularly in the prison sample. There were no differences in relative discriminant validity. Thus, overall, the PPI full-length version showed more evidence of construct validity than did the short form, and the consequences of this psychometric difference should be considered when evaluating the clinical utility of each measure.

Original structures, and fragmentation and reassembly histories of asteroids - Evidence from meteorites

NASA Technical Reports Server (NTRS)

Taylor, G. Jeffrey; Maggiore, Peter; Scott, Edward R. D.; Rubin, Alan E.; Keil, Klaus

1987-01-01

The validity of an onion shell model (OSM) for chondrite parent asteroids was assessed using metallographic cooling rates (MCR) derived from the compositions of metallic Fe-Ni grains. The hypothesis evaluated was that the hottest materials in chondrites would have been buried the deepest and cooled the slowest. The survey covered breccia from regolith and 13 different chondrites. The MCRs agreed well with cooling rates predicted by fission-track thermometry and Ar-40/Ar-39 ages. The OSM predicts an inverse correlation between the cooling rate and the petrographic type. Low correlations found between the MCRs and petrographic type indicate that chondrite parent asteroids were not assembled with onion shell structures.

Assessing Dependency using Self-report and Indirect Measures: Examining the Significance of Discrepancies

PubMed Central

Cogswell, Alex; Alloy, Lauren B.; Karpinski, Andrew; Grant, David

2011-01-01

The present study addressed convergence between self-report and indirect approaches to assessing dependency. The study was moderately successful in validating an implicit measure, which was found to be reliable, orthogonal to two self-report instruments, and predictive of external criteria. This study also examined discrepancies between scores on self-report and implicit measures, and has implications for their significance. The possibility that discrepancies themselves are pathological was not supported, although discrepancies were associated with particular personality profiles. Finally, this study offered additional evidence for the relation between dependency and depressive symptomatology, and identified implicit dependency as contributing unique variance in predicting past major depression. PMID:20552505

Validation of variants in SLC28A3 and UGT1A6 as genetic markers predictive of anthracycline-induced cardiotoxicity in children.

PubMed

Visscher, H; Ross, C J D; Rassekh, S R; Sandor, G S S; Caron, H N; van Dalen, E C; Kremer, L C; van der Pal, H J; Rogers, P C; Rieder, M J; Carleton, B C; Hayden, M R

2013-08-01

The use of anthracyclines as effective antineoplastic drugs is limited by the occurrence of cardiotoxicity. Multiple genetic variants predictive of anthracycline-induced cardiotoxicity (ACT) in children were recently identified. The current study was aimed to assess replication of these findings in an independent cohort of children. . Twenty-three variants were tested for association with ACT in an independent cohort of 218 patients. Predictive models including genetic and clinical risk factors were constructed in the original cohort and assessed in the current replication cohort. . We confirmed the association of rs17863783 in UGT1A6 and ACT in the replication cohort (P = 0.0062, odds ratio (OR) 7.98). Additional evidence for association of rs7853758 (P = 0.058, OR 0.46) and rs885004 (P = 0.058, OR 0.42) in SLC28A3 was found (combined P = 1.6 × 10(-5) and P = 3.0 × 10(-5), respectively). A previously constructed prediction model did not significantly improve risk prediction in the replication cohort over clinical factors alone. However, an improved prediction model constructed using replicated genetic variants as well as clinical factors discriminated significantly better between cases and controls than clinical factors alone in both original (AUC 0.77 vs. 0.68, P = 0.0031) and replication cohort (AUC 0.77 vs. 0.69, P = 0.060). . We validated genetic variants in two genes predictive of ACT in an independent cohort. A prediction model combining replicated genetic variants as well as clinical risk factors might be able to identify high- and low-risk patients who could benefit from alternative treatment options. Copyright © 2013 Wiley Periodicals, Inc.

‘Emotional Intelligence’: Lessons from Lesions

PubMed Central

Hogeveen, J.; Salvi, C.; Grafman, J.

2018-01-01

‘Emotional intelligence’ (EI) is one of the most highly used psychological terms in popular nomenclature, yet its construct, divergent, and predictive validities are contentiously debated. Despite this debate, the EI construct is composed of a set of emotional abilities – recognizing emotional states in the self and others, using emotions to guide thought and behavior, understanding how emotions shape behavior, and emotion regulation – that undoubtedly influence important social and personal outcomes. In this review, evidence from human lesion studies is reviewed in order to provide insight into the necessary brain regions for each of these core emotional abilities. Critically, we consider how this neuropsychological evidence might help to guide efforts to define and measure EI. PMID:27647325

'Emotional Intelligence': Lessons from Lesions.

PubMed

Hogeveen, J; Salvi, C; Grafman, J

2016-10-01

'Emotional intelligence' (EI) is one of the most highly used psychological terms in popular nomenclature, yet its construct, divergent, and predictive validities are contentiously debated. Despite this debate, the EI construct is composed of a set of emotional abilities - recognizing emotional states in the self and others, using emotions to guide thought and behavior, understanding how emotions shape behavior, and emotion regulation - that undoubtedly influence important social and personal outcomes. In this review, evidence from human lesion studies is reviewed in order to provide insight into the necessary brain regions for each of these core emotional abilities. Critically, we consider how this neuropsychological evidence might help to guide efforts to define and measure EI. Copyright © 2016 Elsevier Ltd. All rights reserved.

Development of Decision Support Formulas for the Prediction of Bladder Outlet Obstruction and Prostatic Surgery in Patients With Lower Urinary Tract Symptom/Benign Prostatic Hyperplasia: Part II, External Validation and Usability Testing of a Smartphone App.

PubMed

Choo, Min Soo; Jeong, Seong Jin; Cho, Sung Yong; Yoo, Changwon; Jeong, Chang Wook; Ku, Ja Hyeon; Oh, Seung-June

2017-04-01

We aimed to externally validate the prediction model we developed for having bladder outlet obstruction (BOO) and requiring prostatic surgery using 2 independent data sets from tertiary referral centers, and also aimed to validate a mobile app for using this model through usability testing. Formulas and nomograms predicting whether a subject has BOO and needs prostatic surgery were validated with an external validation cohort from Seoul National University Bundang Hospital and Seoul Metropolitan Government-Seoul National University Boramae Medical Center between January 2004 and April 2015. A smartphone-based app was developed, and 8 young urologists were enrolled for usability testing to identify any human factor issues of the app. A total of 642 patients were included in the external validation cohort. No significant differences were found in the baseline characteristics of major parameters between the original (n=1,179) and the external validation cohort, except for the maximal flow rate. Predictions of requiring prostatic surgery in the validation cohort showed a sensitivity of 80.6%, a specificity of 73.2%, a positive predictive value of 49.7%, and a negative predictive value of 92.0%, and area under receiver operating curve of 0.84. The calibration plot indicated that the predictions have good correspondence. The decision curve showed also a high net benefit. Similar evaluation results using the external validation cohort were seen in the predictions of having BOO. Overall results of the usability test demonstrated that the app was user-friendly with no major human factor issues. External validation of these newly developed a prediction model demonstrated a moderate level of discrimination, adequate calibration, and high net benefit gains for predicting both having BOO and requiring prostatic surgery. Also a smartphone app implementing the prediction model was user-friendly with no major human factor issue.

Preschool Neuropsychological Measures as Predictors of Later Attention Deficit Hyperactivity Disorder

PubMed Central

Breaux, Rosanna P.; Griffith, Shayl F.; Harvey, Elizabeth A.

2016-01-01

The present study examined preschool neuropsychological measures as predictors of school-age attention deficit hyperactivity disorder (ADHD). Participants included 168 children (91 males) who completed neuropsychological measures at ages 3 and 4, and who were evaluated for ADHD and oppositional defiant disorder at age 6. The Conners’ Kiddie Continuous Performance Test (K-CPT), NEPSY Statue subtest, and a delay aversion task significantly distinguished at-risk children who later did and did not meet criteria for ADHD, with poor to fair overall predictive power, specificity, and sensitivity. However, only the K-CPT ADHD Confidence Index and battery added incremental predictive validity beyond early ADHD symptoms. This battery approach, which required impairment on at least 2 of the 3 significant measures, yielded fair overall predictive power, specificity, and sensitivity, and correctly classified 67% of children. In addition, there was some support for the specificity hypothesis, with evidence that cool executive function measures (K-CPT and Statue subtest) tended to predict inattentive symptoms. These findings suggest that neuropsychological deficits are evident by preschool-age in children with ADHD, but neuropsychological tests may still misclassify approximately one-third of children if used alone. Thus, neuropsychological measures may be a useful component of early ADHD assessments, but should be used with caution and in combination with other assessment methods. PMID:26936037

Severity of complicated versus uncomplicated subthreshold depression: New evidence on the "Monotonicity Thesis" from the national comorbidity survey.

PubMed

Wakefield, Jerome C; Schmitz, Mark F

2017-04-01

"Complicated" subthreshold depression (CsD) includes at least one of six pathosuggestive "complicated" symptoms: >6 months duration, marked role impairment, sense of worthlessness, suicidal ideation, psychotic ideation, and psychomotor retardation. "Uncomplicated" subthreshold depression (UsD) has no complicated features. Whereas studies show that complicated (CMDD) versus uncomplicated (UMDD) major depression differ substantially in severity and prognosis, UsD and CsD severity has not been previously compared. This study evaluates UsD and CsD pathology validator levels and examines whether the complicated/uncomplicated distinction offers incremental concurrent validity over the standard number-of-symptoms dimension as a depression severity measure. Using nationally representative community data from the National Comorbidity Survey, seven depression lifetime history subgroups were identified: one MDD screener symptom (n=1432); UsD (n=430); CsD (n=611); UMDD (n=182); and CMDD with 5-6 symptoms (n=518), 7 symptoms (n=217), and 8-9 symptoms (n=291). Severity was evaluated using five concurrent pathology validators: suicide attempt, interference with life, help seeking, hospitalization, and generalized anxiety disorder. CsD validator levels are substantially higher than both UsD and UMDD levels, and similar to mild CMDD, disconfirming the "monotonicity thesis" that severity increase with symptom number. Complicated/uncomplicated status predicts severity, and when complicatedness is controlled, number of symptoms no longer predicts validator levels. Diagnoses were based on respondents' fallible retrospective symptom reports during a lay-administered structured interview, which may not yield diagnoses comparable to clinicians' assessments. CsD is more severe than UsD and comparable to mild MDD. Complicated status more validly indicates depression severity than the standard number-of-symptoms measure. Copyright © 2017 Elsevier B.V. All rights reserved.

Predictive Performance Assessment: Trait and State Dimensions Should not be Confused

NASA Astrophysics Data System (ADS)

Pattyn, N.; Migeotte, P.-F.; Morais, J.; Cluydts, R.; Soetens, E.; Meeusen, R.; de Schutter, G.; Nederhof, E.; Kolinsky, R.

2008-06-01

One of the major aims of performance investigation is to obtain a measure predicting real-life performance, in order to prevent consequences of a potential decrement. Whereas the predictive validity of such assessment has been extensively described for long-term outcomes, as is the case for testing in selection context, equivalent evidence is lacking regarding the short-term predictive value of cognitive testing, i.e., whether these results reflect real-life performance on an immediately subsequent task. In this series of experiments, we investigated both medium-term and short-term predictive value of psychophysiological testing with regard to real-life performance in two operational settings: military student pilots with regard to their success on an evaluation flight, and special forces candidates with regard to their performance on their training course. Our results showed some relationships between test performance and medium-term outcomes. However, no short-term predictive value could be identified for cognitive testing, despite the fact physiological data showed interesting trends. We recommend a critical distinction between "state" and "trait" dimensions of performance with regard to the predictive value of testing.

LTDNA Evidence on Trial

PubMed Central

Roberts, Paul

2016-01-01

Adopting the interpretative/hermeneutical method typical of much legal scholarship, this article considers two sets of issues pertaining to LTDNA profiles as evidence in criminal proceedings. The section titled Expert Evidence as Forensic Epistemic Warrant addresses some rather large questions about the epistemic status and probative value of expert testimony in general. It sketches a theoretical model of expert evidence, highlighting five essential criteria: (1) expert competence; (2) disciplinary domain; (3) methodological validity; (4) materiality; and (5) legal admissibility. This generic model of expert authority, highlighting law's fundamentally normative character, applies to all modern forms of criminal adjudication, across Europe and farther afield. The section titled LTDNA Evidence in UK Criminal Trials then examines English and Northern Irish courts' attempts to get to grips with LTDNA evidence in recent cases. Better appreciating the ways in which UK courts have addressed the challenges of LTDNA evidence may offer some insights into parallel developments in other legal systems. Appellate court rulings follow a predictable judicial logic, which might usefully be studied and reflected upon by any forensic scientist or statistician seeking to operate effectively in criminal proceedings. Whilst each legal jurisdiction has its own unique blend of jurisprudence, institutions, cultures and historical traditions, there is considerable scope for comparative analysis and cross-jurisdictional borrowing and instruction. In the spirit of promoting more nuanced and sophisticated international interdisciplinary dialogue, this article examines UK judicial approaches to LTDNA evidence and begins to elucidate their underlying institutional logic. Legal argument and broader policy debates are not confined to considerations of scientific validity, contamination risks and evidential integrity, or associated judgments of legal admissibility or exclusion. They also crucially concern the manner in which LTDNA profiling results are presented and explained to factfinders in criminal trials. PMID:27826316

LTDNA Evidence on Trial.

PubMed

Roberts, Paul

2016-01-01

Adopting the interpretative/hermeneutical method typical of much legal scholarship, this article considers two sets of issues pertaining to LTDNA profiles as evidence in criminal proceedings. The section titled Expert Evidence as Forensic Epistemic Warrant addresses some rather large questions about the epistemic status and probative value of expert testimony in general. It sketches a theoretical model of expert evidence, highlighting five essential criteria: (1) expert competence; (2) disciplinary domain; (3) methodological validity; (4) materiality; and (5) legal admissibility. This generic model of expert authority, highlighting law's fundamentally normative character, applies to all modern forms of criminal adjudication, across Europe and farther afield. The section titled LTDNA Evidence in UK Criminal Trials then examines English and Northern Irish courts' attempts to get to grips with LTDNA evidence in recent cases. Better appreciating the ways in which UK courts have addressed the challenges of LTDNA evidence may offer some insights into parallel developments in other legal systems. Appellate court rulings follow a predictable judicial logic, which might usefully be studied and reflected upon by any forensic scientist or statistician seeking to operate effectively in criminal proceedings. Whilst each legal jurisdiction has its own unique blend of jurisprudence, institutions, cultures and historical traditions, there is considerable scope for comparative analysis and cross-jurisdictional borrowing and instruction. In the spirit of promoting more nuanced and sophisticated international interdisciplinary dialogue, this article examines UK judicial approaches to LTDNA evidence and begins to elucidate their underlying institutional logic. Legal argument and broader policy debates are not confined to considerations of scientific validity, contamination risks and evidential integrity, or associated judgments of legal admissibility or exclusion. They also crucially concern the manner in which LTDNA profiling results are presented and explained to factfinders in criminal trials.

Design process and preliminary psychometric study of a video game to detect cognitive impairment in senior adults.

PubMed

Valladares-Rodriguez, Sonia; Perez-Rodriguez, Roberto; Facal, David; Fernandez-Iglesias, Manuel J; Anido-Rifon, Luis; Mouriño-Garcia, Marcos

2017-01-01

Assessment of episodic memory has been traditionally used to evaluate potential cognitive impairments in senior adults. Typically, episodic memory evaluation is based on personal interviews and pen-and-paper tests. This article presents the design, development and a preliminary validation of a novel digital game to assess episodic memory intended to overcome the limitations of traditional methods, such as the cost of its administration, its intrusive character, the lack of early detection capabilities, the lack of ecological validity, the learning effect and the existence of confounding factors. Our proposal is based on the gamification of the California Verbal Learning Test (CVLT) and it has been designed to comply with the psychometric characteristics of reliability and validity. Two qualitative focus groups and a first pilot experiment were carried out to validate the proposal. A more ecological, non-intrusive and better administrable tool to perform cognitive assessment was developed. Initial evidence from the focus groups and pilot experiment confirmed the developed game's usability and offered promising results insofar its psychometric validity is concerned. Moreover, the potential of this game for the cognitive classification of senior adults was confirmed, and administration time is dramatically reduced with respect to pen-and-paper tests. Additional research is needed to improve the resolution of the game for the identification of specific cognitive impairments, as well as to achieve a complete validation of the psychometric properties of the digital game. Initial evidence show that serious games can be used as an instrument to assess the cognitive status of senior adults, and even to predict the onset of mild cognitive impairments or Alzheimer's disease.

Design process and preliminary psychometric study of a video game to detect cognitive impairment in senior adults

PubMed Central

Perez-Rodriguez, Roberto; Facal, David; Fernandez-Iglesias, Manuel J.; Anido-Rifon, Luis; Mouriño-Garcia, Marcos

2017-01-01

Introduction Assessment of episodic memory has been traditionally used to evaluate potential cognitive impairments in senior adults. Typically, episodic memory evaluation is based on personal interviews and pen-and-paper tests. This article presents the design, development and a preliminary validation of a novel digital game to assess episodic memory intended to overcome the limitations of traditional methods, such as the cost of its administration, its intrusive character, the lack of early detection capabilities, the lack of ecological validity, the learning effect and the existence of confounding factors. Materials and Methods Our proposal is based on the gamification of the California Verbal Learning Test (CVLT) and it has been designed to comply with the psychometric characteristics of reliability and validity. Two qualitative focus groups and a first pilot experiment were carried out to validate the proposal. Results A more ecological, non-intrusive and better administrable tool to perform cognitive assessment was developed. Initial evidence from the focus groups and pilot experiment confirmed the developed game’s usability and offered promising results insofar its psychometric validity is concerned. Moreover, the potential of this game for the cognitive classification of senior adults was confirmed, and administration time is dramatically reduced with respect to pen-and-paper tests. Limitations Additional research is needed to improve the resolution of the game for the identification of specific cognitive impairments, as well as to achieve a complete validation of the psychometric properties of the digital game. Conclusion Initial evidence show that serious games can be used as an instrument to assess the cognitive status of senior adults, and even to predict the onset of mild cognitive impairments or Alzheimer’s disease. PMID:28674661

Integrating Validity Theory with Use of Measurement Instruments in Clinical Settings

PubMed Central

Kelly, P Adam; O'Malley, Kimberly J; Kallen, Michael A; Ford, Marvella E

2005-01-01

Objective To present validity concepts in a conceptual framework useful for research in clinical settings. Principal Findings We present a three-level decision rubric for validating measurement instruments, to guide health services researchers step-by-step in gathering and evaluating validity evidence within their specific situation. We address construct precision, the capacity of an instrument to measure constructs it purports to measure and differentiate from other, unrelated constructs; quantification precision, the reliability of the instrument; and translation precision, the ability to generalize scores from an instrument across subjects from the same or similar populations. We illustrate with specific examples, such as an approach to validating a measurement instrument for veterans when prior evidence of instrument validity for this population does not exist. Conclusions Validity should be viewed as a property of the interpretations and uses of scores from an instrument, not of the instrument itself: how scores are used and the consequences of this use are integral to validity. Our advice is to liken validation to building a court case, including discovering evidence, weighing the evidence, and recognizing when the evidence is weak and more evidence is needed. PMID:16178998

A user-friendly risk-score for predicting in-hospital cardiac arrest among patients admitted with suspected non ST-elevation acute coronary syndrome - The SAFER-score.

PubMed

Faxén, Jonas; Hall, Marlous; Gale, Chris P; Sundström, Johan; Lindahl, Bertil; Jernberg, Tomas; Szummer, Karolina

2017-12-01

To develop a simple risk-score model for predicting in-hospital cardiac arrest (CA) among patients hospitalized with suspected non-ST elevation acute coronary syndrome (NSTE-ACS). Using the Swedish Web-system for Enhancement and Development of Evidence-based care in Heart disease Evaluated According to Recommended Therapies (SWEDEHEART), we identified patients (n=242 303) admitted with suspected NSTE-ACS between 2008 and 2014. Logistic regression was used to assess the association between 26 candidate variables and in-hospital CA. A risk-score model was developed and validated using a temporal cohort (n=126 073) comprising patients from SWEDEHEART between 2005 and 2007 and an external cohort (n=276 109) comprising patients from the Myocardial Ischaemia National Audit Project (MINAP) between 2008 and 2013. The incidence of in-hospital CA for NSTE-ACS and non-ACS was lower in the SWEDEHEART-derivation cohort than in MINAP (1.3% and 0.5% vs. 2.3% and 2.3%). A seven point, five variable risk score (age ≥60 years (1 point), ST-T abnormalities (2 points), Killip Class >1 (1 point), heart rate <50 or ≥100bpm (1 point), and systolic blood pressure <100mmHg (2 points) was developed. Model discrimination was good in the derivation cohort (c-statistic 0.72) and temporal validation cohort (c-statistic 0.74), and calibration was reasonable with a tendency towards overestimation of risk with a higher sum of score points. External validation showed moderate discrimination (c-statistic 0.65) and calibration showed a general underestimation of predicted risk. A simple points score containing five variables readily available on admission predicts in-hospital CA for patients with suspected NSTE-ACS. Copyright © 2017 Elsevier B.V. All rights reserved.

Predicting neutropenia risk in patients with cancer using electronic data.

PubMed

Pawloski, Pamala A; Thomas, Avis J; Kane, Sheryl; Vazquez-Benitez, Gabriela; Shapiro, Gary R; Lyman, Gary H

2017-04-01

Clinical guidelines recommending the use of myeloid growth factors are largely based on the prescribed chemotherapy regimen. The guidelines suggest that oncologists consider patient-specific characteristics when prescribing granulocyte-colony stimulating factor (G-CSF) prophylaxis; however, a mechanism to quantify individual patient risk is lacking. Readily available electronic health record (EHR) data can provide patient-specific information needed for individualized neutropenia risk estimation. An evidence-based, individualized neutropenia risk estimation algorithm has been developed. This study evaluated the automated extraction of EHR chemotherapy treatment data and externally validated the neutropenia risk prediction model. A retrospective cohort of adult patients with newly diagnosed breast, colorectal, lung, lymphoid, or ovarian cancer who received the first cycle of a cytotoxic chemotherapy regimen from 2008 to 2013 were recruited from a single cancer clinic. Electronically extracted EHR chemotherapy treatment data were validated by chart review. Neutropenia risk stratification was conducted and risk model performance was assessed using calibration and discrimination. Chemotherapy treatment data electronically extracted from the EHR were verified by chart review. The neutropenia risk prediction tool classified 126 patients (57%) as being low risk for febrile neutropenia, 44 (20%) as intermediate risk, and 51 (23%) as high risk. The model was well calibrated (Hosmer-Lemeshow goodness-of-fit test = 0.24). Discrimination was adequate and slightly less than in the original internal validation (c-statistic 0.75 vs 0.81). Chemotherapy treatment data were electronically extracted from the EHR successfully. The individualized neutropenia risk prediction model performed well in our retrospective external cohort. © The Author 2016. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For Permissions, please email: journals.permissions@oup.com

Measurement properties of the upright motor control test for adults with stroke: a systematic review.

PubMed

Gorgon, Edward James R; Lazaro, Rolando T

2016-01-01

The Upright Motor Control Test (UMCT) has been used in clinical practice and research to assess functional strength of the hemiparetic lower limb in adults with stroke. It is unclear if evidence is sufficient to warrant its use. The purpose of this systematic review was to synthesize available evidence on the measurement properties of the UMCT for stroke rehabilitation. Electronic databases that indexed biomedical literature were systematically searched from inception until October 2015 (week 4): Embase, PubMed, Web of Science, CINAHL, PEDro, Cochrane Library, Scopus, ScienceDirect, SPORTDiscus, LILACS, DOAJ, and Google Scholar. All studies that had used the UMCT in the time period covered underwent hand searching for any additional study. Observational studies involving adults with stroke that explored any measurement property of the UMCT were included. The COnsensus-based Standards for the selection of health Measurement INstruments was used to assess the methodological quality of included studies. The CanChild Outcome Measures Rating Form was used for extracting data on measurement properties and clinical utility. The search yielded three methodologic studies that addressed criterion-related validity and contruct validity. Two studies of fair methodological quality demonstrated moderate-level evidence that Knee Extension and Knee Flexion subtest scores were predictive of community-level and household-level ambulation. One study of fair methodological quality provided limited-level evidence for the correlation of Knee Extension subtest scores with a laboratory measure of ground reaction forces. No published studies formally assessed reliability, responsiveness, or clinical utility. Limited information on responsiveness and clinical utility dimensions could be inferred from the included studies. The UMCT is a practical assessment tool for voluntary control or functional strength of the hemiparetic lower limb in standing in adults with stroke. Although different levels of evidence suggest that the Knee Extension and Knee Flexion subtests may possess criterion and construct validity, the lack of published literature examining content validity, reliability, and responsiveness raises questions regarding the use of the UMCT in routine clinical practice. These key findings highlight the need to further investigate the UMCT's measurement properties toward enhancing its standardization.

Predictive validity of the Hendrich fall risk model II in an acute geriatric unit.

PubMed

Ivziku, Dhurata; Matarese, Maria; Pedone, Claudio

2011-04-01

Falls are the most common adverse events reported in acute care hospitals, and older patients are the most likely to fall. The risk of falling cannot be completely eliminated, but it can be reduced through the implementation of a fall prevention program. A major evidence-based intervention to prevent falls has been the use of fall-risk assessment tools. Many tools have been increasingly developed in recent years, but most instruments have not been investigated regarding reliability, validity and clinical usefulness. This study intends to evaluate the predictive validity and inter-rater reliability of Hendrich fall risk model II (HFRM II) in order to identify older patients at risk of falling in geriatric units and recommend its use in clinical practice. A prospective descriptive design was used. The study was carried out in a geriatric acute care unit of an Italian University hospital. All over 65 years old patients consecutively admitted to a geriatric acute care unit of an Italian University hospital over 8-month period were enrolled. The patients enrolled were screened for the falls risk by nurses with the HFRM II within 24h of admission. The falls occurring during the patient's hospital stay were registered. Inter-rater reliability, area under the ROC curve, sensitivity, specificity, positive and negative predictive values and time for the administration were evaluated. 179 elderly patients were included. The inter-rater reliability was 0.87 (95% CI 0.71-1.00). The administration time was about 1min. The most frequently reported risk factors were depression, incontinence, vertigo. Sensitivity and specificity were respectively 86% and 43%. The optimal cut-off score for screening at risk patients was 5 with an area under the ROC curve of 0.72. The risk factors more strongly associated with falls were confusion and depression. As falls of older patients are a common problem in acute care settings it is necessary that the nurses use specific validate and reliable fall risk assessment tools in order to implement the most effective prevention measures. Our findings provided supporting evidence to the choice of the HFRM II to screen older patients at risk of falling in acute care settings. Copyright © 2010 Elsevier Ltd. All rights reserved.

Validity of the MCAT in Predicting Performance in the First Two Years of Medical School.

ERIC Educational Resources Information Center

Jones, Robert F.; Thomae-Forgues, Maria

1984-01-01

The first systematic summary of predictive validity research on the new Medical College Admission Test (MCAT) is presented. The results show that MCAT scores have significant predictive validity with respect to first- and second-year medical school course grades. Further directions for MCAT validity research are described. (Author/MLW)

Evidence for validity of five secondary data sources for enumerating retail food outlets in seven American Indian communities in North Carolina.

PubMed

Fleischhacker, Sheila E; Rodriguez, Daniel A; Evenson, Kelly R; Henley, Amanda; Gizlice, Ziya; Soto, Dolly; Ramachandran, Gowri

2012-11-22

Most studies on the local food environment have used secondary sources to describe the food environment, such as government food registries or commercial listings (e.g., Reference USA). Most of the studies exploring evidence for validity of secondary retail food data have used on-site verification and have not conducted analysis by data source (e.g., sensitivity of Reference USA) or by food outlet type (e.g., sensitivity of Reference USA for convenience stores). Few studies have explored the food environment in American Indian communities. To advance the science on measuring the food environment, we conducted direct, on-site observations of a wide range of food outlets in multiple American Indian communities, without a list guiding the field observations, and then compared our findings to several types of secondary data. Food outlets located within seven State Designated Tribal Statistical Areas in North Carolina (NC) were gathered from online Yellow Pages, Reference USA, Dun & Bradstreet, local health departments, and the NC Department of Agriculture and Consumer Services. All TIGER/Line 2009 roads (>1,500 miles) were driven in six of the more rural tribal areas and, for the largest tribe, all roads in two of its cities were driven. Sensitivity, positive predictive value, concordance, and kappa statistics were calculated to compare secondary data sources to primary data. 699 food outlets were identified during primary data collection. Match rate for primary data and secondary data differed by type of food outlet observed, with the highest match rates found for grocery stores (97%), general merchandise stores (96%), and restaurants (91%). Reference USA exhibited almost perfect sensitivity (0.89). Local health department data had substantial sensitivity (0.66) and was almost perfect when focusing only on restaurants (0.91). Positive predictive value was substantial for Reference USA (0.67) and moderate for local health department data (0.49). Evidence for validity was comparatively lower for Dun & Bradstreet, online Yellow Pages, and the NC Department of Agriculture. Secondary data sources both over- and under-represented the food environment; they were particularly problematic for identifying convenience stores and specialty markets. More attention is needed to improve the validity of existing data sources, especially for rural local food environments.

Evidence for validity of five secondary data sources for enumerating retail food outlets in seven American Indian Communities in North Carolina

PubMed Central

2012-01-01

Background Most studies on the local food environment have used secondary sources to describe the food environment, such as government food registries or commercial listings (e.g., Reference USA). Most of the studies exploring evidence for validity of secondary retail food data have used on-site verification and have not conducted analysis by data source (e.g., sensitivity of Reference USA) or by food outlet type (e.g., sensitivity of Reference USA for convenience stores). Few studies have explored the food environment in American Indian communities. To advance the science on measuring the food environment, we conducted direct, on-site observations of a wide range of food outlets in multiple American Indian communities, without a list guiding the field observations, and then compared our findings to several types of secondary data. Methods Food outlets located within seven State Designated Tribal Statistical Areas in North Carolina (NC) were gathered from online Yellow Pages, Reference USA, Dun & Bradstreet, local health departments, and the NC Department of Agriculture and Consumer Services. All TIGER/Line 2009 roads (>1,500 miles) were driven in six of the more rural tribal areas and, for the largest tribe, all roads in two of its cities were driven. Sensitivity, positive predictive value, concordance, and kappa statistics were calculated to compare secondary data sources to primary data. Results 699 food outlets were identified during primary data collection. Match rate for primary data and secondary data differed by type of food outlet observed, with the highest match rates found for grocery stores (97%), general merchandise stores (96%), and restaurants (91%). Reference USA exhibited almost perfect sensitivity (0.89). Local health department data had substantial sensitivity (0.66) and was almost perfect when focusing only on restaurants (0.91). Positive predictive value was substantial for Reference USA (0.67) and moderate for local health department data (0.49). Evidence for validity was comparatively lower for Dun & Bradstreet, online Yellow Pages, and the NC Department of Agriculture. Conclusions Secondary data sources both over- and under-represented the food environment; they were particularly problematic for identifying convenience stores and specialty markets. More attention is needed to improve the validity of existing data sources, especially for rural local food environments. PMID:23173781

Development and validation of a new population-based simulation model of osteoarthritis in New Zealand.

PubMed

Wilson, R; Abbott, J H

2018-04-01

To describe the construction and preliminary validation of a new population-based microsimulation model developed to analyse the health and economic burden and cost-effectiveness of treatments for knee osteoarthritis (OA) in New Zealand (NZ). We developed the New Zealand Management of Osteoarthritis (NZ-MOA) model, a discrete-time state-transition microsimulation model of the natural history of radiographic knee OA. In this article, we report on the model structure, derivation of input data, validation of baseline model parameters against external data sources, and validation of model outputs by comparison of the predicted population health loss with previous estimates. The NZ-MOA model simulates both the structural progression of radiographic knee OA and the stochastic development of multiple disease symptoms. Input parameters were sourced from NZ population-based data where possible, and from international sources where NZ-specific data were not available. The predicted distributions of structural OA severity and health utility detriments associated with OA were externally validated against other sources of evidence, and uncertainty resulting from key input parameters was quantified. The resulting lifetime and current population health-loss burden was consistent with estimates of previous studies. The new NZ-MOA model provides reliable estimates of the health loss associated with knee OA in the NZ population. The model structure is suitable for analysis of the effects of a range of potential treatments, and will be used in future work to evaluate the cost-effectiveness of recommended interventions within the NZ healthcare system. Copyright © 2018 Osteoarthritis Research Society International. Published by Elsevier Ltd. All rights reserved.

Biomarkers and surrogate endpoints in glaucoma clinical trials.

PubMed

Medeiros, Felipe A

2015-05-01

Surrogate endpoints are often used as replacements for true clinically relevant endpoints in several areas of medicine, as they enable faster and less expensive clinical trials. However, without proper validation, the use of surrogates may lead to incorrect conclusions about the efficacy and safety of treatments. This article reviews the general requirements for validating surrogate endpoints and provides a critical assessment of the use of intraocular pressure (IOP), visual fields, and structural measurements of the optic nerve as surrogate endpoints in glaucoma clinical trials. A valid surrogate endpoint must be able to predict the clinically relevant endpoint and fully capture the effect of an intervention on that endpoint. Despite its widespread use in clinical trials, no proper validation of IOP as a surrogate endpoint has ever been conducted for any class of IOP-lowering treatments. Evidence has accumulated with regard to the role of imaging measurements of optic nerve damage as surrogate endpoints in glaucoma. These measurements are predictive of functional losses in the disease and may explain, at least in part, treatment effects on clinically relevant endpoints. The use of composite endpoints in glaucoma trials may overcome weaknesses of the use of structural or functional endpoints in isolation. Unless research is dedicated to fully develop and validate suitable endpoints that can be used in glaucoma clinical trials, we run the risk of inappropriate judgments about the value of new therapies. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

A comparative investigation of seven indirect attitude measures.

PubMed

Bar-Anan, Yoav; Nosek, Brian A

2014-09-01

We compared the psychometric qualities of seven indirect attitude measures across three attitude domains (race, politics, and self-esteem) with a large sample (N = 23,413). We compared the measures on internal consistency, sensitivity to known effects, relationships with indirect and direct measures of the same topic, the reliability and validity of single-category attitude measurement, their ability to detect meaningful variance among people with nonextreme attitudes, and their robustness to the exclusion of misbehaving or well-behaving participants. All seven indirect measures correlated with each other and with direct measures of the same topic. These relations were always weak for self-esteem, moderate for race, and strong for politics. This pattern suggests that some of the sources of variation in the reliability and predictive validity of the indirect measures is a function of the concepts rather than the methods. The Implicit Association Test (IAT) and Brief IAT (BIAT) showed the best overall psychometric quality, followed by the Go–No-Go association task, Single-Target IAT (ST-IAT), Affective Misattribution Procedure (AMP), Sorting Paired Features task, and Evaluative Priming. The AMP showed a steep decline in its psychometric qualities when people with extreme attitude scores were removed. Single-category attitude scores computed for the IAT and BIAT showed good relationships with other attitude measures but no evidence of discriminant validity between paired categories. The other measures, especially the AMP and ST-IAT, showed better evidence for discriminant validity. These results inform us on the validity of the measures as attitude assessments, but do not speak to the implicitness of the measured constructs.

The Stroke Assessment of Fall Risk (SAFR): predictive validity in inpatient stroke rehabilitation.

PubMed

Breisinger, Terry P; Skidmore, Elizabeth R; Niyonkuru, Christian; Terhorst, Lauren; Campbell, Grace B

2014-12-01

To evaluate relative accuracy of a newly developed Stroke Assessment of Fall Risk (SAFR) for classifying fallers and non-fallers, compared with a health system fall risk screening tool, the Fall Harm Risk Screen. Prospective quality improvement study conducted at an inpatient stroke rehabilitation unit at a large urban university hospital. Patients admitted for inpatient stroke rehabilitation (N = 419) with imaging or clinical evidence of ischemic or hemorrhagic stroke, between 1 August 2009 and 31 July 2010. Not applicable. Sensitivity, specificity, and area under the curve for Receiver Operating Characteristic Curves of both scales' classifications, based on fall risk score completed upon admission to inpatient stroke rehabilitation. A total of 68 (16%) participants fell at least once. The SAFR was significantly more accurate than the Fall Harm Risk Screen (p < 0.001), with area under the curve of 0.73, positive predictive value of 0.29, and negative predictive value of 0.94. For the Fall Harm Risk Screen, area under the curve was 0.56, positive predictive value was 0.19, and negative predictive value was 0.86. Sensitivity and specificity of the SAFR (0.78 and 0.63, respectively) was higher than the Fall Harm Risk Screen (0.57 and 0.48, respectively). An evidence-derived, population-specific fall risk assessment may more accurately predict fallers than a general fall risk screen for stroke rehabilitation patients. While the SAFR improves upon the accuracy of a general assessment tool, additional refinement may be warranted. © The Author(s) 2014.

Prefrontal EEG alpha asymmetry changes while observing disaster happening to other people: cardiac correlates and prediction of emotional impact.

PubMed

Papousek, Ilona; Weiss, Elisabeth M; Schulter, Günter; Fink, Andreas; Reiser, Eva M; Lackner, Helmut K

2014-12-01

Changes of EEG alpha asymmetry in terms of increased right versus left sided activity in prefrontal cortex are considered to index activation of the withdrawal/avoidance motivational system. The present study aimed to add evidence of the validity of individual differences in the EEG alpha asymmetry response and their relevance regarding the impact of emotional events. The magnitude of the EEG alpha asymmetry response while watching a film consisting of scenes of real injury and death correlated with components of transient cardiac responses to sudden horrifying events happening to persons in the film which index withdrawal/avoidance motivation and heightened attention and perceptual intake. Additionally, it predicted greater mood deterioration following the film and film-related intrusive memories and avoidance over the following week. The study provides further evidence for prefrontal EEG alpha asymmetry changes in response to relevant stimuli reflecting an individual's sensitivity to negative social-emotional cues encountered in everyday life. Copyright © 2014 Elsevier B.V. All rights reserved.

Transcriptomic and macroevolutionary evidence for phenotypic uncoupling between frog life history phases

PubMed Central

Wollenberg Valero, Katharina C.; Garcia-Porta, Joan; Rodríguez, Ariel; Arias, Mónica; Shah, Abhijeet; Randrianiaina, Roger Daniel; Brown, Jason L.; Glaw, Frank; Amat, Felix; Künzel, Sven; Metzler, Dirk; Isokpehi, Raphael D.; Vences, Miguel

2017-01-01

Anuran amphibians undergo major morphological transitions during development, but the contribution of their markedly different life-history phases to macroevolution has rarely been analysed. Here we generate testable predictions for coupling versus uncoupling of phenotypic evolution of tadpole and adult life-history phases, and for the underlying expression of genes related to morphological feature formation. We test these predictions by combining evidence from gene expression in two distantly related frogs, Xenopus laevis and Mantidactylus betsileanus, with patterns of morphological evolution in the entire radiation of Madagascan mantellid frogs. Genes linked to morphological structure formation are expressed in a highly phase-specific pattern, suggesting uncoupling of phenotypic evolution across life-history phases. This gene expression pattern agrees with uncoupled rates of trait evolution among life-history phases in the mantellids, which we show to have undergone an adaptive radiation. Our results validate a prevalence of uncoupling in the evolution of tadpole and adult phenotypes of frogs. PMID:28504275

How to predict clinical relapse in inflammatory bowel disease patients

PubMed Central

Liverani, Elisa; Scaioli, Eleonora; Digby, Richard John; Bellanova, Matteo; Belluzzi, Andrea

2016-01-01

Inflammatory bowel diseases have a natural course characterized by alternating periods of remission and relapse. Disease flares occur in a random way and are currently unpredictable for the most part. Predictors of benign or unfavourable clinical course are required to facilitate treatment decisions and to avoid overtreatment. The present article provides a literature review of the current evidence on the main clinical, genetic, endoscopic, histologic, serologic and fecal markers to predict aggressiveness of inflammatory bowel disease and discuss their prognostic role, both in Crohn’s disease and ulcerative colitis. No single marker seems to be reliable alone as a flare predictor, even in light of promising evidence regarding the role of fecal markers, in particular fecal calprotectin, which has reported good results recently. In order to improve our daily clinical practice, validated prognostic scores should be elaborated, integrating clinical and biological markers of prognosis. Finally, we propose an algorithm considering clinical history and biological markers to intercept patients with high risk of clinical relapse. PMID:26811644

Evidence-based medicine and big genomic data.

PubMed

Ioannidis, John P A; Khoury, Muin J

2018-05-01

Genomic and other related big data (Big Genomic Data, BGD for short) are ushering a new era of precision medicine. This overview discusses whether principles of evidence-based medicine hold true for BGD and how they should be operationalized in the current era. Major evidence-based medicine principles include the systematic identification, description and analysis of the validity and utility of BGD, the combination of individual clinical expertise with individual patient needs and preferences, and the focus on obtaining experimental evidence, whenever possible. BGD emphasize information of single patients with an overemphasis on N-of-1 trials to personalize treatment. However, large-scale comparative population data remain indispensable for meaningful translation of BGD personalized information. The impact of BGD on population health depends on its ability to affect large segments of the population. While several frameworks have been proposed to facilitate and standardize decision making for use of genomic tests, there are new caveats that arise from BGD that extend beyond the limitations that were applicable for more simple genetic tests. Non-evidence-based use of BGD may be harmful and result in major waste of healthcare resources. Randomized controlled trials will continue to be the strongest arbitrator for the clinical utility of genomic technologies, including BGD. Research on BGD needs to focus not only on finding robust predictive associations (clinical validity) but also more importantly on evaluating the balance of health benefits and potential harms (clinical utility), as well as implementation challenges. Appropriate features of such useful research on BGD are discussed.

Adaptation and validation of the Diabetes Management Self-Efficacy Scale to Brazilian Portuguese 1

PubMed Central

Pace, Ana Emilia; Gomes, Lilian Cristiane; Bertolin, Daniela Comelis; Loureiro, Helena Maria Almeira Macedo; Bijl, Jaap Van Der; Shortridge-Baggett, Lillie M.

2017-01-01

Objective: to perform the cultural adaptation and validation of the Diabetes Management Self-efficacy Scale for Patients with Type 2 Diabetes Mellitus with a Brazilian population sample. Method: cross-sectional methodological study in which the adaptation and validation process included the stages recommended in the literature. Construct validity and reliability were assessed with 200 adults with type 2 diabetes mellitus. Results: the items indicated by the panel of judges and by the target population were adjusted in the cultural adaptation to improve clarity and understanding. The instrument's four factors remained in the confirmatory factor analysis with factor loadings of items greater than 0.30, except for factor 4; convergent validity, verified by the multitrait-multimethod analysis, presented inter-item correlations from 0.37 to 0.92, while for discriminant validity, 100% of the items presented greater correlation in their own factors. Cronbach's coefficient alpha for the total scale was 0.78, ranging from 0.57 to 0.86 among factors. Conclusion: semantic, cultural, conceptual and idiomatic equivalences were achieved and the instrument's Brazilian version also presented psychometric properties that showed evidence of reliability and validity. Thus, it can be applied both in clinical practice and research. Self-efficacy is useful for planning and assessing educational interventions, as well as predicting behavior modification in self-care. PMID:28562700

Quality appraisal of generic self-reported instruments measuring health-related productivity changes: a systematic review

PubMed Central

2014-01-01

Background Health impairments can result in disability and changed work productivity imposing considerable costs for the employee, employer and society as a whole. A large number of instruments exist to measure health-related productivity changes; however their methodological quality remains unclear. This systematic review critically appraised the measurement properties in generic self-reported instruments that measure health-related productivity changes to recommend appropriate instruments for use in occupational and economic health practice. Methods PubMed, PsycINFO, Econlit and Embase were systematically searched for studies whereof: (i) instruments measured health-related productivity changes; (ii) the aim was to evaluate instrument measurement properties; (iii) instruments were generic; (iv) ratings were self-reported; (v) full-texts were available. Next, methodological quality appraisal was based on COSMIN elements: (i) internal consistency; (ii) reliability; (iii) measurement error; (iv) content validity; (v) structural validity; (vi) hypotheses testing; (vii) cross-cultural validity; (viii) criterion validity; and (ix) responsiveness. Recommendations are based on evidence syntheses. Results This review included 25 articles assessing the reliability, validity and responsiveness of 15 different generic self-reported instruments measuring health-related productivity changes. Most studies evaluated criterion validity, none evaluated cross-cultural validity and information on measurement error is lacking. The Work Limitation Questionnaire (WLQ) was most frequently evaluated with moderate respectively strong positive evidence for content and structural validity and negative evidence for reliability, hypothesis testing and responsiveness. Less frequently evaluated, the Stanford Presenteeism Scale (SPS) showed strong positive evidence for internal consistency and structural validity, and moderate positive evidence for hypotheses testing and criterion validity. The Productivity and Disease Questionnaire (PRODISQ) yielded strong positive evidence for content validity, evidence for other properties is lacking. The other instruments resulted in mostly fair-to-poor quality ratings with limited evidence. Conclusions Decisions based on the content of the instrument, usage purpose, target country and population, and available evidence are recommended. Until high-quality studies are in place to accurately assess the measurement properties of the currently available instruments, the WLQ and, in a Dutch context, the PRODISQ are cautiously preferred based on its strong positive evidence for content validity. Based on its strong positive evidence for internal consistency and structural validity, the SPS is cautiously recommended. PMID:24495301

Antisocial personality disorder and psychopathy in women: a literature review on the reliability and validity of assessment instruments.

PubMed

Dolan, Mairead; Völlm, Birgit

2009-01-01

Crime rates are low in women compared to men. The two disorders most commonly associated with offending behaviour, antisocial personality disorder (ASPD) and psychopathy, are also less prevalent in female samples. However, developments in forensic psychiatry have often ignored gender, and the utility of constructs such as psychopathy and their assessment instruments in female samples remains unclear. This article presents a review of studies looking at rates of ASPD and psychopathy and on the reliability and validity of assessment instruments of these disorders in women. Gender differences in symptom patterns will be considered. The literature seems to suggest that DSM-IV criteria for ASPD may lead to an underestimation of the prevalence of the disorder in women due to the requirement of childhood conduct disorder symptoms. The Psychopathy Checklist-Revised (PCL-R) is a valid and reliable instrument to identify psychopathy in women but there are gender differences in the factor structure and item loadings on this measure. Research to date seems to suggest a three-factor model may be most strongly supported in females. Preliminary evidence suggests the PCL-R may have some value in predicting future offending while the PCL:SV may be useful in predicting institutional violence. Clinical implications are discussed.

Validation of the sperm class analyser CASA system for sperm counting in a busy diagnostic semen analysis laboratory.

PubMed

Dearing, Chey G; Kilburn, Sally; Lindsay, Kevin S

2014-03-01

Sperm counts have been linked to several fertility outcomes making them an essential parameter of semen analysis. It has become increasingly recognised that Computer-Assisted Semen Analysis (CASA) provides improved precision over manual methods but that systems are seldom validated robustly for use. The objective of this study was to gather the evidence to validate or reject the Sperm Class Analyser (SCA) as a tool for routine sperm counting in a busy laboratory setting. The criteria examined were comparison with the Improved Neubauer and Leja 20-μm chambers, within and between field precision, sperm concentration linearity from a stock diluted in semen and media, accuracy against internal and external quality material, assessment of uneven flow effects and a receiver operating characteristic (ROC) analysis to predict fertility in comparison with the Neubauer method. This work demonstrates that SCA CASA technology is not a standalone 'black box', but rather a tool for well-trained staff that allows rapid, high-number sperm counting providing errors are identified and corrected. The system will produce accurate, linear, precise results, with less analytical variance than manual methods that correlate well against the Improved Neubauer chamber. The system provides superior predictive potential for diagnosing fertility problems.

Application of a New Resource-Constrained Triage Method to Military-Age Victims

DTIC Science & Technology

2009-12-01

evidence based, does not consider resources, and has been shown to be scientifically and opera- tionally flawed.’ General P . K. Carlton, former USAF Surgeon...metric that can be used to predict sur- vival probability) P ^ (t) = the survival probability of victims with original SCORE s treated in time period t. n...function coefficients were derived on the design set, and validated on the test set. The logistic function has the form: P ^ - 1/(1 + e"), where P ^ is

DOE Office of Scientific and Technical Information (OSTI.GOV)

Dinh, Nam; Athe, Paridhi; Jones, Christopher

The Virtual Environment for Reactor Applications (VERA) code suite is assessed in terms of capability and credibility against the Consortium for Advanced Simulation of Light Water Reactors (CASL) Verification and Validation Plan (presented herein) in the context of three selected challenge problems: CRUD-Induced Power Shift (CIPS), Departure from Nucleate Boiling (DNB), and Pellet-Clad Interaction (PCI). Capability refers to evidence of required functionality for capturing phenomena of interest while capability refers to the evidence that provides confidence in the calculated results. For this assessment, each challenge problem defines a set of phenomenological requirements against which the VERA software is assessed. Thismore » approach, in turn, enables the focused assessment of only those capabilities relevant to the challenge problem. The evaluation of VERA against the challenge problem requirements represents a capability assessment. The mechanism for assessment is the Sandia-developed Predictive Capability Maturity Model (PCMM) that, for this assessment, evaluates VERA on 8 major criteria: (1) Representation and Geometric Fidelity, (2) Physics and Material Model Fidelity, (3) Software Quality Assurance and Engineering, (4) Code Verification, (5) Solution Verification, (6) Separate Effects Model Validation, (7) Integral Effects Model Validation, and (8) Uncertainty Quantification. For each attribute, a maturity score from zero to three is assigned in the context of each challenge problem. The evaluation of these eight elements constitutes the credibility assessment for VERA.« less

How to test validity in orthodontic research: a mixed dentition analysis example.

PubMed

Donatelli, Richard E; Lee, Shin-Jae

2015-02-01

The data used to test the validity of a prediction method should be different from the data used to generate the prediction model. In this study, we explored whether an independent data set is mandatory for testing the validity of a new prediction method and how validity can be tested without independent new data. Several validation methods were compared in an example using the data from a mixed dentition analysis with a regression model. The validation errors of real mixed dentition analysis data and simulation data were analyzed for increasingly large data sets. The validation results of both the real and the simulation studies demonstrated that the leave-1-out cross-validation method had the smallest errors. The largest errors occurred in the traditional simple validation method. The differences between the validation methods diminished as the sample size increased. The leave-1-out cross-validation method seems to be an optimal validation method for improving the prediction accuracy in a data set with limited sample sizes. Copyright © 2015 American Association of Orthodontists. Published by Elsevier Inc. All rights reserved.

Performance in grade 12 mathematics and science predicts student nurses' performance in first year science modules at a university in the Western Cape.

PubMed

Mthimunye, Katlego D T; Daniels, Felicity M

2017-10-26

The demand for highly qualified and skilled nurses is increasing in South Africa as well as around the world. Having a background in science can create a significant advantage for students wishing to enrol for an undergraduate nursing qualification because nursing as profession is grounded in scientific evidence. The aim of this study was to investigate the predictive validity of grade 12 mathematics and science on the academic performance of first year student nurses in science modules. A quantitative research method using a cross-sectional predictive design was employed in this study. The participants included first year Bachelor of Nursing students enrolled at a university in the Western Cape, South Africa. Descriptive and inferential statistics were performed to analyse the data by using the IBM Statistical Package for Social Sciences versions 24. Descriptive analysis of all variables was performed as well as the Spearman's rank correlation test to describe the relationship among the study variables. Standard multiple linear regressions analysis was performed to determine the predictive validity of grade 12 mathematics and science on the academic performance of first year student nurses in science modules. The results of this study showed that grade 12 physical science is not a significant predictor (p > 0.062) of performance in first year science modules. The multiple linear regression revealed that grade 12 mathematics and life science grades explained 37.1% to 38.1% (R2 = 0.381 and adj R2 = 0.371) of the variation in the first year science grade distributions. Based on the results of the study it is evident that performance in grade 12 mathematics (β = 2.997) and life science (β = 3.175) subjects is a significant predictor (p < 0.001) of the performance in first year science modules for student nurses at the university identified for this study.

How effective are risk assessments/measures for predicting future aggressive behaviour in adults with intellectual disabilities (ID): A systematic review and meta-analysis.

PubMed

Lofthouse, Rachael; Golding, Laura; Totsika, Vasiliki; Hastings, Richard; Lindsay, William

2017-12-01

Risk assessments assist professionals in the identification and management of risk of aggression. The present study aimed to systematically review evidence on the efficacy of assessments for managing the risk of physical aggression in adults with intellectual disabilities (ID). A literature search was conducted using the databases PsycINFO, EMBASE, MEDLINE, Web of Science, and Google Scholar. Electronic and hand searches identified 14 studies that met the inclusion criteria. Standardised mean difference effect sizes Area Under Curve (AUC) were calculated for studies. Random effects subgroup analysis was used to compare different types of risk measures (Actuarial, Structured Professional Judgment and dynamic), and prospective vs. catch-up longitudinal study designs. Overall, evidence of predictive validity was found for risk measures with ID populations: (AUC)=0.724, 95% CI [0.681, 0.768]. There was no variation in the performance of different types of risk measures, or different study design. Risk assessment measures predict the likelihood of aggression in ID population and are comparable to those in mainstream populations. Further meta-analysis is necessary when risk measures are more established in this population. Copyright © 2017 Elsevier Ltd. All rights reserved.

The Quest for Evidence for Proton Therapy: Model-Based Approach and Precision Medicine

DOE Office of Scientific and Technical Information (OSTI.GOV)

Widder, Joachim, E-mail: j.widder@umcg.nl; Schaaf, Arjen van der; Lambin, Philippe

Purpose: Reducing dose to normal tissues is the advantage of protons versus photons. We aimed to describe a method for translating this reduction into a clinically relevant benefit. Methods and Materials: Dutch scientific and health care governance bodies have recently issued landmark reports regarding generation of relevant evidence for new technologies in health care including proton therapy. An approach based on normal tissue complication probability (NTCP) models has been adopted to select patients who are most likely to experience fewer (serious) adverse events achievable by state-of-the-art proton treatment. Results: By analogy with biologically targeted therapies, the technology needs to be testedmore » in enriched cohorts of patients exhibiting the decisive predictive marker: difference in normal tissue dosimetric signatures between proton and photon treatment plans. Expected clinical benefit is then estimated by virtue of multifactorial NTCP models. In this sense, high-tech radiation therapy falls under precision medicine. As a consequence, randomizing nonenriched populations between photons and protons is predictably inefficient and likely to produce confusing results. Conclusions: Validating NTCP models in appropriately composed cohorts treated with protons should be the primary research agenda leading to urgently needed evidence for proton therapy.« less

The effectiveness of physiologically based early warning or track and trigger systems after triage in adult patients presenting to emergency departments: a systematic review.

PubMed

Wuytack, Francesca; Meskell, Pauline; Conway, Aislinn; McDaid, Fiona; Santesso, Nancy; Hickey, Fergal G; Gillespie, Paddy; Raymakers, Adam J N; Smith, Valerie; Devane, Declan

2017-12-06

Changes to physiological parameters precede deterioration of ill patients. Early warning and track and trigger systems (TTS) use routine physiological measurements with pre-specified thresholds to identify deteriorating patients and trigger appropriate and timely escalation of care. Patients presenting to the emergency department (ED) are undiagnosed, undifferentiated and of varying acuity, yet the effectiveness and cost-effectiveness of using early warning systems and TTS in this setting is unclear. We aimed to systematically review the evidence on the use, development/validation, clinical effectiveness and cost-effectiveness of physiologically based early warning systems and TTS for the detection of deterioration in adult patients presenting to EDs. We searched for any study design in scientific databases and grey literature resources up to March 2016. Two reviewers independently screened results and conducted quality assessment. One reviewer extracted data with independent verification of 50% by a second reviewer. Only information available in English was included. Due to the heterogeneity of reporting across studies, results were synthesised narratively and in evidence tables. We identified 6397 citations of which 47 studies and 1 clinical trial registration were included. Although early warning systems are increasingly used in EDs, compliance varies. One non-randomised controlled trial found that using an early warning system in the ED may lead to a change in patient management but may not reduce adverse events; however, this is uncertain, considering the very low quality of evidence. Twenty-eight different early warning systems were developed/validated in 36 studies. There is relatively good evidence on the predictive ability of certain early warning systems on mortality and ICU/hospital admission. No health economic data were identified. Early warning systems seem to predict adverse outcomes in adult patients of varying acuity presenting to the ED but there is a lack of high quality comparative studies to examine the effect of using early warning systems on patient outcomes. Such studies should include health economics assessments.

A new framework to enhance the interpretation of external validation studies of clinical prediction models.

PubMed

Debray, Thomas P A; Vergouwe, Yvonne; Koffijberg, Hendrik; Nieboer, Daan; Steyerberg, Ewout W; Moons, Karel G M

2015-03-01

It is widely acknowledged that the performance of diagnostic and prognostic prediction models should be assessed in external validation studies with independent data from "different but related" samples as compared with that of the development sample. We developed a framework of methodological steps and statistical methods for analyzing and enhancing the interpretation of results from external validation studies of prediction models. We propose to quantify the degree of relatedness between development and validation samples on a scale ranging from reproducibility to transportability by evaluating their corresponding case-mix differences. We subsequently assess the models' performance in the validation sample and interpret the performance in view of the case-mix differences. Finally, we may adjust the model to the validation setting. We illustrate this three-step framework with a prediction model for diagnosing deep venous thrombosis using three validation samples with varying case mix. While one external validation sample merely assessed the model's reproducibility, two other samples rather assessed model transportability. The performance in all validation samples was adequate, and the model did not require extensive updating to correct for miscalibration or poor fit to the validation settings. The proposed framework enhances the interpretation of findings at external validation of prediction models. Copyright © 2015 The Authors. Published by Elsevier Inc. All rights reserved.

Validity of vaccination cards and parental recall to estimate vaccination coverage: a systematic review of the literature.

PubMed

Miles, Melody; Ryman, Tove K; Dietz, Vance; Zell, Elizabeth; Luman, Elizabeth T

2013-03-15

Immunization programs frequently rely on household vaccination cards, parental recall, or both to calculate vaccination coverage. This information is used at both the global and national level for planning and allocating performance-based funds. However, the validity of household-derived coverage sources has not yet been widely assessed or discussed. To advance knowledge on the validity of different sources of immunization coverage, we undertook a global review of literature. We assessed concordance, sensitivity, specificity, positive and negative predictive value, and coverage percentage point difference when subtracting household vaccination source from a medical provider source. Median coverage difference per paper ranged from -61 to +1 percentage points between card versus provider sources and -58 to +45 percentage points between recall versus provider source. When card and recall sources were combined, median coverage difference ranged from -40 to +56 percentage points. Overall, concordance, sensitivity, specificity, positive and negative predictive value showed poor agreement, providing evidence that household vaccination information may not be reliable, and should be interpreted with care. While only 5 papers (11%) included in this review were from low-middle income countries, low-middle income countries often rely more heavily on household vaccination information for decision making. Recommended actions include strengthening quality of child-level data and increasing investments to improve vaccination card availability and card marking. There is also an urgent need for additional validation studies of vaccine coverage in low and middle income countries. Copyright © 2013. Published by Elsevier Ltd.

Validity and validation of expert (Q)SAR systems.

PubMed

Hulzebos, E; Sijm, D; Traas, T; Posthumus, R; Maslankiewicz, L

2005-08-01

At a recent workshop in Setubal (Portugal) principles were drafted to assess the suitability of (quantitative) structure-activity relationships ((Q)SARs) for assessing the hazards and risks of chemicals. In the present study we applied some of the Setubal principles to test the validity of three (Q)SAR expert systems and validate the results. These principles include a mechanistic basis, the availability of a training set and validation. ECOSAR, BIOWIN and DEREK for Windows have a mechanistic or empirical basis. ECOSAR has a training set for each QSAR. For half of the structural fragments the number of chemicals in the training set is >4. Based on structural fragments and log Kow, ECOSAR uses linear regression to predict ecotoxicity. Validating ECOSAR for three 'valid' classes results in predictivity of > or = 64%. BIOWIN uses (non-)linear regressions to predict the probability of biodegradability based on fragments and molecular weight. It has a large training set and predicts non-ready biodegradability well. DEREK for Windows predictions are supported by a mechanistic rationale and literature references. The structural alerts in this program have been developed with a training set of positive and negative toxicity data. However, to support the prediction only a limited number of chemicals in the training set is presented to the user. DEREK for Windows predicts effects by 'if-then' reasoning. The program predicts best for mutagenicity and carcinogenicity. Each structural fragment in ECOSAR and DEREK for Windows needs to be evaluated and validated separately.

Prediction of psychosis across protocols and risk cohorts using automated language analysis

PubMed Central

Corcoran, Cheryl M.; Carrillo, Facundo; Fernández‐Slezak, Diego; Bedi, Gillinder; Klim, Casimir; Javitt, Daniel C.; Bearden, Carrie E.; Cecchi, Guillermo A.

2018-01-01

Language and speech are the primary source of data for psychiatrists to diagnose and treat mental disorders. In psychosis, the very structure of language can be disturbed, including semantic coherence (e.g., derailment and tangentiality) and syntactic complexity (e.g., concreteness). Subtle disturbances in language are evident in schizophrenia even prior to first psychosis onset, during prodromal stages. Using computer‐based natural language processing analyses, we previously showed that, among English‐speaking clinical (e.g., ultra) high‐risk youths, baseline reduction in semantic coherence (the flow of meaning in speech) and in syntactic complexity could predict subsequent psychosis onset with high accuracy. Herein, we aimed to cross‐validate these automated linguistic analytic methods in a second larger risk cohort, also English‐speaking, and to discriminate speech in psychosis from normal speech. We identified an automated machine‐learning speech classifier – comprising decreased semantic coherence, greater variance in that coherence, and reduced usage of possessive pronouns – that had an 83% accuracy in predicting psychosis onset (intra‐protocol), a cross‐validated accuracy of 79% of psychosis onset prediction in the original risk cohort (cross‐protocol), and a 72% accuracy in discriminating the speech of recent‐onset psychosis patients from that of healthy individuals. The classifier was highly correlated with previously identified manual linguistic predictors. Our findings support the utility and validity of automated natural language processing methods to characterize disturbances in semantics and syntax across stages of psychotic disorder. The next steps will be to apply these methods in larger risk cohorts to further test reproducibility, also in languages other than English, and identify sources of variability. This technology has the potential to improve prediction of psychosis outcome among at‐risk youths and identify linguistic targets for remediation and preventive intervention. More broadly, automated linguistic analysis can be a powerful tool for diagnosis and treatment across neuropsychiatry. PMID:29352548

Prediction of psychosis across protocols and risk cohorts using automated language analysis.

PubMed

Corcoran, Cheryl M; Carrillo, Facundo; Fernández-Slezak, Diego; Bedi, Gillinder; Klim, Casimir; Javitt, Daniel C; Bearden, Carrie E; Cecchi, Guillermo A

2018-02-01

Language and speech are the primary source of data for psychiatrists to diagnose and treat mental disorders. In psychosis, the very structure of language can be disturbed, including semantic coherence (e.g., derailment and tangentiality) and syntactic complexity (e.g., concreteness). Subtle disturbances in language are evident in schizophrenia even prior to first psychosis onset, during prodromal stages. Using computer-based natural language processing analyses, we previously showed that, among English-speaking clinical (e.g., ultra) high-risk youths, baseline reduction in semantic coherence (the flow of meaning in speech) and in syntactic complexity could predict subsequent psychosis onset with high accuracy. Herein, we aimed to cross-validate these automated linguistic analytic methods in a second larger risk cohort, also English-speaking, and to discriminate speech in psychosis from normal speech. We identified an automated machine-learning speech classifier - comprising decreased semantic coherence, greater variance in that coherence, and reduced usage of possessive pronouns - that had an 83% accuracy in predicting psychosis onset (intra-protocol), a cross-validated accuracy of 79% of psychosis onset prediction in the original risk cohort (cross-protocol), and a 72% accuracy in discriminating the speech of recent-onset psychosis patients from that of healthy individuals. The classifier was highly correlated with previously identified manual linguistic predictors. Our findings support the utility and validity of automated natural language processing methods to characterize disturbances in semantics and syntax across stages of psychotic disorder. The next steps will be to apply these methods in larger risk cohorts to further test reproducibility, also in languages other than English, and identify sources of variability. This technology has the potential to improve prediction of psychosis outcome among at-risk youths and identify linguistic targets for remediation and preventive intervention. More broadly, automated linguistic analysis can be a powerful tool for diagnosis and treatment across neuropsychiatry. © 2018 World Psychiatric Association.

Prospective validation of pathologic complete response models in rectal cancer: Transferability and reproducibility.

PubMed

van Soest, Johan; Meldolesi, Elisa; van Stiphout, Ruud; Gatta, Roberto; Damiani, Andrea; Valentini, Vincenzo; Lambin, Philippe; Dekker, Andre

2017-09-01

Multiple models have been developed to predict pathologic complete response (pCR) in locally advanced rectal cancer patients. Unfortunately, validation of these models normally omit the implications of cohort differences on prediction model performance. In this work, we will perform a prospective validation of three pCR models, including information whether this validation will target transferability or reproducibility (cohort differences) of the given models. We applied a novel methodology, the cohort differences model, to predict whether a patient belongs to the training or to the validation cohort. If the cohort differences model performs well, it would suggest a large difference in cohort characteristics meaning we would validate the transferability of the model rather than reproducibility. We tested our method in a prospective validation of three existing models for pCR prediction in 154 patients. Our results showed a large difference between training and validation cohort for one of the three tested models [Area under the Receiver Operating Curve (AUC) cohort differences model: 0.85], signaling the validation leans towards transferability. Two out of three models had a lower AUC for validation (0.66 and 0.58), one model showed a higher AUC in the validation cohort (0.70). We have successfully applied a new methodology in the validation of three prediction models, which allows us to indicate if a validation targeted transferability (large differences between training/validation cohort) or reproducibility (small cohort differences). © 2017 American Association of Physicists in Medicine.

Early Therapy Intensity Level (TIL) Predicts Mortality in Spontaneous Intracerebral Hemorrhage.

PubMed

Ziai, Wendy C; Siddiqui, Aazim A; Ullman, Natalie; Herrick, Daniel B; Yenokyan, Gayane; McBee, Nichol; Lane, Karen; Hanley, Daniel F

2015-10-01

Outcome from spontaneous intracerebral hemorrhage (sICH) may depend on patient-care variability. We developed as ICH-specific therapy intensity level (TIL) metric using evidence-based elements in a high severity sICH cohort. This is a cohort study of 170 patients with sICH and any intraventricular hemorrhage treated in 2 academic neuroICUs. Pre-defined quality indicators were identified based on current guidelines, scientific evidence, and likelihood of care documentation in first 72 h of hospital admission. We assessed performance on each indicator and association with discharge mortality. Significant indicators were aggregated to develop a TIL score. The predictive validity of the best fit TIL score was tested with threefold cross-validation of multivariate logistic regression models of in-hospital survival and good outcome (modified Rankin score 0-3). Median ICH score was 3; discharge mortality was 51.2%. Five/19 tested variables were significantly associated with lower discharge mortality: no DNR/withdrawal of treatment within 24 h of admission, target glucose within 4 h of high glucose, no recurrent hyperpyrexia, clinical reversal of herniation/intracranial pressure >20 mmHg within 60 min of detection, and reversal of INR (<1.4) within 2 h of first elevation. One point was given for each or if not applicable. Median TIL score was significantly higher in survivors versus non-survivors (5[1] vs. 3[1]; P < 0.001). A 4-point aggregated TIL score was most predictive of discharge survival (area under receiving operating characteristic curve 0.85, 95% CI 0.80-0.90) and good outcome (AUC 0.84) and was an independent predictor of both (survival: OR 7.10; 95% CI 3.57-14.11; P < 0.001; good outcome: OR 3.10; 95% CI 1.06-8.79; P < 0.001). A simplified TIL score using evidenced-based patient-care parameters within first 3 days of admission after sICH was significantly associated with early mortality and good outcome. The next step is prospective validation of the simplified TIL score in a large clinical trial.

Time to first cigarette in the morning as an index of ability to quit smoking: Implications for nicotine dependence

PubMed Central

Baker, Timothy B.; Piper, Megan E.; McCarthy, Danielle E.; Bolt, Daniel M.; Smith, Stevens S.; Kim, Su-Young; Colby, Suzanne; Conti, David; Giovino, Gary A.; Hatsukami, Dorothy; Hyland, Andrew; Krishnan-Sarin, Suchitra; Niaura, Raymond; Perkins, Kenneth A.; Toll, Benjamin A.

2010-01-01

An inability to maintain abstinence is a key indicator of tobacco dependence. Unfortunately, little evidence exists regarding the ability of the major tobacco dependence measures to predict smoking cessation outcome. This paper used data from four placebo-controlled smoking cessation trials and one international epidemiologic study to determine relations between the Fagerström Test for Nicotine Dependence (FTND; Heatherton et al., 1991), the Heaviness of Smoking Index (HSI; Kozlowski et al., 1994), the Nicotine Dependence Syndrome Scale (NDSS; Shiffman et al., 2004) and the Wisconsin Inventory of Smoking Dependence Motives (WISDM; Piper et al. 2004) with cessation success. Results showed that much of the predictive validity of the FTND could be attributed to its first item, time to first cigarette in the morning, and this item had greater validity than any other single measure. Thus, the time to first cigarette item appears to tap a pattern of heavy, uninterrupted, and automatic smoking and may be a good single-item measure of nicotine dependence. PMID:18067032

Rapid antibiotic-resistance predictions from genome sequence data for Staphylococcus aureus and Mycobacterium tuberculosis

PubMed Central

Bradley, Phelim; Gordon, N. Claire; Walker, Timothy M.; Dunn, Laura; Heys, Simon; Huang, Bill; Earle, Sarah; Pankhurst, Louise J.; Anson, Luke; de Cesare, Mariateresa; Piazza, Paolo; Votintseva, Antonina A.; Golubchik, Tanya; Wilson, Daniel J.; Wyllie, David H.; Diel, Roland; Niemann, Stefan; Feuerriegel, Silke; Kohl, Thomas A.; Ismail, Nazir; Omar, Shaheed V.; Smith, E. Grace; Buck, David; McVean, Gil; Walker, A. Sarah; Peto, Tim E. A.; Crook, Derrick W.; Iqbal, Zamin

2015-01-01

The rise of antibiotic-resistant bacteria has led to an urgent need for rapid detection of drug resistance in clinical samples, and improvements in global surveillance. Here we show how de Bruijn graph representation of bacterial diversity can be used to identify species and resistance profiles of clinical isolates. We implement this method for Staphylococcus aureus and Mycobacterium tuberculosis in a software package (‘Mykrobe predictor') that takes raw sequence data as input, and generates a clinician-friendly report within 3 minutes on a laptop. For S. aureus, the error rates of our method are comparable to gold-standard phenotypic methods, with sensitivity/specificity of 99.1%/99.6% across 12 antibiotics (using an independent validation set, n=470). For M. tuberculosis, our method predicts resistance with sensitivity/specificity of 82.6%/98.5% (independent validation set, n=1,609); sensitivity is lower here, probably because of limited understanding of the underlying genetic mechanisms. We give evidence that minor alleles improve detection of extremely drug-resistant strains, and demonstrate feasibility of the use of emerging single-molecule nanopore sequencing techniques for these purposes. PMID:26686880

Adaptation and validation of the Evidence-Based Practice Belief and Implementation scales for French-speaking Swiss nurses and allied healthcare providers.

PubMed

Verloo, Henk; Desmedt, Mario; Morin, Diane

2017-09-01

To evaluate two psychometric properties of the French versions of the Evidence-Based Practice Beliefs and Evidence-Based Practice Implementation scales, namely their internal consistency and construct validity. The Evidence-Based Practice Beliefs and Evidence-Based Practice Implementation scales developed by Melnyk et al. are recognised as valid, reliable instruments in English. However, no psychometric validation for their French versions existed. Secondary analysis of a cross sectional survey. Source data came from a cross-sectional descriptive study sample of 382 nurses and other allied healthcare providers. Cronbach's alpha was used to evaluate internal consistency, and principal axis factor analysis and varimax rotation were computed to determine construct validity. The French Evidence-Based Practice Beliefs and Evidence-Based Practice Implementation scales showed excellent reliability, with Cronbach's alphas close to the scores established by Melnyk et al.'s original versions. Principal axis factor analysis showed medium-to-high factor loading scores without obtaining collinearity. Principal axis factor analysis with varimax rotation of the 16-item Evidence-Based Practice Beliefs scale resulted in a four-factor loading structure. Principal axis factor analysis with varimax rotation of the 17-item Evidence-Based Practice Implementation scale revealed a two-factor loading structure. Further research should attempt to understand why the French Evidence-Based Practice Implementation scale showed a two-factor loading structure but Melnyk et al.'s original has only one. The French versions of the Evidence-Based Practice Beliefs and Evidence-Based Practice Implementation scales can both be considered valid and reliable instruments for measuring Evidence-Based Practice beliefs and implementation. The results suggest that the French Evidence-Based Practice Beliefs and Evidence-Based Practice Implementation scales are valid and reliable and can therefore be used to evaluate the effectiveness of organisational strategies aimed at increasing professionals' confidence in Evidence-Based Practice, supporting its use and implementation. © 2017 John Wiley & Sons Ltd.

Psychometric testing of the breastfeeding self-efficacy scale-short form among adolescents.

PubMed

Dennis, Cindy-Lee; Heaman, Maureen; Mossman, Marion

2011-09-01

This study aimed to (a) psychometrically assess the Breastfeeding Self-Efficacy Scale-Short Form (BSES-SF) antenatally and postnatally among adolescents, and (b) examine the relationship between breast-feeding self-efficacy and adolescent demographic variables. The BSES-SF is used internationally but has not been psychometrically tested with an adolescent population. A methodological study was conducted in which 103 pregnant adolescents were recruited from two prenatal clinics at a tertiary care setting in western Canada. The BSES-SF was administered at 34 weeks' gestation and again at 1 and 4 weeks postpartum. The Cronbach's alpha coefficient was .84 for the antenatal assessment and .93 for the postnatal assessment. Antenatal BSES-SF scores significantly predicted breast-feeding initiation, whereas antenatal and postnatal scores predicted duration and exclusivity to 4 weeks postpartum. Other reliability and validity results are consistent with previous research with adult samples. Study findings indicate prenatal classes and professional support may be particularly important sources of information to increase adolescent breast-feeding self-efficacy. Results provide evidence that the scale may be a valid and reliable measure of breast-feeding self-efficacy among adolescents, predicting breast-feeding initiation, duration, and exclusivity. Copyright © 2011 Society for Adolescent Health and Medicine. Published by Elsevier Inc. All rights reserved.

Brain responses to biological motion predict treatment outcome in young adults with autism receiving Virtual Reality Social Cognition Training: Preliminary findings.

PubMed

Yang, Y J Daniel; Allen, Tandra; Abdullahi, Sebiha M; Pelphrey, Kevin A; Volkmar, Fred R; Chapman, Sandra B

2017-06-01

Autism Spectrum Disorder (ASD) is characterized by remarkable heterogeneity in social, communication, and behavioral deficits, creating a major barrier in identifying effective treatments for a given individual with ASD. To facilitate precision medicine in ASD, we utilized a well-validated biological motion neuroimaging task to identify pretreatment biomarkers that can accurately forecast the response to an evidence-based behavioral treatment, Virtual Reality-Social Cognition Training (VR-SCT). In a preliminary sample of 17 young adults with high-functioning ASD, we identified neural predictors of change in emotion recognition after VR-SCT. The predictors were characterized by the pretreatment brain activations to biological vs. scrambled motion in the neural circuits that support (a) language comprehension and interpretation of incongruent auditory emotions and prosody, and (b) processing socio-emotional experience and interpersonal affective information, as well as emotional regulation. The predictive value of the findings for individual adults with ASD was supported by regression-based multivariate pattern analyses with cross validation. To our knowledge, this is the first pilot study that shows neuroimaging-based predictive biomarkers for treatment effectiveness in adults with ASD. The findings have potentially far-reaching implications for developing more precise and effective treatments for ASD. Copyright © 2017 The Authors. Published by Elsevier Ltd.. All rights reserved.

Early Prediction of Intensive Care Unit-Acquired Weakness: A Multicenter External Validation Study.

PubMed

Witteveen, Esther; Wieske, Luuk; Sommers, Juultje; Spijkstra, Jan-Jaap; de Waard, Monique C; Endeman, Henrik; Rijkenberg, Saskia; de Ruijter, Wouter; Sleeswijk, Mengalvio; Verhamme, Camiel; Schultz, Marcus J; van Schaik, Ivo N; Horn, Janneke

2018-01-01

An early diagnosis of intensive care unit-acquired weakness (ICU-AW) is often not possible due to impaired consciousness. To avoid a diagnostic delay, we previously developed a prediction model, based on single-center data from 212 patients (development cohort), to predict ICU-AW at 2 days after ICU admission. The objective of this study was to investigate the external validity of the original prediction model in a new, multicenter cohort and, if necessary, to update the model. Newly admitted ICU patients who were mechanically ventilated at 48 hours after ICU admission were included. Predictors were prospectively recorded, and the outcome ICU-AW was defined by an average Medical Research Council score <4. In the validation cohort, consisting of 349 patients, we analyzed performance of the original prediction model by assessment of calibration and discrimination. Additionally, we updated the model in this validation cohort. Finally, we evaluated a new prediction model based on all patients of the development and validation cohort. Of 349 analyzed patients in the validation cohort, 190 (54%) developed ICU-AW. Both model calibration and discrimination of the original model were poor in the validation cohort. The area under the receiver operating characteristics curve (AUC-ROC) was 0.60 (95% confidence interval [CI]: 0.54-0.66). Model updating methods improved calibration but not discrimination. The new prediction model, based on all patients of the development and validation cohort (total of 536 patients) had a fair discrimination, AUC-ROC: 0.70 (95% CI: 0.66-0.75). The previously developed prediction model for ICU-AW showed poor performance in a new independent multicenter validation cohort. Model updating methods improved calibration but not discrimination. The newly derived prediction model showed fair discrimination. This indicates that early prediction of ICU-AW is still challenging and needs further attention.

Preliminary validation of the Yale Food Addiction Scale for children.

PubMed

Gearhardt, Ashley N; Roberto, Christina A; Seamans, Marissa J; Corbin, William R; Brownell, Kelly D

2013-12-01

Evidence is growing that an addictive process may play a role in problematic eating behavior. The majority of research on this topic has examined the concept of "food addiction" solely in adult samples. If certain foods have addictive potential, children may be impacted as much as (or more) than adults due to psychological and neurobiological vulnerabilities at younger developmental stages. In the current study, we developed a measure of food addiction in children that reflects the diagnostic indicators of addiction. The content and reading level of the Yale Food Addiction Scale (YFAS) was altered to be appropriate for children (YFAS-C). The YFAS-C and other eating-related measures were administered to study participants to examine the validity and reliability of the YFAS-C. 75 children were recruited from the community ranging from lean to obese. The validation of the YFAC-C provides preliminary support for its convergent validity with like constructs and incremental validity in predicting body mass index. Internal consistency was adequate given the small number of items on the scale. The YFAS-C appears to be a helpful tool for identifying addictive-like eating in children. © 2013.

Validity evidence based on test content.

PubMed

Sireci, Stephen; Faulkner-Bond, Molly

2014-01-01

Validity evidence based on test content is one of the five forms of validity evidence stipulated in the Standards for Educational and Psychological Testing developed by the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. In this paper, we describe the logic and theory underlying such evidence and describe traditional and modern methods for gathering and analyzing content validity data. A comprehensive review of the literature and of the aforementioned Standards is presented. For educational tests and other assessments targeting knowledge and skill possessed by examinees, validity evidence based on test content is necessary for building a validity argument to support the use of a test for a particular purpose. By following the methods described in this article, practitioners have a wide arsenal of tools available for determining how well the content of an assessment is congruent with and appropriate for the specific testing purposes.

The Predictive Validity of the Minnesota Reading Assessment for Students in Postsecondary Vocational Education Programs.

ERIC Educational Resources Information Center

Brown, James M.; Chang, Gerald

1982-01-01

The predictive validity of the Minnesota Reading Assessment (MRA) when used to project potential performance of postsecondary vocational-technical education students was examined. Findings confirmed the MRA to be a valid predictor, although the error in prediction varied between the criterion variables. (Author/GK)

Comparative Predictive Validity of the New MCAT Using Different Admissions Criteria.

ERIC Educational Resources Information Center

Golmon, Melton E.; Berry, Charles A.

1981-01-01

New Medical College Admission Test (MCAT) scores and undergraduate academic achievement were examined for their validity in predicting the performance of two select student populations at Northwestern University Medical School. The data support the hypothesis that New MCAT scores possess substantial predictive validity. (Author/MLW)

Adapting Social Neuroscience Measures for Schizophrenia Clinical Trials, Part 3: Fathoming External Validity

PubMed Central

Olbert, Charles M.

2013-01-01

It is unknown whether measures adapted from social neuroscience linked to specific neural systems will demonstrate relationships to external variables. Four paradigms adapted from social neuroscience were administered to 173 clinically stable outpatients with schizophrenia to determine their relationships to functionally meaningful variables and to investigate their incremental validity beyond standard measures of social and nonsocial cognition. The 4 paradigms included 2 that assess perception of nonverbal social and action cues (basic biological motion and emotion in biological motion) and 2 that involve higher level inferences about self and others’ mental states (self- referential memory and empathic accuracy). Overall, social neuroscience paradigms showed significant relationships to functional capacity but weak relationships to community functioning; the paradigms also showed weak correlations to clinical symptoms. Evidence for incremental validity beyond standard measures of social and nonsocial cognition was mixed with additional predictive power shown for functional capacity but not community functioning. Of the newly adapted paradigms, the empathic accuracy task had the broadest external validity. These results underscore the difficulty of translating developments from neuroscience into clinically useful tasks with functional significance. PMID:24072806

Validity evidence for the Security Scale as a measure of perceived attachment security in adolescence.

PubMed

Van Ryzin, Mark J; Leve, Leslie D

2012-04-01

In this study, the validity of a self-report measure of children's perceived attachment security (the Kerns Security Scale) was tested using adolescents. With regards to predictive validity, the Security Scale was significantly associated with (1) observed mother-adolescent interactions during conflict and (2) parent- and teacher-rated social competence. With regards to convergent validity, the Security Scale was significantly associated with all subscales of the Adult Attachment Scale (i.e., Depend, Anxiety, and Close) as measured 3 years later. Further, these links were found even after controlling for mother-child relationship quality as assessed by the Inventory of Parent and Peer Attachment (IPPA), and chi-square difference tests indicated that the Security Scale was generally a stronger predictor as compared to the IPPA. These results suggest that the Security Scale can be used to assess perceived attachment security across both childhood and adolescence, and thus could contribute significantly to developmental research during this period. Copyright © 2011 The Foundation for Professionals in Services for Adolescents. Published by Elsevier Ltd. All rights reserved.

The multi-dimensional measure of informed choice: a validation study.

PubMed

Michie, Susan; Dormandy, Elizabeth; Marteau, Theresa M

2002-09-01

The aim of this prospective study is to assess the reliability and validity of a multi-dimensional measure of informed choice (MMIC). Participants were 225 pregnant women in two general hospitals in the UK, women receiving low-risk results following serum screening for Down syndrome. The MMIC was administered before testing and the Ottawa Decisional Conflict Scale was administered 6 weeks later. The component scales of the MMIC, knowledge and attitude, were internally consistent (alpha values of 0.68 and 0.78, respectively). Those who made a choice categorised as informed using the MMIC rated their decision 6 weeks later as being more informed, better supported and of higher quality than women whose choice was categorised as uninformed. This provides evidence of predictive validity, whilst the lack of association between the MMIC and anxiety shows construct (discriminant) validity. Thus, the MMIC has been shown to be psychometrically robust in pregnant women offered the choice to undergo prenatal screening for Down syndrome and receiving a low-risk result. Replication of this finding in other groups, facing other decisions, with other outcomes, should be assessed in future research.

Adapting social neuroscience measures for schizophrenia clinical trials, part 3: fathoming external validity.

PubMed

Olbert, Charles M; Penn, David L; Kern, Robert S; Lee, Junghee; Horan, William P; Reise, Steven P; Ochsner, Kevin N; Marder, Stephen R; Green, Michael F

2013-11-01

It is unknown whether measures adapted from social neuroscience linked to specific neural systems will demonstrate relationships to external variables. Four paradigms adapted from social neuroscience were administered to 173 clinically stable outpatients with schizophrenia to determine their relationships to functionally meaningful variables and to investigate their incremental validity beyond standard measures of social and nonsocial cognition. The 4 paradigms included 2 that assess perception of nonverbal social and action cues (basic biological motion and emotion in biological motion) and 2 that involve higher level inferences about self and others' mental states (self-referential memory and empathic accuracy). Overall, social neuroscience paradigms showed significant relationships to functional capacity but weak relationships to community functioning; the paradigms also showed weak correlations to clinical symptoms. Evidence for incremental validity beyond standard measures of social and nonsocial cognition was mixed with additional predictive power shown for functional capacity but not community functioning. Of the newly adapted paradigms, the empathic accuracy task had the broadest external validity. These results underscore the difficulty of translating developments from neuroscience into clinically useful tasks with functional significance.

Diagnostic classification past, present, and future: implications for pharmacotherapy.

PubMed

Howland, Robert H

2013-04-01

Making a diagnosis is a key step in understanding the natural course of a disorder, selecting an appropriate treatment for the disorder, and predicting its response to treatment. Diagnostic proposals can be evaluated in two ways: reliability and validity. The reliability and validity of diagnoses are not one and the same, although establishing reliability is usually a necessary step before being able to evaluate and determine validity. There is little evidence that most psychiatric diagnoses are valid, but the reliability of diagnoses using classification systems developed since 1970 have greatly improved and are important for clinical practice and research. Past and current diagnostic systems have not optimally assisted the search for disorder-specific pathophysiological mechanisms, and they do not provide the specificity that clinicians would like when selecting medication. The Research Domain Criteria project is intended to shift research away from categorical diagnoses to focus on dysregulated neurobiological systems, and this approach ultimately may be more useful for understanding the pathophysiology of mental disorders and improving the development and use of treatment interventions. Copyright 2013, SLACK Incorporated.

Prediction of adult height in girls: the Beunen-Malina-Freitas method.

PubMed

Beunen, Gaston P; Malina, Robert M; Freitas, Duarte L; Thomis, Martine A; Maia, José A; Claessens, Albrecht L; Gouveia, Elvio R; Maes, Hermine H; Lefevre, Johan

2011-12-01

The purpose of this study was to validate and cross-validate the Beunen-Malina-Freitas method for non-invasive prediction of adult height in girls. A sample of 420 girls aged 10-15 years from the Madeira Growth Study were measured at yearly intervals and then 8 years later. Anthropometric dimensions (lengths, breadths, circumferences, and skinfolds) were measured; skeletal age was assessed using the Tanner-Whitehouse 3 method and menarcheal status (present or absent) was recorded. Adult height was measured and predicted using stepwise, forward, and maximum R (2) regression techniques. Multiple correlations, mean differences, standard errors of prediction, and error boundaries were calculated. A sample of the Leuven Longitudinal Twin Study was used to cross-validate the regressions. Age-specific coefficients of determination (R (2)) between predicted and measured adult height varied between 0.57 and 0.96, while standard errors of prediction varied between 1.1 and 3.9 cm. The cross-validation confirmed the validity of the Beunen-Malina-Freitas method in girls aged 12-15 years, but at lower ages the cross-validation was less consistent. We conclude that the Beunen-Malina-Freitas method is valid for the prediction of adult height in girls aged 12-15 years. It is applicable to European populations or populations of European ancestry.

Attentional influences on memory formation: A tale of a not-so-simple story.

PubMed

Ortiz-Tudela, J; Milliken, B; Jiménez, L; Lupiáñez, J

2018-05-01

Is there a learning mechanism triggered by mere expectation violation? Is there some form of memory enhancement inherent to an event mismatching our predictions? Across seven experiments, we explore this issue by means of a validity paradigm. Although our manipulation clearly succeeded in generating an expectation and breaking it, the memory consequences of that expectation mismatch are not so obvious. We report here evidence of a null effect of expectation on memory formation. Our results (1) show that enhanced memory for unexpected events is not easily achieved and (2) call for a reevaluation of previous accounts of memory enhancements based on prediction error or difficulty of processing. Limitations of this study and possible implications for the field are discussed in detail.

Validation of gamma irradiator controls for quality and regulatory compliance

NASA Astrophysics Data System (ADS)

Harding, Rorry B.; Pinteric, Francis J. A.

1995-09-01

Since 1978 the U.S. Food and Drug Administration (FDA) has had both the legal authority and the Current Good Manufacturing Practice (CGMP) regulations in place to require irradiator owners who process medical devices to produce evidence of Irradiation Process Validation. One of the key components of Irradiation Process Validation is the validation of the irradiator controls. However, it is only recently that FDA audits have focused on this component of the process validation. What is Irradiator Control System Validation? What constitutes evidence of control? How do owners obtain evidence? What is the irradiator supplier's role in validation? How does the ISO 9000 Quality Standard relate to the FDA's CGMP requirement for evidence of Control System Validation? This paper presents answers to these questions based on the recent experiences of Nordion's engineering and product management staff who have worked with several US-based irradiator owners. This topic — Validation of Irradiator Controls — is a significant regulatory compliance and operations issue within the irradiator suppliers' and users' community.

The Development and Validation of the Client Expectations of Massage Scale

PubMed Central

Boulanger, Karen T.; Campo, Shelly; Glanville, Jennifer L.; Lowe, John B; Yang, Jingzhen

2012-01-01

Background: Although there is evidence that client expectations influence client outcomes, a valid and reliable scale for measuring the range of client expectations for both massage therapy and the behaviors of their massage therapists does not exist. Understanding how client expectations influence client outcomes would provide insight into how massage achieves its reported effects. Purpose: To develop and validate the Client Expectations of Massage Scale (CEMS), a measure of clients’ clinical, educational, interpersonal, and outcome expectations. Setting: Offices of licensed massage therapists in Iowa. Research Design: A practice-based research methodology was used to collect data from two samples of massage therapy clients. For Sample 1, 21 volunteer massage therapists collected data from their clients before the massage. Factor analysis was conducted to test construct validity and coefficient alpha was used to assess reliability. Correlational analyses with the CEMS, previous measures of client expectations, and the Life Orientation Test–Revised were examined to test the convergent and discriminant validity of the CEMS. For Sample 2, 24 massage therapists distributed study materials for clients to complete before and after a massage therapy session. Structural equation modeling was used to assess the construct, discriminant, and predictive validity of the CEMS. Participants: Sample 1 involved 320 and Sample 2 involved 321 adult massage clients. Intervention: Standard care provided by licensed massage therapists. Main Outcomes: Numeric Rating Scale for pain and Positive and Negative Affect Schedule–Revised (including the Serenity subscale). Results: The CEMS demonstrated good construct, convergent, discriminant and predictive validity, and adequate reliability. Client expectations were generally positive toward massage and their massage therapists. Positive outcome expectations had a positive effect on clients’ changes in pain and serenity. High interpersonal expectations had a negative effect on clients’ changes in serenity. Conclusions: Client expectations contribute to the nonspecific effects of massage therapy. PMID:23087774

Off-duty deviance: Organizational policies and evidence for two prevention strategies.

PubMed

Lyons, Brian D; Hoffman, Brian J; Bommer, William H; Kennedy, Colby L; Hetrick, Andrea L

2016-04-01

Anecdotal evidence suggests that organizations are increasingly concerned with employee off-duty deviance (ODD), yet management research has rarely investigated this type of deviant behavior. We define ODD as behaviors committed outside the workplace or when off-duty that are deviant by organizational and/or societal standards, jeopardize the employee's status within the organization, and threaten the interests and well-being of the organization and its stakeholders. Three studies are presented to better understand the relevance of ODD to modern organizations and then to understand potential approaches to reduce the incidence of ODD. The first study provides a qualitative review of publicly available ODD policies within the Fortune 500; the results showed that 13.4% of the Fortune 500 had a publicly available ODD policy, with the majority prohibiting criminal forms of ODD to protect the firm's reputation. The next 2 studies examine the efficacy of different approaches to reduce criminal ODD: policy adoption and personnel selection. In the second study, a longitudinal, quasi-experimental design showed a significant-albeit modest-reduction in criminal ODD following the adoption of a conduct policy. In the third and final study, a criterion-related validity design supported the predictive validity of general mental ability and prior deviance in predicting criminal ODD. This compendium of studies provides an initial empirical investigation into ODD and offers implications relevant to the deviance literature, policy development, and personnel selection. (c) 2016 APA, all rights reserved).

Do candidate reactions relate to job performance or affect criterion-related validity? A multistudy investigation of relations among reactions, selection test scores, and job performance.

PubMed

McCarthy, Julie M; Van Iddekinge, Chad H; Lievens, Filip; Kung, Mei-Chuan; Sinar, Evan F; Campion, Michael A

2013-09-01

Considerable evidence suggests that how candidates react to selection procedures can affect their test performance and their attitudes toward the hiring organization (e.g., recommending the firm to others). However, very few studies of candidate reactions have examined one of the outcomes organizations care most about: job performance. We attempt to address this gap by developing and testing a conceptual framework that delineates whether and how candidate reactions might influence job performance. We accomplish this objective using data from 4 studies (total N = 6,480), 6 selection procedures (personality tests, job knowledge tests, cognitive ability tests, work samples, situational judgment tests, and a selection inventory), 5 key candidate reactions (anxiety, motivation, belief in tests, self-efficacy, and procedural justice), 2 contexts (industry and education), 3 continents (North America, South America, and Europe), 2 study designs (predictive and concurrent), and 4 occupational areas (medical, sales, customer service, and technological). Consistent with previous research, candidate reactions were related to test scores, and test scores were related to job performance. Further, there was some evidence that reactions affected performance indirectly through their influence on test scores. Finally, in no cases did candidate reactions affect the prediction of job performance by increasing or decreasing the criterion-related validity of test scores. Implications of these findings and avenues for future research are discussed. PsycINFO Database Record (c) 2013 APA, all rights reserved

Is the public being protected? Prevention of suboptimal medical practice through training programs and credentialing examinations.

PubMed

Tamblyn, R

1994-06-01

Governments have traditionally looked to the medical profession for leadership in health planning and have charged the profession with the responsibility of establishing and monitoring standards of medical practice. Training program accreditation and licensure/certification exams have been used as the primary methods of preventing unqualified individuals from entering medical practice. Despite the critical nature of the decision made at the time of licensure/certification, there is no information about the validity of these examinations for predicting subsequent practice and health outcome. In this article, the assumptions implicit in the current use of licensing/certifying examinations are identified, the relevant evidence is reviewed, and the implications of this evidence for current methods of measurement are discussed.

A computer case definition for sudden cardiac death.

PubMed

Chung, Cecilia P; Murray, Katherine T; Stein, C Michael; Hall, Kathi; Ray, Wayne A

2010-06-01

To facilitate studies of medications and sudden cardiac death, we developed and validated a computer case definition for these deaths. The study of community dwelling Tennessee Medicaid enrollees 30-74 years of age utilized a linked database with Medicaid inpatient/outpatient files, state death certificate files, and a state 'all-payers' hospital discharge file. The computerized case definition was developed from a retrospective cohort study of sudden cardiac deaths occurring between 1990 and 1993. Medical records for 926 potential cases had been adjudicated for this study to determine if they met the clinical definition for sudden cardiac death occurring in the community and were likely to be due to ventricular tachyarrhythmias. The computerized case definition included deaths with (1) no evidence of a terminal hospital admission/nursing home stay in any of the data sources; (2) an underlying cause of death code consistent with sudden cardiac death; and (3) no terminal procedures inconsistent with unresuscitated cardiac arrest. This definition was validated in an independent sample of 174 adjudicated deaths occurring between 1994 and 2005. The positive predictive value of the computer case definition was 86.0% in the development sample and 86.8% in the validation sample. The positive predictive value did not vary materially for deaths coded according to the ICO-9 (1994-1998, positive predictive value = 85.1%) or ICD-10 (1999-2005, 87.4%) systems. A computerized Medicaid database, linked with death certificate files and a state hospital discharge database, can be used for a computer case definition of sudden cardiac death. Copyright (c) 2009 John Wiley & Sons, Ltd.

In-vivo detectability index: development and validation of an automated methodology

NASA Astrophysics Data System (ADS)

Smith, Taylor Brunton; Solomon, Justin; Samei, Ehsan

2017-03-01

The purpose of this study was to develop and validate a method to estimate patient-specific detectability indices directly from patients' CT images (i.e., "in vivo"). The method works by automatically extracting noise (NPS) and resolution (MTF) properties from each patient's CT series based on previously validated techniques. Patient images are thresholded into skin-air interfaces to form edge-spread functions, which are further binned, differentiated, and Fourier transformed to form the MTF. The NPS is likewise estimated from uniform areas of the image. These are combined with assumed task functions (reference function: 10 mm disk lesion with contrast of -15 HU) to compute detectability indices for a non-prewhitening matched filter model observer predicting observer performance. The results were compared to those from a previous human detection study on 105 subtle, hypo-attenuating liver lesions, using a two-alternative-forcedchoice (2AFC) method, over 6 dose levels using 16 readers. The in vivo detectability indices estimated for all patient images were compared to binary 2AFC outcomes with a generalized linear mixed-effects statistical model (Probit link function, linear terms only, no interactions, random term for readers). The model showed that the in vivo detectability indices were strongly predictive of 2AFC outcomes (P < 0.05). A linear comparison between the human detection accuracy and model-predicted detection accuracy (for like conditions) resulted in Pearson and Spearman correlations coefficients of 0.86 and 0.87, respectively. These data provide evidence that the in vivo detectability index could potentially be used to automatically estimate and track image quality in a clinical operation.

Quantitative structure-retention relationships of polycyclic aromatic hydrocarbons gas-chromatographic retention indices.

PubMed

Drosos, Juan Carlos; Viola-Rhenals, Maricela; Vivas-Reyes, Ricardo

2010-06-25

Polycyclic aromatic compounds (PAHs) are of concern in environmental chemistry and toxicology. In the present work, a QSRR study was performed for 209 previously reported PAHs using quantum mechanics and other sources descriptors estimated by different approaches. The B3LYP/6-31G* level of theory was used for geometrical optimization and quantum mechanics related variables. A good linear relationship between gas-chromatographic retention index and electronic or topologic descriptors was found by stepwise linear regression analysis. The molecular polarizability (alpha) and the second order molecular connectivity Kier and Hall index ((2)chi) showed evidence of significant correlation with retention index by means of important squared coefficient of determination, (R(2)), values (R(2)=0.950 and 0.962, respectively). A one variable QSRR model is presented for each descriptor and both models demonstrates a significant predictive capacity established using the leave-many-out LMO (excluding 25% of rows) cross validation method's q(2) cross-validation coefficients q(2)(CV-LMO25%), (obtained q(2)(CV-LMO25%) 0.947 and 0.960, respectively). Furthermore, the physicochemical interpretation of selected descriptors allowed detailed explanation of the source of the observed statistical correlation. The model analysis suggests that only one descriptor is sufficient to establish a consistent retention index-structure relationship. Moderate or non-significant improve was observed for quantitative results or statistical validation parameters when introducing more terms in predictive equation. The one parameter QSRR proposed model offers a consistent scheme to predict chromatographic properties of PAHs compounds. Copyright 2010 Elsevier B.V. All rights reserved.

Evaluating shortened versions of the AUDIT as screeners for alcohol use problems in a general population study

PubMed Central

Nayak, Madhabika B.; Bond, Jason C.; Greenfield, Thomas K.

2015-01-01

Background Efficient alcohol screening measures are important to prevent or treat alcohol use disorders (AUDs). Objectives We studied different versions of the Alcohol Use Disorders Identification Test (AUDIT) comparing their performance to the full AUDIT and an AUD measure as screeners for alcohol use problems in Goa, India. Methods Data from a general population study on 743 male drinkers aged 18 to 49 years are reported. Drinkers completed the AUDIT and an AUD measure. We created shorter versions of the AUDIT by a) collapsing AUDIT item responses into 3 and 2 categories and b) deleting 2 items with the lowest factor loadings. Each version was evaluated using factor, reliability and validity, and differential item functioning (DIF) analysis by age, education, standard of living index (SLI), and area of residence. Results A single factor solution was found for each version with lower factor loadings for items on guilt and concern. There were no significant differences among the different AUDIT versions in predicting AUD. No significant DIF was found by education, SLI or area of residence. DIF was observed for the alcohol frequency item by age. Conclusions/Importance The AUDIT may be used with dichotomized response options without loss of predictive validity. A shortened 8-item dichotomized scale can adequately screen for AUDs in Goa when brevity is of paramount importance, although with lower predictive validity. Although the frequency item was endorsed more by older men, there is no evidence that the AUDIT items perform differently in other groups of male drinkers in Goa. PMID:26549791

Effects of people-centred factors on enterprise resource planning implementation project success: empirical evidence from Sri Lanka

NASA Astrophysics Data System (ADS)

Wickramasinghe, Vathsala; Gunawardena, Vathsala

2010-08-01

Extant literature suggests people-centred factors as one of the major areas influencing enterprise resource planning (ERP) implementation project success. Yet, to date, few empirical studies attempted to validate the link between people-centred factors and ERP implementation project success. The purpose of this study is to empirically identify people-centred factors that are critical to ERP implementation projects in Sri Lanka. The study develops and empirically validates a framework for people-centred factors that influence the success of ERP implementation projects. Survey research methodology was used and collected data from 74 ERP implementation projects in Sri Lanka. The people-centred factors of 'project team competence', 'rewards' and 'communication and change' were found to predict significantly the ERP implementation project success.

Problem-solving style and multicultural personality dispositions: a study of construct validity.

PubMed

Houtz, John C; Ponterotto, Joseph G; Burger, Claudia; Marino, Cherylynn

2010-06-01

This exploratory study examined the relationship between problem-solving styles and multicultural personality dispositions among 91 graduate students enrolled in an urban university located in the northeast United States. Problem-solving style was assessed with the three dimensions of the VIEW: an Assessment of Problem Solving Style. Multicultural personality was assessed with the five-factor Multicultural Personality Questionnaire (MPQ); its factors of Cultural Empathy, Open-mindedness, Social Initiative, and Flexibility correlated significantly with Explorer and External problem-solving styles, as predicted. The Emotional Stability subscale also correlated significantly with scores on Explorer style, suggesting that individuals who prefer "thinking in new directions" in problem solving are more likely to report remaining calm under stressful situations. Collectively, study results provided additional evidence of construct validity for the VIEW.

Life beyond MSE and R2 — improving validation of predictive models with observations

NASA Astrophysics Data System (ADS)

Papritz, Andreas; Nussbaum, Madlene

2017-04-01

Machine learning and statistical predictive methods are evaluated by the closeness of predictions to observations of a test dataset. Common criteria for rating predictive methods are bias and mean square error (MSE), characterizing systematic and random prediction errors. Many studies also report R2-values, but their meaning is not always clear (correlation between observations and predictions or MSE skill score; Wilks, 2011). The same criteria are also used for choosing tuning parameters of predictive procedures by cross-validation and bagging (e.g. Hastie et al., 2009). For evident reasons, atmospheric sciences have developed a rich box of tools for forecast verification. Specific criteria have been proposed for evaluating deterministic and probabilistic predictions of binary, multinomial, ordinal and continuous responses (see reviews by Wilks, 2011, Jollie and Stephenson, 2012 and Gneiting et al., 2007). It appears that these techniques are not very well-known in the geosciences community interested in machine learning. In our presentation we review techniques that offer more insight into proximity of data and predictions than bias, MSE and R2 alone. We mention here only examples: (i) Graphing observations vs. predictions is usually more appropriate than the reverse (Piñeiro et al., 2008). (ii) The decomposition of the Brier score score (= MSE for probabilistic predictions of binary yes/no data) into reliability and resolution reveals (conditional) bias and capability of discriminating yes/no observations by the predictions. We illustrate the approaches by applications from digital soil mapping studies. Gneiting, T., Balabdaoui, F., and Raftery, A. E. (2007). Probabilistic forecasts, calibration and sharpness. Journal of the Royal Statistical Society Series B, 69, 243-268. Hastie, T., Tibshirani, R., and Friedman, J. (2009). The Elements of Statistical Learning; Data Mining, Inference and Prediction. Springer, New York, second edition. Jolliffe, I. T. and Stephenson, D. B., editors (2012). Forecast Verification: A Practitioner's Guide in Atmospheric Science. Wiley-Blackwell, second edition. Piñeiro, G., Perelman, S., Guerschman, J., and Paruelo, J. (2008). How to evaluate models: Observed vs. predicted or predicted vs. observed? Ecological Modelling, 216, 316-322. Wilks, D. S. (2011). Statistical Methods in the Atmospheric Sciences. Academic Press, third edition.

The validity of the Brain Injury Cognitive Screen (BICS) as a neuropsychological screening assessment for traumatic and non-traumatic brain injury.

PubMed

Vaughan, Frances L; Neal, Jo Anne; Mulla, Farzana Nizam; Edwards, Barbara; Coetzer, Rudi

2017-04-01

The Brain Injury Cognitive Screen (BICS) was developed as an in-service cognitive assessment battery for acquired brain injury patients entering community rehabilitation. The BICS focuses on domains that are particularly compromised following TBI, and provides a broader and more detailed assessment of executive function, attention and information processing than comparable screening assessments. The BICS also includes brief assessments of perception, naming, and construction, which were predicted to be more sensitive to impairments following non-traumatic brain injury. The studies reported here examine preliminary evidence for its validity in post-acute rehabilitation. In Study 1, TBI patients completed the BICS and were compared with matched controls. Patients with focal lesions and matched controls were compared in Study 2. Study 3 examined demographic effects in a sample of normative data. TBI and focal lesion patients obtained significantly lower composite memory, executive function and attention and information processing BICS scores than healthy controls. Injury severity effects were also obtained. Logistic regression analyses indicated that each group of BICS memory, executive function and attention measures reliably differentiated TBI and focal lesion participants from controls. Design Recall, Prospective Memory, Verbal Fluency, and Visual Search test scores showed significant independent regression effects. Other subtest measures showed evidence of sensitivity to brain injury. The study provides preliminary evidence of the BICS' sensitivity to cognitive impairment caused by acquired brain injury, and its potential clinical utility as a cognitive screen. Further validation based on a revised version of the BICS and more normative data are required.

Improved statistical signal detection in pharmacovigilance by combining multiple strength-of-evidence aspects in vigiRank.

PubMed

Caster, Ola; Juhlin, Kristina; Watson, Sarah; Norén, G Niklas

2014-08-01

Detection of unknown risks with marketed medicines is key to securing the optimal care of individual patients and to reducing the societal burden from adverse drug reactions. Large collections of individual case reports remain the primary source of information and require effective analytics to guide clinical assessors towards likely drug safety signals. Disproportionality analysis is based solely on aggregate numbers of reports and naively disregards report quality and content. However, these latter features are the very fundament of the ensuing clinical assessment. Our objective was to develop and evaluate a data-driven screening algorithm for emerging drug safety signals that accounts for report quality and content. vigiRank is a predictive model for emerging safety signals, here implemented with shrinkage logistic regression to identify predictive variables and estimate their respective contributions. The variables considered for inclusion capture different aspects of strength of evidence, including quality and clinical content of individual reports, as well as trends in time and geographic spread. A reference set of 264 positive controls (historical safety signals from 2003 to 2007) and 5,280 negative controls (pairs of drugs and adverse events not listed in the Summary of Product Characteristics of that drug in 2012) was used for model fitting and evaluation; the latter used fivefold cross-validation to protect against over-fitting. All analyses were performed on a reconstructed version of VigiBase(®) as of 31 December 2004, at around which time most safety signals in our reference set were emerging. The following aspects of strength of evidence were selected for inclusion into vigiRank: the numbers of informative and recent reports, respectively; disproportional reporting; the number of reports with free-text descriptions of the case; and the geographic spread of reporting. vigiRank offered a statistically significant improvement in area under the receiver operating characteristics curve (AUC) over screening based on the Information Component (IC) and raw numbers of reports, respectively (0.775 vs. 0.736 and 0.707, cross-validated). Accounting for multiple aspects of strength of evidence has clear conceptual and empirical advantages over disproportionality analysis. vigiRank is a first-of-its-kind predictive model to factor in report quality and content in first-pass screening to better meet tomorrow's post-marketing drug safety surveillance needs.

Improving medical decisions for incapacitated persons: does focusing on "accurate predictions" lead to an inaccurate picture?

PubMed

Kim, Scott Y H

2014-04-01

The Patient Preference Predictor (PPP) proposal places a high priority on the accuracy of predicting patients' preferences and finds the performance of surrogates inadequate. However, the quest to develop a highly accurate, individualized statistical model has significant obstacles. First, it will be impossible to validate the PPP beyond the limit imposed by 60%-80% reliability of people's preferences for future medical decisions--a figure no better than the known average accuracy of surrogates. Second, evidence supports the view that a sizable minority of persons may not even have preferences to predict. Third, many, perhaps most, people express their autonomy just as much by entrusting their loved ones to exercise their judgment than by desiring to specifically control future decisions. Surrogate decision making faces none of these issues and, in fact, it may be more efficient, accurate, and authoritative than is commonly assumed.

Entraining IDyOT: Timing in the Information Dynamics of Thinking

PubMed Central

Forth, Jamie; Agres, Kat; Purver, Matthew; Wiggins, Geraint A.

2016-01-01

We present a novel hypothetical account of entrainment in music and language, in context of the Information Dynamics of Thinking model, IDyOT. The extended model affords an alternative view of entrainment, and its companion term, pulse, from earlier accounts. The model is based on hierarchical, statistical prediction, modeling expectations of both what an event will be and when it will happen. As such, it constitutes a kind of predictive coding, with a particular novel hypothetical implementation. Here, we focus on the model's mechanism for predicting when a perceptual event will happen, given an existing sequence of past events, which may be musical or linguistic. We propose a range of tests to validate or falsify the model, at various different levels of abstraction, and argue that computational modeling in general, and this model in particular, can offer a means of providing limited but useful evidence for evolutionary hypotheses. PMID:27803682

Reconciled rat and human metabolic networks for comparative toxicogenomics and biomarker predictions

PubMed Central

Blais, Edik M.; Rawls, Kristopher D.; Dougherty, Bonnie V.; Li, Zhuo I.; Kolling, Glynis L.; Ye, Ping; Wallqvist, Anders; Papin, Jason A.

2017-01-01

The laboratory rat has been used as a surrogate to study human biology for more than a century. Here we present the first genome-scale network reconstruction of Rattus norvegicus metabolism, iRno, and a significantly improved reconstruction of human metabolism, iHsa. These curated models comprehensively capture metabolic features known to distinguish rats from humans including vitamin C and bile acid synthesis pathways. After reconciling network differences between iRno and iHsa, we integrate toxicogenomics data from rat and human hepatocytes, to generate biomarker predictions in response to 76 drugs. We validate comparative predictions for xanthine derivatives with new experimental data and literature-based evidence delineating metabolite biomarkers unique to humans. Our results provide mechanistic insights into species-specific metabolism and facilitate the selection of biomarkers consistent with rat and human biology. These models can serve as powerful computational platforms for contextualizing experimental data and making functional predictions for clinical and basic science applications. PMID:28176778

Exploiting proteomic data for genome annotation and gene model validation in Aspergillus niger.

PubMed

Wright, James C; Sugden, Deana; Francis-McIntyre, Sue; Riba-Garcia, Isabel; Gaskell, Simon J; Grigoriev, Igor V; Baker, Scott E; Beynon, Robert J; Hubbard, Simon J

2009-02-04

Proteomic data is a potentially rich, but arguably unexploited, data source for genome annotation. Peptide identifications from tandem mass spectrometry provide prima facie evidence for gene predictions and can discriminate over a set of candidate gene models. Here we apply this to the recently sequenced Aspergillus niger fungal genome from the Joint Genome Institutes (JGI) and another predicted protein set from another A.niger sequence. Tandem mass spectra (MS/MS) were acquired from 1d gel electrophoresis bands and searched against all available gene models using Average Peptide Scoring (APS) and reverse database searching to produce confident identifications at an acceptable false discovery rate (FDR). 405 identified peptide sequences were mapped to 214 different A.niger genomic loci to which 4093 predicted gene models clustered, 2872 of which contained the mapped peptides. Interestingly, 13 (6%) of these loci either had no preferred predicted gene model or the genome annotators' chosen "best" model for that genomic locus was not found to be the most parsimonious match to the identified peptides. The peptides identified also boosted confidence in predicted gene structures spanning 54 introns from different gene models. This work highlights the potential of integrating experimental proteomics data into genomic annotation pipelines much as expressed sequence tag (EST) data has been. A comparison of the published genome from another strain of A.niger sequenced by DSM showed that a number of the gene models or proteins with proteomics evidence did not occur in both genomes, further highlighting the utility of the method.

Multifactorial disease risk calculator: Risk prediction for multifactorial disease pedigrees.

PubMed

Campbell, Desmond D; Li, Yiming; Sham, Pak C

2018-03-01

Construction of multifactorial disease models from epidemiological findings and their application to disease pedigrees for risk prediction is nontrivial for all but the simplest of cases. Multifactorial Disease Risk Calculator is a web tool facilitating this. It provides a user-friendly interface, extending a reported methodology based on a liability-threshold model. Multifactorial disease models incorporating all the following features in combination are handled: quantitative risk factors (including polygenic scores), categorical risk factors (including major genetic risk loci), stratified age of onset curves, and the partition of the population variance in disease liability into genetic, shared, and unique environment effects. It allows the application of such models to disease pedigrees. Pedigree-related outputs are (i) individual disease risk for pedigree members, (ii) n year risk for unaffected pedigree members, and (iii) the disease pedigree's joint liability distribution. Risk prediction for each pedigree member is based on using the constructed disease model to appropriately weigh evidence on disease risk available from personal attributes and family history. Evidence is used to construct the disease pedigree's joint liability distribution. From this, lifetime and n year risk can be predicted. Example disease models and pedigrees are provided at the website and are used in accompanying tutorials to illustrate the features available. The website is built on an R package which provides the functionality for pedigree validation, disease model construction, and risk prediction. Website: http://grass.cgs.hku.hk:3838/mdrc/current. © 2017 WILEY PERIODICALS, INC.

Reconsidering "evidence" for fast-and-frugal heuristics.

PubMed

Hilbig, Benjamin E

2010-12-01

In several recent reviews, authors have argued for the pervasive use of fast-and-frugal heuristics in human judgment. They have provided an overview of heuristics and have reiterated findings corroborating that such heuristics can be very valid strategies leading to high accuracy. They also have reviewed previous work that implies that simple heuristics are actually used by decision makers. Unfortunately, concerning the latter point, these reviews appear to be somewhat incomplete. More important, previous conclusions have been derived from investigations that bear some noteworthy methodological limitations. I demonstrate these by proposing a new heuristic and provide some novel critical findings. Also, I review some of the relevant literature often not-or only partially-considered. Overall, although some fast-and-frugal heuristics indeed seem to predict behavior at times, there is little to no evidence for others. More generally, the empirical evidence available does not warrant the conclusion that heuristics are pervasively used.

Teachers' Grade Assignment and the Predictive Validity of Criterion-Referenced Grades

ERIC Educational Resources Information Center

Thorsen, Cecilia; Cliffordson, Christina

2012-01-01

Research has found that grades are the most valid instruments for predicting educational success. Why grades have better predictive validity than, for example, standardized tests is not yet fully understood. One possible explanation is that grades reflect not only subject-specific knowledge and skills but also individual differences in other…

How the Brain Decides When to Work and When to Rest: Dissociation of Implicit-Reactive from Explicit-Predictive Computational Processes

PubMed Central

Meyniel, Florent; Safra, Lou; Pessiglione, Mathias

2014-01-01

A pervasive case of cost-benefit problem is how to allocate effort over time, i.e. deciding when to work and when to rest. An economic decision perspective would suggest that duration of effort is determined beforehand, depending on expected costs and benefits. However, the literature on exercise performance emphasizes that decisions are made on the fly, depending on physiological variables. Here, we propose and validate a general model of effort allocation that integrates these two views. In this model, a single variable, termed cost evidence, accumulates during effort and dissipates during rest, triggering effort cessation and resumption when reaching bounds. We assumed that such a basic mechanism could explain implicit adaptation, whereas the latent parameters (slopes and bounds) could be amenable to explicit anticipation. A series of behavioral experiments manipulating effort duration and difficulty was conducted in a total of 121 healthy humans to dissociate implicit-reactive from explicit-predictive computations. Results show 1) that effort and rest durations are adapted on the fly to variations in cost-evidence level, 2) that the cost-evidence fluctuations driving the behavior do not match explicit ratings of exhaustion, and 3) that actual difficulty impacts effort duration whereas expected difficulty impacts rest duration. Taken together, our findings suggest that cost evidence is implicitly monitored online, with an accumulation rate proportional to actual task difficulty. In contrast, cost-evidence bounds and dissipation rate might be adjusted in anticipation, depending on explicit task difficulty. PMID:24743711

Diagnostic accuracy of criteria for urinary tract infection in a cohort of nursing home residents.

PubMed

Juthani-Mehta, Manisha; Tinetti, Mary; Perrelli, Eleanor; Towle, Virginia; Van Ness, Peter H; Quagliarello, Vincent

2007-07-01

To prospectively evaluate nursing home residents with suspected urinary tract infection (UTI) to determine whether they met the McGeer, Loeb, or revised Loeb consensus-based criteria and whether any set of criteria was associated with laboratory evidence of UTI, namely bacteriuria (>100,000 colony forming units) plus pyuria (>10 white blood cells). Prospective cohort study. Three New Haven-area nursing homes. Of 611 residents screened, 457 were eligible, 362 consented, and 340 enrolled. Participants underwent prospective surveillance from May 2005 to April 2006 for the development of suspected UTI (defined as a participant's physician or nurse clinically suspecting UTI). One hundred participants with suspected UTI and a urinalysis and urine culture performed were included in the analyses. Participants were identified who met the criteria of McGeer, Loeb, revised Loeb, and laboratory evidence of UTI. Using laboratory evidence of UTI as the outcome, the McGeer criteria demonstrated 30% sensitivity, 82% specificity, 57% positive predictive value (PPV), and 61% negative predictive value (NPV); the Loeb criteria showed 19% sensitivity, 89% specificity, 57% PPV, and 59% NPV; and the revised Loeb criteria demonstrated 30% sensitivity, 79% specificity, 52% PPV, and 60% NPV. All of the consensus-based criteria have similar test characteristics. The diagnostic accuracy of UTI criteria in nursing home residents could be improved, and the data suggest that evidence-based clinical criteria associated with laboratory evidence of UTI need to be identified and validated.

A clinical test of stepping and change of direction to identify multiple falling older adults.

PubMed

Dite, Wayne; Temple, Viviene A

2002-11-01

To establish the reliability and validity of a new clinical test of dynamic standing balance, the Four Square Step Test (FSST), to evaluate its sensitivity, specificity, and predictive value in identifying subjects who fall, and to compare it with 3 established balance and mobility tests. A 3-group comparison performed by using 3 validated tests and 1 new test. A rehabilitation center and university medical school in Australia. Eighty-one community-dwelling adults over the age of 65 years. Subjects were age- and gender-matched to form 3 groups: multiple fallers, nonmultiple fallers, and healthy comparisons. Not applicable. Time to complete the FSST and Timed Up and Go test and the number of steps to complete the Step Test and Functional Reach Test distance. High reliability was found for interrater (n=30, intraclass correlation coefficient [ICC]=.99) and retest reliability (n=20, ICC=.98). Evidence for validity was found through correlation with other existing balance tests. Validity was supported, with the FSST showing significantly better performance scores (P<.01) for each of the healthier and less impaired groups. The FSST also revealed a sensitivity of 85%, a specificity of 88% to 100%, and a positive predictive value of 86%. As a clinical test, the FSST is reliable, valid, easy to score, quick to administer, requires little space, and needs no special equipment. It is unique in that it involves stepping over low objects (2.5cm) and movement in 4 directions. The FSST had higher combined sensitivity and specificity for identifying differences between groups in the selected sample population of older adults than the 3 tests with which it was compared. Copyright 2002 by the American Congress of Rehabilitation Medicine and the American Academy of Physical Medicine and Rehabilitation

Validation of the conceptual research utilization scale: an application of the standards for educational and psychological testing in healthcare.

PubMed

Squires, Janet E; Estabrooks, Carole A; Newburn-Cook, Christine V; Gierl, Mark

2011-05-19

There is a lack of acceptable, reliable, and valid survey instruments to measure conceptual research utilization (CRU). In this study, we investigated the psychometric properties of a newly developed scale (the CRU Scale). We used the Standards for Educational and Psychological Testing as a validation framework to assess four sources of validity evidence: content, response processes, internal structure, and relations to other variables. A panel of nine international research utilization experts performed a formal content validity assessment. To determine response process validity, we conducted a series of one-on-one scale administration sessions with 10 healthcare aides. Internal structure and relations to other variables validity was examined using CRU Scale response data from a sample of 707 healthcare aides working in 30 urban Canadian nursing homes. Principal components analysis and confirmatory factor analyses were conducted to determine internal structure. Relations to other variables were examined using: (1) bivariate correlations; (2) change in mean values of CRU with increasing levels of other kinds of research utilization; and (3) multivariate linear regression. Content validity index scores for the five items ranged from 0.55 to 1.00. The principal components analysis predicted a 5-item 1-factor model. This was inconsistent with the findings from the confirmatory factor analysis, which showed best fit for a 4-item 1-factor model. Bivariate associations between CRU and other kinds of research utilization were statistically significant (p < 0.01) for the latent CRU scale score and all five CRU items. The CRU scale score was also shown to be significant predictor of overall research utilization in multivariate linear regression. The CRU scale showed acceptable initial psychometric properties with respect to responses from healthcare aides in nursing homes. Based on our validity, reliability, and acceptability analyses, we recommend using a reduced (four-item) version of the CRU scale to yield sound assessments of CRU by healthcare aides. Refinement to the wording of one item is also needed. Planned future research will include: latent scale scoring, identification of variables that predict and are outcomes to conceptual research use, and longitudinal work to determine CRU Scale sensitivity to change.

Measuring Black Men’s Police-Based Discrimination Experiences: Development and Validation of the Police and Law Enforcement (PLE) Scale

PubMed Central

English, Devin; Bowleg, Lisa; del Río-González, Ana Maria; Tschann, Jeanne M.; Agans, Robert; Malebranche, David J

2017-01-01

Objectives Although social science research has examined police and law enforcement-perpetrated discrimination against Black men using policing statistics and implicit bias studies, there is little quantitative evidence detailing this phenomenon from the perspective of Black men. Consequently, there is a dearth of research detailing how Black men’s perspectives on police and law enforcement-related stress predict negative physiological and psychological health outcomes. This study addresses these gaps with the qualitative development and quantitative test of the Police and Law Enforcement (PLE) scale. Methods In Study 1, we employed thematic analysis on transcripts of individual qualitative interviews with 90 Black men to assess key themes and concepts and develop quantitative items. In Study 2, we used 2 focus groups comprised of 5 Black men each (n=10), intensive cognitive interviewing with a separate sample of Black men (n=15), and piloting with another sample of Black men (n=13) to assess the ecological validity of the quantitative items. For study 3, we analyzed data from a sample of 633 Black men between the ages of 18 and 65 to test the factor structure of the PLE, as we all as its concurrent validity and convergent/discriminant validity. Results Qualitative analyses and confirmatory factor analyses suggested that a 5-item, 1-factor measure appropriately represented respondents’ experiences of police/law enforcement discrimination. As hypothesized, the PLE was positively associated with measures of racial discrimination and depressive symptoms. Conclusions Preliminary evidence suggests that the PLE is a reliable and valid measure of Black men’s experiences of discrimination with police/law enforcement. PMID:28080104

Screening emergency department patients for opioid drug use: A qualitative systematic review.

PubMed

Sahota, Preet Kaur; Shastry, Siri; Mukamel, Dana B; Murphy, Linda; Yang, Narisu; Lotfipour, Shahram; Chakravarthy, Bharath

2018-05-24

The opioid drug epidemic is a major public health concern and an economic burden in the United States. The purpose of this systematic review is to assess the reliability and validity of screening instruments used in emergency medicine settings to detect opioid use in patients and to assess psychometric data for each screening instrument. PubMed/MEDLINE, PsycINFO, Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, Web of Science, Cumulative Index to Nursing and Allied Health Literature and ClinicalTrials.gov were searched for articles published up to May 2018. The extracted articles were independently screened for eligibility by two reviewers. We extracted 1555 articles for initial screening and 95 articles were assessed for full-text eligibility. Six articles were extracted from the full-text assessment. Six instruments were identified from the final article list: Screener and Opioid Assessment for Patients with Pain - Revised; Drug Abuse Screening Test; Opioid Risk Tool; Current Opioid Misuse Measure; an Emergency Medicine Providers Clinician Assessment Questionnaire; and an Emergency Provider Impression Data Collection Form. Screening instrument characteristics, and reliability and validity data were extracted from the six studies. A meta-analysis was not conducted due to heterogeneity between the studies. There is a lack of validity and reliability evidence in all six articles; and sensitivity, specificity and predictive values varied between the different instruments. These instruments cannot be validated for use in emergency medicine settings. There is no clear evidence to state which screening instruments are appropriate for use in detecting opioid use disorders in emergency medicine patients. There is a need for brief, reliable, valid and feasible opioid use screening instruments in the emergency medicine setting. Copyright © 2018 Elsevier Ltd. All rights reserved.

Measuring Black men's police-based discrimination experiences: Development and validation of the Police and Law Enforcement (PLE) Scale.

PubMed

English, Devin; Bowleg, Lisa; Del Río-González, Ana Maria; Tschann, Jeanne M; Agans, Robert P; Malebranche, David J

2017-04-01

Although social science research has examined police and law enforcement-perpetrated discrimination against Black men using policing statistics and implicit bias studies, there is little quantitative evidence detailing this phenomenon from the perspective of Black men. Consequently, there is a dearth of research detailing how Black men's perspectives on police and law enforcement-related stress predict negative physiological and psychological health outcomes. This study addresses these gaps with the qualitative development and quantitative test of the Police and Law Enforcement (PLE) Scale. In Study 1, we used thematic analysis on transcripts of individual qualitative interviews with 90 Black men to assess key themes and concepts and develop quantitative items. In Study 2, we used 2 focus groups comprised of 5 Black men each (n = 10), intensive cognitive interviewing with a separate sample of Black men (n = 15), and piloting with another sample of Black men (n = 13) to assess the ecological validity of the quantitative items. For Study 3, we analyzed data from a sample of 633 Black men between the ages of 18 and 65 to test the factor structure of the PLE, as we all as its concurrent validity and convergent/discriminant validity. Qualitative analyses and confirmatory factor analyses suggested that a 5-item, 1-factor measure appropriately represented respondents' experiences of police/law enforcement discrimination. As hypothesized, the PLE was positively associated with measures of racial discrimination and depressive symptoms. Preliminary evidence suggests that the PLE is a reliable and valid measure of Black men's experiences of discrimination with police/law enforcement. (PsycINFO Database Record (c) 2017 APA, all rights reserved).

Applications of machine learning in cancer prediction and prognosis.

PubMed

Cruz, Joseph A; Wishart, David S

2007-02-11

Machine learning is a branch of artificial intelligence that employs a variety of statistical, probabilistic and optimization techniques that allows computers to "learn" from past examples and to detect hard-to-discern patterns from large, noisy or complex data sets. This capability is particularly well-suited to medical applications, especially those that depend on complex proteomic and genomic measurements. As a result, machine learning is frequently used in cancer diagnosis and detection. More recently machine learning has been applied to cancer prognosis and prediction. This latter approach is particularly interesting as it is part of a growing trend towards personalized, predictive medicine. In assembling this review we conducted a broad survey of the different types of machine learning methods being used, the types of data being integrated and the performance of these methods in cancer prediction and prognosis. A number of trends are noted, including a growing dependence on protein biomarkers and microarray data, a strong bias towards applications in prostate and breast cancer, and a heavy reliance on "older" technologies such artificial neural networks (ANNs) instead of more recently developed or more easily interpretable machine learning methods. A number of published studies also appear to lack an appropriate level of validation or testing. Among the better designed and validated studies it is clear that machine learning methods can be used to substantially (15-25%) improve the accuracy of predicting cancer susceptibility, recurrence and mortality. At a more fundamental level, it is also evident that machine learning is also helping to improve our basic understanding of cancer development and progression.

Applications of Machine Learning in Cancer Prediction and Prognosis

PubMed Central

Cruz, Joseph A.; Wishart, David S.

2006-01-01

Machine learning is a branch of artificial intelligence that employs a variety of statistical, probabilistic and optimization techniques that allows computers to “learn” from past examples and to detect hard-to-discern patterns from large, noisy or complex data sets. This capability is particularly well-suited to medical applications, especially those that depend on complex proteomic and genomic measurements. As a result, machine learning is frequently used in cancer diagnosis and detection. More recently machine learning has been applied to cancer prognosis and prediction. This latter approach is particularly interesting as it is part of a growing trend towards personalized, predictive medicine. In assembling this review we conducted a broad survey of the different types of machine learning methods being used, the types of data being integrated and the performance of these methods in cancer prediction and prognosis. A number of trends are noted, including a growing dependence on protein biomarkers and microarray data, a strong bias towards applications in prostate and breast cancer, and a heavy reliance on “older” technologies such artificial neural networks (ANNs) instead of more recently developed or more easily interpretable machine learning methods. A number of published studies also appear to lack an appropriate level of validation or testing. Among the better designed and validated studies it is clear that machine learning methods can be used to substantially (15–25%) improve the accuracy of predicting cancer susceptibility, recurrence and mortality. At a more fundamental level, it is also evident that machine learning is also helping to improve our basic understanding of cancer development and progression. PMID:19458758

LMethyR-SVM: Predict Human Enhancers Using Low Methylated Regions based on Weighted Support Vector Machines.

PubMed

Xu, Jingting; Hu, Hong; Dai, Yang

The identification of enhancers is a challenging task. Various types of epigenetic information including histone modification have been utilized in the construction of enhancer prediction models based on a diverse panel of machine learning schemes. However, DNA methylation profiles generated from the whole genome bisulfite sequencing (WGBS) have not been fully explored for their potential in enhancer prediction despite the fact that low methylated regions (LMRs) have been implied to be distal active regulatory regions. In this work, we propose a prediction framework, LMethyR-SVM, using LMRs identified from cell-type-specific WGBS DNA methylation profiles and a weighted support vector machine learning framework. In LMethyR-SVM, the set of cell-type-specific LMRs is further divided into three sets: reliable positive, like positive and likely negative, according to their resemblance to a small set of experimentally validated enhancers in the VISTA database based on an estimated non-parametric density distribution. Then, the prediction model is obtained by solving a weighted support vector machine. We demonstrate the performance of LMethyR-SVM by using the WGBS DNA methylation profiles derived from the human embryonic stem cell type (H1) and the fetal lung fibroblast cell type (IMR90). The predicted enhancers are highly conserved with a reasonable validation rate based on a set of commonly used positive markers including transcription factors, p300 binding and DNase-I hypersensitive sites. In addition, we show evidence that the large fraction of the LMethyR-SVM predicted enhancers are not predicted by ChromHMM in H1 cell type and they are more enriched for the FANTOM5 enhancers. Our work suggests that low methylated regions detected from the WGBS data are useful as complementary resources to histone modification marks in developing models for the prediction of cell-type-specific enhancers.

Validation of learning assessments: A primer.

PubMed

Peeters, Michael J; Martin, Beth A

2017-09-01

The Accreditation Council for Pharmacy Education's Standards 2016 has placed greater emphasis on validating educational assessments. In this paper, we describe validity, reliability, and validation principles, drawing attention to the conceptual change that highlights one validity with multiple evidence sources; to this end, we recommend abandoning historical (confusing) terminology associated with the term validity. Further, we describe and apply Kane's framework (scoring, generalization, extrapolation, and implications) for the process of validation, with its inferences and conclusions from varied uses of assessment instruments by different colleges and schools of pharmacy. We then offer five practical recommendations that can improve reporting of validation evidence in pharmacy education literature. We describe application of these recommendations, including examples of validation evidence in the context of pharmacy education. After reading this article, the reader should be able to understand the current concept of validation, and use a framework as they validate and communicate their own institution's learning assessments. Copyright © 2017 Elsevier Inc. All rights reserved.

The stroke impairment assessment set: its internal consistency and predictive validity.

PubMed

Tsuji, T; Liu, M; Sonoda, S; Domen, K; Chino, N

2000-07-01

To study the scale quality and predictive validity of the Stroke Impairment Assessment Set (SIAS) developed for stroke outcome research. Rasch analysis of the SIAS; stepwise multiple regression analysis to predict discharge functional independence measure (FIM) raw scores from demographic data, the SIAS scores, and the admission FIM scores; cross-validation of the prediction rule. Tertiary rehabilitation center in Japan. One hundred ninety stroke inpatients for the study of the scale quality and the predictive validity; a second sample of 116 stroke inpatients for the cross-validation study. Mean square fit statistics to study the degree of fit to the unidimensional model; logits to express item difficulties; discharge FIM scores for the study of predictive validity. The degree of misfit was acceptable except for the shoulder range of motion (ROM), pain, visuospatial function, and speech items; and the SIAS items could be arranged on a common unidimensional scale. The difficulty patterns were identical at admission and at discharge except for the deep tendon reflexes, ROM, and pain items. They were also similar for the right- and left-sided brain lesion groups except for the speech and visuospatial items. For the prediction of the discharge FIM scores, the independent variables selected were age, the SIAS total scores, and the admission FIM scores; and the adjusted R2 was .64 (p < .0001). Stability of the predictive equation was confirmed in the cross-validation sample (R2 = .68, p < .001). The unidimensionality of the SIAS was confirmed, and the SIAS total scores proved useful for stroke outcome prediction.

Truth and Evidence in Validity Theory

ERIC Educational Resources Information Center

Borsboom, Denny; Markus, Keith A.

2013-01-01

According to Kane (this issue), "the validity of a proposed interpretation or use depends on how well the evidence supports" the claims being made. Because truth and evidence are distinct, this means that the validity of a test score interpretation could be high even though the interpretation is false. As an illustration, we discuss the case of…

Desire thinking as a confounder in the relationship between mindfulness and craving: Evidence from a cross-cultural validation of the Desire Thinking Questionnaire.

PubMed

Chakroun-Baggioni, Nadia; Corman, Maya; Spada, Marcantonio M; Caselli, Gabriele; Gierski, Fabien

2017-10-01

Desire thinking and mindfulness have been associated with craving. The aim of the present study was to validate the French version of the Desire Thinking Questionnaire (DTQ) and to investigate the relationship between mindfulness, desire thinking and craving among a sample of university students. Four hundred and ninety six university students completed the DTQ and measures of mindfulness, craving and alcohol use. Results from confirmatory factor analyses showed that the two-factor structure proposed in the original DTQ exhibited suitable goodness-of-fit statistics. The DTQ also demonstrated good internal reliability, temporal stability and predictive validity. A set of linear regressions revealed that desire thinking had a confounding effect in the relationship between mindfulness and craving. The confounding role of desire thinking in the relationship between mindfulness and craving suggests that interrupting desire thinking may be a viable clinical option aimed at reducing craving. Copyright © 2017 Elsevier B.V. All rights reserved.

Preliminary development of the adolescent students' basic psychological needs at school scale.

PubMed

Tian, Lili; Han, Mengmeng; Huebner, E Scott

2014-04-01

The aim of the present study was to develop and provide evidence for the validity of a new measure of adolescent students' psychological need satisfaction at school, using a sample of Chinese students. We conducted four studies with four independent samples (total n = 1872). The first study aimed to develop items for the new instrument and to ascertain its factorial structure using exploratory factor analysis procedures. The second study aimed to examine the instrument's factorial structure using confirmatory factor analysis procedures as well as to assess its internal consistency reliability, convergent and divergent validity. The third study aimed to assess its measurement invariance across gender and age. The fourth study aimed to test its test-retest reliability over time and predictive validity. These preliminary results showed that the new instrument has promising psychometric properties. The potential contributions of the new instrument for future research and educational practices were discussed. Copyright © 2014 The Foundation for Professionals in Services for Adolescents. Published by Elsevier Ltd. All rights reserved.

Factor structure and clinical correlates of the Food Thought Suppression Inventory within treatment seeking obese women with binge eating disorder

PubMed Central

Barnes, Rachel D.; Sawaoka, Takuya; White, Marney A.; Masheb, Robin M.; Grilo, Carlos M.

2013-01-01

Prior research on the relations among eating behaviors and thought suppression is limited to a measure of general thought suppression, the White Bear Suppression Inventory. To address this limitation, researchers recently validated the Food Thought Suppression Inventory (FTSI). Analyses using this measure suggest that food thought suppression is distinct from and is more predictive of eating disorder psychopathology than is general thought suppression. The FTSI, however, has not yet been validated in clinical samples. The purpose of the current study is to examine the factor structure and clinical correlates of the FTSI within treatment seeking obese women with binge eating disorder (BED; N = 128). Analyses revealed a valid and reliable one-factor measure of food thought suppression that was related to higher levels of eating and general psychopathology. The findings provide evidence for the use of the FTSI with obese women with BED. Future research should examine the psychometric properties of the FTSI within larger and more diverse samples. PMID:23265399

Understanding and Measuring Health Care Insecurity

PubMed Central

Tomsik, Philip E.; Smith, Samantha; Mason, Mary Jane; Zyzanski, Stephen J.; Stange, Kurt C.; Werner, James J.; Flocke, Susan A.

2015-01-01

Purpose To define the concept of “health care insecurity,” validate a new self-report measure, and examine the impact of beginning care at a free clinic on uninsured patients’ health care insecurity. Methods Consecutive new patients presenting at a free clinic completed 15 items assessing domains of health care insecurity (HCI) at their first visit and again four to eight weeks later. Psychometrics and change of the HCI measure were examined. Results The HCI measure was found to have high internal consistency (α=0.94). Evidence of concurrent validity was indicated by negative correlation with VR-12 health-related quality of life physical and mental health components and positive correlation with the Perceived Stress Scale. Predictive validity was shown among the 83% of participants completing follow-up: HCI decreased after beginning care at a free clinic (p<.001). Conclusion Reliably assessing patient experience of health care insecurity is feasible and has potential to inform efforts to improve quality and access to care among underserved populations. PMID:25418245

Prediction of rodent carcinogenic potential of naturally occurring chemicals in the human diet using high-throughput QSAR predictive modeling

DOE Office of Scientific and Technical Information (OSTI.GOV)

Valerio, Luis G.; Arvidson, Kirk B.; Chanderbhan, Ronald F.

2007-07-01

Consistent with the U.S. Food and Drug Administration (FDA) Critical Path Initiative, predictive toxicology software programs employing quantitative structure-activity relationship (QSAR) models are currently under evaluation for regulatory risk assessment and scientific decision support for highly sensitive endpoints such as carcinogenicity, mutagenicity and reproductive toxicity. At the FDA's Center for Food Safety and Applied Nutrition's Office of Food Additive Safety and the Center for Drug Evaluation and Research's Informatics and Computational Safety Analysis Staff (ICSAS), the use of computational SAR tools for both qualitative and quantitative risk assessment applications are being developed and evaluated. One tool of current interest ismore » MDL-QSAR predictive discriminant analysis modeling of rodent carcinogenicity, which has been previously evaluated for pharmaceutical applications by the FDA ICSAS. The study described in this paper aims to evaluate the utility of this software to estimate the carcinogenic potential of small, organic, naturally occurring chemicals found in the human diet. In addition, a group of 19 known synthetic dietary constituents that were positive in rodent carcinogenicity studies served as a control group. In the test group of naturally occurring chemicals, 101 were found to be suitable for predictive modeling using this software's discriminant analysis modeling approach. Predictions performed on these compounds were compared to published experimental evidence of each compound's carcinogenic potential. Experimental evidence included relevant toxicological studies such as rodent cancer bioassays, rodent anti-carcinogenicity studies, genotoxic studies, and the presence of chemical structural alerts. Statistical indices of predictive performance were calculated to assess the utility of the predictive modeling method. Results revealed good predictive performance using this software's rodent carcinogenicity module of over 1200 chemicals, comprised primarily of pharmaceutical, industrial and some natural products developed under an FDA-MDL cooperative research and development agreement (CRADA). The predictive performance for this group of dietary natural products and the control group was 97% sensitivity and 80% concordance. Specificity was marginal at 53%. This study finds that the in silico QSAR analysis employing this software's rodent carcinogenicity database is capable of identifying the rodent carcinogenic potential of naturally occurring organic molecules found in the human diet with a high degree of sensitivity. It is the first study to demonstrate successful QSAR predictive modeling of naturally occurring carcinogens found in the human diet using an external validation test. Further test validation of this software and expansion of the training data set for dietary chemicals will help to support the future use of such QSAR methods for screening and prioritizing the risk of dietary chemicals when actual animal data are inadequate, equivocal, or absent.« less

A resolved two-way coupled CFD/6-DOF approach for predicting embolus transport and the embolus-trapping efficiency of IVC filters.

PubMed

Aycock, Kenneth I; Campbell, Robert L; Manning, Keefe B; Craven, Brent A

2017-06-01

Inferior vena cava (IVC) filters are medical devices designed to provide a mechanical barrier to the passage of emboli from the deep veins of the legs to the heart and lungs. Despite decades of development and clinical use, IVC filters still fail to prevent the passage of all hazardous emboli. The objective of this study is to (1) develop a resolved two-way computational model of embolus transport, (2) provide verification and validation evidence for the model, and (3) demonstrate the ability of the model to predict the embolus-trapping efficiency of an IVC filter. Our model couples computational fluid dynamics simulations of blood flow to six-degree-of-freedom simulations of embolus transport and resolves the interactions between rigid, spherical emboli and the blood flow using an immersed boundary method. Following model development and numerical verification and validation of the computational approach against benchmark data from the literature, embolus transport simulations are performed in an idealized IVC geometry. Centered and tilted filter orientations are considered using a nonlinear finite element-based virtual filter placement procedure. A total of 2048 coupled CFD/6-DOF simulations are performed to predict the embolus-trapping statistics of the filter. The simulations predict that the embolus-trapping efficiency of the IVC filter increases with increasing embolus diameter and increasing embolus-to-blood density ratio. Tilted filter placement is found to decrease the embolus-trapping efficiency compared with centered filter placement. Multiple embolus-trapping locations are predicted for the IVC filter, and the trapping locations are predicted to shift upstream and toward the vessel wall with increasing embolus diameter. Simulations of the injection of successive emboli into the IVC are also performed and reveal that the embolus-trapping efficiency decreases with increasing thrombus load in the IVC filter. In future work, the computational tool could be used to investigate IVC filter design improvements, the effect of patient anatomy on embolus transport and IVC filter embolus-trapping efficiency, and, with further development and validation, optimal filter selection and placement on a patient-specific basis.

Do behavioral scientists really understand HIV-related sexual risk behavior? A systematic review of longitudinal and experimental studies predicting sexual behavior.

PubMed

Huebner, David M; Perry, Nicholas S

2015-10-01

Behavioral interventions to reduce sexual risk behavior depend on strong health behavior theory. By identifying the psychosocial variables that lead causally to sexual risk, theories provide interventionists with a guide for how to change behavior. However, empirical research is critical to determining whether a particular theory adequately explains sexual risk behavior. A large body of cross-sectional evidence, which has been reviewed elsewhere, supports the notion that certain theory-based constructs (e.g., self-efficacy) are correlates of sexual behavior. However, given the limitations of inferring causality from correlational research, it is essential that we review the evidence from more methodologically rigorous studies (i.e., longitudinal and experimental designs). This systematic review identified 44 longitudinal studies in which investigators attempted to predict sexual risk from psychosocial variables over time. We also found 134 experimental studies (i.e., randomized controlled trials of HIV interventions), but of these only 9 (6.7 %) report the results of mediation analyses that might provide evidence for the validity of health behavior theories in predicting sexual behavior. Results show little convergent support across both types of studies for most traditional, theoretical predictors of sexual behavior. This suggests that the field must expand the body of empirical work that utilizes the most rigorous study designs to test our theoretical assumptions. The inconsistent results of existing research would indicate that current theoretical models of sexual risk behavior are inadequate, and may require expansion or adaptation.

Integrating the acoustics of running speech into the pure tone audiogram: a step from audibility to intelligibility and disability.

PubMed

Corthals, Paul

2008-01-01

The aim of the present study is to construct a simple method for visualizing and quantifying the audibility of speech on the audiogram and to predict speech intelligibility. The proposed method involves a series of indices on the audiogram form reflecting the sound pressure level distribution of running speech. The indices that coincide with a patient's pure tone thresholds reflect speech audibility and give evidence of residual functional hearing capacity. Two validation studies were conducted among sensorineurally hearing-impaired participants (n = 56 and n = 37, respectively) to investigate the relation with speech recognition ability and hearing disability. The potential of the new audibility indices as predictors for speech reception thresholds is comparable to the predictive potential of the ANSI 1968 articulation index and the ANSI 1997 speech intelligibility index. The sum of indices or a weighted combination can explain considerable proportions of variance in speech reception results for sentences in quiet free field conditions. The proportions of variance that can be explained in questionnaire results on hearing disability are less, presumably because the threshold indices almost exclusively reflect message audibility and much less the psychosocial consequences of hearing deficits. The outcomes underpin the validity of the new audibility indexing system, even though the proposed method may be better suited for predicting relative performance across a set of conditions than for predicting absolute speech recognition performance. (c) 2007 S. Karger AG, Basel

A predictive model of hospitalization risk among disabled medicaid enrollees.

PubMed

McAna, John F; Crawford, Albert G; Novinger, Benjamin W; Sidorov, Jaan; Din, Franklin M; Maio, Vittorio; Louis, Daniel Z; Goldfarb, Neil I

2013-05-01

To identify Medicaid patients, based on 1 year of administrative data, who were at high risk of admission to a hospital in the next year, and who were most likely to benefit from outreach and targeted interventions. Observational cohort study for predictive modeling. Claims, enrollment, and eligibility data for 2007 from a state Medicaid program were used to provide the independent variables for a logistic regression model to predict inpatient stays in 2008 for fully covered, continuously enrolled, disabled members. The model was developed using a 50% random sample from the state and was validated against the other 50%. Further validation was carried out by applying the parameters from the model to data from a second state's disabled Medicaid population. The strongest predictors in the model developed from the first 50% sample were over age 65 years, inpatient stay(s) in 2007, and higher Charlson Comorbidity Index scores. The areas under the receiver operating characteristic curve for the model based on the 50% state sample and its application to the 2 other samples ranged from 0.79 to 0.81. Models developed independently for all 3 samples were as high as 0.86. The results show a consistent trend of more accurate prediction of hospitalization with increasing risk score. This is a fairly robust method for targeting Medicaid members with a high probability of future avoidable hospitalizations for possible case management or other interventions. Comparison with a second state's Medicaid program provides additional evidence for the usefulness of the model.

Design of novel quinazolinone derivatives as inhibitors for 5HT7 receptor.

PubMed

Chitta, Aparna; Jatavath, Mohan Babu; Fatima, Sabiha; Manga, Vijjulatha

2012-02-01

To study the pharmacophore properties of quinazolinone derivatives as 5HT(7) inhibitors, 3D QSAR methodologies, namely Comparative Molecular Field Analysis (CoMFA) and Comparative Molecular Similarity Indices Analysis (CoMSIA) were applied, partial least square (PLS) analysis was performed and QSAR models were generated. The derived model showed good statistical reliability in terms of predicting the 5HT(7) inhibitory activity of the quinazolione derivative, based on molecular property fields like steric, electrostatic, hydrophobic, hydrogen bond donor and hydrogen bond acceptor fields. This is evident from statistical parameters like q(2) (cross validated correlation coefficient) of 0.642, 0.602 and r(2) (conventional correlation coefficient) of 0.937, 0.908 for CoMFA and CoMSIA respectively. The predictive ability of the models to determine 5HT(7) antagonistic activity is validated using a test set of 26 molecules that were not included in the training set and the predictive r(2) obtained for the test set was 0.512 & 0.541. Further, the results of the derived model are illustrated by means of contour maps, which give an insight into the interaction of the drug with the receptor. The molecular fields so obtained served as the basis for the design of twenty new ligands. In addition, ADME (Adsorption, Distribution, Metabolism and Elimination) have been calculated in order to predict the relevant pharmaceutical properties, and the results are in conformity with required drug like properties.

Predicting Blunt Cerebrovascular Injury in Pediatric Trauma: Validation of the “Utah Score”

PubMed Central

Ravindra, Vijay M.; Bollo, Robert J.; Sivakumar, Walavan; Akbari, Hassan; Naftel, Robert P.; Limbrick, David D.; Jea, Andrew; Gannon, Stephen; Shannon, Chevis; Birkas, Yekaterina; Yang, George L.; Prather, Colin T.; Kestle, John R.

2017-01-01

Abstract Risk factors for blunt cerebrovascular injury (BCVI) may differ between children and adults, suggesting that children at low risk for BCVI after trauma receive unnecessary computed tomography angiography (CTA) and high-dose radiation. We previously developed a score for predicting pediatric BCVI based on retrospective cohort analysis. Our objective is to externally validate this prediction score with a retrospective multi-institutional cohort. We included patients who underwent CTA for traumatic cranial injury at four pediatric Level I trauma centers. Each patient in the validation cohort was scored using the “Utah Score” and classified as high or low risk. Before analysis, we defined a misclassification rate <25% as validating the Utah Score. Six hundred forty-five patients (mean age 8.6 ± 5.4 years; 63.4% males) underwent screening for BCVI via CTA. The validation cohort was 411 patients from three sites compared with the training cohort of 234 patients. Twenty-two BCVIs (5.4%) were identified in the validation cohort. The Utah Score was significantly associated with BCVIs in the validation cohort (odds ratio 8.1 [3.3, 19.8], p < 0.001) and discriminated well in the validation cohort (area under the curve 72%). When the Utah Score was applied to the validation cohort, the sensitivity was 59%, specificity was 85%, positive predictive value was 18%, and negative predictive value was 97%. The Utah Score misclassified 16.6% of patients in the validation cohort. The Utah Score for predicting BCVI in pediatric trauma patients was validated with a low misclassification rate using a large, independent, multicenter cohort. Its implementation in the clinical setting may reduce the use of CTA in low-risk patients. PMID:27297774

COMBREX-DB: an experiment centered database of protein function: knowledge, predictions and knowledge gaps.

PubMed

Chang, Yi-Chien; Hu, Zhenjun; Rachlin, John; Anton, Brian P; Kasif, Simon; Roberts, Richard J; Steffen, Martin

2016-01-04

The COMBREX database (COMBREX-DB; combrex.bu.edu) is an online repository of information related to (i) experimentally determined protein function, (ii) predicted protein function, (iii) relationships among proteins of unknown function and various types of experimental data, including molecular function, protein structure, and associated phenotypes. The database was created as part of the novel COMBREX (COMputational BRidges to EXperiments) effort aimed at accelerating the rate of gene function validation. It currently holds information on ∼ 3.3 million known and predicted proteins from over 1000 completely sequenced bacterial and archaeal genomes. The database also contains a prototype recommendation system for helping users identify those proteins whose experimental determination of function would be most informative for predicting function for other proteins within protein families. The emphasis on documenting experimental evidence for function predictions, and the prioritization of uncharacterized proteins for experimental testing distinguish COMBREX from other publicly available microbial genomics resources. This article describes updates to COMBREX-DB since an initial description in the 2011 NAR Database Issue. © The Author(s) 2015. Published by Oxford University Press on behalf of Nucleic Acids Research.

The Predictive Validity of Teacher Candidate Letters of Reference

ERIC Educational Resources Information Center

Mason, Richard W.; Schroeder, Mark P.

2014-01-01

Letters of reference are widely used as an essential part of the hiring process of newly licensed teachers. While the predictive validity of these letters of reference has been called into question it has never been empirically studied. The current study examined the predictive validity of the quality of letters of reference for forty-one student…

Predictive Validity of Curriculum-Based Measures for English Learners at Varying English Proficiency Levels

ERIC Educational Resources Information Center

Kim, Jennifer Sun; Vanderwood, Michael L.; Lee, Catherine Y.

2016-01-01

This study examined the predictive validity of curriculum-based measures in reading for Spanish-speaking English learners (ELs) at various levels of English proficiency. Third-grade Spanish-speaking EL students were screened during the fall using DIBELS Oral Reading Fluency (DORF) and Daze. Predictive validity was examined in relation to spring…

Should essays and other "open-ended"-type questions retain a place in written summative assessment in clinical medicine?

PubMed

Hift, Richard J

2014-11-28

Written assessments fall into two classes: constructed-response or open-ended questions, such as the essay and a number of variants of the short-answer question, and selected-response or closed-ended questions; typically in the form of multiple-choice. It is widely believed that constructed response written questions test higher order cognitive processes in a manner that multiple-choice questions cannot, and consequently have higher validity. An extensive review of the literature suggests that in summative assessment neither premise is evidence-based. Well-structured open-ended and multiple-choice questions appear equivalent in their ability to assess higher cognitive functions, and performance in multiple-choice assessments may correlate more highly than the open-ended format with competence demonstrated in clinical practice following graduation. Studies of construct validity suggest that both formats measure essentially the same dimension, at least in mathematics, the physical sciences, biology and medicine. The persistence of the open-ended format in summative assessment may be due to the intuitive appeal of the belief that synthesising an answer to an open-ended question must be both more cognitively taxing and similar to actual experience than is selecting a correct response. I suggest that cognitive-constructivist learning theory would predict that a well-constructed context-rich multiple-choice item represents a complex problem-solving exercise which activates a sequence of cognitive processes which closely parallel those required in clinical practice, hence explaining the high validity of the multiple-choice format. The evidence does not support the proposition that the open-ended assessment format is superior to the multiple-choice format, at least in exit-level summative assessment, in terms of either its ability to test higher-order cognitive functioning or its validity. This is explicable using a theory of mental models, which might predict that the multiple-choice format will have higher validity, a statement for which some empiric support exists. Given the superior reliability and cost-effectiveness of the multiple-choice format consideration should be given to phasing out open-ended format questions in summative assessment. Whether the same applies to non-exit-level assessment and formative assessment is a question which remains to be answered; particularly in terms of the educational effect of testing, an area which deserves intensive study.

Development and validation of SEER (Seeking, Engaging with and Evaluating Research): a measure of policymakers' capacity to engage with and use research.

PubMed

Brennan, Sue E; McKenzie, Joanne E; Turner, Tari; Redman, Sally; Makkar, Steve; Williamson, Anna; Haynes, Abby; Green, Sally E

2017-01-17

Capacity building strategies are widely used to increase the use of research in policy development. However, a lack of well-validated measures for policy contexts has hampered efforts to identify priorities for capacity building and to evaluate the impact of strategies. We aimed to address this gap by developing SEER (Seeking, Engaging with and Evaluating Research), a self-report measure of individual policymakers' capacity to engage with and use research. We used the SPIRIT Action Framework to identify pertinent domains and guide development of items for measuring each domain. Scales covered (1) individual capacity to use research (confidence in using research, value placed on research, individual perceptions of the value their organisation places on research, supporting tools and systems), (2) actions taken to engage with research and researchers, and (3) use of research to inform policy (extent and type of research use). A sample of policymakers engaged in health policy development provided data to examine scale reliability (internal consistency, test-retest) and validity (relation to measures of similar concepts, relation to a measure of intention to use research, internal structure of the individual capacity scales). Response rates were 55% (150/272 people, 12 agencies) for the validity and internal consistency analyses, and 54% (57/105 people, 9 agencies) for test-retest reliability. The individual capacity scales demonstrated adequate internal consistency reliability (alpha coefficients > 0.7, all four scales) and test-retest reliability (intra-class correlation coefficients > 0.7 for three scales and 0.59 for fourth scale). Scores on individual capacity scales converged as predicted with measures of similar concepts (moderate correlations of > 0.4), and confirmatory factor analysis provided evidence that the scales measured related but distinct concepts. Items in each of these four scales related as predicted to concepts in the measurement model derived from the SPIRIT Action Framework. Evidence about the reliability and validity of the research engagement actions and research use scales was equivocal. Initial testing of SEER suggests that the four individual capacity scales may be used in policy settings to examine current capacity and identify areas for capacity building. The relation between capacity, research engagement actions and research use requires further investigation.

10-Year Coronary Heart Disease Risk Prediction Using Coronary Artery Calcium and Traditional Risk Factors: Derivation in the MESA (Multi-Ethnic Study of Atherosclerosis) With Validation in the HNR (Heinz Nixdorf Recall) Study and the DHS (Dallas Heart Study).

PubMed

McClelland, Robyn L; Jorgensen, Neal W; Budoff, Matthew; Blaha, Michael J; Post, Wendy S; Kronmal, Richard A; Bild, Diane E; Shea, Steven; Liu, Kiang; Watson, Karol E; Folsom, Aaron R; Khera, Amit; Ayers, Colby; Mahabadi, Amir-Abbas; Lehmann, Nils; Jöckel, Karl-Heinz; Moebus, Susanne; Carr, J Jeffrey; Erbel, Raimund; Burke, Gregory L

2015-10-13

Several studies have demonstrated the tremendous potential of using coronary artery calcium (CAC) in addition to traditional risk factors for coronary heart disease (CHD) risk prediction. However, to date, no risk score incorporating CAC has been developed. The goal of this study was to derive and validate a novel risk score to estimate 10-year CHD risk using CAC and traditional risk factors. Algorithm development was conducted in the MESA (Multi-Ethnic Study of Atherosclerosis), a prospective community-based cohort study of 6,814 participants age 45 to 84 years, who were free of clinical heart disease at baseline and followed for 10 years. MESA is sex balanced and included 39% non-Hispanic whites, 12% Chinese Americans, 28% African Americans, and 22% Hispanic Americans. External validation was conducted in the HNR (Heinz Nixdorf Recall Study) and the DHS (Dallas Heart Study). Inclusion of CAC in the MESA risk score offered significant improvements in risk prediction (C-statistic 0.80 vs. 0.75; p < 0.0001). External validation in both the HNR and DHS studies provided evidence of very good discrimination and calibration. Harrell's C-statistic was 0.779 in HNR and 0.816 in DHS. Additionally, the difference in estimated 10-year risk between events and nonevents was approximately 8% to 9%, indicating excellent discrimination. Mean calibration, or calibration-in-the-large, was excellent for both studies, with average predicted 10-year risk within one-half of a percent of the observed event rate. An accurate estimate of 10-year CHD risk can be obtained using traditional risk factors and CAC. The MESA risk score, which is available online on the MESA web site for easy use, can be used to aid clinicians when communicating risk to patients and when determining risk-based treatment strategies. Copyright © 2015 American College of Cardiology Foundation. Published by Elsevier Inc. All rights reserved.

PIV-measured versus CFD-predicted flow dynamics in anatomically realistic cerebral aneurysm models.

PubMed

Ford, Matthew D; Nikolov, Hristo N; Milner, Jaques S; Lownie, Stephen P; Demont, Edwin M; Kalata, Wojciech; Loth, Francis; Holdsworth, David W; Steinman, David A

2008-04-01

Computational fluid dynamics (CFD) modeling of nominally patient-specific cerebral aneurysms is increasingly being used as a research tool to further understand the development, prognosis, and treatment of brain aneurysms. We have previously developed virtual angiography to indirectly validate CFD-predicted gross flow dynamics against the routinely acquired digital subtraction angiograms. Toward a more direct validation, here we compare detailed, CFD-predicted velocity fields against those measured using particle imaging velocimetry (PIV). Two anatomically realistic flow-through phantoms, one a giant internal carotid artery (ICA) aneurysm and the other a basilar artery (BA) tip aneurysm, were constructed of a clear silicone elastomer. The phantoms were placed within a computer-controlled flow loop, programed with representative flow rate waveforms. PIV images were collected on several anterior-posterior (AP) and lateral (LAT) planes. CFD simulations were then carried out using a well-validated, in-house solver, based on micro-CT reconstructions of the geometries of the flow-through phantoms and inlet/outlet boundary conditions derived from flow rates measured during the PIV experiments. PIV and CFD results from the central AP plane of the ICA aneurysm showed a large stable vortex throughout the cardiac cycle. Complex vortex dynamics, captured by PIV and CFD, persisted throughout the cardiac cycle on the central LAT plane. Velocity vector fields showed good overall agreement. For the BA, aneurysm agreement was more compelling, with both PIV and CFD similarly resolving the dynamics of counter-rotating vortices on both AP and LAT planes. Despite the imposition of periodic flow boundary conditions for the CFD simulations, cycle-to-cycle fluctuations were evident in the BA aneurysm simulations, which agreed well, in terms of both amplitudes and spatial distributions, with cycle-to-cycle fluctuations measured by PIV in the same geometry. The overall good agreement between PIV and CFD suggests that CFD can reliably predict the details of the intra-aneurysmal flow dynamics observed in anatomically realistic in vitro models. Nevertheless, given the various modeling assumptions, this does not prove that they are mimicking the actual in vivo hemodynamics, and so validations against in vivo data are encouraged whenever possible.

Propeller aircraft interior noise model utilization study and validation

NASA Technical Reports Server (NTRS)

Pope, L. D.

1984-01-01

Utilization and validation of a computer program designed for aircraft interior noise prediction is considered. The program, entitled PAIN (an acronym for Propeller Aircraft Interior Noise), permits (in theory) predictions of sound levels inside propeller driven aircraft arising from sidewall transmission. The objective of the work reported was to determine the practicality of making predictions for various airplanes and the extent of the program's capabilities. The ultimate purpose was to discern the quality of predictions for tonal levels inside an aircraft occurring at the propeller blade passage frequency and its harmonics. The effort involved three tasks: (1) program validation through comparisons of predictions with scale-model test results; (2) development of utilization schemes for large (full scale) fuselages; and (3) validation through comparisons of predictions with measurements taken in flight tests on a turboprop aircraft. Findings should enable future users of the program to efficiently undertake and correctly interpret predictions.

A simplified approach to the pooled analysis of calibration of clinical prediction rules for systematic reviews of validation studies

PubMed Central

Dimitrov, Borislav D; Motterlini, Nicola; Fahey, Tom

2015-01-01

Objective Estimating calibration performance of clinical prediction rules (CPRs) in systematic reviews of validation studies is not possible when predicted values are neither published nor accessible or sufficient or no individual participant or patient data are available. Our aims were to describe a simplified approach for outcomes prediction and calibration assessment and evaluate its functionality and validity. Study design and methods: Methodological study of systematic reviews of validation studies of CPRs: a) ABCD2 rule for prediction of 7 day stroke; and b) CRB-65 rule for prediction of 30 day mortality. Predicted outcomes in a sample validation study were computed by CPR distribution patterns (“derivation model”). As confirmation, a logistic regression model (with derivation study coefficients) was applied to CPR-based dummy variables in the validation study. Meta-analysis of validation studies provided pooled estimates of “predicted:observed” risk ratios (RRs), 95% confidence intervals (CIs), and indexes of heterogeneity (I2) on forest plots (fixed and random effects models), with and without adjustment of intercepts. The above approach was also applied to the CRB-65 rule. Results Our simplified method, applied to ABCD2 rule in three risk strata (low, 0–3; intermediate, 4–5; high, 6–7 points), indicated that predictions are identical to those computed by univariate, CPR-based logistic regression model. Discrimination was good (c-statistics =0.61–0.82), however, calibration in some studies was low. In such cases with miscalibration, the under-prediction (RRs =0.73–0.91, 95% CIs 0.41–1.48) could be further corrected by intercept adjustment to account for incidence differences. An improvement of both heterogeneities and P-values (Hosmer-Lemeshow goodness-of-fit test) was observed. Better calibration and improved pooled RRs (0.90–1.06), with narrower 95% CIs (0.57–1.41) were achieved. Conclusion Our results have an immediate clinical implication in situations when predicted outcomes in CPR validation studies are lacking or deficient by describing how such predictions can be obtained by everyone using the derivation study alone, without any need for highly specialized knowledge or sophisticated statistics. PMID:25931829

Predicting workplace aggression and violence.

PubMed

Barling, Julian; Dupré, Kathryne E; Kelloway, E Kevin

2009-01-01

Consistent with the relative recency of research on workplace aggression and the considerable media attention given to high-profile incidents, numerous myths about the nature of workplace aggression have emerged. In this review, we examine these myths from an evidence-based perspective, bringing greater clarity to our understanding of the predictors of workplace aggression. We conclude by pointing to the need for more research focusing on construct validity and prevention issues as well as for methodologies that minimize the likelihood of mono-method bias and that strengthen the ability to make causal inferences.

Utilization of a quantitative mammalian cell mutation system, CHO/HGPRT, in experimental mutagenesis and genetic toxicology

DOE Office of Scientific and Technical Information (OSTI.GOV)

Hsie, A. W.; Couch, D. B.; O'Neill, J. P.

1977-01-01

Development of the CHO/HGPRT system is described and a host-mediated CHO/HGPRT assay is discussed. The following topics are discussed: evidence for the genetic origin of mutation induction in the CHO/HGPRT system; dose-response relationship for EMS-mediated mutation induction and cell lethality; apparent dosimetry of EMS-induced mutagenesis; structure-activity relationship of alkylating agents and ICR compounds; mutagenicity and cytotoxicity of congeners of two classes of nitrosi compounds; and preliminary validation of the CHO/HGPRT assay in predicting chemical carcinogenicity. (HLW)

The predictive validity of three versions of the MCAT in relation to performance in medical school, residency, and licensing examinations: a longitudinal study of 36 classes of Jefferson Medical College.

PubMed

Callahan, Clara A; Hojat, Mohammadreza; Veloski, Jon; Erdmann, James B; Gonnella, Joseph S

2010-06-01

The Medical College Admission Test (MCAT) has undergone several revisions for content and validity since its inception. With another comprehensive review pending, this study examines changes in the predictive validity of the MCAT's three recent versions. Study participants were 7,859 matriculants in 36 classes entering Jefferson Medical College between 1970 and 2005; 1,728 took the pre-1978 version of the MCAT; 3,032 took the 1978-1991 version, and 3,099 took the post-1991 version. MCAT subtest scores were the predictors, and performance in medical school, attrition, scores on the medical licensing examinations, and ratings of clinical competence in the first year of residency were the criterion measures. No significant improvement in validity coefficients was observed for performance in medical school or residency. Validity coefficients for all three versions of the MCAT in predicting Part I/Step 1 remained stable (in the mid-0.40s, P < .01). A systematic decline was observed in the validity coefficients of the MCAT versions in predicting Part II/Step 2. It started at 0.47 for the pre-1978 version, decreased to between 0.42 and 0.40 for the 1978-1991 versions, and to 0.37 for the post-1991 version. Validity coefficients for the MCAT versions in predicting Part III/Step 3 remained near 0.30. These were generally larger for women than men. Although the findings support the short- and long-term predictive validity of the MCAT, opportunities to strengthen it remain. Subsequent revisions should increase the test's ability to predict performance on United States Medical Licensing Examination Step 2 and must minimize the differential validity for gender.

Predicted Biological Activity of Purchasable Chemical Space

PubMed Central

2017-01-01

Whereas 400 million distinct compounds are now purchasable within the span of a few weeks, the biological activities of most are unknown. To facilitate access to new chemistry for biology, we have combined the Similarity Ensemble Approach (SEA) with the maximum Tanimoto similarity to the nearest bioactive to predict activity for every commercially available molecule in ZINC. This method, which we label SEA+TC, outperforms both SEA and a naïve-Bayesian classifier via predictive performance on a 5-fold cross-validation of ChEMBL’s bioactivity data set (version 21). Using this method, predictions for over 40% of compounds (>160 million) have either high significance (pSEA ≥ 40), high similarity (ECFP4MaxTc ≥ 0.4), or both, for one or more of 1382 targets well described by ligands in the literature. Using a further 1347 less-well-described targets, we predict activities for an additional 11 million compounds. To gauge whether these predictions are sensible, we investigate 75 predictions for 50 drugs lacking a binding affinity annotation in ChEMBL. The 535 million predictions for over 171 million compounds at 2629 targets are linked to purchasing information and evidence to support each prediction and are freely available via https://zinc15.docking.org and https://files.docking.org. PMID:29193970

Fluoroscopically Guided Diagnostic and Therapeutic Intra-Articular Sacroiliac Joint Injections: A Systematic Review.

PubMed

Kennedy, David J; Engel, Andrew; Kreiner, D Scott; Nampiaparampil, Devi; Duszynski, Belinda; MacVicar, John

2015-08-01

To assess the validity of fluoroscopically guided diagnostic intra-articular injections of local anesthetic and effectiveness of intra-articular steroid injections in treating sacroiliac joint (SIJ) pain. Systematic review. Ten reviewers independently assessed 45 publications on diagnostic validity or effectiveness of fluoroscopically guided intra-articular SIJ injections. For diagnostic injections, the primary outcome was validity; for therapeutic injections, analgesia. Secondary outcomes were also described. Of 45 articles reviewed, 39 yielded diagnostic data on physical exam findings, provocation tests, and SIJ injections for diagnosing SIJ pain, and 15 addressed therapeutic effectiveness. When confirmed by comparative local anesthetic blocks with a high degree of pain relief, no single physical exam maneuver predicts response to diagnostic injections. When at least three physical exam findings are present, sensitivity, and specificity increases significantly. The prevalence of SIJ pain is likely 20-30% among patients that have suspected SIJ pain based on history and physical examination. This estimate may be higher in certain subgroups such as the elderly and fusion patients. Two randomized controlled trials and multiple observational studies supported the effectiveness of therapeutic sacroiliac joint injections. Based on this literature, it is unclear whether image-guided intra-articular diagnostic injections of local anesthetic predict positive responses to therapeutic agents. The overall quality of evidence is moderate for the effectiveness of therapeutic SIJ injections. Wiley Periodicals, Inc.

Risk mapping of West Nile virus circulation in Spain, 2015.

PubMed

Sánchez-Gómez, Amaya; Amela, Carmen; Fernández-Carrión, Eduardo; Martínez-Avilés, Marta; Sánchez-Vizcaíno, José Manuel; Sierra-Moros, María José

2017-05-01

West Nile fever is an emergent disease in Europe. The objective of this study was to conduct a predictive risk mapping of West Nile Virus (WNV) circulation in Spain based on historical data of WNV circulation. Areas of Spain with evidence of WNV circulation were mapped based on data from notifications to the surveillance systems and a literature review. A logistic regression-based spatial model was used to assess the probability of WNV circulation. Data were analyzed at municipality level. Mean temperatures of the period from June to October, presence of wetlands and presence of Special Protection Areas for birds were considered as potential predictors. Two predictors of WNV circulation were identified: higher temperature [adjusted odds ratio (AOR) 2.07, 95% CI 1.82-2.35, p<0.01] and presence of wetlands (3.37, 95% CI 1.89-5.99, p<0.01). Model validations indicated good predictions: area under the ROC curve was 0.895 (95% CI 0.870-0.919) for internal validation and 0.895 (95% CI 0.840-0.951) for external validation. This model could support improvements of WNV risk- based surveillance in Spain. The importance of a comprehensive surveillance for WNF, including human, animal and potential vectors is highlighted, which could additionally result in model refinements. Copyright © 2017 Elsevier B.V. All rights reserved.

Early Prognostication Markers in Cardiac Arrest Patients Treated with Hypothermia

PubMed Central

Karapetkova, Maria; Koenig, Matthew A.; Jia, Xiaofeng

2015-01-01

Background and purpose Established prognostication markers, such as clinical findings, electroencephalography (EEG), and biochemical markers, used by clinicians to predict neurologic outcome after cardiac arrest (CA) are altered under therapeutic hypothermia (TH) conditions and their validity remains uncertain. Methods MEDLINE and EMBASE were searched for evidence on the current standards for neurologic outcome prediction for out-of-hospital CA patients treated with TH and the validity of a wide range of prognostication markers. Relevant studies that suggested one or several established biomarkers, and multimodal approaches for prognostication were included and reviewed. Results While the prognostic accuracy of various tests has been questioned after TH, pupillary light reflexes and somatosensory evoked potentials (SSEP) are still strongly associated with negative outcome for early prognostication. Increasingly, EEG background activity has also been identified as a valid predictor for outcome after 72 hours after CA and a preferred prognostic method in clinical settings. Neuroimaging techniques, such as MRI and CT, can identify functional and structural brain injury, but are not readily available at the patient’s bedside because of limited availability and high costs. Conclusions A multimodal algorithm composed of neurological examination, EEG-based quantitative testing, and SSEP, in conjunction with newer MRI sequences, if available, holds promise for accurate prognostication in CA patients treated with TH. In order to avoid premature withdrawal of care, prognostication should be performed later than 72 hours after CA. PMID:26228521

Early prognostication markers in cardiac arrest patients treated with hypothermia.

PubMed

Karapetkova, M; Koenig, M A; Jia, X

2016-03-01

Established prognostication markers, such as clinical findings, electroencephalography (EEG) and biochemical markers, used by clinicians to predict neurological outcome after cardiac arrest (CA) are altered under therapeutic hypothermia (TH) conditions and their validity remains uncertain. MEDLINE and Embase were searched for evidence on the current standards for neurological outcome prediction for out-of-hospital CA patients treated with TH and the validity of a wide range of prognostication markers. Relevant studies that suggested one or several established biomarkers and multimodal approaches for prognostication are included and reviewed. Whilst the prognostic accuracy of various tests after TH has been questioned, pupillary light reflexes and somatosensory evoked potentials are still strongly associated with negative outcome for early prognostication. Increasingly, EEG background activity has also been identified as a valid predictor for outcome after 72 h after CA and a preferred prognostic method in clinical settings. Neuroimaging techniques, such as magnetic resonance imaging and computed tomography, can identify functional and structural brain injury but are not readily available at the patient's bedside because of limited availability and high costs. A multimodal algorithm composed of neurological examination, EEG-based quantitative testing and somatosensory evoked potentials, in conjunction with newer magnetic resonance imaging sequences, if available, holds promise for accurate prognostication in CA patients treated with TH. In order to avoid premature withdrawal of care, prognostication should be performed more than 72 h after CA. © 2015 EAN.

Development and validation of the alcohol Expectancy Questionnaire Short Form (EQ-SF).

PubMed

Mezquita, Laura; Camacho, Laura; Suso-Ribera, Carlos; Ortet, Generós; Ibáñez, Manuel I

2018-01-15

Alcohol expectancies are proximal variables to alcohol use and misuse. In recent decades, different measures have been developed to assess this construct. One of the most frequently used and recommended instruments is the Expectancy Questionnaire (EQ; Leigh y Stacy, 1993). Our aim is to develop a short version of the EQ (EQ-SF) for suitable use in time-limited administrations. Two samples, adolescents (N = 514, 57.20% females) and adults (N = 548, 61.50% females), completed the EQ together with alcohol-use measures. Different item selection strategies were applied to select the 24 items. The EQ-SF structure was explored using confirmatory factor analysis, and measurement invariance was tested running a multi-group analysis comparing groups by sex and age. Reliability was tested using Cronbach's alpha and omega coefficients. Concurrent validity was investigated with regression analyses. The EQ-SF showed acceptable between-groups measurement invariance. Alphas and omegas ranged from .77 to .93. Positive expectancies predicted both alcohol use and alcohol-related problems. Negative expectancies predicted alcohol-related problems. Sex and age moderated these associations. Males with high positive alcohol expectancies showed higher alcohol consumption than females, while adults with high negative alcohol expectancies showed greater alcohol-related problems than adolescents. Different evidence on the validity and reliability of the EQ-SF suggest that it is a suitable instrument to assess alcohol expectancies in the Spanish population.

Health Sciences-Evidence Based Practice questionnaire (HS-EBP) for measuring transprofessional evidence-based practice: Creation, development and psychometric validation

PubMed Central

Fernández-Domínguez, Juan Carlos; de Pedro-Gómez, Joan Ernest; Morales-Asencio, José Miguel; Sastre-Fullana, Pedro; Sesé-Abad, Albert

2017-01-01

Introduction Most of the EBP measuring instruments available to date present limitations both in the operationalisation of the construct and also in the rigour of their psychometric development, as revealed in the literature review performed. The aim of this paper is to provide rigorous and adequate reliability and validity evidence of the scores of a new transdisciplinary psychometric tool, the Health Sciences Evidence-Based Practice (HS-EBP), for measuring the construct EBP in Health Sciences professionals. Methods A pilot study and a subsequent two-stage validation test sample were conducted to progressively refine the instrument until a reduced 60-item version with a five-factor latent structure. Reliability was analysed through both Cronbach’s alpha coefficient and intraclass correlations (ICC). Latent structure was contrasted using confirmatory factor analysis (CFA) following a model comparison aproach. Evidence of criterion validity of the scores obtained was achieved by considering attitudinal resistance to change, burnout, and quality of professional life as criterion variables; while convergent validity was assessed using the Spanish version of the Evidence-Based Practice Questionnaire (EBPQ-19). Results Adequate evidence of both reliability and ICC was obtained for the five dimensions of the questionnaire. According to the CFA model comparison, the best fit corresponded to the five-factor model (RMSEA = 0.049; CI 90% RMSEA = [0.047; 0.050]; CFI = 0.99). Adequate criterion and convergent validity evidence was also provided. Finally, the HS-EBP showed the capability to find differences between EBP training levels as an important evidence of decision validity. Conclusions Reliability and validity evidence obtained regarding the HS-EBP confirm the adequate operationalisation of the EBP construct as a process put into practice to respond to every clinical situation arising in the daily practice of professionals in health sciences (transprofessional). The tool could be useful for EBP individual assessment and for evaluating the impact of specific interventions to improve EBP. PMID:28486533

Health Sciences-Evidence Based Practice questionnaire (HS-EBP) for measuring transprofessional evidence-based practice: Creation, development and psychometric validation.

PubMed

Fernández-Domínguez, Juan Carlos; de Pedro-Gómez, Joan Ernest; Morales-Asencio, José Miguel; Bennasar-Veny, Miquel; Sastre-Fullana, Pedro; Sesé-Abad, Albert

2017-01-01

Most of the EBP measuring instruments available to date present limitations both in the operationalisation of the construct and also in the rigour of their psychometric development, as revealed in the literature review performed. The aim of this paper is to provide rigorous and adequate reliability and validity evidence of the scores of a new transdisciplinary psychometric tool, the Health Sciences Evidence-Based Practice (HS-EBP), for measuring the construct EBP in Health Sciences professionals. A pilot study and a subsequent two-stage validation test sample were conducted to progressively refine the instrument until a reduced 60-item version with a five-factor latent structure. Reliability was analysed through both Cronbach's alpha coefficient and intraclass correlations (ICC). Latent structure was contrasted using confirmatory factor analysis (CFA) following a model comparison aproach. Evidence of criterion validity of the scores obtained was achieved by considering attitudinal resistance to change, burnout, and quality of professional life as criterion variables; while convergent validity was assessed using the Spanish version of the Evidence-Based Practice Questionnaire (EBPQ-19). Adequate evidence of both reliability and ICC was obtained for the five dimensions of the questionnaire. According to the CFA model comparison, the best fit corresponded to the five-factor model (RMSEA = 0.049; CI 90% RMSEA = [0.047; 0.050]; CFI = 0.99). Adequate criterion and convergent validity evidence was also provided. Finally, the HS-EBP showed the capability to find differences between EBP training levels as an important evidence of decision validity. Reliability and validity evidence obtained regarding the HS-EBP confirm the adequate operationalisation of the EBP construct as a process put into practice to respond to every clinical situation arising in the daily practice of professionals in health sciences (transprofessional). The tool could be useful for EBP individual assessment and for evaluating the impact of specific interventions to improve EBP.

Performance of genomic prediction within and across generations in maritime pine.

PubMed

Bartholomé, Jérôme; Van Heerwaarden, Joost; Isik, Fikret; Boury, Christophe; Vidal, Marjorie; Plomion, Christophe; Bouffier, Laurent

2016-08-11

Genomic selection (GS) is a promising approach for decreasing breeding cycle length in forest trees. Assessment of progeny performance and of the prediction accuracy of GS models over generations is therefore a key issue. A reference population of maritime pine (Pinus pinaster) with an estimated effective inbreeding population size (status number) of 25 was first selected with simulated data. This reference population (n = 818) covered three generations (G0, G1 and G2) and was genotyped with 4436 single-nucleotide polymorphism (SNP) markers. We evaluated the effects on prediction accuracy of both the relatedness between the calibration and validation sets and validation on the basis of progeny performance. Pedigree-based (best linear unbiased prediction, ABLUP) and marker-based (genomic BLUP and Bayesian LASSO) models were used to predict breeding values for three different traits: circumference, height and stem straightness. On average, the ABLUP model outperformed genomic prediction models, with a maximum difference in prediction accuracies of 0.12, depending on the trait and the validation method. A mean difference in prediction accuracy of 0.17 was found between validation methods differing in terms of relatedness. Including the progenitors in the calibration set reduced this difference in prediction accuracy to 0.03. When only genotypes from the G0 and G1 generations were used in the calibration set and genotypes from G2 were used in the validation set (progeny validation), prediction accuracies ranged from 0.70 to 0.85. This study suggests that the training of prediction models on parental populations can predict the genetic merit of the progeny with high accuracy: an encouraging result for the implementation of GS in the maritime pine breeding program.

Evidence of Construct Validity in Published Achievement Tests.

ERIC Educational Resources Information Center

Nolet, Victor; Tindal, Gerald

Valid interpretation of test scores is the shared responsibility of the test designer and the test user. Test publishers must provide evidence of the validity of the decisions their tests are intended to support, while test users are responsible for analyzing this evidence and subsequently using the test in the manner indicated by the publisher.…

Could Fractional Exhaled Nitric Oxide Test be Useful in Predicting Inhaled Corticosteroid Responsiveness in Chronic Cough? A Systematic Review.

PubMed

Song, Woo-Jung; Won, Ha-Kyeong; Moon, Sung-Do; Chung, Soo-Jie; Kang, Sung-Yoon; Sohn, Kyoung-Hee; Kim, Ju-Young; Kim, Byung-Keun; Lim, Kyung-Hwan; Kim, Mi-Yeong; Yang, Min-Suk; Park, Heung-Woo; Chang, Yoon-Seok; Lee, Byung-Jae; Morice, Alyn H; Cho, Sang-Heon

Fractional exhaled nitric oxide (Feno) is a safe and convenient test for assessing T H 2 airway inflammation, which is potentially useful in the management of patients with chronic cough. To summarize the current evidence on the diagnostic usefulness of Feno for predicting inhaled corticosteroid (ICS) responsiveness in patients with chronic cough. A systematic literature review was conducted to identify articles published in peer-reviewed journals up to February 2015, without language restriction. We included studies that reported the usefulness of Feno (index test) for predicting ICS responsiveness (reference standard) in patients with chronic cough (target condition). The data were extracted to construct a 2 × 2 accuracy table. Study quality was assessed with Quality Assessment of Diagnostic Accuracy Studies 2. We identified 5 original studies (2 prospective and 3 retrospective studies). We identified considerable heterogeneities in study design and outcome definitions, and thus were unable to perform a meta-analysis. The proportion of ICS responders ranged from 44% to 59%. Sensitivity and specificity ranged from 53% to 90%, and from 63% to 97%, respectively. The reported area under the curve ranged from about 0.60 to 0.87; however, studies with a prospective design and a lower prevalence of asthma had lower area under the curve values. None measured placebo effects or objective cough frequency. We did not find strong evidence to support the use of Feno tests for predicting ICS responsiveness in chronic cough. Further studies need to have a randomized, placebo-controlled design, and should use validated measurement tools for cough. Standardization would facilitate the development of clinical evidence. Copyright © 2016. Published by Elsevier Inc.

Macro- and Micro-Validation: Beyond the "Five Sources" Framework for Classifying Validation Evidence and Analysis

ERIC Educational Resources Information Center

Newton, Paul E.

2016-01-01

This paper argues that the dominant framework for conceptualizing validation evidence and analysis--the "five sources" framework from the 1999 "Standards"--is seriously limited. Its limitation raises a significant barrier to understanding the nature of comprehensive validation, and this presents a significant threat to…

A simple risk scoring system for prediction of relapse after inpatient alcohol treatment.

PubMed

Pedersen, Mads Uffe; Hesse, Morten

2009-01-01

Predicting relapse after alcoholism treatment can be useful in targeting patients for aftercare services. However, a valid and practical instrument for predicting relapse risk does not exist. Based on a prospective study of alcoholism treatment, we developed the Risk of Alcoholic Relapse Scale (RARS) using items taken from the Addiction Severity Index and some basic demographic information. The RARS was cross-validated using two non-overlapping samples, and tested for its ability to predict relapse across different models of treatment. The RARS predicted relapse to drinking within 6 months after alcoholism treatment in both the original and the validation sample, and in a second validation sample it predicted admission to new treatment 3 years after treatment. The RARS can identify patients at high risk of relapse who need extra aftercare and support after treatment.