Validation of biomarkers to predict response to immunotherapy in cancer: Volume I - pre-analytical and analytical validation.

PubMed

Masucci, Giuseppe V; Cesano, Alessandra; Hawtin, Rachael; Janetzki, Sylvia; Zhang, Jenny; Kirsch, Ilan; Dobbin, Kevin K; Alvarez, John; Robbins, Paul B; Selvan, Senthamil R; Streicher, Howard Z; Butterfield, Lisa H; Thurin, Magdalena

2016-01-01

Immunotherapies have emerged as one of the most promising approaches to treat patients with cancer. Recently, there have been many clinical successes using checkpoint receptor blockade, including T cell inhibitory receptors such as cytotoxic T-lymphocyte-associated antigen 4 (CTLA-4) and programmed cell death-1 (PD-1). Despite demonstrated successes in a variety of malignancies, responses only typically occur in a minority of patients in any given histology. Additionally, treatment is associated with inflammatory toxicity and high cost. Therefore, determining which patients would derive clinical benefit from immunotherapy is a compelling clinical question. Although numerous candidate biomarkers have been described, there are currently three FDA-approved assays based on PD-1 ligand expression (PD-L1) that have been clinically validated to identify patients who are more likely to benefit from a single-agent anti-PD-1/PD-L1 therapy. Because of the complexity of the immune response and tumor biology, it is unlikely that a single biomarker will be sufficient to predict clinical outcomes in response to immune-targeted therapy. Rather, the integration of multiple tumor and immune response parameters, such as protein expression, genomics, and transcriptomics, may be necessary for accurate prediction of clinical benefit. Before a candidate biomarker and/or new technology can be used in a clinical setting, several steps are necessary to demonstrate its clinical validity. Although regulatory guidelines provide general roadmaps for the validation process, their applicability to biomarkers in the cancer immunotherapy field is somewhat limited. Thus, Working Group 1 (WG1) of the Society for Immunotherapy of Cancer (SITC) Immune Biomarkers Task Force convened to address this need. In this two volume series, we discuss pre-analytical and analytical (Volume I) as well as clinical and regulatory (Volume II) aspects of the validation process as applied to predictive biomarkers

Predictive Validity Study of the APS Writing and Reading Tests [and] Validating Placement Rules for the APS Writing Test.

ERIC Educational Resources Information Center

College of the Canyons, Valencia, CA. Office of Institutional Development.

California's College of the Canyons has used the College Board Assessment and Placement Services (APS) test to assess students' abilities in basic and college English since spring 1993. These two reports summarize data from a May 1994 study of the predictive validity of the APS writing and reading tests and a June 1994 effort to validate the cut…

Validation of behave fire behavior predictions in oak savannas

USGS Publications Warehouse

Grabner, Keith W.; Dwyer, John; Cutter, Bruce E.

1997-01-01

Prescribed fire is a valuable tool in the restoration and management of oak savannas. BEHAVE, a fire behavior prediction system developed by the United States Forest Service, can be a useful tool when managing oak savannas with prescribed fire. BEHAVE predictions of fire rate-of-spread and flame length were validated using four standardized fuel models: Fuel Model 1 (short grass), Fuel Model 2 (timber and grass), Fuel Model 3 (tall grass), and Fuel Model 9 (hardwood litter). Also, a customized oak savanna fuel model (COSFM) was created and validated. Results indicate that standardized fuel model 2 and the COSFM reliably estimate mean rate-of-spread (MROS). The COSFM did not appreciably reduce MROS variation when compared to fuel model 2. Fuel models 1, 3, and 9 did not reliably predict MROS. Neither the standardized fuel models nor the COSFM adequately predicted flame lengths. We concluded that standardized fuel model 2 should be used with BEHAVE when predicting fire rates-of-spread in established oak savannas.

A new framework to enhance the interpretation of external validation studies of clinical prediction models.

PubMed

Debray, Thomas P A; Vergouwe, Yvonne; Koffijberg, Hendrik; Nieboer, Daan; Steyerberg, Ewout W; Moons, Karel G M

2015-03-01

It is widely acknowledged that the performance of diagnostic and prognostic prediction models should be assessed in external validation studies with independent data from "different but related" samples as compared with that of the development sample. We developed a framework of methodological steps and statistical methods for analyzing and enhancing the interpretation of results from external validation studies of prediction models. We propose to quantify the degree of relatedness between development and validation samples on a scale ranging from reproducibility to transportability by evaluating their corresponding case-mix differences. We subsequently assess the models' performance in the validation sample and interpret the performance in view of the case-mix differences. Finally, we may adjust the model to the validation setting. We illustrate this three-step framework with a prediction model for diagnosing deep venous thrombosis using three validation samples with varying case mix. While one external validation sample merely assessed the model's reproducibility, two other samples rather assessed model transportability. The performance in all validation samples was adequate, and the model did not require extensive updating to correct for miscalibration or poor fit to the validation settings. The proposed framework enhances the interpretation of findings at external validation of prediction models. Copyright © 2015 The Authors. Published by Elsevier Inc. All rights reserved.

Testing the Predictive Validity of the Hendrich II Fall Risk Model.

PubMed

Jung, Hyesil; Park, Hyeoun-Ae

2018-03-01

Cumulative data on patient fall risk have been compiled in electronic medical records systems, and it is possible to test the validity of fall-risk assessment tools using these data between the times of admission and occurrence of a fall. The Hendrich II Fall Risk Model scores assessed during three time points of hospital stays were extracted and used for testing the predictive validity: (a) upon admission, (b) when the maximum fall-risk score from admission to falling or discharge, and (c) immediately before falling or discharge. Predictive validity was examined using seven predictive indicators. In addition, logistic regression analysis was used to identify factors that significantly affect the occurrence of a fall. Among the different time points, the maximum fall-risk score assessed between admission and falling or discharge showed the best predictive performance. Confusion or disorientation and having a poor ability to rise from a sitting position were significant risk factors for a fall.

Experimental validation of predicted cancer genes using FRET

NASA Astrophysics Data System (ADS)

Guala, Dimitri; Bernhem, Kristoffer; Ait Blal, Hammou; Jans, Daniel; Lundberg, Emma; Brismar, Hjalmar; Sonnhammer, Erik L. L.

2018-07-01

Huge amounts of data are generated in genome wide experiments, designed to investigate diseases with complex genetic causes. Follow up of all potential leads produced by such experiments is currently cost prohibitive and time consuming. Gene prioritization tools alleviate these constraints by directing further experimental efforts towards the most promising candidate targets. Recently a gene prioritization tool called MaxLink was shown to outperform other widely used state-of-the-art prioritization tools in a large scale in silico benchmark. An experimental validation of predictions made by MaxLink has however been lacking. In this study we used Fluorescence Resonance Energy Transfer, an established experimental technique for detection of protein-protein interactions, to validate potential cancer genes predicted by MaxLink. Our results provide confidence in the use of MaxLink for selection of new targets in the battle with polygenic diseases.

Individualized prediction of perineural invasion in colorectal cancer: development and validation of a radiomics prediction model.

PubMed

Huang, Yanqi; He, Lan; Dong, Di; Yang, Caiyun; Liang, Cuishan; Chen, Xin; Ma, Zelan; Huang, Xiaomei; Yao, Su; Liang, Changhong; Tian, Jie; Liu, Zaiyi

2018-02-01

To develop and validate a radiomics prediction model for individualized prediction of perineural invasion (PNI) in colorectal cancer (CRC). After computed tomography (CT) radiomics features extraction, a radiomics signature was constructed in derivation cohort (346 CRC patients). A prediction model was developed to integrate the radiomics signature and clinical candidate predictors [age, sex, tumor location, and carcinoembryonic antigen (CEA) level]. Apparent prediction performance was assessed. After internal validation, independent temporal validation (separate from the cohort used to build the model) was then conducted in 217 CRC patients. The final model was converted to an easy-to-use nomogram. The developed radiomics nomogram that integrated the radiomics signature and CEA level showed good calibration and discrimination performance [Harrell's concordance index (c-index): 0.817; 95% confidence interval (95% CI): 0.811-0.823]. Application of the nomogram in validation cohort gave a comparable calibration and discrimination (c-index: 0.803; 95% CI: 0.794-0.812). Integrating the radiomics signature and CEA level into a radiomics prediction model enables easy and effective risk assessment of PNI in CRC. This stratification of patients according to their PNI status may provide a basis for individualized auxiliary treatment.

Two-Speed Gearbox Dynamic Simulation Predictions and Test Validation

NASA Technical Reports Server (NTRS)

Lewicki, David G.; DeSmidt, Hans; Smith, Edward C.; Bauman, Steven W.

2010-01-01

Dynamic simulations and experimental validation tests were performed on a two-stage, two-speed gearbox as part of the drive system research activities of the NASA Fundamental Aeronautics Subsonics Rotary Wing Project. The gearbox was driven by two electromagnetic motors and had two electromagnetic, multi-disk clutches to control output speed. A dynamic model of the system was created which included a direct current electric motor with proportional-integral-derivative (PID) speed control, a two-speed gearbox with dual electromagnetically actuated clutches, and an eddy current dynamometer. A six degree-of-freedom model of the gearbox accounted for the system torsional dynamics and included gear, clutch, shaft, and load inertias as well as shaft flexibilities and a dry clutch stick-slip friction model. Experimental validation tests were performed on the gearbox in the NASA Glenn gear noise test facility. Gearbox output speed and torque as well as drive motor speed and current were compared to those from the analytical predictions. The experiments correlate very well with the predictions, thus validating the dynamic simulation methodologies.

Exact Analysis of Squared Cross-Validity Coefficient in Predictive Regression Models

ERIC Educational Resources Information Center

Shieh, Gwowen

2009-01-01

In regression analysis, the notion of population validity is of theoretical interest for describing the usefulness of the underlying regression model, whereas the presumably more important concept of population cross-validity represents the predictive effectiveness for the regression equation in future research. It appears that the inference…

Systematic review of the concurrent and predictive validity of MRI biomarkers in OA

PubMed Central

Hunter, D.J.; Zhang, W.; Conaghan, Philip G.; Hirko, K.; Menashe, L.; Li, L.; Reichmann, W.M.; Losina, E.

2012-01-01

SUMMARY Objective To summarize literature on the concurrent and predictive validity of MRI-based measures of osteoarthritis (OA) structural change. Methods An online literature search was conducted of the OVID, EMBASE, CINAHL, PsychInfo and Cochrane databases of articles published up to the time of the search, April 2009. 1338 abstracts obtained with this search were preliminarily screened for relevance by two reviewers. Of these, 243 were selected for data extraction for this analysis on validity as well as separate reviews on discriminate validity and diagnostic performance. Of these 142 manuscripts included data pertinent to concurrent validity and 61 manuscripts for the predictive validity review. For this analysis we extracted data on criterion (concurrent and predictive) validity from both longitudinal and cross-sectional studies for all synovial joint tissues as it relates to MRI measurement in OA. Results Concurrent validity of MRI in OA has been examined compared to symptoms, radiography, histology/pathology, arthroscopy, CT, and alignment. The relation of bone marrow lesions, synovitis and effusion to pain was moderate to strong. There was a weak or no relation of cartilage morphology or meniscal tears to pain. The relation of cartilage morphology to radiographic OA and radiographic joint space was inconsistent. There was a higher frequency of meniscal tears, synovitis and other features in persons with radiographic OA. The relation of cartilage to other constructs including histology and arthroscopy was stronger. Predictive validity of MRI in OA has been examined for ability to predict total knee replacement (TKR), change in symptoms, radiographic progression as well as MRI progression. Quantitative cartilage volume change and presence of cartilage defects or bone marrow lesions are potential predictors of TKR. Conclusion MRI has inherent strengths and unique advantages in its ability to visualize multiple individual tissue pathologies relating to pain

Observations on CFD Verification and Validation from the AIAA Drag Prediction Workshops

NASA Technical Reports Server (NTRS)

Morrison, Joseph H.; Kleb, Bil; Vassberg, John C.

2014-01-01

The authors provide observations from the AIAA Drag Prediction Workshops that have spanned over a decade and from a recent validation experiment at NASA Langley. These workshops provide an assessment of the predictive capability of forces and moments, focused on drag, for transonic transports. It is very difficult to manage the consistency of results in a workshop setting to perform verification and validation at the scientific level, but it may be sufficient to assess it at the level of practice. Observations thus far: 1) due to simplifications in the workshop test cases, wind tunnel data are not necessarily the “correct” results that CFD should match, 2) an average of core CFD data are not necessarily a better estimate of the true solution as it is merely an average of other solutions and has many coupled sources of variation, 3) outlier solutions should be investigated and understood, and 4) the DPW series does not have the systematic build up and definition on both the computational and experimental side that is required for detailed verification and validation. Several observations regarding the importance of the grid, effects of physical modeling, benefits of open forums, and guidance for validation experiments are discussed. The increased variation in results when predicting regions of flow separation and increased variation due to interaction effects, e.g., fuselage and horizontal tail, point out the need for validation data sets for these important flow phenomena. Experiences with a recent validation experiment at NASA Langley are included to provide guidance on validation experiments.

Parent- and Self-Reported Dimensions of Oppositionality in Youth: Construct Validity, Concurrent Validity, and the Prediction of Criminal Outcomes in Adulthood

ERIC Educational Resources Information Center

Aebi, Marcel; Plattner, Belinda; Metzke, Christa Winkler; Bessler, Cornelia; Steinhausen, Hans-Christoph

2013-01-01

Background: Different dimensions of oppositional defiant disorder (ODD) have been found as valid predictors of further mental health problems and antisocial behaviors in youth. The present study aimed at testing the construct, concurrent, and predictive validity of ODD dimensions derived from parent- and self-report measures. Method: Confirmatory…

Validity of the MCAT in Predicting Performance in the First Two Years of Medical School.

ERIC Educational Resources Information Center

Jones, Robert F.; Thomae-Forgues, Maria

1984-01-01

The first systematic summary of predictive validity research on the new Medical College Admission Test (MCAT) is presented. The results show that MCAT scores have significant predictive validity with respect to first- and second-year medical school course grades. Further directions for MCAT validity research are described. (Author/MLW)

Cross-validation of the Beunen-Malina method to predict adult height.

PubMed

Beunen, Gaston P; Malina, Robert M; Freitas, Duarte I; Maia, José A; Claessens, Albrecht L; Gouveia, Elvio R; Lefevre, Johan

2010-08-01

The purpose of this study was to cross-validate the Beunen-Malina method for non-invasive prediction of adult height. Three hundred and eight boys aged 13, 14, 15 and 16 years from the Madeira Growth Study were observed at annual intervals in 1996, 1997 and 1998 and re-measured 7-8 years later. Height, sitting height and the triceps and subscapular skinfolds were measured; skeletal age was assessed using the Tanner-Whitehouse 2 method. Adult height was measured and predicted using the Beunen-Malina method. Maturity groups were classified using relative skeletal age (skeletal age minus chronological age). Pearson correlations, mean differences and standard errors of estimate (SEE) were calculated. Age-specific correlations between predicted and measured adult height vary between 0.70 and 0.85, while age-specific SEE varies between 3.3 and 4.7 cm. The correlations and SEE are similar to those obtained in the development of the original Beunen-Malina method. The Beunen-Malina method is a valid method to predict adult height in adolescent boys and can be used in European populations or populations from European ancestry. Percentage of predicted adult height is a non-invasive valid method to assess biological maturity.

Criterion for evaluating the predictive ability of nonlinear regression models without cross-validation.

PubMed

Kaneko, Hiromasa; Funatsu, Kimito

2013-09-23

We propose predictive performance criteria for nonlinear regression models without cross-validation. The proposed criteria are the determination coefficient and the root-mean-square error for the midpoints between k-nearest-neighbor data points. These criteria can be used to evaluate predictive ability after the regression models are updated, whereas cross-validation cannot be performed in such a situation. The proposed method is effective and helpful in handling big data when cross-validation cannot be applied. By analyzing data from numerical simulations and quantitative structural relationships, we confirm that the proposed criteria enable the predictive ability of the nonlinear regression models to be appropriately quantified.

External validation of the Cairns Prediction Model (CPM) to predict conversion from laparoscopic to open cholecystectomy.

PubMed

Hu, Alan Shiun Yew; Donohue, Peter O'; Gunnarsson, Ronny K; de Costa, Alan

2018-03-14

Valid and user-friendly prediction models for conversion to open cholecystectomy allow for proper planning prior to surgery. The Cairns Prediction Model (CPM) has been in use clinically in the original study site for the past three years, but has not been tested at other sites. A retrospective, single-centred study collected ultrasonic measurements and clinical variables alongside with conversion status from consecutive patients who underwent laparoscopic cholecystectomy from 2013 to 2016 in The Townsville Hospital, North Queensland, Australia. An area under the curve (AUC) was calculated to externally validate of the CPM. Conversion was necessary in 43 (4.2%) out of 1035 patients. External validation showed an area under the curve of 0.87 (95% CI 0.82-0.93, p = 1.1 × 10 -14 ). In comparison with most previously published models, which have an AUC of approximately 0.80 or less, the CPM has the highest AUC of all published prediction models both for internal and external validation. Crown Copyright © 2018. Published by Elsevier Inc. All rights reserved.

Testing the Predictive Validity and Construct of Pathological Video Game Use

PubMed Central

Groves, Christopher L.; Gentile, Douglas; Tapscott, Ryan L.; Lynch, Paul J.

2015-01-01

Three studies assessed the construct of pathological video game use and tested its predictive validity. Replicating previous research, Study 1 produced evidence of convergent validity in 8th and 9th graders (N = 607) classified as pathological gamers. Study 2 replicated and extended the findings of Study 1 with college undergraduates (N = 504). Predictive validity was established in Study 3 by measuring cue reactivity to video games in college undergraduates (N = 254), such that pathological gamers were more emotionally reactive to and provided higher subjective appraisals of video games than non-pathological gamers and non-gamers. The three studies converged to show that pathological video game use seems similar to other addictions in its patterns of correlations with other constructs. Conceptual and definitional aspects of Internet Gaming Disorder are discussed. PMID:26694472

Predictive value and construct validity of the work functioning screener-healthcare (WFS-H).

PubMed

Boezeman, Edwin J; Nieuwenhuijsen, Karen; Sluiter, Judith K

2016-05-25

To test the predictive value and convergent construct validity of a 6-item work functioning screener (WFS-H). Healthcare workers (249 nurses) completed a questionnaire containing the work functioning screener (WFS-H) and a work functioning instrument (NWFQ) measuring the following: cognitive aspects of task execution and general incidents, avoidance behavior, conflicts and irritation with colleagues, impaired contact with patients and their family, and level of energy and motivation. Productivity and mental health were also measured. Negative and positive predictive values, AUC values, and sensitivity and specificity were calculated to examine the predictive value of the screener. Correlation analysis was used to examine the construct validity. The screener had good predictive value, since the results showed that a negative screener score is a strong indicator of work functioning not hindered by mental health problems (negative predictive values: 94%-98%; positive predictive values: 21%-36%; AUC:.64-.82; sensitivity: 42%-76%; and specificity 85%-87%). The screener has good construct validity due to moderate, but significant (p<.001), associations with productivity (r=.51), mental health (r=.48), and distress (r=.47). The screener (WFS-H) had good predictive value and good construct validity. Its score offers occupational health professionals a helpful preliminary insight into the work functioning of healthcare workers.

The Predictive Validity of the University Student Selection Examination

ERIC Educational Resources Information Center

Karakaya, Ismail; Tavsancil, Ezel

2008-01-01

The main purpose of this study is to investigate the predictive validity of the 2003 University Student Selection Examination (OSS). For this purpose, freshman grade point average (FGPA) in higher education was predicted by raw scores, standard scores, and placement scores (YEP). This study has been conducted on a research group. In this study,…

Differential Predictive Validity of a Preschool Battery Across Race and Sex.

ERIC Educational Resources Information Center

Reynolds, Cecil R.

Determination of the fairness of preschool tests for use with children of varying cultural backgrounds is the major objective of this study. The predictive validity of a battery of preschool tests, chosen to represent the core areas of preschool assessment, across race and sex, was evaluated. Validity of the battery was examined over a 12-month…

Validation and Inter-comparison Against Observations of GODAE Ocean View Ocean Prediction Systems

NASA Astrophysics Data System (ADS)

Xu, J.; Davidson, F. J. M.; Smith, G. C.; Lu, Y.; Hernandez, F.; Regnier, C.; Drevillon, M.; Ryan, A.; Martin, M.; Spindler, T. D.; Brassington, G. B.; Oke, P. R.

2016-02-01

For weather forecasts, validation of forecast performance is done at the end user level as well as by the meteorological forecast centers. In the development of Ocean Prediction Capacity, the same level of care for ocean forecast performance and validation is needed. Herein we present results from a validation against observations of 6 Global Ocean Forecast Systems under the GODAE OceanView International Collaboration Network. These systems include the Global Ocean Ice Forecast System (GIOPS) developed by the Government of Canada, two systems PSY3 and PSY4 from the French Mercator-Ocean Ocean Forecasting Group, the FOAM system from UK met office, HYCOM-RTOFS from NOAA/NCEP/NWA of USA, and the Australian Bluelink-OceanMAPS system from the CSIRO, the Australian Meteorological Bureau and the Australian Navy.The observation data used in the comparison are sea surface temperature, sub-surface temperature, sub-surface salinity, sea level anomaly, and sea ice total concentration data. Results of the inter-comparison demonstrate forecast performance limits, strengths and weaknesses of each of the six systems. This work establishes validation protocols and routines by which all new prediction systems developed under the CONCEPTS Collaborative Network will be benchmarked prior to approval for operations. This includes anticipated delivery of CONCEPTS regional prediction systems over the next two years including a pan Canadian 1/12th degree resolution ice ocean prediction system and limited area 1/36th degree resolution prediction systems. The validation approach of comparing forecasts to observations at the time and location of the observation is called Class 4 metrics. It has been adopted by major international ocean prediction centers, and will be recommended to JCOMM-WMO as routine validation approach for operational oceanography worldwide.

The Predictive Validity of Dynamic Assessment: A Review

ERIC Educational Resources Information Center

Caffrey, Erin; Fuchs, Douglas; Fuchs, Lynn S.

2008-01-01

The authors report on a mixed-methods review of 24 studies that explores the predictive validity of dynamic assessment (DA). For 15 of the studies, they conducted quantitative analyses using Pearson's correlation coefficients. They descriptively examined the remaining studies to determine if their results were consistent with findings from the…

The predictive validity of safety climate.

PubMed

Johnson, Stephen E

2007-01-01

Safety professionals have increasingly turned their attention to social science for insight into the causation of industrial accidents. One social construct, safety climate, has been examined by several researchers [Cooper, M. D., & Phillips, R. A. (2004). Exploratory analysis of the safety climate and safety behavior relationship. Journal of Safety Research, 35(5), 497-512; Gillen, M., Baltz, D., Gassel, M., Kirsch, L., & Vacarro, D. (2002). Perceived safety climate, job Demands, and coworker support among union and nonunion injured construction workers. Journal of Safety Research, 33(1), 33-51; Neal, A., & Griffin, M. A. (2002). Safety climate and safety behaviour. Australian Journal of Management, 27, 66-76; Zohar, D. (2000). A group-level model of safety climate: Testing the effect of group climate on microaccidents in manufacturing jobs. Journal of Applied Psychology, 85(4), 587-596; Zohar, D., & Luria, G. (2005). A multilevel model of safety climate: Cross-level relationships between organization and group-level climates. Journal of Applied Psychology, 90(4), 616-628] who have documented its importance as a factor explaining the variation of safety-related outcomes (e.g., behavior, accidents). Researchers have developed instruments for measuring safety climate and have established some degree of psychometric reliability and validity. The problem, however, is that predictive validity has not been firmly established, which reduces the credibility of safety climate as a meaningful social construct. The research described in this article addresses this problem and provides additional support for safety climate as a viable construct and as a predictive indicator of safety-related outcomes. This study used 292 employees at three locations of a heavy manufacturing organization to complete the 16 item Zohar Safety Climate Questionnaire (ZSCQ) [Zohar, D., & Luria, G. (2005). A multilevel model of safety climate: Cross-level relationships between organization and group

Predictive Validation of an Influenza Spread Model

PubMed Central

Hyder, Ayaz; Buckeridge, David L.; Leung, Brian

2013-01-01

Background Modeling plays a critical role in mitigating impacts of seasonal influenza epidemics. Complex simulation models are currently at the forefront of evaluating optimal mitigation strategies at multiple scales and levels of organization. Given their evaluative role, these models remain limited in their ability to predict and forecast future epidemics leading some researchers and public-health practitioners to question their usefulness. The objective of this study is to evaluate the predictive ability of an existing complex simulation model of influenza spread. Methods and Findings We used extensive data on past epidemics to demonstrate the process of predictive validation. This involved generalizing an individual-based model for influenza spread and fitting it to laboratory-confirmed influenza infection data from a single observed epidemic (1998–1999). Next, we used the fitted model and modified two of its parameters based on data on real-world perturbations (vaccination coverage by age group and strain type). Simulating epidemics under these changes allowed us to estimate the deviation/error between the expected epidemic curve under perturbation and observed epidemics taking place from 1999 to 2006. Our model was able to forecast absolute intensity and epidemic peak week several weeks earlier with reasonable reliability and depended on the method of forecasting-static or dynamic. Conclusions Good predictive ability of influenza epidemics is critical for implementing mitigation strategies in an effective and timely manner. Through the process of predictive validation applied to a current complex simulation model of influenza spread, we provided users of the model (e.g. public-health officials and policy-makers) with quantitative metrics and practical recommendations on mitigating impacts of seasonal influenza epidemics. This methodology may be applied to other models of communicable infectious diseases to test and potentially improve their predictive

Validation of Accelerometer Prediction Equations in Children with Chronic Disease.

PubMed

Stephens, Samantha; Takken, Tim; Esliger, Dale W; Pullenayegum, Eleanor; Beyene, Joseph; Tremblay, Mark; Schneiderman, Jane; Biggar, Doug; Longmuir, Pat; McCrindle, Brian; Abad, Audrey; Ignas, Dan; Van Der Net, Janjaap; Feldman, Brian

2016-02-01

The purpose of this study was to assess the criterion validity of existing accelerometer-based energy expenditure (EE) prediction equations among children with chronic conditions, and to develop new prediction equations. Children with congenital heart disease (CHD), cystic fibrosis (CF), dermatomyositis (JDM), juvenile arthritis (JA), inherited muscle disease (IMD), and hemophilia (HE) completed 7 tasks while EE was measured using indirect calorimetry with counts determined by accelerometer. Agreement between predicted EE and measured EE was assessed. Disease-specific equations and cut points were developed and cross-validated. In total, 196 subjects participated. One participant dropped out before testing due to time constraints, while 15 CHD, 32 CF, 31 JDM, 31 JA, 30 IMD, 28 HE, and 29 healthy controls completed the study. Agreement between predicted and measured EE varied across disease group and ranged from (ICC) .13-.46. Disease-specific prediction equations exhibited a range of results (ICC .62-.88) (SE 0.45-0.78). In conclusion, poor agreement was demonstrated using current prediction equations in children with chronic conditions. Disease-specific equations and cut points were developed.

Taking the Next Step: Combining Incrementally Valid Indicators to Improve Recidivism Prediction

ERIC Educational Resources Information Center

Walters, Glenn D.

2011-01-01

The possibility of combining indicators to improve recidivism prediction was evaluated in a sample of released federal prisoners randomly divided into a derivation subsample (n = 550) and a cross-validation subsample (n = 551). Five incrementally valid indicators were selected from five domains: demographic (age), historical (prior convictions),…

Predicting functional outcomes among college drinkers: reliability and predictive validity of the Young Adult Alcohol Consequences Questionnaire.

PubMed

Read, Jennifer P; Merrill, Jennifer E; Kahler, Christopher W; Strong, David R

2007-11-01

Heavy drinking and associated consequences are widespread among U.S. college students. Recently, Read et al. (Read, J. P., Kahler, C. W., Strong, D., & Colder, C. R. (2006). Development and preliminary validation of the Young Adult Alcohol Consequences Questionnaire. Journal of Studies on Alcohol, 67, 169-178) developed the Young Adult Alcohol Consequences Questionnaire (YAACQ) to assess the broad range of consequences that may result from heavy drinking in the college milieu. In the present study, we sought to add to the psychometric validation of this measure by employing a prospective design to examine the test-retest reliability, concurrent validity, and predictive validity of the YAACQ. We also sought to examine the utility of the YAACQ administered early in the semester in the prediction of functional outcomes later in the semester, including the persistence of heavy drinking, and academic functioning. Ninety-two college students (48 females) completed a self-report assessment battery during the first weeks of the Fall semester, and approximately one week later. Additionally, 64 subjects (37 females) participated at an optional third time point at the end of the semester. Overall, the YAACQ demonstrated strong internal consistency, test-retest reliability, and concurrent and predictive validity. YAACQ scores also were predictive of both drinking frequency, and "binge" drinking frequency. YAACQ total scores at baseline were an early indicator of academic performance later in the semester, with greater number of total consequences experienced being negatively associated with end-of-semester grade point average. Specific YAACQ subscale scores (Impaired Control, Dependence Symptoms, Blackout Drinking) showed unique prediction of persistent drinking and academic outcomes.

Investigating Postgraduate College Admission Interviews: Generalizability Theory Reliability and Incremental Predictive Validity

ERIC Educational Resources Information Center

Arce-Ferrer, Alvaro J.; Castillo, Irene Borges

2007-01-01

The use of face-to-face interviews is controversial for college admissions decisions in light of the lack of availability of validity and reliability evidence for most college admission processes. This study investigated reliability and incremental predictive validity of a face-to-face postgraduate college admission interview with a sample of…

Role of learning potential in cognitive remediation: Construct and predictive validity.

PubMed

Davidson, Charlie A; Johannesen, Jason K; Fiszdon, Joanna M

2016-03-01

The construct, convergent, discriminant, and predictive validity of Learning Potential (LP) was evaluated in a trial of cognitive remediation for adults with schizophrenia-spectrum disorders. LP utilizes a dynamic assessment approach to prospectively estimate an individual's learning capacity if provided the opportunity for specific related learning. LP was assessed in 75 participants at study entry, of whom 41 completed an eight-week cognitive remediation (CR) intervention, and 22 received treatment-as-usual (TAU). LP was assessed in a "test-train-test" verbal learning paradigm. Incremental predictive validity was assessed as the degree to which LP predicted memory skill acquisition above and beyond prediction by static verbal learning ability. Examination of construct validity confirmed that LP scores reflected use of trained semantic clustering strategy. LP scores correlated with executive functioning and education history, but not other demographics or symptom severity. Following the eight-week active phase, TAU evidenced little substantial change in skill acquisition outcomes, which related to static baseline verbal learning ability but not LP. For the CR group, LP significantly predicted skill acquisition in domains of verbal and visuospatial memory, but not auditory working memory. Furthermore, LP predicted skill acquisition incrementally beyond relevant background characteristics, symptoms, and neurocognitive abilities. Results suggest that LP assessment can significantly improve prediction of specific skill acquisition with cognitive training, particularly for the domain assessed, and thereby may prove useful in individualization of treatment. Published by Elsevier B.V.

Modern modeling techniques had limited external validity in predicting mortality from traumatic brain injury.

PubMed

van der Ploeg, Tjeerd; Nieboer, Daan; Steyerberg, Ewout W

2016-10-01

Prediction of medical outcomes may potentially benefit from using modern statistical modeling techniques. We aimed to externally validate modeling strategies for prediction of 6-month mortality of patients suffering from traumatic brain injury (TBI) with predictor sets of increasing complexity. We analyzed individual patient data from 15 different studies including 11,026 TBI patients. We consecutively considered a core set of predictors (age, motor score, and pupillary reactivity), an extended set with computed tomography scan characteristics, and a further extension with two laboratory measurements (glucose and hemoglobin). With each of these sets, we predicted 6-month mortality using default settings with five statistical modeling techniques: logistic regression (LR), classification and regression trees, random forests (RFs), support vector machines (SVM) and neural nets. For external validation, a model developed on one of the 15 data sets was applied to each of the 14 remaining sets. This process was repeated 15 times for a total of 630 validations. The area under the receiver operating characteristic curve (AUC) was used to assess the discriminative ability of the models. For the most complex predictor set, the LR models performed best (median validated AUC value, 0.757), followed by RF and support vector machine models (median validated AUC value, 0.735 and 0.732, respectively). With each predictor set, the classification and regression trees models showed poor performance (median validated AUC value, <0.7). The variability in performance across the studies was smallest for the RF- and LR-based models (inter quartile range for validated AUC values from 0.07 to 0.10). In the area of predicting mortality from TBI, nonlinear and nonadditive effects are not pronounced enough to make modern prediction methods beneficial. Copyright © 2016 Elsevier Inc. All rights reserved.

Review and evaluation of performance measures for survival prediction models in external validation settings.

PubMed

Rahman, M Shafiqur; Ambler, Gareth; Choodari-Oskooei, Babak; Omar, Rumana Z

2017-04-18

When developing a prediction model for survival data it is essential to validate its performance in external validation settings using appropriate performance measures. Although a number of such measures have been proposed, there is only limited guidance regarding their use in the context of model validation. This paper reviewed and evaluated a wide range of performance measures to provide some guidelines for their use in practice. An extensive simulation study based on two clinical datasets was conducted to investigate the performance of the measures in external validation settings. Measures were selected from categories that assess the overall performance, discrimination and calibration of a survival prediction model. Some of these have been modified to allow their use with validation data, and a case study is provided to describe how these measures can be estimated in practice. The measures were evaluated with respect to their robustness to censoring and ease of interpretation. All measures are implemented, or are straightforward to implement, in statistical software. Most of the performance measures were reasonably robust to moderate levels of censoring. One exception was Harrell's concordance measure which tended to increase as censoring increased. We recommend that Uno's concordance measure is used to quantify concordance when there are moderate levels of censoring. Alternatively, Gönen and Heller's measure could be considered, especially if censoring is very high, but we suggest that the prediction model is re-calibrated first. We also recommend that Royston's D is routinely reported to assess discrimination since it has an appealing interpretation. The calibration slope is useful for both internal and external validation settings and recommended to report routinely. Our recommendation would be to use any of the predictive accuracy measures and provide the corresponding predictive accuracy curves. In addition, we recommend to investigate the characteristics

Predicting the ungauged basin: model validation and realism assessment

NASA Astrophysics Data System (ADS)

van Emmerik, Tim; Mulder, Gert; Eilander, Dirk; Piet, Marijn; Savenije, Hubert

2016-04-01

The hydrological decade on Predictions in Ungauged Basins (PUB) [1] led to many new insights in model development, calibration strategies, data acquisition and uncertainty analysis. Due to a limited amount of published studies on genuinely ungauged basins, model validation and realism assessment of model outcome has not been discussed to a great extent. With this study [2] we aim to contribute to the discussion on how one can determine the value and validity of a hydrological model developed for an ungauged basin. As in many cases no local, or even regional, data are available, alternative methods should be applied. Using a PUB case study in a genuinely ungauged basin in southern Cambodia, we give several examples of how one can use different types of soft data to improve model design, calibrate and validate the model, and assess the realism of the model output. A rainfall-runoff model was coupled to an irrigation reservoir, allowing the use of additional and unconventional data. The model was mainly forced with remote sensing data, and local knowledge was used to constrain the parameters. Model realism assessment was done using data from surveys. This resulted in a successful reconstruction of the reservoir dynamics, and revealed the different hydrological characteristics of the two topographical classes. We do not present a generic approach that can be transferred to other ungauged catchments, but we aim to show how clever model design and alternative data acquisition can result in a valuable hydrological model for ungauged catchments. [1] Sivapalan, M., Takeuchi, K., Franks, S., Gupta, V., Karambiri, H., Lakshmi, V., et al. (2003). IAHS decade on predictions in ungauged basins (PUB), 2003-2012: shaping an exciting future for the hydrological sciences. Hydrol. Sci. J. 48, 857-880. doi: 10.1623/hysj.48.6.857.51421 [2] van Emmerik, T., Mulder, G., Eilander, D., Piet, M. and Savenije, H. (2015). Predicting the ungauged basin: model validation and realism assessment

Predictive Validity of Curriculum-Based Measures for English Learners at Varying English Proficiency Levels

ERIC Educational Resources Information Center

Kim, Jennifer Sun; Vanderwood, Michael L.; Lee, Catherine Y.

2016-01-01

This study examined the predictive validity of curriculum-based measures in reading for Spanish-speaking English learners (ELs) at various levels of English proficiency. Third-grade Spanish-speaking EL students were screened during the fall using DIBELS Oral Reading Fluency (DORF) and Daze. Predictive validity was examined in relation to spring…

Validating Inertial Confinement Fusion (ICF) predictive capability using perturbed capsules

NASA Astrophysics Data System (ADS)

Schmitt, Mark; Magelssen, Glenn; Tregillis, Ian; Hsu, Scott; Bradley, Paul; Dodd, Evan; Cobble, James; Flippo, Kirk; Offerman, Dustin; Obrey, Kimberly; Wang, Yi-Ming; Watt, Robert; Wilke, Mark; Wysocki, Frederick; Batha, Steven

2009-11-01

Achieving ignition on NIF is a monumental step on the path toward utilizing fusion as a controlled energy source. Obtaining robust ignition requires accurate ICF models to predict the degradation of ignition caused by heterogeneities in capsule construction and irradiation. LANL has embarked on a project to induce controlled defects in capsules to validate our ability to predict their effects on fusion burn. These efforts include the validation of feature-driven hydrodynamics and mix in a convergent geometry. This capability is needed to determine the performance of capsules imploded under less-than-optimum conditions on future IFE facilities. LANL's recently initiated Defect Implosion Experiments (DIME) conducted at Rochester's Omega facility are providing input for these efforts. Recent simulation and experimental results will be shown.

Predictive validity of the Biomedical Admissions Test: an evaluation and case study.

PubMed

McManus, I C; Ferguson, Eamonn; Wakeford, Richard; Powis, David; James, David

2011-01-01

There has been an increase in the use of pre-admission selection tests for medicine. Such tests need to show good psychometric properties. Here, we use a paper by Emery and Bell [2009. The predictive validity of the Biomedical Admissions Test for pre-clinical examination performance. Med Educ 43:557-564] as a case study to evaluate and comment on the reporting of psychometric data in the field of medical student selection (and the comments apply to many papers in the field). We highlight pitfalls when reliability data are not presented, how simple zero-order associations can lead to inaccurate conclusions about the predictive validity of a test, and how biases need to be explored and reported. We show with BMAT that it is the knowledge part of the test which does all the predictive work. We show that without evidence of incremental validity it is difficult to assess the value of any selection tests for medicine.

Geographic and temporal validity of prediction models: Different approaches were useful to examine model performance

PubMed Central

Austin, Peter C.; van Klaveren, David; Vergouwe, Yvonne; Nieboer, Daan; Lee, Douglas S.; Steyerberg, Ewout W.

2017-01-01

Objective Validation of clinical prediction models traditionally refers to the assessment of model performance in new patients. We studied different approaches to geographic and temporal validation in the setting of multicenter data from two time periods. Study Design and Setting We illustrated different analytic methods for validation using a sample of 14,857 patients hospitalized with heart failure at 90 hospitals in two distinct time periods. Bootstrap resampling was used to assess internal validity. Meta-analytic methods were used to assess geographic transportability. Each hospital was used once as a validation sample, with the remaining hospitals used for model derivation. Hospital-specific estimates of discrimination (c-statistic) and calibration (calibration intercepts and slopes) were pooled using random effects meta-analysis methods. I2 statistics and prediction interval width quantified geographic transportability. Temporal transportability was assessed using patients from the earlier period for model derivation and patients from the later period for model validation. Results Estimates of reproducibility, pooled hospital-specific performance, and temporal transportability were on average very similar, with c-statistics of 0.75. Between-hospital variation was moderate according to I2 statistics and prediction intervals for c-statistics. Conclusion This study illustrates how performance of prediction models can be assessed in settings with multicenter data at different time periods. PMID:27262237

Efficient strategies for leave-one-out cross validation for genomic best linear unbiased prediction.

PubMed

Cheng, Hao; Garrick, Dorian J; Fernando, Rohan L

2017-01-01

A random multiple-regression model that simultaneously fit all allele substitution effects for additive markers or haplotypes as uncorrelated random effects was proposed for Best Linear Unbiased Prediction, using whole-genome data. Leave-one-out cross validation can be used to quantify the predictive ability of a statistical model. Naive application of Leave-one-out cross validation is computationally intensive because the training and validation analyses need to be repeated n times, once for each observation. Efficient Leave-one-out cross validation strategies are presented here, requiring little more effort than a single analysis. Efficient Leave-one-out cross validation strategies is 786 times faster than the naive application for a simulated dataset with 1,000 observations and 10,000 markers and 99 times faster with 1,000 observations and 100 markers. These efficiencies relative to the naive approach using the same model will increase with increases in the number of observations. Efficient Leave-one-out cross validation strategies are presented here, requiring little more effort than a single analysis.

Validity of the Student Risk Screening Scale: Evidence of Predictive Validity in a Diverse, Suburban Elementary Setting

ERIC Educational Resources Information Center

Menzies, Holly M.; Lane, Kathleen Lynne

2012-01-01

In this study the authors examined the psychometric properties of the "Student Risk Screening Scale" (SRSS), including predictive validity in terms of student outcomes in behavioral and academic domains. The school, a diverse, suburban school in Southern California, administered the SRSS at three time points as part of regular school…

The Predictive Validity of the Minnesota Reading Assessment for Students in Postsecondary Vocational Education Programs.

ERIC Educational Resources Information Center

Brown, James M.; Chang, Gerald

1982-01-01

The predictive validity of the Minnesota Reading Assessment (MRA) when used to project potential performance of postsecondary vocational-technical education students was examined. Findings confirmed the MRA to be a valid predictor, although the error in prediction varied between the criterion variables. (Author/GK)

Predictive validity of the Braden Scale, Norton Scale, and Waterlow Scale in the Czech Republic.

PubMed

Šateková, Lenka; Žiaková, Katarína; Zeleníková, Renáta

2017-02-01

The aim of this study was to determine the predictive validity of the Braden, Norton, and Waterlow scales in 2 long-term care departments in the Czech Republic. Assessing the risk for developing pressure ulcers is the first step in their prevention. At present, many scales are used in clinical practice, but most of them have not been properly validated yet (for example, the Modified Norton Scale in the Czech Republic). In the Czech Republic, only the Braden Scale has been validated so far. This is a prospective comparative instrument testing study. A random sample of 123 patients was recruited. The predictive validity of the pressure ulcer risk assessment scales was evaluated based on sensitivity, specificity, positive and negative predictive values, and the area under the receiver operating characteristic curve. The data were collected from April to August 2014. In the present study, the best predictive validity values were observed for the Norton Scale, followed by the Braden Scale and the Waterlow Scale, in that order. We recommended that the above 3 pressure ulcer risk assessment scales continue to be evaluated in the Czech clinical setting. © 2016 John Wiley & Sons Australia, Ltd.

Predicting Pilot Error in Nextgen: Pilot Performance Modeling and Validation Efforts

NASA Technical Reports Server (NTRS)

Wickens, Christopher; Sebok, Angelia; Gore, Brian; Hooey, Becky

2012-01-01

We review 25 articles presenting 5 general classes of computational models to predict pilot error. This more targeted review is placed within the context of the broader review of computational models of pilot cognition and performance, including such aspects as models of situation awareness or pilot-automation interaction. Particular emphasis is placed on the degree of validation of such models against empirical pilot data, and the relevance of the modeling and validation efforts to Next Gen technology and procedures.

Implementation and Initial Validation of the APS English Test [and] The APS English-Writing Test at Golden West College: Evidence for Predictive Validity.

ERIC Educational Resources Information Center

Isonio, Steven

In May 1991, Golden West College (California) conducted a validation study of the English portion of the Assessment and Placement Services for Community Colleges (APS), followed by a predictive validity study in July 1991. The initial study was designed to aid in the implementation of the new test at GWC by comparing data on APS use at other…

Predicting child maltreatment: A meta-analysis of the predictive validity of risk assessment instruments.

PubMed

van der Put, Claudia E; Assink, Mark; Boekhout van Solinge, Noëlle F

2017-11-01

Risk assessment is crucial in preventing child maltreatment since it can identify high-risk cases in need of child protection intervention. Despite widespread use of risk assessment instruments in child welfare, it is unknown how well these instruments predict maltreatment and what instrument characteristics are associated with higher levels of predictive validity. Therefore, a multilevel meta-analysis was conducted to examine the predictive accuracy of (characteristics of) risk assessment instruments. A literature search yielded 30 independent studies (N=87,329) examining the predictive validity of 27 different risk assessment instruments. From these studies, 67 effect sizes could be extracted. Overall, a medium significant effect was found (AUC=0.681), indicating a moderate predictive accuracy. Moderator analyses revealed that onset of maltreatment can be better predicted than recurrence of maltreatment, which is a promising finding for early detection and prevention of child maltreatment. In addition, actuarial instruments were found to outperform clinical instruments. To bring risk and needs assessment in child welfare to a higher level, actuarial instruments should be further developed and strengthened by distinguishing risk assessment from needs assessment and by integrating risk assessment with case management. Copyright © 2017 Elsevier Ltd. All rights reserved.

The Validity of College Grade Prediction Equations Over Time.

ERIC Educational Resources Information Center

Sawyer, Richard L.; Maxey, James

A sample of 260 colleges was surveyed during the years 1972-1976 to determine the validity of predicting college freshmen grades from standardized test scores and high school grades using the American College Testing (ACT) Assessment Program, an evaluative and placement service for students and educators involved in the transition from high school…

Thirty-Year Stability and Predictive Validity of Vocational Interests

ERIC Educational Resources Information Center

Rottinghaus, Patrick J.; Coon, Kristin L.; Gaffey, Abigail R.; Zytowski, Donald G.

2007-01-01

This study reports a 30-year follow-up of 107 former high school juniors and seniors from a rural Midwestern community who completed the Kuder Occupational Interest Survey (KOIS) in 1975 and 2005. Absolute, intra-individual, and test-retest stability of interests, and predictive validity of occupations were examined. Results showed minor absolute…

Acute Brain Dysfunction: Development and Validation of a Daily Prediction Model.

PubMed

Marra, Annachiara; Pandharipande, Pratik P; Shotwell, Matthew S; Chandrasekhar, Rameela; Girard, Timothy D; Shintani, Ayumi K; Peelen, Linda M; Moons, Karl G M; Dittus, Robert S; Ely, E Wesley; Vasilevskis, Eduard E

2018-03-24

The goal of this study was to develop and validate a dynamic risk model to predict daily changes in acute brain dysfunction (ie, delirium and coma), discharge, and mortality in ICU patients. Using data from a multicenter prospective ICU cohort, a daily acute brain dysfunction-prediction model (ABD-pm) was developed by using multinomial logistic regression that estimated 15 transition probabilities (from one of three brain function states [normal, delirious, or comatose] to one of five possible outcomes [normal, delirious, comatose, ICU discharge, or died]) using baseline and daily risk factors. Model discrimination was assessed by using predictive characteristics such as negative predictive value (NPV). Calibration was assessed by plotting empirical vs model-estimated probabilities. Internal validation was performed by using a bootstrap procedure. Data were analyzed from 810 patients (6,711 daily transitions). The ABD-pm included individual risk factors: mental status, age, preexisting cognitive impairment, baseline and daily severity of illness, and daily administration of sedatives. The model yielded very high NPVs for "next day" delirium (NPV: 0.823), coma (NPV: 0.892), normal cognitive state (NPV: 0.875), ICU discharge (NPV: 0.905), and mortality (NPV: 0.981). The model demonstrated outstanding calibration when predicting the total number of patients expected to be in any given state across predicted risk. We developed and internally validated a dynamic risk model that predicts the daily risk for one of three cognitive states, ICU discharge, or mortality. The ABD-pm may be useful for predicting the proportion of patients for each outcome state across entire ICU populations to guide quality, safety, and care delivery activities. Copyright © 2018 American College of Chest Physicians. Published by Elsevier Inc. All rights reserved.

Genomic prediction in animals and plants: simulation of data, validation, reporting, and benchmarking.

PubMed

Daetwyler, Hans D; Calus, Mario P L; Pong-Wong, Ricardo; de Los Campos, Gustavo; Hickey, John M

2013-02-01

The genomic prediction of phenotypes and breeding values in animals and plants has developed rapidly into its own research field. Results of genomic prediction studies are often difficult to compare because data simulation varies, real or simulated data are not fully described, and not all relevant results are reported. In addition, some new methods have been compared only in limited genetic architectures, leading to potentially misleading conclusions. In this article we review simulation procedures, discuss validation and reporting of results, and apply benchmark procedures for a variety of genomic prediction methods in simulated and real example data. Plant and animal breeding programs are being transformed by the use of genomic data, which are becoming widely available and cost-effective to predict genetic merit. A large number of genomic prediction studies have been published using both simulated and real data. The relative novelty of this area of research has made the development of scientific conventions difficult with regard to description of the real data, simulation of genomes, validation and reporting of results, and forward in time methods. In this review article we discuss the generation of simulated genotype and phenotype data, using approaches such as the coalescent and forward in time simulation. We outline ways to validate simulated data and genomic prediction results, including cross-validation. The accuracy and bias of genomic prediction are highlighted as performance indicators that should be reported. We suggest that a measure of relatedness between the reference and validation individuals be reported, as its impact on the accuracy of genomic prediction is substantial. A large number of methods were compared in example simulated and real (pine and wheat) data sets, all of which are publicly available. In our limited simulations, most methods performed similarly in traits with a large number of quantitative trait loci (QTL), whereas in traits

Genomic Prediction in Animals and Plants: Simulation of Data, Validation, Reporting, and Benchmarking

PubMed Central

Daetwyler, Hans D.; Calus, Mario P. L.; Pong-Wong, Ricardo; de los Campos, Gustavo; Hickey, John M.

2013-01-01

The genomic prediction of phenotypes and breeding values in animals and plants has developed rapidly into its own research field. Results of genomic prediction studies are often difficult to compare because data simulation varies, real or simulated data are not fully described, and not all relevant results are reported. In addition, some new methods have been compared only in limited genetic architectures, leading to potentially misleading conclusions. In this article we review simulation procedures, discuss validation and reporting of results, and apply benchmark procedures for a variety of genomic prediction methods in simulated and real example data. Plant and animal breeding programs are being transformed by the use of genomic data, which are becoming widely available and cost-effective to predict genetic merit. A large number of genomic prediction studies have been published using both simulated and real data. The relative novelty of this area of research has made the development of scientific conventions difficult with regard to description of the real data, simulation of genomes, validation and reporting of results, and forward in time methods. In this review article we discuss the generation of simulated genotype and phenotype data, using approaches such as the coalescent and forward in time simulation. We outline ways to validate simulated data and genomic prediction results, including cross-validation. The accuracy and bias of genomic prediction are highlighted as performance indicators that should be reported. We suggest that a measure of relatedness between the reference and validation individuals be reported, as its impact on the accuracy of genomic prediction is substantial. A large number of methods were compared in example simulated and real (pine and wheat) data sets, all of which are publicly available. In our limited simulations, most methods performed similarly in traits with a large number of quantitative trait loci (QTL), whereas in traits

Experimental validation of boundary element methods for noise prediction

NASA Technical Reports Server (NTRS)

Seybert, A. F.; Oswald, Fred B.

1992-01-01

Experimental validation of methods to predict radiated noise is presented. A combined finite element and boundary element model was used to predict the vibration and noise of a rectangular box excited by a mechanical shaker. The predicted noise was compared to sound power measured by the acoustic intensity method. Inaccuracies in the finite element model shifted the resonance frequencies by about 5 percent. The predicted and measured sound power levels agree within about 2.5 dB. In a second experiment, measured vibration data was used with a boundary element model to predict noise radiation from the top of an operating gearbox. The predicted and measured sound power for the gearbox agree within about 3 dB.

Validating spatiotemporal predictions of an important pest of small grains.

PubMed

Merrill, Scott C; Holtzer, Thomas O; Peairs, Frank B; Lester, Philip J

2015-01-01

Arthropod pests are typically managed using tactics applied uniformly to the whole field. Precision pest management applies tactics under the assumption that within-field pest pressure differences exist. This approach allows for more precise and judicious use of scouting resources and management tactics. For example, a portion of a field delineated as attractive to pests may be selected to receive extra monitoring attention. Likely because of the high variability in pest dynamics, little attention has been given to developing precision pest prediction models. Here, multimodel synthesis was used to develop a spatiotemporal model predicting the density of a key pest of wheat, the Russian wheat aphid, Diuraphis noxia (Kurdjumov). Spatially implicit and spatially explicit models were synthesized to generate spatiotemporal pest pressure predictions. Cross-validation and field validation were used to confirm model efficacy. A strong within-field signal depicting aphid density was confirmed with low prediction errors. Results show that the within-field model predictions will provide higher-quality information than would be provided by traditional field scouting. With improvements to the broad-scale model component, the model synthesis approach and resulting tool could improve pest management strategy and provide a template for the development of spatially explicit pest pressure models. © 2014 Society of Chemical Industry.

Developing and validating risk prediction models in an individual participant data meta-analysis

PubMed Central

2014-01-01

Background Risk prediction models estimate the risk of developing future outcomes for individuals based on one or more underlying characteristics (predictors). We review how researchers develop and validate risk prediction models within an individual participant data (IPD) meta-analysis, in order to assess the feasibility and conduct of the approach. Methods A qualitative review of the aims, methodology, and reporting in 15 articles that developed a risk prediction model using IPD from multiple studies. Results The IPD approach offers many opportunities but methodological challenges exist, including: unavailability of requested IPD, missing patient data and predictors, and between-study heterogeneity in methods of measurement, outcome definitions and predictor effects. Most articles develop their model using IPD from all available studies and perform only an internal validation (on the same set of data). Ten of the 15 articles did not allow for any study differences in baseline risk (intercepts), potentially limiting their model’s applicability and performance in some populations. Only two articles used external validation (on different data), including a novel method which develops the model on all but one of the IPD studies, tests performance in the excluded study, and repeats by rotating the omitted study. Conclusions An IPD meta-analysis offers unique opportunities for risk prediction research. Researchers can make more of this by allowing separate model intercept terms for each study (population) to improve generalisability, and by using ‘internal-external cross-validation’ to simultaneously develop and validate their model. Methodological challenges can be reduced by prospectively planned collaborations that share IPD for risk prediction. PMID:24397587

Prediction of Primary Care Depression Outcomes at Six Months: Validation of DOC-6 ©.

PubMed

Angstman, Kurt B; Garrison, Gregory M; Gonzalez, Cesar A; Cozine, Daniel W; Cozine, Elizabeth W; Katzelnick, David J

2017-01-01

The goal of this study was to develop and validate an assessment tool for adult primary care patients diagnosed with depression to determine predictive probability of clinical outcomes at 6 months. We retrospectively reviewed 3096 adult patients enrolled in collaborative care management (CCM) for depression. Patients enrolled on or before December 31, 2013, served as the training set (n = 2525), whereas those enrolled after that date served as the preliminary validation set (n = 571). Six variables (2 demographic and 4 clinical) were statistically significant in determining clinical outcomes. Using the validation data set, the remission classifier produced the receiver operating characteristics (ROC) curve with a c-statistic or area under the curve (AUC) of 0.62 with predicted probabilities than ranged from 14.5% to 79.1%, with a median of 50.6%. The persistent depressive symptoms (PDS) classifier produced an ROC curve with a c-statistic or AUC of 0.67 and predicted probabilities that ranged from 5.5% to 73.1%, with a median of 23.5%. We were able to identify readily available variables and then validated these in the prediction of depression remission and PDS at 6 months. The DOC-6 tool may be used to predict which patients may be at risk for worse outcomes. © Copyright 2017 by the American Board of Family Medicine.

Temporal and external validation of a prediction model for adverse outcomes among inpatients with diabetes.

PubMed

Adderley, N J; Mallett, S; Marshall, T; Ghosh, S; Rayman, G; Bellary, S; Coleman, J; Akiboye, F; Toulis, K A; Nirantharakumar, K

2018-06-01

To temporally and externally validate our previously developed prediction model, which used data from University Hospitals Birmingham to identify inpatients with diabetes at high risk of adverse outcome (mortality or excessive length of stay), in order to demonstrate its applicability to other hospital populations within the UK. Temporal validation was performed using data from University Hospitals Birmingham and external validation was performed using data from both the Heart of England NHS Foundation Trust and Ipswich Hospital. All adult inpatients with diabetes were included. Variables included in the model were age, gender, ethnicity, admission type, intensive therapy unit admission, insulin therapy, albumin, sodium, potassium, haemoglobin, C-reactive protein, estimated GFR and neutrophil count. Adverse outcome was defined as excessive length of stay or death. Model discrimination in the temporal and external validation datasets was good. In temporal validation using data from University Hospitals Birmingham, the area under the curve was 0.797 (95% CI 0.785-0.810), sensitivity was 70% (95% CI 67-72) and specificity was 75% (95% CI 74-76). In external validation using data from Heart of England NHS Foundation Trust, the area under the curve was 0.758 (95% CI 0.747-0.768), sensitivity was 73% (95% CI 71-74) and specificity was 66% (95% CI 65-67). In external validation using data from Ipswich, the area under the curve was 0.736 (95% CI 0.711-0.761), sensitivity was 63% (95% CI 59-68) and specificity was 69% (95% CI 67-72). These results were similar to those for the internally validated model derived from University Hospitals Birmingham. The prediction model to identify patients with diabetes at high risk of developing an adverse event while in hospital performed well in temporal and external validation. The externally validated prediction model is a novel tool that can be used to improve care pathways for inpatients with diabetes. Further research to assess

Validity of the SAT® for Predicting First-Year Grades: 2009 SAT Validity Sample. Statistical Report No. 2012-2

ERIC Educational Resources Information Center

Patterson, Brian F.; Mattern, Krista D.

2009-01-01

In an effort to continuously monitor the validity of the SAT for predicting first-year college grades, the College Board has continued its multi-year effort to recruit four-year colleges and universities (henceforth, "institutions") to provide data on the cohorts of first-time, first-year students entering in the fall semester beginning…

CFD Validation Studies for Hypersonic Flow Prediction

NASA Technical Reports Server (NTRS)

Gnoffo, Peter A.

2001-01-01

A series of experiments to measure pressure and heating for code validation involving hypersonic, laminar, separated flows was conducted at the Calspan-University at Buffalo Research Center (CUBRC) in the Large Energy National Shock (LENS) tunnel. The experimental data serves as a focus for a code validation session but are not available to the authors until the conclusion of this session. The first set of experiments considered here involve Mach 9.5 and Mach 11.3 N2 flow over a hollow cylinder-flare with 30 degree flare angle at several Reynolds numbers sustaining laminar, separated flow. Truncated and extended flare configurations are considered. The second set of experiments, at similar conditions, involves flow over a sharp, double cone with fore-cone angle of 25 degrees and aft-cone angle of 55 degrees. Both sets of experiments involve 30 degree compressions. Location of the separation point in the numerical simulation is extremely sensitive to the level of grid refinement in the numerical predictions. The numerical simulations also show a significant influence of Reynolds number on extent of separation. Flow unsteadiness was easily introduced into the double cone simulations using aggressive relaxation parameters that normally promote convergence.

CFD Validation Studies for Hypersonic Flow Prediction

NASA Technical Reports Server (NTRS)

Gnoffo, Peter A.

2001-01-01

A series of experiments to measure pressure and heating for code validation involving hypersonic, laminar, separated flows was conducted at the Calspan-University at Buffalo Research Center (CUBRC) in the Large Energy National Shock (LENS) tunnel. The experimental data serves as a focus for a code validation session but are not available to the authors until the conclusion of this session. The first set of experiments considered here involve Mach 9.5 and Mach 11.3 N, flow over a hollow cylinder-flare with 30 deg flare angle at several Reynolds numbers sustaining laminar, separated flow. Truncated and extended flare configurations are considered. The second set of experiments, at similar conditions, involves flow over a sharp, double cone with fore-cone angle of 25 deg and aft-cone angle of 55 deg. Both sets of experiments involve 30 deg compressions. Location of the separation point in the numerical simulation is extremely sensitive to the level of grid refinement in the numerical predictions. The numerical simulations also show a significant influence of Reynolds number on extent of separation. Flow unsteadiness was easily introduced into the double cone simulations using aggressive relaxation parameters that normally promote convergence.

A Public-Private Partnership Develops and Externally Validates a 30-Day Hospital Readmission Risk Prediction Model

PubMed Central

Choudhry, Shahid A.; Li, Jing; Davis, Darcy; Erdmann, Cole; Sikka, Rishi; Sutariya, Bharat

2013-01-01

Introduction: Preventing the occurrence of hospital readmissions is needed to improve quality of care and foster population health across the care continuum. Hospitals are being held accountable for improving transitions of care to avert unnecessary readmissions. Advocate Health Care in Chicago and Cerner (ACC) collaborated to develop all-cause, 30-day hospital readmission risk prediction models to identify patients that need interventional resources. Ideally, prediction models should encompass several qualities: they should have high predictive ability; use reliable and clinically relevant data; use vigorous performance metrics to assess the models; be validated in populations where they are applied; and be scalable in heterogeneous populations. However, a systematic review of prediction models for hospital readmission risk determined that most performed poorly (average C-statistic of 0.66) and efforts to improve their performance are needed for widespread usage. Methods: The ACC team incorporated electronic health record data, utilized a mixed-method approach to evaluate risk factors, and externally validated their prediction models for generalizability. Inclusion and exclusion criteria were applied on the patient cohort and then split for derivation and internal validation. Stepwise logistic regression was performed to develop two predictive models: one for admission and one for discharge. The prediction models were assessed for discrimination ability, calibration, overall performance, and then externally validated. Results: The ACC Admission and Discharge Models demonstrated modest discrimination ability during derivation, internal and external validation post-recalibration (C-statistic of 0.76 and 0.78, respectively), and reasonable model fit during external validation for utility in heterogeneous populations. Conclusions: The ACC Admission and Discharge Models embody the design qualities of ideal prediction models. The ACC plans to continue its partnership to

Beware of external validation! - A Comparative Study of Several Validation Techniques used in QSAR Modelling.

PubMed

Majumdar, Subhabrata; Basak, Subhash C

2018-04-26

Proper validation is an important aspect of QSAR modelling. External validation is one of the widely used validation methods in QSAR where the model is built on a subset of the data and validated on the rest of the samples. However, its effectiveness for datasets with a small number of samples but large number of predictors remains suspect. Calculating hundreds or thousands of molecular descriptors using currently available software has become the norm in QSAR research, owing to computational advances in the past few decades. Thus, for n chemical compounds and p descriptors calculated for each molecule, the typical chemometric dataset today has high value of p but small n (i.e. n < p). Motivated by the evidence of inadequacies of external validation in estimating the true predictive capability of a statistical model in recent literature, this paper performs an extensive and comparative study of this method with several other validation techniques. We compared four validation methods: leave-one-out, K-fold, external and multi-split validation, using statistical models built using the LASSO regression, which simultaneously performs variable selection and modelling. We used 300 simulated datasets and one real dataset of 95 congeneric amine mutagens for this evaluation. External validation metrics have high variation among different random splits of the data, hence are not recommended for predictive QSAR models. LOO has the overall best performance among all validation methods applied in our scenario. Results from external validation are too unstable for the datasets we analyzed. Based on our findings, we recommend using the LOO procedure for validating QSAR predictive models built on high-dimensional small-sample data. Copyright© Bentham Science Publishers; For any queries, please email at epub@benthamscience.org.

The predictive validity of ideal partner preferences: a review and meta-analysis.

PubMed

Eastwick, Paul W; Luchies, Laura B; Finkel, Eli J; Hunt, Lucy L

2014-05-01

A central element of interdependence theory is that people have standards against which they compare their current outcomes, and one ubiquitous standard in the mating domain is the preference for particular attributes in a partner (ideal partner preferences). This article reviews research on the predictive validity of ideal partner preferences and presents a new integrative model that highlights when and why ideals succeed or fail to predict relational outcomes. Section 1 examines predictive validity by reviewing research on sex differences in the preference for physical attractiveness and earning prospects. Men and women reliably differ in the extent to which these qualities affect their romantic evaluations of hypothetical targets. Yet a new meta-analysis spanning the attraction and relationships literatures (k = 97) revealed that physical attractiveness predicted romantic evaluations with a moderate-to-strong effect size (r = ∼.40) for both sexes, and earning prospects predicted romantic evaluations with a small effect size (r = ∼.10) for both sexes. Sex differences in the correlations were small (r difference = .03) and uniformly nonsignificant. Section 2 reviews research on individual differences in ideal partner preferences, drawing from several theoretical traditions to explain why ideals predict relational evaluations at different relationship stages. Furthermore, this literature also identifies alternative measures of ideal partner preferences that have stronger predictive validity in certain theoretically sensible contexts. Finally, a discussion highlights a new framework for conceptualizing the appeal of traits, the difference between live and hypothetical interactions, and the productive interplay between mating research and broader psychological theories.

Validation of Fatigue Modeling Predictions in Aviation Operations

NASA Technical Reports Server (NTRS)

Gregory, Kevin; Martinez, Siera; Flynn-Evans, Erin

2017-01-01

Bio-mathematical fatigue models that predict levels of alertness and performance are one potential tool for use within integrated fatigue risk management approaches. A number of models have been developed that provide predictions based on acute and chronic sleep loss, circadian desynchronization, and sleep inertia. Some are publicly available and gaining traction in settings such as commercial aviation as a means of evaluating flight crew schedules for potential fatigue-related risks. Yet, most models have not been rigorously evaluated and independently validated for the operations to which they are being applied and many users are not fully aware of the limitations in which model results should be interpreted and applied.

Survey of statistical techniques used in validation studies of air pollution prediction models

DOE Office of Scientific and Technical Information (OSTI.GOV)

Bornstein, R D; Anderson, S F

1979-03-01

Statistical techniques used by meteorologists to validate predictions made by air pollution models are surveyed. Techniques are divided into the following three groups: graphical, tabular, and summary statistics. Some of the practical problems associated with verification are also discussed. Characteristics desired in any validation program are listed and a suggested combination of techniques that possesses many of these characteristics is presented.

Predictive Validity of Explicit and Implicit Threat Overestimation in Contamination Fear

PubMed Central

Green, Jennifer S.; Teachman, Bethany A.

2012-01-01

We examined the predictive validity of explicit and implicit measures of threat overestimation in relation to contamination-fear outcomes using structural equation modeling. Undergraduate students high in contamination fear (N = 56) completed explicit measures of contamination threat likelihood and severity, as well as looming vulnerability cognitions, in addition to an implicit measure of danger associations with potential contaminants. Participants also completed measures of contamination-fear symptoms, as well as subjective distress and avoidance during a behavioral avoidance task, and state looming vulnerability cognitions during an exposure task. The latent explicit (but not implicit) threat overestimation variable was a significant and unique predictor of contamination fear symptoms and self-reported affective and cognitive facets of contamination fear. On the contrary, the implicit (but not explicit) latent measure predicted behavioral avoidance (at the level of a trend). Results are discussed in terms of differential predictive validity of implicit versus explicit markers of threat processing and multiple fear response systems. PMID:24073390

Preventing patient absenteeism: validation of a predictive overbooking model.

PubMed

Reid, Mark W; Cohen, Samuel; Wang, Hank; Kaung, Aung; Patel, Anish; Tashjian, Vartan; Williams, Demetrius L; Martinez, Bibiana; Spiegel, Brennan M R

2015-12-01

To develop a model that identifies patients at high risk for missing scheduled appointments ("no-shows" and cancellations) and to project the impact of predictive overbooking in a gastrointestinal endoscopy clinic-an exemplar resource-intensive environment with a high no-show rate. We retrospectively developed an algorithm that uses electronic health record (EHR) data to identify patients who do not show up to their appointments. Next, we prospectively validated the algorithm at a Veterans Administration healthcare network clinic. We constructed a multivariable logistic regression model that assigned a no-show risk score optimized by receiver operating characteristic curve analysis. Based on these scores, we created a calendar of projected open slots to offer to patients and compared the daily performance of predictive overbooking with fixed overbooking and typical "1 patient, 1 slot" scheduling. Data from 1392 patients identified several predictors of no-show, including previous absenteeism, comorbid disease burden, and current diagnoses of mood and substance use disorders. The model correctly classified most patients during the development (area under the curve [AUC] = 0.80) and validation phases (AUC = 0.75). Prospective testing in 1197 patients found that predictive overbooking averaged 0.51 unused appointments per day versus 6.18 for typical booking (difference = -5.67; 95% CI, -6.48 to -4.87; P < .0001). Predictive overbooking could have increased service utilization from 62% to 97% of capacity, with only rare clinic overflows. Information from EHRs can accurately predict whether patients will no-show. This method can be used to overbook appointments, thereby maximizing service utilization while staying within clinic capacity.

A Case for Transforming the Criterion of a Predictive Validity Study

ERIC Educational Resources Information Center

Patterson, Brian F.; Kobrin, Jennifer L.

2011-01-01

This study presents a case for applying a transformation (Box and Cox, 1964) of the criterion used in predictive validity studies. The goals of the transformation were to better meet the assumptions of the linear regression model and to reduce the residual variance of fitted (i.e., predicted) values. Using data for the 2008 cohort of first-time,…

The Predictive Validity of the Metropolitan Readiness Tests, 1976 Edition.

ERIC Educational Resources Information Center

Nagle, Richard J.

1979-01-01

A sample of 176 first-grade children was tested on the Metropolitan Readiness Tests, 1976 Edition (MRT), during the initial month of school and was retested eight months later on the Stanford Achievement Test. Results demonstrated substantial validity of the MRT for predicting first-grade achievement. (Author/CTM)

Understanding Interrater Reliability and Validity of Risk Assessment Tools Used to Predict Adverse Clinical Events.

PubMed

Siedlecki, Sandra L; Albert, Nancy M

This article will describe how to assess interrater reliability and validity of risk assessment tools, using easy-to-follow formulas, and to provide calculations that demonstrate principles discussed. Clinical nurse specialists should be able to identify risk assessment tools that provide high-quality interrater reliability and the highest validity for predicting true events of importance to clinical settings. Making best practice recommendations for assessment tool use is critical to high-quality patient care and safe practices that impact patient outcomes and nursing resources. Optimal risk assessment tool selection requires knowledge about interrater reliability and tool validity. The clinical nurse specialist will understand the reliability and validity issues associated with risk assessment tools, and be able to evaluate tools using basic calculations. Risk assessment tools are developed to objectively predict quality and safety events and ultimately reduce the risk of event occurrence through preventive interventions. To ensure high-quality tool use, clinical nurse specialists must critically assess tool properties. The better the tool's ability to predict adverse events, the more likely that event risk is mediated. Interrater reliability and validity assessment is relatively an easy skill to master and will result in better decisions when selecting or making recommendations for risk assessment tool use.

Predictive and concurrent validity of the Braden scale in long-term care: a meta-analysis.

PubMed

Wilchesky, Machelle; Lungu, Ovidiu

2015-01-01

Pressure ulcer prevention is an important long-term care (LTC) quality indicator. While the Braden Scale is a recommended risk assessment tool, there is a paucity of information specifically pertaining to its validity within the LTC setting. We, therefore, undertook a systematic review and meta-analysis comparing Braden Scale predictive and concurrent validity within this context. We searched the Medline, EMBASE, PsychINFO and PubMed databases from 1985-2014 for studies containing the requisite information to analyze tool validity. Our initial search yielded 3,773 articles. Eleven datasets emanating from nine published studies describing 40,361 residents met all meta-analysis inclusion criteria and were analyzed using random effects models. Pooled sensitivity, specificity, positive predictive value (PPV), and negative predictive values were 86%, 38%, 28%, and 93%, respectively. Specificity was poorer in concurrent samples as compared with predictive samples (38% vs. 72%), while PPV was low in both sample types (25 and 37%). Though random effects model results showed that the Scale had good overall predictive ability [RR, 4.33; 95% CI, 3.28-5.72], none of the concurrent samples were found to have "optimal" sensitivity and specificity. In conclusion, the appropriateness of the Braden Scale in LTC is questionable given its low specificity and PPV, in particular in concurrent validity studies. Future studies should further explore the extent to which the apparent low validity of the Scale in LTC is due to the choice of cutoff point and/or preventive strategies implemented by LTC staff as a matter of course. © 2015 by the Wound Healing Society.

The Kuder Occupational Interest Inventory as a Moderator of Its Predictive Validity.

ERIC Educational Resources Information Center

Hansen, Chris J.; Zytowski, Donald G.

1979-01-01

A measure of the extent to which the Kuder Occupational Interest Survey (KOIS) was predictive of occupational membership for an individual was correlated with KOIS item and scale scores. Results indicated that the KOIS was a moderator of its own predictive validity. (Author/JKS)

NNvPDB: Neural Network based Protein Secondary Structure Prediction with PDB Validation.

PubMed

Sakthivel, Seethalakshmi; S K M, Habeeb

2015-01-01

The predicted secondary structural states are not cross validated by any of the existing servers. Hence, information on the level of accuracy for every sequence is not reported by the existing servers. This was overcome by NNvPDB, which not only reported greater Q3 but also validates every prediction with the homologous PDB entries. NNvPDB is based on the concept of Neural Network, with a new and different approach of training the network every time with five PDB structures that are similar to query sequence. The average accuracy for helix is 76%, beta sheet is 71% and overall (helix, sheet and coil) is 66%. http://bit.srmuniv.ac.in/cgi-bin/bit/cfpdb/nnsecstruct.pl.

Responsiveness and predictive validity of the tablet-based symbol digit modalities test in patients with stroke.

PubMed

Hsiao, Pei-Chi; Yu, Wan-Hui; Lee, Shih-Chieh; Chen, Mei-Hsiang; Hsieh, Ching-Lin

2018-06-14

The responsiveness and predictive validity of the Tablet-based Symbol Digit Modalities Test (T-SDMT) are unknown, which limits the utility of the T-SDMT in both clinical and research settings. The purpose of this study was to examine the responsiveness and predictive validity of the T-SDMT in inpatients with stroke. A follow-up, repeated-assessments design. One rehabilitation unit at a local medical center. A total of 50 inpatients receiving rehabilitation completed T-SDMT assessments at admission to and discharge from a rehabilitation ward. The median follow-up period was 14 days. The Barthel index (BI) was assessed at discharge and was used as the criterion of the predictive validity. The mean changes in the T-SDMT scores between admission and discharge were statistically significant (paired t-test = 3.46, p = 0.001). The T-SDMT scores showed a nearly moderate standardized response mean (0.49). A moderate association (Pearson's r = 0.47) was found between the scores of the T-SDMT at admission and those of the BI at discharge, indicating good predictive validity of the T-SDMT. Our results support the responsiveness and predictive validity of the T-SDMT in patients with stroke receiving rehabilitation in hospitals. This study provides empirical evidence supporting the use of the T-SDMT as an outcome measure for assessing processingspeed in inpatients with stroke. The scores of the T-SDMT could be used to predict basic activities of daily living function in inpatients with stroke.

A Model for Investigating Predictive Validity at Highly Selective Institutions.

ERIC Educational Resources Information Center

Gross, Alan L.; And Others

A statistical model for investigating predictive validity at highly selective institutions is described. When the selection ratio is small, one must typically deal with a data set containing relatively large amounts of missing data on both criterion and predictor variables. Standard statistical approaches are based on the strong assumption that…

The Validity of Conscientiousness Is Overestimated in the Prediction of Job Performance.

PubMed

Kepes, Sven; McDaniel, Michael A

2015-01-01

Sensitivity analyses refer to investigations of the degree to which the results of a meta-analysis remain stable when conditions of the data or the analysis change. To the extent that results remain stable, one can refer to them as robust. Sensitivity analyses are rarely conducted in the organizational science literature. Despite conscientiousness being a valued predictor in employment selection, sensitivity analyses have not been conducted with respect to meta-analytic estimates of the correlation (i.e., validity) between conscientiousness and job performance. To address this deficiency, we reanalyzed the largest collection of conscientiousness validity data in the personnel selection literature and conducted a variety of sensitivity analyses. Publication bias analyses demonstrated that the validity of conscientiousness is moderately overestimated (by around 30%; a correlation difference of about .06). The misestimation of the validity appears to be due primarily to suppression of small effects sizes in the journal literature. These inflated validity estimates result in an overestimate of the dollar utility of personnel selection by millions of dollars and should be of considerable concern for organizations. The fields of management and applied psychology seldom conduct sensitivity analyses. Through the use of sensitivity analyses, this paper documents that the existing literature overestimates the validity of conscientiousness in the prediction of job performance. Our data show that effect sizes from journal articles are largely responsible for this overestimation.

Predictive validity of pre-admission assessments on medical student performance.

PubMed

Dabaliz, Al-Awwab; Kaadan, Samy; Dabbagh, M Marwan; Barakat, Abdulaziz; Shareef, Mohammad Abrar; Al-Tannir, Mohamad; Obeidat, Akef; Mohamed, Ayman

2017-11-24

To examine the predictive validity of pre-admission variables on students' performance in a medical school in Saudi Arabia. In this retrospective study, we collected admission and college performance data for 737 students in preclinical and clinical years. Data included high school scores and other standardized test scores, such as those of the National Achievement Test and the General Aptitude Test. Additionally, we included the scores of the Test of English as a Foreign Language (TOEFL) and the International English Language Testing System (IELTS) exams. Those datasets were then compared with college performance indicators, namely the cumulative Grade Point Average (cGPA) and progress test, using multivariate linear regression analysis. In preclinical years, both the National Achievement Test (p=0.04, B=0.08) and TOEFL (p=0.017, B=0.01) scores were positive predictors of cGPA, whereas the General Aptitude Test (p=0.048, B=-0.05) negatively predicted cGPA. Moreover, none of the pre-admission variables were predictive of progress test performance in the same group. On the other hand, none of the pre-admission variables were predictive of cGPA in clinical years. Overall, cGPA strongly predict-ed students' progress test performance (p<0.001 and B=19.02). Only the National Achievement Test and TOEFL significantly predicted performance in preclinical years. However, these variables do not predict progress test performance, meaning that they do not predict the functional knowledge reflected in the progress test. We report various strengths and deficiencies in the current medical college admission criteria, and call for employing more sensitive and valid ones that predict student performance and functional knowledge, especially in the clinical years.

The Predictive Validity of Savry Ratings for Assessing Youth Offenders in Singapore

PubMed Central

Chu, Chi Meng; Goh, Mui Leng; Chong, Dominic

2015-01-01

Empirical support for the usage of the SAVRY has been reported in studies conducted in many Western contexts, but not in a Singaporean context. This study compared the predictive validity of the SAVRY ratings for violent and general recidivism against the Youth Level of Service/Case Management Inventory (YLS/CMI) ratings within the Singaporean context. Using a sample of 165 male young offenders (Mfollow-up = 4.54 years), results showed that the SAVRY Total Score and Summary Risk Rating, as well as YLS/CMI Total Score and Overall Risk Rating, predicted violent and general recidivism. SAVRY Protective Total Score was only significantly predictive of desistance from general recidivism, and did not show incremental predictive validity for violent and general recidivism over the SAVRY Total Score. Overall, the results suggest that the SAVRY is suited (to varying degrees) for assessing the risk of violent and general recidivism in young offenders within the Singaporean context, but might not be better than the YLS/CMI. PMID:27231403

Validation of the Social Appearance Anxiety Scale: factor, convergent, and divergent validity.

PubMed

Levinson, Cheri A; Rodebaugh, Thomas L

2011-09-01

The Social Appearance Anxiety Scale (SAAS) was created to assess fear of overall appearance evaluation. Initial psychometric work indicated that the measure had a single-factor structure and exhibited excellent internal consistency, test-retest reliability, and convergent validity. In the current study, the authors further examined the factor, convergent, and divergent validity of the SAAS in two samples of undergraduates. In Study 1 (N = 323), the authors tested the factor structure, convergent, and divergent validity of the SAAS with measures of the Big Five personality traits, negative affect, fear of negative evaluation, and social interaction anxiety. In Study 2 (N = 118), participants completed a body evaluation that included measurements of height, weight, and body fat content. The SAAS exhibited excellent convergent and divergent validity with self-report measures (i.e., self-esteem, trait anxiety, ethnic identity, and sympathy), predicted state anxiety experienced during the body evaluation, and predicted body fat content. In both studies, results confirmed a single-factor structure as the best fit to the data. These results lend additional support for the use of the SAAS as a valid measure of social appearance anxiety.

Derivation and validation of in-hospital mortality prediction models in ischaemic stroke patients using administrative data.

PubMed

Lee, Jason; Morishima, Toshitaka; Kunisawa, Susumu; Sasaki, Noriko; Otsubo, Tetsuya; Ikai, Hiroshi; Imanaka, Yuichi

2013-01-01

Stroke and other cerebrovascular diseases are a major cause of death and disability. Predicting in-hospital mortality in ischaemic stroke patients can help to identify high-risk patients and guide treatment approaches. Chart reviews provide important clinical information for mortality prediction, but are laborious and limiting in sample sizes. Administrative data allow for large-scale multi-institutional analyses but lack the necessary clinical information for outcome research. However, administrative claims data in Japan has seen the recent inclusion of patient consciousness and disability information, which may allow more accurate mortality prediction using administrative data alone. The aim of this study was to derive and validate models to predict in-hospital mortality in patients admitted for ischaemic stroke using administrative data. The sample consisted of 21,445 patients from 176 Japanese hospitals, who were randomly divided into derivation and validation subgroups. Multivariable logistic regression models were developed using 7- and 30-day and overall in-hospital mortality as dependent variables. Independent variables included patient age, sex, comorbidities upon admission, Japan Coma Scale (JCS) score, Barthel Index score, modified Rankin Scale (mRS) score, and admissions after hours and on weekends/public holidays. Models were developed in the derivation subgroup, and coefficients from these models were applied to the validation subgroup. Predictive ability was analysed using C-statistics; calibration was evaluated with Hosmer-Lemeshow χ(2) tests. All three models showed predictive abilities similar or surpassing that of chart review-based models. The C-statistics were highest in the 7-day in-hospital mortality prediction model, at 0.906 and 0.901 in the derivation and validation subgroups, respectively. For the 30-day in-hospital mortality prediction models, the C-statistics for the derivation and validation subgroups were 0.893 and 0

Validation of finite element and boundary element methods for predicting structural vibration and radiated noise

NASA Technical Reports Server (NTRS)

Seybert, A. F.; Wu, X. F.; Oswald, Fred B.

1992-01-01

Analytical and experimental validation of methods to predict structural vibration and radiated noise are presented. A rectangular box excited by a mechanical shaker was used as a vibrating structure. Combined finite element method (FEM) and boundary element method (BEM) models of the apparatus were used to predict the noise radiated from the box. The FEM was used to predict the vibration, and the surface vibration was used as input to the BEM to predict the sound intensity and sound power. Vibration predicted by the FEM model was validated by experimental modal analysis. Noise predicted by the BEM was validated by sound intensity measurements. Three types of results are presented for the total radiated sound power: (1) sound power predicted by the BEM modeling using vibration data measured on the surface of the box; (2) sound power predicted by the FEM/BEM model; and (3) sound power measured by a sound intensity scan. The sound power predicted from the BEM model using measured vibration data yields an excellent prediction of radiated noise. The sound power predicted by the combined FEM/BEM model also gives a good prediction of radiated noise except for a shift of the natural frequencies that are due to limitations in the FEM model.

Development and validation of a predictive model for excessive postpartum blood loss: A retrospective, cohort study.

PubMed

Rubio-Álvarez, Ana; Molina-Alarcón, Milagros; Arias-Arias, Ángel; Hernández-Martínez, Antonio

2018-03-01

postpartum haemorrhage is one of the leading causes of maternal morbidity and mortality worldwide. Despite the use of uterotonics agents as preventive measure, it remains a challenge to identify those women who are at increased risk of postpartum bleeding. to develop and to validate a predictive model to assess the risk of excessive bleeding in women with vaginal birth. retrospective cohorts study. "Mancha-Centro Hospital" (Spain). the elaboration of the predictive model was based on a derivation cohort consisting of 2336 women between 2009 and 2011. For validation purposes, a prospective cohort of 953 women between 2013 and 2014 were employed. Women with antenatal fetal demise, multiple pregnancies and gestations under 35 weeks were excluded METHODS: we used a multivariate analysis with binary logistic regression, Ridge Regression and areas under the Receiver Operating Characteristic curves to determine the predictive ability of the proposed model. there was 197 (8.43%) women with excessive bleeding in the derivation cohort and 63 (6.61%) women in the validation cohort. Predictive factors in the final model were: maternal age, primiparity, duration of the first and second stages of labour, neonatal birth weight and antepartum haemoglobin levels. Accordingly, the predictive ability of this model in the derivation cohort was 0.90 (95% CI: 0.85-0.93), while it remained 0.83 (95% CI: 0.74-0.92) in the validation cohort. this predictive model is proved to have an excellent predictive ability in the derivation cohort, and its validation in a latter population equally shows a good ability for prediction. This model can be employed to identify women with a higher risk of postpartum haemorrhage. Copyright © 2017 Elsevier Ltd. All rights reserved.

External model validation of binary clinical risk prediction models in cardiovascular and thoracic surgery.

PubMed

Hickey, Graeme L; Blackstone, Eugene H

2016-08-01

Clinical risk-prediction models serve an important role in healthcare. They are used for clinical decision-making and measuring the performance of healthcare providers. To establish confidence in a model, external model validation is imperative. When designing such an external model validation study, thought must be given to patient selection, risk factor and outcome definitions, missing data, and the transparent reporting of the analysis. In addition, there are a number of statistical methods available for external model validation. Execution of a rigorous external validation study rests in proper study design, application of suitable statistical methods, and transparent reporting. Copyright © 2016 The American Association for Thoracic Surgery. Published by Elsevier Inc. All rights reserved.

Development and validation of the ORACLE score to predict risk of osteoporosis.

PubMed

Richy, Florent; Deceulaer, Fréderic; Ethgen, Olivier; Bruyère, Olivier; Reginster, Jean-Yves

2004-11-01

To develop and validate a composite index, the Osteoporosis Risk Assessment by Composite Linear Estimate (ORACLE), that includes risk factors and ultrasonometric outcomes to screen for osteoporosis. Two cohorts of postmenopausal women aged 45 years and older participated in the development (n = 407) and the validation (n = 202) of ORACLE. Their bone mineral density was determined by dual energy x-ray absorptiometry and quantitative ultrasonometry (QUS), and their historical and clinical risk factors were assessed (January to June 2003). Logistic regression analysis was used to select significant predictors of bone mineral density, whereas receiver operating characteristic (ROC) analysis was used to assess the discriminatory performance of ORACLE. The final logistic regression model retained 4 biometric or historical variables and 1 ultrasonometric outcome. The ROC areas under the curves (AUCs) for ORACLE were 84% for the prediction of osteoporosis and 78% for low bone mass. A sensitivity of 90% corresponded to a specificity of 50% for identification of women at risk of developing osteoporosis. The corresponding positive and negative predictive values were 86% and 54%, respectively, in the development cohort. In the validation cohort, the AUCs for identification of osteoporosis and low bone mass were 81% and 76% for ORACLE, 69% and 64% for QUS T score, 71% and 68% for QUS ultrasonometric bone profile index, and 76% and 75% for Osteoporosis Self-assessment Tool, respectively. ORACLE had the best discriminatory performance in identifying osteoporosis compared with the other approaches (P < .05). ORACLE exhibited the highest discriminatory properties compared with ultrasonography alone or other previously validated risk indices. It may be helpful to enhance the predictive value of QUS.

External validation of a 5-year survival prediction model after elective abdominal aortic aneurysm repair.

PubMed

DeMartino, Randall R; Huang, Ying; Mandrekar, Jay; Goodney, Philip P; Oderich, Gustavo S; Kalra, Manju; Bower, Thomas C; Cronenwett, Jack L; Gloviczki, Peter

2018-01-01

The benefit of prophylactic repair of abdominal aortic aneurysms (AAAs) is based on the risk of rupture exceeding the risk of death from other comorbidities. The purpose of this study was to validate a 5-year survival prediction model for patients undergoing elective repair of asymptomatic AAA <6.5 cm to assist in optimal selection of patients. All patients undergoing elective repair for asymptomatic AAA <6.5 cm (open or endovascular) from 2002 to 2011 were identified from a single institutional database (validation group). We assessed the ability of a prior published Vascular Study Group of New England (VSGNE) model (derivation group) to predict survival in our cohort. The model was assessed for discrimination (concordance index), calibration (calibration slope and calibration in the large), and goodness of fit (score test). The VSGNE derivation group consisted of 2367 patients (70% endovascular). Major factors associated with survival in the derivation group were age, coronary disease, chronic obstructive pulmonary disease, renal function, and antiplatelet and statin medication use. Our validation group consisted of 1038 patients (59% endovascular). The validation group was slightly older (74 vs 72 years; P < .01) and had a higher proportion of men (76% vs 68%; P < .01). In addition, the derivation group had higher rates of advanced cardiac disease, chronic obstructive pulmonary disease, and baseline creatinine concentration (1.2 vs 1.1 mg/dL; P < .01). Despite slight differences in preoperative patient factors, 5-year survival was similar between validation and derivation groups (75% vs 77%; P = .33). The concordance index of the validation group was identical between derivation and validation groups at 0.659 (95% confidence interval, 0.63-0.69). Our validation calibration in the large value was 1.02 (P = .62, closer to 1 indicating better calibration), calibration slope of 0.84 (95% confidence interval, 0.71-0.97), and score test of P = .57 (>.05

Development and validation of immune dysfunction score to predict 28-day mortality of sepsis patients

PubMed Central

Fang, Wen-Feng; Douglas, Ivor S.; Chen, Yu-Mu; Lin, Chiung-Yu; Kao, Hsu-Ching; Fang, Ying-Tang; Huang, Chi-Han; Chang, Ya-Ting; Huang, Kuo-Tung; Wang, Yi-His; Wang, Chin-Chou

2017-01-01

Background Sepsis-induced immune dysfunction ranging from cytokines storm to immunoparalysis impacts outcomes. Monitoring immune dysfunction enables better risk stratification and mortality prediction and is mandatory before widely application of immunoadjuvant therapies. We aimed to develop and validate a scoring system according to patients’ immune dysfunction status for 28-day mortality prediction. Methods A prospective observational study from a cohort of adult sepsis patients admitted to ICU between August 2013 and June 2016 at Kaohsiung Chang Gung Memorial Hospital in Taiwan. We evaluated immune dysfunction status through measurement of baseline plasma Cytokine levels, Monocyte human leukocyte-DR expression by flow cytometry, and stimulated immune response using post LPS stimulated cytokine elevation ratio. An immune dysfunction score was created for 28-day mortality prediction and was validated. Results A total of 151 patients were enrolled. Data of the first consecutive 106 septic patients comprised the training cohort, and of other 45 patients comprised the validation cohort. Among the 106 patients, 21 died and 85 were still alive on day 28 after ICU admission. (mortality rate, 19.8%). Independent predictive factors revealed via multivariate logistic regression analysis included segmented neutrophil-to-monocyte ratio, granulocyte-colony stimulating factor, interleukin-10, and monocyte human leukocyte antigen-antigen D–related levels, all of which were selected to construct the score, which predicted 28-day mortality with area under the curve of 0.853 and 0.789 in the training and validation cohorts, respectively. Conclusions The immune dysfunction scoring system developed here included plasma granulocyte-colony stimulating factor level, interleukin-10 level, serum segmented neutrophil-to-monocyte ratio, and monocyte human leukocyte antigen-antigen D–related expression appears valid and reproducible for predicting 28-day mortality. PMID:29073262

Job Embeddedness Demonstrates Incremental Validity When Predicting Turnover Intentions for Australian University Employees

PubMed Central

Heritage, Brody; Gilbert, Jessica M.; Roberts, Lynne D.

2016-01-01

Job embeddedness is a construct that describes the manner in which employees can be enmeshed in their jobs, reducing their turnover intentions. Recent questions regarding the properties of quantitative job embeddedness measures, and their predictive utility, have been raised. Our study compared two competing reflective measures of job embeddedness, examining their convergent, criterion, and incremental validity, as a means of addressing these questions. Cross-sectional quantitative data from 246 Australian university employees (146 academic; 100 professional) was gathered. Our findings indicated that the two compared measures of job embeddedness were convergent when total scale scores were examined. Additionally, job embeddedness was capable of demonstrating criterion and incremental validity, predicting unique variance in turnover intention. However, this finding was not readily apparent with one of the compared job embeddedness measures, which demonstrated comparatively weaker evidence of validity. We discuss the theoretical and applied implications of these findings, noting that job embeddedness has a complementary place among established determinants of turnover intention. PMID:27199817

Development and validation of a predictive equation for lean body mass in children and adolescents.

PubMed

Foster, Bethany J; Platt, Robert W; Zemel, Babette S

2012-05-01

Lean body mass (LBM) is not easy to measure directly in the field or clinical setting. Equations to predict LBM from simple anthropometric measures, which account for the differing contributions of fat and lean to body weight at different ages and levels of adiposity, would be useful to both human biologists and clinicians. To develop and validate equations to predict LBM in children and adolescents across the entire range of the adiposity spectrum. Dual energy X-ray absorptiometry was used to measure LBM in 836 healthy children (437 females) and linear regression was used to develop sex-specific equations to estimate LBM from height, weight, age, body mass index (BMI) for age z-score and population ancestry. Equations were validated using bootstrapping methods and in a local independent sample of 332 children and in national data collected by NHANES. The mean difference between measured and predicted LBM was - 0.12% (95% limits of agreement - 11.3% to 8.5%) for males and - 0.14% ( - 11.9% to 10.9%) for females. Equations performed equally well across the entire adiposity spectrum, as estimated by BMI z-score. Validation indicated no over-fitting. LBM was predicted within 5% of measured LBM in the validation sample. The equations estimate LBM accurately from simple anthropometric measures.

Base Flow Model Validation

NASA Technical Reports Server (NTRS)

Sinha, Neeraj; Brinckman, Kevin; Jansen, Bernard; Seiner, John

2011-01-01

A method was developed of obtaining propulsive base flow data in both hot and cold jet environments, at Mach numbers and altitude of relevance to NASA launcher designs. The base flow data was used to perform computational fluid dynamics (CFD) turbulence model assessments of base flow predictive capabilities in order to provide increased confidence in base thermal and pressure load predictions obtained from computational modeling efforts. Predictive CFD analyses were used in the design of the experiments, available propulsive models were used to reduce program costs and increase success, and a wind tunnel facility was used. The data obtained allowed assessment of CFD/turbulence models in a complex flow environment, working within a building-block procedure to validation, where cold, non-reacting test data was first used for validation, followed by more complex reacting base flow validation.

The Reliability and Predictive Validity of the Stalking Risk Profile.

PubMed

McEwan, Troy E; Shea, Daniel E; Daffern, Michael; MacKenzie, Rachel D; Ogloff, James R P; Mullen, Paul E

2018-03-01

This study assessed the reliability and validity of the Stalking Risk Profile (SRP), a structured measure for assessing stalking risks. The SRP was administered at the point of assessment or retrospectively from file review for 241 adult stalkers (91% male) referred to a community-based forensic mental health service. Interrater reliability was high for stalker type, and moderate-to-substantial for risk judgments and domain scores. Evidence for predictive validity and discrimination between stalking recidivists and nonrecidivists for risk judgments depended on follow-up duration. Discrimination was moderate (area under the curve = 0.66-0.68) and positive and negative predictive values good over the full follow-up period ( Mdn = 170.43 weeks). At 6 months, discrimination was better than chance only for judgments related to stalking of new victims (area under the curve = 0.75); however, high-risk stalkers still reoffended against their original victim(s) 2 to 4 times as often as low-risk stalkers. Implications for the clinical utility and refinement of the SRP are discussed.

Assessment of predictive performance in incomplete data by combining internal validation and multiple imputation.

PubMed

Wahl, Simone; Boulesteix, Anne-Laure; Zierer, Astrid; Thorand, Barbara; van de Wiel, Mark A

2016-10-26

Missing values are a frequent issue in human studies. In many situations, multiple imputation (MI) is an appropriate missing data handling strategy, whereby missing values are imputed multiple times, the analysis is performed in every imputed data set, and the obtained estimates are pooled. If the aim is to estimate (added) predictive performance measures, such as (change in) the area under the receiver-operating characteristic curve (AUC), internal validation strategies become desirable in order to correct for optimism. It is not fully understood how internal validation should be combined with multiple imputation. In a comprehensive simulation study and in a real data set based on blood markers as predictors for mortality, we compare three combination strategies: Val-MI, internal validation followed by MI on the training and test parts separately, MI-Val, MI on the full data set followed by internal validation, and MI(-y)-Val, MI on the full data set omitting the outcome followed by internal validation. Different validation strategies, including bootstrap und cross-validation, different (added) performance measures, and various data characteristics are considered, and the strategies are evaluated with regard to bias and mean squared error of the obtained performance estimates. In addition, we elaborate on the number of resamples and imputations to be used, and adopt a strategy for confidence interval construction to incomplete data. Internal validation is essential in order to avoid optimism, with the bootstrap 0.632+ estimate representing a reliable method to correct for optimism. While estimates obtained by MI-Val are optimistically biased, those obtained by MI(-y)-Val tend to be pessimistic in the presence of a true underlying effect. Val-MI provides largely unbiased estimates, with a slight pessimistic bias with increasing true effect size, number of covariates and decreasing sample size. In Val-MI, accuracy of the estimate is more strongly improved by

Validated Risk Score for Predicting 6-Month Mortality in Infective Endocarditis.

PubMed

Park, Lawrence P; Chu, Vivian H; Peterson, Gail; Skoutelis, Athanasios; Lejko-Zupa, Tatjana; Bouza, Emilio; Tattevin, Pierre; Habib, Gilbert; Tan, Ren; Gonzalez, Javier; Altclas, Javier; Edathodu, Jameela; Fortes, Claudio Querido; Siciliano, Rinaldo Focaccia; Pachirat, Orathai; Kanj, Souha; Wang, Andrew

2016-04-18

Host factors and complications have been associated with higher mortality in infective endocarditis (IE). We sought to develop and validate a model of clinical characteristics to predict 6-month mortality in IE. Using a large multinational prospective registry of definite IE (International Collaboration on Endocarditis [ICE]-Prospective Cohort Study [PCS], 2000-2006, n=4049), a model to predict 6-month survival was developed by Cox proportional hazards modeling with inverse probability weighting for surgery treatment and was internally validated by the bootstrapping method. This model was externally validated in an independent prospective registry (ICE-PLUS, 2008-2012, n=1197). The 6-month mortality was 971 of 4049 (24.0%) in the ICE-PCS cohort and 342 of 1197 (28.6%) in the ICE-PLUS cohort. Surgery during the index hospitalization was performed in 48.1% and 54.0% of the cohorts, respectively. In the derivation model, variables related to host factors (age, dialysis), IE characteristics (prosthetic or nosocomial IE, causative organism, left-sided valve vegetation), and IE complications (severe heart failure, stroke, paravalvular complication, and persistent bacteremia) were independently associated with 6-month mortality, and surgery was associated with a lower risk of mortality (Harrell's C statistic 0.715). In the validation model, these variables had similar hazard ratios (Harrell's C statistic 0.682), with a similar, independent benefit of surgery (hazard ratio 0.74, 95% CI 0.62-0.89). A simplified risk model was developed by weight adjustment of these variables. Six-month mortality after IE is ≈25% and is predicted by host factors, IE characteristics, and IE complications. Surgery during the index hospitalization is associated with lower mortality but is performed less frequently in the highest risk patients. A simplified risk model may be used to identify specific risk subgroups in IE. © 2016 The Authors. Published on behalf of the American Heart

The Validity of Conscientiousness Is Overestimated in the Prediction of Job Performance

PubMed Central

2015-01-01

Introduction Sensitivity analyses refer to investigations of the degree to which the results of a meta-analysis remain stable when conditions of the data or the analysis change. To the extent that results remain stable, one can refer to them as robust. Sensitivity analyses are rarely conducted in the organizational science literature. Despite conscientiousness being a valued predictor in employment selection, sensitivity analyses have not been conducted with respect to meta-analytic estimates of the correlation (i.e., validity) between conscientiousness and job performance. Methods To address this deficiency, we reanalyzed the largest collection of conscientiousness validity data in the personnel selection literature and conducted a variety of sensitivity analyses. Results Publication bias analyses demonstrated that the validity of conscientiousness is moderately overestimated (by around 30%; a correlation difference of about .06). The misestimation of the validity appears to be due primarily to suppression of small effects sizes in the journal literature. These inflated validity estimates result in an overestimate of the dollar utility of personnel selection by millions of dollars and should be of considerable concern for organizations. Conclusion The fields of management and applied psychology seldom conduct sensitivity analyses. Through the use of sensitivity analyses, this paper documents that the existing literature overestimates the validity of conscientiousness in the prediction of job performance. Our data show that effect sizes from journal articles are largely responsible for this overestimation. PMID:26517553

Predictive validity of pre-admission assessments on medical student performance

PubMed Central

Dabaliz, Al-Awwab; Kaadan, Samy; Dabbagh, M. Marwan; Barakat, Abdulaziz; Shareef, Mohammad Abrar; Al-Tannir, Mohamad; Obeidat, Akef

2017-01-01

Objectives To examine the predictive validity of pre-admission variables on students’ performance in a medical school in Saudi Arabia.  Methods In this retrospective study, we collected admission and college performance data for 737 students in preclinical and clinical years. Data included high school scores and other standardized test scores, such as those of the National Achievement Test and the General Aptitude Test. Additionally, we included the scores of the Test of English as a Foreign Language (TOEFL) and the International English Language Testing System (IELTS) exams. Those datasets were then compared with college performance indicators, namely the cumulative Grade Point Average (cGPA) and progress test, using multivariate linear regression analysis. Results In preclinical years, both the National Achievement Test (p=0.04, B=0.08) and TOEFL (p=0.017, B=0.01) scores were positive predictors of cGPA, whereas the General Aptitude Test (p=0.048, B=-0.05) negatively predicted cGPA. Moreover, none of the pre-admission variables were predictive of progress test performance in the same group. On the other hand, none of the pre-admission variables were predictive of cGPA in clinical years. Overall, cGPA strongly predict-ed students’ progress test performance (p<0.001 and B=19.02). Conclusions Only the National Achievement Test and TOEFL significantly predicted performance in preclinical years. However, these variables do not predict progress test performance, meaning that they do not predict the functional knowledge reflected in the progress test. We report various strengths and deficiencies in the current medical college admission criteria, and call for employing more sensitive and valid ones that predict student performance and functional knowledge, especially in the clinical years. PMID:29176032

Initial Retrieval Validation from the Joint Airborne IASI Validation Experiment (JAIVEx)

NASA Technical Reports Server (NTRS)

Zhou, Daniel K.; Liu, Xu; Smith, WIlliam L.; Larar, Allen M.; Taylor, Jonathan P.; Revercomb, Henry E.; Mango, Stephen A.; Schluessel, Peter; Calbet, Xavier

2007-01-01

The Joint Airborne IASI Validation Experiment (JAIVEx) was conducted during April 2007 mainly for validation of the Infrared Atmospheric Sounding Interferometer (IASI) on the MetOp satellite, but also included a strong component focusing on validation of the Atmospheric InfraRed Sounder (AIRS) aboard the AQUA satellite. The cross validation of IASI and AIRS is important for the joint use of their data in the global Numerical Weather Prediction process. Initial inter-comparisons of geophysical products have been conducted from different aspects, such as using different measurements from airborne ultraspectral Fourier transform spectrometers (specifically, the NPOESS Airborne Sounder Testbed Interferometer (NAST-I) and the Scanning-High resolution Interferometer Sounder (S-HIS) aboard the NASA WB-57 aircraft), UK Facility for Airborne Atmospheric Measurements (FAAM) BAe146-301 aircraft insitu instruments, dedicated dropsondes, radiosondes, and ground based Raman Lidar. An overview of the JAIVEx retrieval validation plan and some initial results of this field campaign are presented.

Validation of predictive equations for weight and height using a metric tape.

PubMed

Rabito, E I; Mialich, M S; Martínez, E Z; García, R W D; Jordao, A A; Marchini, J S

2008-01-01

Weight and height measurements are important data for the evaluation of nutritional status but some situations prevent the execution of these measurements in the standard manner, using special equipment or an estimate by predictive equations. Predictive equations of height and weight requiring only a metric tape as an instrument have been recently developed. To validate three predictive equations for weight and two for height by Rabito and evaluating their agreement with the equations proposed by Chumlea. The following data were collected: sex, age and anthropometric measurements, ie, weight (kg), height (m), subscapular skinfold (mm), calf (cm), arm (cm) and abdominal (cm) circumferences, arm length (cm), and half span (cm). Data were analyzed statistically using the Lin coefficient to test the agreement between the equations and the St. Laurent coefficient to compare the estimated weight and height values with real values. 100 adults (age 48 +/- 18 years) admitted to the University Hospital (HCFMRP/USP) were evaluated. Equations I: W(kg) = 0.5030 (AC) + 0.5634 (AbC) + 1.3180 (CC) +0.0339 (SSSF) - 43.1560 and II: W (kg) = 0.4808 (AC) + 0.5646 (AbC) +1.3160 (CC) - 42.2450 showed the highest coefficients of agreement for weight and equations IV and V showed the highest coefficients of agreement for height. The St. Laurent coefficient indicated that equations III and V were valid for weight and height, respectively. Among the validated equations, the number III W (kg) = 0.5759 (AC) + 0.5263 (AbC) +1.2452 (CC) - 4.8689 (S) - 32.9241 and VH (m) = 63,525 -3,237(S) - 0,06904 (A) + 1,293 (HS) are recommended for height or weight because of their easy use for hospitalized patients and the equations be validated in other situations.

Applicability of Monte Carlo cross validation technique for model development and validation using generalised least squares regression

NASA Astrophysics Data System (ADS)

Haddad, Khaled; Rahman, Ataur; A Zaman, Mohammad; Shrestha, Surendra

2013-03-01

SummaryIn regional hydrologic regression analysis, model selection and validation are regarded as important steps. Here, the model selection is usually based on some measurements of goodness-of-fit between the model prediction and observed data. In Regional Flood Frequency Analysis (RFFA), leave-one-out (LOO) validation or a fixed percentage leave out validation (e.g., 10%) is commonly adopted to assess the predictive ability of regression-based prediction equations. This paper develops a Monte Carlo Cross Validation (MCCV) technique (which has widely been adopted in Chemometrics and Econometrics) in RFFA using Generalised Least Squares Regression (GLSR) and compares it with the most commonly adopted LOO validation approach. The study uses simulated and regional flood data from the state of New South Wales in Australia. It is found that when developing hydrologic regression models, application of the MCCV is likely to result in a more parsimonious model than the LOO. It has also been found that the MCCV can provide a more realistic estimate of a model's predictive ability when compared with the LOO.

Development and Validation of an Empiric Tool to Predict Favorable Neurologic Outcomes Among PICU Patients.

PubMed

Gupta, Punkaj; Rettiganti, Mallikarjuna; Gossett, Jeffrey M; Daufeldt, Jennifer; Rice, Tom B; Wetzel, Randall C

2018-01-01

To create a novel tool to predict favorable neurologic outcomes during ICU stay among children with critical illness. Logistic regression models using adaptive lasso methodology were used to identify independent factors associated with favorable neurologic outcomes. A mixed effects logistic regression model was used to create the final prediction model including all predictors selected from the lasso model. Model validation was performed using a 10-fold internal cross-validation approach. Virtual Pediatric Systems (VPS, LLC, Los Angeles, CA) database. Patients less than 18 years old admitted to one of the participating ICUs in the Virtual Pediatric Systems database were included (2009-2015). None. A total of 160,570 patients from 90 hospitals qualified for inclusion. Of these, 1,675 patients (1.04%) were associated with a decline in Pediatric Cerebral Performance Category scale by at least 2 between ICU admission and ICU discharge (unfavorable neurologic outcome). The independent factors associated with unfavorable neurologic outcome included higher weight at ICU admission, higher Pediatric Index of Morality-2 score at ICU admission, cardiac arrest, stroke, seizures, head/nonhead trauma, use of conventional mechanical ventilation and high-frequency oscillatory ventilation, prolonged hospital length of ICU stay, and prolonged use of mechanical ventilation. The presence of chromosomal anomaly, cardiac surgery, and utilization of nitric oxide were associated with favorable neurologic outcome. The final online prediction tool can be accessed at https://soipredictiontool.shinyapps.io/GNOScore/. Our model predicted 139,688 patients with favorable neurologic outcomes in an internal validation sample when the observed number of patients with favorable neurologic outcomes was among 139,591 patients. The area under the receiver operating curve for the validation model was 0.90. This proposed prediction tool encompasses 20 risk factors into one probability to predict

Neurocognition and community outcome in schizophrenia: long-term predictive validity.

PubMed

Fujii, Daryl E; Wylie, A Michael

2003-02-01

The present study examined the predictive validity of neuropsychological measures to functional outcome in 26 schizophrenic patients 15-plus year post-testing. Outcome measures included score on the Resource Associated Functional Level Scale (RAFLS), number of state hospital admissions, and total duration of state hospital inpatient stay. Results of several stepwise multiple regressions revealed that verbal memory significantly predicted RAFLS score, accounting for nearly half of the variance. Trails B significantly predicted duration of state hospital inpatient status. Discussion focused on the utility of these measures for clinicians and system planners. Copyright 2002 Elsevier Science B.V.

Development and External Validation of a Melanoma Risk Prediction Model Based on Self-assessed Risk Factors.

PubMed

Vuong, Kylie; Armstrong, Bruce K; Weiderpass, Elisabete; Lund, Eiliv; Adami, Hans-Olov; Veierod, Marit B; Barrett, Jennifer H; Davies, John R; Bishop, D Timothy; Whiteman, David C; Olsen, Catherine M; Hopper, John L; Mann, Graham J; Cust, Anne E; McGeechan, Kevin

2016-08-01

Identifying individuals at high risk of melanoma can optimize primary and secondary prevention strategies. To develop and externally validate a risk prediction model for incident first-primary cutaneous melanoma using self-assessed risk factors. We used unconditional logistic regression to develop a multivariable risk prediction model. Relative risk estimates from the model were combined with Australian melanoma incidence and competing mortality rates to obtain absolute risk estimates. A risk prediction model was developed using the Australian Melanoma Family Study (629 cases and 535 controls) and externally validated using 4 independent population-based studies: the Western Australia Melanoma Study (511 case-control pairs), Leeds Melanoma Case-Control Study (960 cases and 513 controls), Epigene-QSkin Study (44 544, of which 766 with melanoma), and Swedish Women's Lifestyle and Health Cohort Study (49 259 women, of which 273 had melanoma). We validated model performance internally and externally by assessing discrimination using the area under the receiver operating curve (AUC). Additionally, using the Swedish Women's Lifestyle and Health Cohort Study, we assessed model calibration and clinical usefulness. The risk prediction model included hair color, nevus density, first-degree family history of melanoma, previous nonmelanoma skin cancer, and lifetime sunbed use. On internal validation, the AUC was 0.70 (95% CI, 0.67-0.73). On external validation, the AUC was 0.66 (95% CI, 0.63-0.69) in the Western Australia Melanoma Study, 0.67 (95% CI, 0.65-0.70) in the Leeds Melanoma Case-Control Study, 0.64 (95% CI, 0.62-0.66) in the Epigene-QSkin Study, and 0.63 (95% CI, 0.60-0.67) in the Swedish Women's Lifestyle and Health Cohort Study. Model calibration showed close agreement between predicted and observed numbers of incident melanomas across all deciles of predicted risk. In the external validation setting, there was higher net benefit when using the risk prediction

The Predictive Validity of Four Intelligence Tests for School Grades: A Small Sample Longitudinal Study

PubMed Central

Gygi, Jasmin T.; Hagmann-von Arx, Priska; Schweizer, Florine; Grob, Alexander

2017-01-01

Intelligence is considered the strongest single predictor of scholastic achievement. However, little is known regarding the predictive validity of well-established intelligence tests for school grades. We analyzed the predictive validity of four widely used intelligence tests in German-speaking countries: The Intelligence and Development Scales (IDS), the Reynolds Intellectual Assessment Scales (RIAS), the Snijders-Oomen Nonverbal Intelligence Test (SON-R 6-40), and the Wechsler Intelligence Scale for Children (WISC-IV), which were individually administered to 103 children (Mage = 9.17 years) enrolled in regular school. School grades were collected longitudinally after 3 years (averaged school grades, mathematics, and language) and were available for 54 children (Mage = 11.77 years). All four tests significantly predicted averaged school grades. Furthermore, the IDS and the RIAS predicted both mathematics and language, while the SON-R 6-40 predicted mathematics. The WISC-IV showed no significant association with longitudinal scholastic achievement when mathematics and language were analyzed separately. The results revealed the predictive validity of currently used intelligence tests for longitudinal scholastic achievement in German-speaking countries and support their use in psychological practice, in particular for predicting averaged school grades. However, this conclusion has to be considered as preliminary due to the small sample of children observed. PMID:28348543

The Predictive Validity of CBM Writing Indices for Eighth-Grade Students

ERIC Educational Resources Information Center

Amato, Janelle M.; Watkins, Marley W.

2011-01-01

Curriculum-based measurement (CBM) is an alternative to traditional assessment techniques. Technical work has begun to identify CBM writing indices that are psychometrically sound for monitoring older students' writing proficiency. This study examined the predictive validity of CBM writing indices in a sample of 447 eighth-grade students.…

Concurrent and Predictive Validity of the Phelps Kindergarten Readiness Scale-II

ERIC Educational Resources Information Center

Duncan, Jennifer; Rafter, Erin M.

2005-01-01

The purpose of this research was to establish the concurrent and predictive validity of the Phelps Kindergarten Readiness Scale, Second Edition (PKRS-II; L. Phelps, 2003). Seventy-four kindergarten students of diverse ethnic backgrounds enrolled in a northeastern suburban school participated in the study. The concurrent administration of the…

Predictive validity and correlates of self-assessed resilience among U.S. Army soldiers.

PubMed

Campbell-Sills, Laura; Kessler, Ronald C; Ursano, Robert J; Sun, Xiaoying; Taylor, Charles T; Heeringa, Steven G; Nock, Matthew K; Sampson, Nancy A; Jain, Sonia; Stein, Murray B

2018-02-01

Self-assessment of resilience could prove valuable to military and other organizations whose personnel confront foreseen stressors. We evaluated the validity of self-assessed resilience among U.S. Army soldiers, including whether predeployment perceived resilience predicted postdeployment emotional disorder. Resilience was assessed via self-administered questionnaire among new soldiers reporting for basic training (N = 35,807) and experienced soldiers preparing to deploy to Afghanistan (N = 8,558). Concurrent validity of self-assessed resilience was evaluated among recruits by estimating its association with past-month emotional disorder. Predictive validity was examined among 3,526 experienced soldiers with no lifetime emotional disorder predeployment. Predictive models estimated associations of predeployment resilience with incidence of emotional disorder through 9 months postdeployment and with marked improvement in coping at 3 months postdeployment. Weights-adjusted regression models incorporated stringent controls for risk factors. Soldiers characterized themselves as very resilient on average [M = 14.34, SD = 4.20 (recruits); M = 14.75, SD = 4.31 (experienced soldiers); theoretical range = 0-20]. Demographic characteristics exhibited only modest associations with resilience, while severity of childhood maltreatment was negatively associated with resilience in both samples. Among recruits, resilience was inversely associated with past-month emotional disorder [adjusted odds ratio (AOR) = 0.65, 95% CI = 0.62-0.68, P < .0005 (per standard score increase)]. Among deployed soldiers, greater predeployment resilience was associated with decreased incidence of emotional disorder (AOR = 0.91; 95% CI = 0.84-0.98; P = .016) and increased odds of improved coping (AOR = 1.36; 95% CI = 1.24-1.49; P < .0005) postdeployment. Findings supported validity of self-assessed resilience among soldiers, although its predictive effect on incidence of

Development and validation of a prediction model for functional decline in older medical inpatients.

PubMed

Takada, Toshihiko; Fukuma, Shingo; Yamamoto, Yosuke; Tsugihashi, Yukio; Nagano, Hiroyuki; Hayashi, Michio; Miyashita, Jun; Azuma, Teruhisa; Fukuhara, Shunichi

2018-05-17

To prevent functional decline in older inpatients, identification of high-risk patients is crucial. The aim of this study was to develop and validate a prediction model to assess the risk of functional decline in older medical inpatients. In this retrospective cohort study, patients ≥65 years admitted acutely to medical wards were included. The healthcare database of 246 acute care hospitals (n = 229,913) was used for derivation, and two acute care hospitals (n = 1767 and 5443, respectively) were used for validation. Data were collected using a national administrative claims and discharge database. Functional decline was defined as a decline of the Katz score at discharge compared with on admission. About 6% of patients in the derivation cohort and 9% and 2% in each validation cohort developed functional decline. A model with 7 items, age, body mass index, living in a nursing home, ambulance use, need for assistance in walking, dementia, and bedsore, was developed. On internal validation, it demonstrated a c-statistic of 0.77 (95% confidence interval (CI) = 0.767-0.771) and good fit on the calibration plot. On external validation, the c-statistics were 0.79 (95% CI = 0.77-0.81) and 0.75 (95% CI = 0.73-0.77) for each cohort, respectively. Calibration plots showed good fit in one cohort and overestimation in the other one. A prediction model for functional decline in older medical inpatients was derived and validated. It is expected that use of the model would lead to early identification of high-risk patients and introducing early intervention. Copyright © 2018 Elsevier B.V. All rights reserved.

A valid model for predicting responsible nerve roots in lumbar degenerative disease with diagnostic doubt.

PubMed

Li, Xiaochuan; Bai, Xuedong; Wu, Yaohong; Ruan, Dike

2016-03-15

To construct and validate a model to predict responsible nerve roots in lumbar degenerative disease with diagnostic doubt (DD). From January 2009-January 2013, 163 patients with DD were assigned to the construction (n = 106) or validation sample (n = 57) according to different admission times to hospital. Outcome was assessed according to the Japanese Orthopedic Association (JOA) recovery rate as excellent, good, fair, and poor. The first two results were considered as effective clinical outcome (ECO). Baseline patient and clinical characteristics were considered as secondary variables. A multivariate logistic regression model was used to construct a model with the ECO as a dependent variable and other factors as explanatory variables. The odds ratios (ORs) of each risk factor were adjusted and transformed into a scoring system. Area under the curve (AUC) was calculated and validated in both internal and external samples. Moreover, calibration plot and predictive ability of this scoring system were also tested for further validation. Patients with DD with ECOs in both construction and validation models were around 76 % (76.4 and 75.5 % respectively). more preoperative visual analog pain scale (VAS) score (OR = 1.56, p < 0.01), stenosis levels of L4/5 or L5/S1 (OR = 1.44, p = 0.04), stenosis locations with neuroforamen (OR = 1.95, p = 0.01), neurological deficit (OR = 1.62, p = 0.01), and more VAS improvement of selective nerve route block (SNRB) (OR = 3.42, p = 0.02). the internal area under the curve (AUC) was 0.85, and the external AUC was 0.72, with a good calibration plot of prediction accuracy. Besides, the predictive ability of ECOs was not different from the actual results (p = 0.532). We have constructed and validated a predictive model for confirming responsible nerve roots in patients with DD. The associated risk factors were preoperative VAS score, stenosis levels of L4/5 or L5/S1, stenosis locations

Predictive Validity of a Student Self-Report Screener of Behavioral and Emotional Risk in an Urban High School

ERIC Educational Resources Information Center

Dowdy, Erin; Harrell-Williams, Leigh; Dever, Bridget V.; Furlong, Michael J.; Moore, Stephanie; Raines, Tara; Kamphaus, Randy W.

2016-01-01

Increasingly, schools are implementing school-based screening for risk of behavioral and emotional problems; hence, foundational evidence supporting the predictive validity of screening instruments is important to assess. This study examined the predictive validity of the Behavior Assessment System for Children-2 Behavioral and Emotional Screening…

Predicting risk behaviors: development and validation of a diagnostic scale.

PubMed

Witte, K; Cameron, K A; McKeon, J K; Berkowitz, J M

1996-01-01

The goal of this study was to develop and validate the Risk Behavior Diagnosis (RBD) Scale for use by health care providers and practitioners interested in promoting healthy behaviors. Theoretically guided by the Extended Parallel Process Model (EPPM; a fear appeal theory), the RBD scale was designed to work in conjunction with an easy-to-use formula to determine which types of health risk messages would be most appropriate for a given individual or audience. Because some health risk messages promote behavior change and others backfire, this type of scale offers guidance to practitioners on how to develop the best persuasive message possible to motivate healthy behaviors. The results of the study demonstrate the RBD scale to have a high degree of content, construct, and predictive validity. Specific examples and practical suggestions are offered to facilitate use of the scale for health practitioners.

Validating a Predictive Model of Acute Advanced Imaging Biomarkers in Ischemic Stroke.

PubMed

Bivard, Andrew; Levi, Christopher; Lin, Longting; Cheng, Xin; Aviv, Richard; Spratt, Neil J; Lou, Min; Kleinig, Tim; O'Brien, Billy; Butcher, Kenneth; Zhang, Jingfen; Jannes, Jim; Dong, Qiang; Parsons, Mark

2017-03-01

Advanced imaging to identify tissue pathophysiology may provide more accurate prognostication than the clinical measures used currently in stroke. This study aimed to derive and validate a predictive model for functional outcome based on acute clinical and advanced imaging measures. A database of prospectively collected sub-4.5 hour patients with ischemic stroke being assessed for thrombolysis from 5 centers who had computed tomographic perfusion and computed tomographic angiography before a treatment decision was assessed. Individual variable cut points were derived from a classification and regression tree analysis. The optimal cut points for each assessment variable were then used in a backward logic regression to predict modified Rankin scale (mRS) score of 0 to 1 and 5 to 6. The variables remaining in the models were then assessed using a receiver operating characteristic curve analysis. Overall, 1519 patients were included in the study, 635 in the derivation cohort and 884 in the validation cohort. The model was highly accurate at predicting mRS score of 0 to 1 in all patients considered for thrombolysis therapy (area under the curve [AUC] 0.91), those who were treated (AUC 0.88) and those with recanalization (AUC 0.89). Next, the model was highly accurate at predicting mRS score of 5 to 6 in all patients considered for thrombolysis therapy (AUC 0.91), those who were treated (0.89) and those with recanalization (AUC 0.91). The odds ratio of thrombolysed patients who met the model criteria achieving mRS score of 0 to 1 was 17.89 (4.59-36.35, P <0.001) and for mRS score of 5 to 6 was 8.23 (2.57-26.97, P <0.001). This study has derived and validated a highly accurate model at predicting patient outcome after ischemic stroke. © 2017 American Heart Association, Inc.

Development and validation of multivariable predictive model for thromboembolic events in lymphoma patients.

PubMed

Antic, Darko; Milic, Natasa; Nikolovski, Srdjan; Todorovic, Milena; Bila, Jelena; Djurdjevic, Predrag; Andjelic, Bosko; Djurasinovic, Vladislava; Sretenovic, Aleksandra; Vukovic, Vojin; Jelicic, Jelena; Hayman, Suzanne; Mihaljevic, Biljana

2016-10-01

Lymphoma patients are at increased risk of thromboembolic events but thromboprophylaxis in these patients is largely underused. We sought to develop and validate a simple model, based on individual clinical and laboratory patient characteristics that would designate lymphoma patients at risk for thromboembolic event. The study population included 1,820 lymphoma patients who were treated in the Lymphoma Departments at the Clinics of Hematology, Clinical Center of Serbia and Clinical Center Kragujevac. The model was developed using data from a derivation cohort (n = 1,236), and further assessed in the validation cohort (n = 584). Sixty-five patients (5.3%) in the derivation cohort and 34 (5.8%) patients in the validation cohort developed thromboembolic events. The variables independently associated with risk for thromboembolism were: previous venous and/or arterial events, mediastinal involvement, BMI>30 kg/m(2) , reduced mobility, extranodal localization, development of neutropenia and hemoglobin level < 100g/L. Based on the risk model score, the population was divided into the following risk categories: low (score 0-1), intermediate (score 2-3), and high (score >3). For patients classified at risk (intermediate and high-risk scores), the model produced negative predictive value of 98.5%, positive predictive value of 25.1%, sensitivity of 75.4%, and specificity of 87.5%. A high-risk score had positive predictive value of 65.2%. The diagnostic performance measures retained similar values in the validation cohort. Developed prognostic Thrombosis Lymphoma - ThroLy score is more specific for lymphoma patients than any other available score targeting thrombosis in cancer patients. Am. J. Hematol. 91:1014-1019, 2016. © 2016 Wiley Periodicals, Inc. © 2016 Wiley Periodicals, Inc.

Validation of the thermophysiological model by Fiala for prediction of local skin temperatures

NASA Astrophysics Data System (ADS)

Martínez, Natividad; Psikuta, Agnes; Kuklane, Kalev; Quesada, José Ignacio Priego; de Anda, Rosa María Cibrián Ortiz; Soriano, Pedro Pérez; Palmer, Rosario Salvador; Corberán, José Miguel; Rossi, René Michel; Annaheim, Simon

2016-12-01

The most complete and realistic physiological data are derived from direct measurements during human experiments; however, they present some limitations such as ethical concerns, time and cost burden. Thermophysiological models are able to predict human thermal response in a wide range of environmental conditions, but their use is limited due to lack of validation. The aim of this work was to validate the thermophysiological model by Fiala for prediction of local skin temperatures against a dedicated database containing 43 different human experiments representing a wide range of conditions. The validation was conducted based on root-mean-square deviation (rmsd) and bias. The thermophysiological model by Fiala showed a good precision when predicting core and mean skin temperature (rmsd 0.26 and 0.92 °C, respectively) and also local skin temperatures for most body sites (average rmsd for local skin temperatures 1.32 °C). However, an increased deviation of the predictions was observed for the forehead skin temperature (rmsd of 1.63 °C) and for the thigh during exercising exposures (rmsd of 1.41 °C). Possible reasons for the observed deviations are lack of information on measurement circumstances (hair, head coverage interference) or an overestimation of the sweat evaporative cooling capacity for the head and thigh, respectively. This work has highlighted the importance of collecting details about the clothing worn and how and where the sensors were attached to the skin for achieving more precise results in the simulations.

Predictive Validity and Accuracy of Oral Reading Fluency for English Learners

ERIC Educational Resources Information Center

Vanderwood, Michael L.; Tung, Catherine Y.; Checca, C. Jason

2014-01-01

The predictive validity and accuracy of an oral reading fluency (ORF) measure for a statewide assessment in English language arts was examined for second-grade native English speakers (NESs) and English learners (ELs) with varying levels of English proficiency. In addition to comparing ELs with native English speakers, the impact of English…

Test-Retest Reliability and Predictive Validity of the Implicit Association Test in Children

ERIC Educational Resources Information Center

Rae, James R.; Olson, Kristina R.

2018-01-01

The Implicit Association Test (IAT) is increasingly used in developmental research despite minimal evidence of whether children's IAT scores are reliable across time or predictive of behavior. When test-retest reliability and predictive validity have been assessed, the results have been mixed, and because these studies have differed on many…

Validation of Groundwater Models: Meaningful or Meaningless?

NASA Astrophysics Data System (ADS)

Konikow, L. F.

2003-12-01

Although numerical simulation models are valuable tools for analyzing groundwater systems, their predictive accuracy is limited. People who apply groundwater flow or solute-transport models, as well as those who make decisions based on model results, naturally want assurance that a model is "valid." To many people, model validation implies some authentication of the truth or accuracy of the model. History matching is often presented as the basis for model validation. Although such model calibration is a necessary modeling step, it is simply insufficient for model validation. Because of parameter uncertainty and solution non-uniqueness, declarations of validation (or verification) of a model are not meaningful. Post-audits represent a useful means to assess the predictive accuracy of a site-specific model, but they require the existence of long-term monitoring data. Model testing may yield invalidation, but that is an opportunity to learn and to improve the conceptual and numerical models. Examples of post-audits and of the application of a solute-transport model to a radioactive waste disposal site illustrate deficiencies in model calibration, prediction, and validation.

A Supervised Learning Process to Validate Online Disease Reports for Use in Predictive Models.

PubMed

Patching, Helena M M; Hudson, Laurence M; Cooke, Warrick; Garcia, Andres J; Hay, Simon I; Roberts, Mark; Moyes, Catherine L

2015-12-01

Pathogen distribution models that predict spatial variation in disease occurrence require data from a large number of geographic locations to generate disease risk maps. Traditionally, this process has used data from public health reporting systems; however, using online reports of new infections could speed up the process dramatically. Data from both public health systems and online sources must be validated before they can be used, but no mechanisms exist to validate data from online media reports. We have developed a supervised learning process to validate geolocated disease outbreak data in a timely manner. The process uses three input features, the data source and two metrics derived from the location of each disease occurrence. The location of disease occurrence provides information on the probability of disease occurrence at that location based on environmental and socioeconomic factors and the distance within or outside the current known disease extent. The process also uses validation scores, generated by disease experts who review a subset of the data, to build a training data set. The aim of the supervised learning process is to generate validation scores that can be used as weights going into the pathogen distribution model. After analyzing the three input features and testing the performance of alternative processes, we selected a cascade of ensembles comprising logistic regressors. Parameter values for the training data subset size, number of predictors, and number of layers in the cascade were tested before the process was deployed. The final configuration was tested using data for two contrasting diseases (dengue and cholera), and 66%-79% of data points were assigned a validation score. The remaining data points are scored by the experts, and the results inform the training data set for the next set of predictors, as well as going to the pathogen distribution model. The new supervised learning process has been implemented within our live site and is

Experimental validation of finite element and boundary element methods for predicting structural vibration and radiated noise

NASA Technical Reports Server (NTRS)

Seybert, A. F.; Wu, T. W.; Wu, X. F.

1994-01-01

This research report is presented in three parts. In the first part, acoustical analyses were performed on modes of vibration of the housing of a transmission of a gear test rig developed by NASA. The modes of vibration of the transmission housing were measured using experimental modal analysis. The boundary element method (BEM) was used to calculate the sound pressure and sound intensity on the surface of the housing and the radiation efficiency of each mode. The radiation efficiency of each of the transmission housing modes was then compared to theoretical results for a finite baffled plate. In the second part, analytical and experimental validation of methods to predict structural vibration and radiated noise are presented. A rectangular box excited by a mechanical shaker was used as a vibrating structure. Combined finite element method (FEM) and boundary element method (BEM) models of the apparatus were used to predict the noise level radiated from the box. The FEM was used to predict the vibration, while the BEM was used to predict the sound intensity and total radiated sound power using surface vibration as the input data. Vibration predicted by the FEM model was validated by experimental modal analysis; noise predicted by the BEM was validated by measurements of sound intensity. Three types of results are presented for the total radiated sound power: sound power predicted by the BEM model using vibration data measured on the surface of the box; sound power predicted by the FEM/BEM model; and sound power measured by an acoustic intensity scan. In the third part, the structure used in part two was modified. A rib was attached to the top plate of the structure. The FEM and BEM were then used to predict structural vibration and radiated noise respectively. The predicted vibration and radiated noise were then validated through experimentation.

Predictive validity of the Sødring Motor Evaluation of Stroke Patients (SMES).

PubMed

Wyller, T B; Sødring, K M; Sveen, U; Ljunggren, A E; Bautz-Holter, E

1996-12-01

The Sødring Motor Evaluation of Stroke Patients (SMES) has been developed as an instrument for the evaluation by physiotherapists of motor function and activities in stroke patients. The predictive validity of the instrument was studied in a consecutive sample of 93 acute stroke patients, assessed in the acute phase and after one year. The outcome measures were: survival, residence at home or in institution, the Barthel ADL index (dichotomized at 19/20), and the Frenchay Activities Index (FAI) (dichotomized at 9/10). The SMES, scored in the acute phase, demonstrated a marginally significant predictive power regarding survival, but was a highly significant predictor regarding the other outcomes. The adjusted odds ratio for a good versus a poor outcome for patients in the upper versus the lower tertile of the SMES arm subscore was 5.4 (95% confidence interval 0.9-59) for survival, 11.5 (2.1-88) for living at home, 86.3 (11-infinity) for a high Barthel score, and 31.4 (5.2-288) for a high FAI score. We conclude that SMES has high predictive validity.

External validation of the NUn score for predicting anastomotic leakage after oesophageal resection.

PubMed

Paireder, Matthias; Jomrich, Gerd; Asari, Reza; Kristo, Ivan; Gleiss, Andreas; Preusser, Matthias; Schoppmann, Sebastian F

2017-08-29

Early detection of anastomotic leakage (AL) after oesophageal resection for malignancy is crucial. This retrospective study validates a risk score, predicting AL, which includes C-reactive protein, albumin and white cell count in patients undergoing oesophageal resection between 2003 and 2014. For validation of the NUn score a receiver operating characteristic (ROC) curve is estimated. Area under the ROC curve (AUC) is reported with 95% confidence interval (CI). Among 258 patients (79.5% male) 32 patients showed signs of anastomotic leakage (12.4%). NUn score in our data has a median of 9.3 (range 6.2-17.6). The odds ratio for AL was 1.31 (CI 1.03-1.67; p = 0.028). AUC for AL was 0.59 (CI 0.47-0.72). Using the original cutoff value of 10, the sensitivity was 45.2% an the specificity was 73.8%. This results in a positive predictive value of 19.4% and a negative predictive value of 90.6%. The proportion of variation in AL occurrence, which is explained by the NUn score, was 2.5% (PEV = 0.025). This study provides evidence for an external validation of a simple risk score for AL after oesophageal resection. In this cohort, the NUn score is not useful due to its poor discrimination.

Perioperative Respiratory Adverse Events in Pediatric Ambulatory Anesthesia: Development and Validation of a Risk Prediction Tool.

PubMed

Subramanyam, Rajeev; Yeramaneni, Samrat; Hossain, Mohamed Monir; Anneken, Amy M; Varughese, Anna M

2016-05-01

Perioperative respiratory adverse events (PRAEs) are the most common cause of serious adverse events in children receiving anesthesia. Our primary aim of this study was to develop and validate a risk prediction tool for the occurrence of PRAE from the onset of anesthesia induction until discharge from the postanesthesia care unit in children younger than 18 years undergoing elective ambulatory anesthesia for surgery and radiology. The incidence of PRAE was studied. We analyzed data from 19,059 patients from our department's quality improvement database. The predictor variables were age, sex, ASA physical status, morbid obesity, preexisting pulmonary disorder, preexisting neurologic disorder, and location of ambulatory anesthesia (surgery or radiology). Composite PRAE was defined as the presence of any 1 of the following events: intraoperative bronchospasm, intraoperative laryngospasm, postoperative apnea, postoperative laryngospasm, postoperative bronchospasm, or postoperative prolonged oxygen requirement. Development and validation of the risk prediction tool for PRAE were performed using a split sampling technique to split the database into 2 independent cohorts based on the year when the patient received ambulatory anesthesia for surgery and radiology using logistic regression. A risk score was developed based on the regression coefficients from the validation tool. The performance of the risk prediction tool was assessed by using tests of discrimination and calibration. The overall incidence of composite PRAE was 2.8%. The derivation cohort included 8904 patients, and the validation cohort included 10,155 patients. The risk of PRAE was 3.9% in the development cohort and 1.8% in the validation cohort. Age ≤ 3 years (versus >3 years), ASA physical status II or III (versus ASA physical status I), morbid obesity, preexisting pulmonary disorder, and surgery (versus radiology) significantly predicted the occurrence of PRAE in a multivariable logistic regression

A simplified approach to the pooled analysis of calibration of clinical prediction rules for systematic reviews of validation studies

PubMed Central

Dimitrov, Borislav D; Motterlini, Nicola; Fahey, Tom

2015-01-01

Objective Estimating calibration performance of clinical prediction rules (CPRs) in systematic reviews of validation studies is not possible when predicted values are neither published nor accessible or sufficient or no individual participant or patient data are available. Our aims were to describe a simplified approach for outcomes prediction and calibration assessment and evaluate its functionality and validity. Study design and methods: Methodological study of systematic reviews of validation studies of CPRs: a) ABCD2 rule for prediction of 7 day stroke; and b) CRB-65 rule for prediction of 30 day mortality. Predicted outcomes in a sample validation study were computed by CPR distribution patterns (“derivation model”). As confirmation, a logistic regression model (with derivation study coefficients) was applied to CPR-based dummy variables in the validation study. Meta-analysis of validation studies provided pooled estimates of “predicted:observed” risk ratios (RRs), 95% confidence intervals (CIs), and indexes of heterogeneity (I2) on forest plots (fixed and random effects models), with and without adjustment of intercepts. The above approach was also applied to the CRB-65 rule. Results Our simplified method, applied to ABCD2 rule in three risk strata (low, 0–3; intermediate, 4–5; high, 6–7 points), indicated that predictions are identical to those computed by univariate, CPR-based logistic regression model. Discrimination was good (c-statistics =0.61–0.82), however, calibration in some studies was low. In such cases with miscalibration, the under-prediction (RRs =0.73–0.91, 95% CIs 0.41–1.48) could be further corrected by intercept adjustment to account for incidence differences. An improvement of both heterogeneities and P-values (Hosmer-Lemeshow goodness-of-fit test) was observed. Better calibration and improved pooled RRs (0.90–1.06), with narrower 95% CIs (0.57–1.41) were achieved. Conclusion Our results have an immediate clinical

Development and validation of a novel predictive scoring model for microvascular invasion in patients with hepatocellular carcinoma.

PubMed

Zhao, Hui; Hua, Ye; Dai, Tu; He, Jian; Tang, Min; Fu, Xu; Mao, Liang; Jin, Huihan; Qiu, Yudong

2017-03-01

Microvascular invasion (MVI) in patients with hepatocellular carcinoma (HCC) cannot be accurately predicted preoperatively. This study aimed to establish a predictive scoring model of MVI in solitary HCC patients without macroscopic vascular invasion. A total of 309 consecutive HCC patients who underwent curative hepatectomy were divided into the derivation (n=206) and validation cohort (n=103). A predictive scoring model of MVI was established according to the valuable predictors in the derivation cohort based on multivariate logistic regression analysis. The performance of the predictive model was evaluated in the derivation and validation cohorts. Preoperative imaging features on CECT, such as intratumoral arteries, non-nodular type of HCC and absence of radiological tumor capsule were independent predictors for MVI. The predictive scoring model was established according to the β coefficients of the 3 predictors. Area under receiver operating characteristic (AUROC) of the predictive scoring model was 0.872 (95% CI, 0.817-0.928) and 0.856 (95% CI, 0.771-0.940) in the derivation and validation cohorts. The positive and negative predictive values were 76.5% and 88.0% in the derivation cohort and 74.4% and 88.3% in the validation cohort. The performance of the model was similar between the patients with tumor size ≤5cm and >5cm in AUROC (P=0.910). The predictive scoring model based on intratumoral arteries, non-nodular type of HCC, and absence of the radiological tumor capsule on preoperative CECT is of great value in the prediction of MVI regardless of tumor size. Copyright © 2017 Elsevier B.V. All rights reserved.

Development and validation of classifiers and variable subsets for predicting nursing home admission.

PubMed

Nuutinen, Mikko; Leskelä, Riikka-Leena; Suojalehto, Ella; Tirronen, Anniina; Komssi, Vesa

2017-04-13

In previous years a substantial number of studies have identified statistically important predictors of nursing home admission (NHA). However, as far as we know, the analyses have been done at the population-level. No prior research has analysed the prediction accuracy of a NHA model for individuals. This study is an analysis of 3056 longer-term home care customers in the city of Tampere, Finland. Data were collected from the records of social and health service usage and RAI-HC (Resident Assessment Instrument - Home Care) assessment system during January 2011 and September 2015. The aim was to find out the most efficient variable subsets to predict NHA for individuals and validate the accuracy. The variable subsets of predicting NHA were searched by sequential forward selection (SFS) method, a variable ranking metric and the classifiers of logistic regression (LR), support vector machine (SVM) and Gaussian naive Bayes (GNB). The validation of the results was guaranteed using randomly balanced data sets and cross-validation. The primary performance metrics for the classifiers were the prediction accuracy and AUC (average area under the curve). The LR and GNB classifiers achieved 78% accuracy for predicting NHA. The most important variables were RAI MAPLE (Method for Assigning Priority Levels), functional impairment (RAI IADL, Activities of Daily Living), cognitive impairment (RAI CPS, Cognitive Performance Scale), memory disorders (diagnoses G30-G32 and F00-F03) and the use of community-based health-service and prior hospital use (emergency visits and periods of care). The accuracy of the classifier for individuals was high enough to convince the officials of the city of Tampere to integrate the predictive model based on the findings of this study as a part of home care information system. Further work need to be done to evaluate variables that are modifiable and responsive to interventions.

Broadband Fan Noise Prediction System for Turbofan Engines. Volume 3; Validation and Test Cases

NASA Technical Reports Server (NTRS)

Morin, Bruce L.

2010-01-01

Pratt & Whitney has developed a Broadband Fan Noise Prediction System (BFaNS) for turbofan engines. This system computes the noise generated by turbulence impinging on the leading edges of the fan and fan exit guide vane, and noise generated by boundary-layer turbulence passing over the fan trailing edge. BFaNS has been validated on three fan rigs that were tested during the NASA Advanced Subsonic Technology Program (AST). The predicted noise spectra agreed well with measured data. The predicted effects of fan speed, vane count, and vane sweep also agreed well with measurements. The noise prediction system consists of two computer programs: Setup_BFaNS and BFaNS. Setup_BFaNS converts user-specified geometry and flow-field information into a BFaNS input file. From this input file, BFaNS computes the inlet and aft broadband sound power spectra generated by the fan and FEGV. The output file from BFaNS contains the inlet, aft and total sound power spectra from each noise source. This report is the third volume of a three-volume set documenting the Broadband Fan Noise Prediction System: Volume 1: Setup_BFaNS User s Manual and Developer s Guide; Volume 2: BFaNS User s Manual and Developer s Guide; and Volume 3: Validation and Test Cases. The present volume begins with an overview of the Broadband Fan Noise Prediction System, followed by validation studies that were done on three fan rigs. It concludes with recommended improvements and additional studies for BFaNS.

A collaborative environment for developing and validating predictive tools for protein biophysical characteristics

NASA Astrophysics Data System (ADS)

Johnston, Michael A.; Farrell, Damien; Nielsen, Jens Erik

2012-04-01

The exchange of information between experimentalists and theoreticians is crucial to improving the predictive ability of theoretical methods and hence our understanding of the related biology. However many barriers exist which prevent the flow of information between the two disciplines. Enabling effective collaboration requires that experimentalists can easily apply computational tools to their data, share their data with theoreticians, and that both the experimental data and computational results are accessible to the wider community. We present a prototype collaborative environment for developing and validating predictive tools for protein biophysical characteristics. The environment is built on two central components; a new python-based integration module which allows theoreticians to provide and manage remote access to their programs; and PEATDB, a program for storing and sharing experimental data from protein biophysical characterisation studies. We demonstrate our approach by integrating PEATSA, a web-based service for predicting changes in protein biophysical characteristics, into PEATDB. Furthermore, we illustrate how the resulting environment aids method development using the Potapov dataset of experimentally measured ΔΔGfold values, previously employed to validate and train protein stability prediction algorithms.

Incremental Validity of Biographical Data in the Prediction of En Route Air Traffic Control Specialist Technical Skills

DTIC Science & Technology

2012-07-01

Incremental Validity of Biographical Data in the Prediction of En Route Air Traffic Control Specialist Technical Skills Dana Broach Civil Aerospace...Medical Institute Federal Aviation Administration Oklahoma City, OK 73125 July 2012 Final Report DOT/FAA/AM- 12 /8 Office of Aerospace Medicine...FAA/AM- 12 /8 4. Title and Subtitle 5. Report Date July 2012 Incremental Validity of Biographical Data in the Prediction of En Route Air

Independent external validation of predictive models for urinary dysfunction following external beam radiotherapy of the prostate: Issues in model development and reporting.

PubMed

Yahya, Noorazrul; Ebert, Martin A; Bulsara, Max; Kennedy, Angel; Joseph, David J; Denham, James W

2016-08-01

Most predictive models are not sufficiently validated for prospective use. We performed independent external validation of published predictive models for urinary dysfunctions following radiotherapy of the prostate. Multivariable models developed to predict atomised and generalised urinary symptoms, both acute and late, were considered for validation using a dataset representing 754 participants from the TROG 03.04-RADAR trial. Endpoints and features were harmonised to match the predictive models. The overall performance, calibration and discrimination were assessed. 14 models from four publications were validated. The discrimination of the predictive models in an independent external validation cohort, measured using the area under the receiver operating characteristic (ROC) curve, ranged from 0.473 to 0.695, generally lower than in internal validation. 4 models had ROC >0.6. Shrinkage was required for all predictive models' coefficients ranging from -0.309 (prediction probability was inverse to observed proportion) to 0.823. Predictive models which include baseline symptoms as a feature produced the highest discrimination. Two models produced a predicted probability of 0 and 1 for all patients. Predictive models vary in performance and transferability illustrating the need for improvements in model development and reporting. Several models showed reasonable potential but efforts should be increased to improve performance. Baseline symptoms should always be considered as potential features for predictive models. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

Prediction, Detection, and Validation of Isotope Clusters in Mass Spectrometry Data

PubMed Central

Treutler, Hendrik; Neumann, Steffen

2016-01-01

Mass spectrometry is a key analytical platform for metabolomics. The precise quantification and identification of small molecules is a prerequisite for elucidating the metabolism and the detection, validation, and evaluation of isotope clusters in LC-MS data is important for this task. Here, we present an approach for the improved detection of isotope clusters using chemical prior knowledge and the validation of detected isotope clusters depending on the substance mass using database statistics. We find remarkable improvements regarding the number of detected isotope clusters and are able to predict the correct molecular formula in the top three ranks in 92% of the cases. We make our methodology freely available as part of the Bioconductor packages xcms version 1.50.0 and CAMERA version 1.30.0. PMID:27775610

A Unified Model of Performance: Validation of its Predictions across Different Sleep/Wake Schedules

PubMed Central

Ramakrishnan, Sridhar; Wesensten, Nancy J.; Balkin, Thomas J.; Reifman, Jaques

2016-01-01

Study Objectives: Historically, mathematical models of human neurobehavioral performance developed on data from one sleep study were limited to predicting performance in similar studies, restricting their practical utility. We recently developed a unified model of performance (UMP) to predict the effects of the continuum of sleep loss—from chronic sleep restriction (CSR) to total sleep deprivation (TSD) challenges—and validated it using data from two studies of one laboratory. Here, we significantly extended this effort by validating the UMP predictions across a wide range of sleep/wake schedules from different studies and laboratories. Methods: We developed the UMP on psychomotor vigilance task (PVT) lapse data from one study encompassing four different CSR conditions (7 d of 3, 5, 7, and 9 h of sleep/night), and predicted performance in five other studies (from four laboratories), including different combinations of TSD (40 to 88 h), CSR (2 to 6 h of sleep/night), control (8 to 10 h of sleep/night), and nap (nocturnal and diurnal) schedules. Results: The UMP accurately predicted PVT performance trends across 14 different sleep/wake conditions, yielding average prediction errors between 7% and 36%, with the predictions lying within 2 standard errors of the measured data 87% of the time. In addition, the UMP accurately predicted performance impairment (average error of 15%) for schedules (TSD and naps) not used in model development. Conclusions: The unified model of performance can be used as a tool to help design sleep/wake schedules to optimize the extent and duration of neurobehavioral performance and to accelerate recovery after sleep loss. Citation: Ramakrishnan S, Wesensten NJ, Balkin TJ, Reifman J. A unified model of performance: validation of its predictions across different sleep/wake schedules. SLEEP 2016;39(1):249–262. PMID:26518594

A Comparative Study of Adolescent Risk Assessment Instruments: Predictive and Incremental Validity

ERIC Educational Resources Information Center

Welsh, Jennifer L.; Schmidt, Fred; McKinnon, Lauren; Chattha, H. K.; Meyers, Joanna R.

2008-01-01

Promising new adolescent risk assessment tools are being incorporated into clinical practice but currently possess limited evidence of predictive validity regarding their individual and/or combined use in risk assessments. The current study compares three structured adolescent risk instruments, Youth Level of Service/Case Management Inventory…

Recidivism in female offenders: PCL-R lifestyle factor and VRAG show predictive validity in a German sample.

PubMed

Eisenbarth, Hedwig; Osterheider, Michael; Nedopil, Norbert; Stadtland, Cornelis

2012-01-01

A clear and structured approach to evidence-based and gender-specific risk assessment of violence in female offenders is high on political and mental health agendas. However, most data on the factors involved in risk-assessment instruments are based on data of male offenders. The aim of the present study was to validate the use of the Psychopathy Checklist Revised (PCL-R), the HCR-20 and the Violence Risk Appraisal Guide (VRAG) for the prediction of recidivism in German female offenders. This study is part of the Munich Prognosis Project (MPP). It focuses on a subsample of female delinquents (n = 80) who had been referred for forensic-psychiatric evaluation prior to sentencing. The mean time at risk was 8 years (SD = 5 years; range: 1-18 years). During this time, 31% (n = 25) of the female offenders were reconvicted, 5% (n = 4) for violent and 26% (n = 21) for non-violent re-offenses. The predictive validity of the PCL-R for general recidivism was calculated. Analysis with receiver-operating characteristics revealed that the PCL-R total score, the PCL-R antisocial lifestyle factor, the PCL-R lifestyle factor and the PCL-R impulsive and irresponsible behavioral style factor had a moderate predictive validity for general recidivism (area under the curve, AUC = 0.66, p = 0.02). The VRAG has also demonstrated predictive validity (AUC = 0.72, p = 0.02), whereas the HCR-20 showed no predictive validity. These results appear to provide the first evidence that the PCL-R total score and the antisocial lifestyle factor are predictive for general female recidivism, as has been shown consistently for male recidivists. The implications of these findings for crime prevention, prognosis in women, and future research are discussed. Copyright © 2012 John Wiley & Sons, Ltd.

Validation of BEHAVE fire behavior predictions in oak savannas using five fuel models

Treesearch

Keith Grabner; John Dwyer; Bruce Cutter

1997-01-01

Prescribed fire is a valuable tool in the restoration and management of oak savannas. BEHAVE, a fire behavior prediction system developed by the United States Forest Service, can be a useful tool when managing oak savannas with prescribed fire. BEHAVE predictions of fire rate-of-spread and flame length were validated using four standardized fuel models: Fuel Model 1 (...

Cross Cultural Adaptation, Validity, and Reliability of the Farsi Breastfeeding Attrition Prediction Tools in Iranian Pregnant Women

PubMed Central

Mortazavi, Forough; Mousavi, Seyed Abbas; Chaman, Reza; Khosravi, Ahmad; Janke, Jill R.

2015-01-01

Background: The rate of exclusive breastfeeding in Iran is decreasing. The breastfeeding attrition prediction tools (BAPT) have been validated and used in predicting premature weaning. Objectives: We aimed to translate the BAPT into Farsi, assess its content validity, and examine its reliability and validity to identify exclusive breastfeeding discontinuation in Iran. Materials and Methods: The BAPT was translated into Farsi and the content validity of the Farsi version of the BAPT was assessed. It was administered to 356 pregnant women in the third trimester of pregnancy, who were residents of a city in northeast of Iran. The structural integrity of the four-factor model was assessed in confirmatory factor analysis (CFA) and exploratory factor analysis (EFA). Reliability was assessed using Cronbach’s alpha coefficient and item-subscale correlations. Validity was assessed using the known-group comparison (128 with vs. 228 without breastfeeding experience) and predictive validity (80 successes vs. 265 failures in exclusive breastfeeding). Results: The internal consistency of the whole instrument (49 items) was 0.775. CFA provided an acceptable fit to the a priori four-factor model (Chi-square/df = 1.8, Root Mean Square Error of Approximation (RMSEA) = 0.049, Standardized Root Mean Square Residual (SRMR) = 0.064, Comparative Fit Index (CFI) = 0.911). The difference in means of breastfeeding control (BFC) between the participants with and without breastfeeding experience was significant (P < 0.001). In addition, the total score of BAPT and the score of Breast Feeding Control (BFC) subscale were higher in women who were on exclusive breastfeeding than women who were not, at four months postpartum (P < 0.05). Conclusions: This study validated the Farsi version of BAPT. It is useful for researchers who want to use it in Iran to identify women at higher risks of Exclusive Breast Feeding (EBF) discontinuation. PMID:26019910

Development and validation of an ICD-10-based disability predictive index for patients admitted to hospitals with trauma.

PubMed

Wada, Tomoki; Yasunaga, Hideo; Yamana, Hayato; Matsui, Hiroki; Fushimi, Kiyohide; Morimura, Naoto

2018-03-01

There was no established disability predictive measurement for patients with trauma that could be used in administrative claims databases. The aim of the present study was to develop and validate a diagnosis-based disability predictive index for severe physical disability at discharge using the International Classification of Diseases, 10th revision (ICD-10) coding. This retrospective observational study used the Diagnosis Procedure Combination database in Japan. Patients who were admitted to hospitals with trauma and discharged alive from 01 April 2010 to 31 March 2015 were included. Pediatric patients under 15 years old were excluded. Data for patients admitted to hospitals from 01 April 2010 to 31 March 2013 was used for development of a disability predictive index (derivation cohort), while data for patients admitted to hospitals from 01 April 2013 to 31 March 2015 was used for the internal validation (validation cohort). The outcome of interest was severe physical disability defined as the Barthel Index score of <60 at discharge. Trauma-related ICD-10 codes were categorized into 36 injury groups with reference to the categorization used in the Global Burden of Diseases study 2013. A multivariable logistic regression analysis was performed for the outcome using the injury groups and patient baseline characteristics including patient age, sex, and Charlson Comorbidity Index (CCI) score in the derivation cohort. A score corresponding to a regression coefficient was assigned to each injury group. The disability predictive index for each patient was defined as the sum of the scores. The predictive performance of the index was validated using the receiver operating characteristic curve analysis in the validation cohort. The derivation cohort included 1,475,158 patients, while the validation cohort included 939,659 patients. Of the 939,659 patients, 235,382 (25.0%) were discharged with severe physical disability. The c-statistics of the disability predictive index

Unsteady Aerodynamic Validation Experiences From the Aeroelastic Prediction Workshop

NASA Technical Reports Server (NTRS)

Heeg, Jennifer; Chawlowski, Pawel

2014-01-01

The AIAA Aeroelastic Prediction Workshop (AePW) was held in April 2012, bringing together communities of aeroelasticians, computational fluid dynamicists and experimentalists. The extended objective was to assess the state of the art in computational aeroelastic methods as practical tools for the prediction of static and dynamic aeroelastic phenomena. As a step in this process, workshop participants analyzed unsteady aerodynamic and weakly-coupled aeroelastic cases. Forced oscillation and unforced system experiments and computations have been compared for three configurations. This paper emphasizes interpretation of the experimental data, computational results and their comparisons from the perspective of validation of unsteady system predictions. The issues examined in detail are variability introduced by input choices for the computations, post-processing, and static aeroelastic modeling. The final issue addressed is interpreting unsteady information that is present in experimental data that is assumed to be steady, and the resulting consequences on the comparison data sets.

Validation of the DECAF score to predict hospital mortality in acute exacerbations of COPD

PubMed Central

Echevarria, C; Steer, J; Heslop-Marshall, K; Stenton, SC; Hickey, PM; Hughes, R; Wijesinghe, M; Harrison, RN; Steen, N; Simpson, AJ; Gibson, GJ; Bourke, SC

2016-01-01

Background Hospitalisation due to acute exacerbations of COPD (AECOPD) is common, and subsequent mortality high. The DECAF score was derived for accurate prediction of mortality and risk stratification to inform patient care. We aimed to validate the DECAF score, internally and externally, and to compare its performance to other predictive tools. Methods The study took place in the two hospitals within the derivation study (internal validation) and in four additional hospitals (external validation) between January 2012 and May 2014. Consecutive admissions were identified by screening admissions and searching coding records. Admission clinical data, including DECAF indices, and mortality were recorded. The prognostic value of DECAF and other scores were assessed by the area under the receiver operator characteristic (AUROC) curve. Results In the internal and external validation cohorts, 880 and 845 patients were recruited. Mean age was 73.1 (SD 10.3) years, 54.3% were female, and mean (SD) FEV1 45.5 (18.3) per cent predicted. Overall mortality was 7.7%. The DECAF AUROC curve for inhospital mortality was 0.83 (95% CI 0.78 to 0.87) in the internal cohort and 0.82 (95% CI 0.77 to 0.87) in the external cohort, and was superior to other prognostic scores for inhospital or 30-day mortality. Conclusions DECAF is a robust predictor of mortality, using indices routinely available on admission. Its generalisability is supported by consistent strong performance; it can identify low-risk patients (DECAF 0–1) potentially suitable for Hospital at Home or early supported discharge services, and high-risk patients (DECAF 3–6) for escalation planning or appropriate early palliation. Trial registration number UKCRN ID 14214. PMID:26769015

Validation of Skeletal Muscle cis-Regulatory Module Predictions Reveals Nucleotide Composition Bias in Functional Enhancers

PubMed Central

Kwon, Andrew T.; Chou, Alice Yi; Arenillas, David J.; Wasserman, Wyeth W.

2011-01-01

We performed a genome-wide scan for muscle-specific cis-regulatory modules (CRMs) using three computational prediction programs. Based on the predictions, 339 candidate CRMs were tested in cell culture with NIH3T3 fibroblasts and C2C12 myoblasts for capacity to direct selective reporter gene expression to differentiated C2C12 myotubes. A subset of 19 CRMs validated as functional in the assay. The rate of predictive success reveals striking limitations of computational regulatory sequence analysis methods for CRM discovery. Motif-based methods performed no better than predictions based only on sequence conservation. Analysis of the properties of the functional sequences relative to inactive sequences identifies nucleotide sequence composition can be an important characteristic to incorporate in future methods for improved predictive specificity. Muscle-related TFBSs predicted within the functional sequences display greater sequence conservation than non-TFBS flanking regions. Comparison with recent MyoD and histone modification ChIP-Seq data supports the validity of the functional regions. PMID:22144875

Clinical Nomograms to Predict Stone-Free Rates after Shock-Wave Lithotripsy: Development and Internal-Validation

PubMed Central

Kim, Jung Kwon; Ha, Seung Beom; Jeon, Chan Hoo; Oh, Jong Jin; Cho, Sung Yong; Oh, Seung-June; Kim, Hyeon Hoe; Jeong, Chang Wook

2016-01-01

Purpose Shock-wave lithotripsy (SWL) is accepted as the first line treatment modality for uncomplicated upper urinary tract stones; however, validated prediction models with regards to stone-free rates (SFRs) are still needed. We aimed to develop nomograms predicting SFRs after the first and within the third session of SWL. Computed tomography (CT) information was also modeled for constructing nomograms. Materials and Methods From March 2006 to December 2013, 3028 patients were treated with SWL for ureter and renal stones at our three tertiary institutions. Four cohorts were constructed: Total-development, Total-validation, CT-development, and CT-validation cohorts. The nomograms were developed using multivariate logistic regression models with selected significant variables in a univariate logistic regression model. A C-index was used to assess the discrimination accuracy of nomograms and calibration plots were used to analyze the consistency of prediction. Results The SFR, after the first and within the third session, was 48.3% and 68.8%, respectively. Significant variables were sex, stone location, stone number, and maximal stone diameter in the Total-development cohort, and mean Hounsfield unit (HU) and grade of hydronephrosis (HN) were additional parameters in the CT-development cohort. The C-indices were 0.712 and 0.723 for after the first and within the third session of SWL in the Total-development cohort, and 0.755 and 0.756, in the CT-development cohort, respectively. The calibration plots showed good correspondences. Conclusions We constructed and validated nomograms to predict SFR after SWL. To the best of our knowledge, these are the first graphical nomograms to be modeled with CT information. These may be useful for patient counseling and treatment decision-making. PMID:26890006

The predictive validity of the BioMedical Admissions Test for pre-clinical examination performance.

PubMed

Emery, Joanne L; Bell, John F

2009-06-01

Some medical courses in the UK have many more applicants than places and almost all applicants have the highest possible previous and predicted examination grades. The BioMedical Admissions Test (BMAT) was designed to assist in the student selection process specifically for a number of 'traditional' medical courses with clear pre-clinical and clinical phases and a strong focus on science teaching in the early years. It is intended to supplement the information provided by examination results, interviews and personal statements. This paper reports on the predictive validity of the BMAT and its predecessor, the Medical and Veterinary Admissions Test. Results from the earliest 4 years of the test (2000-2003) were matched to the pre-clinical examination results of those accepted onto the medical course at the University of Cambridge. Correlation and logistic regression analyses were performed for each cohort. Section 2 of the test ('Scientific Knowledge') correlated more strongly with examination marks than did Section 1 ('Aptitude and Skills'). It also had a stronger relationship with the probability of achieving the highest examination class. The BMAT and its predecessor demonstrate predictive validity for the pre-clinical years of the medical course at the University of Cambridge. The test identifies important differences in skills and knowledge between candidates, not shown by their previous attainment, which predict their examination performance. It is thus a valid source of additional admissions information for medical courses with a strong scientific emphasis when previous attainment is very high.

A Longitudinal Study of the Predictive Validity of a Kindergarten Screening Battery.

ERIC Educational Resources Information Center

Kilgallon, Mary K.; Mueller, Richard J.

Test validity was studied in nine subtests of a kindergarten screening battery used to predict reading comprehension for children up to five years after entering kindergarten. The independent variables were kindergarteners' scores on the: (1) Otis-Lennon Mental Ability Test; (2) Bender Visual Motor Gestalt Test; (3) Detroit Tests of Learning…

Validated Questionnaire of Maternal Attitude and Knowledge for Predicting Caries Risk in Children: Epidemiological Study in North Jakarta, Indonesia.

PubMed

Laksmiastuti, Sri Ratna; Budiardjo, Sarworini Bagio; Sutadi, Heriandi

2017-06-01

Predicting caries risk in children can be done by identifying caries risk factors. It is an important measure which contributes to best understanding of the cariogenic profile of the patient. Identification could be done by clinical examination and answering the questionnaire. We arrange the study to verify the questionnaire validation for predicting caries risk in children. The study was conducted on 62 pairs of mothers and their children, aged between 3 and 5 years. The questionnaire consists of 10 questions concerning mothers' attitude and knowledge about oral health. The reliability and validity test is based on Cronbach's alpha and correlation coefficient value. All question are reliable (Cronbach's alpha = 0.873) and valid (Corrected item-total item correlation >0.4). Five questionnaires of mother's attitude about oral health and five questionnaires of mother's knowledge about oral health are reliable and valid for predicting caries risk in children.

A new test set for validating predictions of protein-ligand interaction.

PubMed

Nissink, J Willem M; Murray, Chris; Hartshorn, Mike; Verdonk, Marcel L; Cole, Jason C; Taylor, Robin

2002-12-01

We present a large test set of protein-ligand complexes for the purpose of validating algorithms that rely on the prediction of protein-ligand interactions. The set consists of 305 complexes with protonation states assigned by manual inspection. The following checks have been carried out to identify unsuitable entries in this set: (1) assessing the involvement of crystallographically related protein units in ligand binding; (2) identification of bad clashes between protein side chains and ligand; and (3) assessment of structural errors, and/or inconsistency of ligand placement with crystal structure electron density. In addition, the set has been pruned to assure diversity in terms of protein-ligand structures, and subsets are supplied for different protein-structure resolution ranges. A classification of the set by protein type is available. As an illustration, validation results are shown for GOLD and SuperStar. GOLD is a program that performs flexible protein-ligand docking, and SuperStar is used for the prediction of favorable interaction sites in proteins. The new CCDC/Astex test set is freely available to the scientific community (http://www.ccdc.cam.ac.uk). Copyright 2002 Wiley-Liss, Inc.

Reliability and Validity of the Load-Velocity Relationship to Predict the 1RM Back Squat.

PubMed

Banyard, Harry G; Nosaka, Kazunori; Haff, G Gregory

2017-07-01

Banyard, HG, Nosaka, K, and Haff, GG. Reliability and validity of the load-velocity relationship to predict the 1RM back squat. J Strength Cond Res 31(7): 1897-1904, 2017-This study investigated the reliability and validity of the load-velocity relationship to predict the free-weight back squat one repetition maximum (1RM). Seventeen strength-trained males performed three 1RM assessments on 3 separate days. All repetitions were performed to full depth with maximal concentric effort. Predicted 1RMs were calculated by entering the mean concentric velocity of the 1RM (V1RM) into an individualized linear regression equation, which was derived from the load-velocity relationship of 3 (20, 40, 60% of 1RM), 4 (20, 40, 60, 80% of 1RM), or 5 (20, 40, 60, 80, 90% of 1RM) incremental warm-up sets. The actual 1RM (140.3 ± 27.2 kg) was very stable between 3 trials (ICC = 0.99; SEM = 2.9 kg; CV = 2.1%; ES = 0.11). Predicted 1RM from 5 warm-up sets up to and including 90% of 1RM was the most reliable (ICC = 0.92; SEM = 8.6 kg; CV = 5.7%; ES = -0.02) and valid (r = 0.93; SEE = 10.6 kg; CV = 7.4%; ES = 0.71) of the predicted 1RM methods. However, all predicted 1RMs were significantly different (p ≤ 0.05; ES = 0.71-1.04) from the actual 1RM. Individual variation for the actual 1RM was small between trials ranging from -5.6 to 4.8% compared with the most accurate predictive method up to 90% of 1RM, which was more variable (-5.5 to 27.8%). Importantly, the V1RM (0.24 ± 0.06 m·s) was unreliable between trials (ICC = 0.42; SEM = 0.05 m·s; CV = 22.5%; ES = 0.14). The load-velocity relationship for the full depth free-weight back squat showed moderate reliability and validity but could not accurately predict 1RM, which was stable between trials. Thus, the load-velocity relationship 1RM prediction method used in this study cannot accurately modify sessional training loads because of large V1RM variability.

Further Validation of the Coach Identity Prominence Scale

ERIC Educational Resources Information Center

Pope, J. Paige; Hall, Craig R.

2014-01-01

This study was designed to examine select psychometric properties of the Coach Identity Prominence Scale (CIPS), including the reliability, factorial validity, convergent validity, discriminant validity, and predictive validity. Coaches (N = 338) who averaged 37 (SD = 12.27) years of age, had a mean of 13 (SD = 9.90) years of coaching experience,…

Validation of the 4P's Plus screen for substance use in pregnancy validation of the 4P's Plus.

PubMed

Chasnoff, I J; Wells, A M; McGourty, R F; Bailey, L K

2007-12-01

The purpose of this study is to validate the 4P's Plus screen for substance use in pregnancy. A total of 228 pregnant women enrolled in prenatal care underwent screening with the 4P's Plus and received a follow-up clinical assessment for substance use. Statistical analyses regarding reliability, sensitivity, specificity, and positive and negative predictive validity of the 4Ps Plus were conducted. The overall reliability for the five-item measure was 0.62. Seventy-four (32.5%) of the women had a positive screen. Sensitivity and specificity were very good, at 87 and 76%, respectively. Positive predictive validity was low (36%), but negative predictive validity was quite high (97%). Of the 31 women who had a positive clinical assessment, 45% were using less than 1 day per week. The 4P's Plus reliably and effectively screens pregnant women for risk of substance use, including those women typically missed by other perinatal screening methodologies.

The East London glaucoma prediction score: web-based validation of glaucoma risk screening tool

PubMed Central

Stephen, Cook; Benjamin, Longo-Mbenza

2013-01-01

AIM It is difficult for Optometrists and General Practitioners to know which patients are at risk. The East London glaucoma prediction score (ELGPS) is a web based risk calculator that has been developed to determine Glaucoma risk at the time of screening. Multiple risk factors that are available in a low tech environment are assessed to provide a risk assessment. This is extremely useful in settings where access to specialist care is difficult. Use of the calculator is educational. It is a free web based service. Data capture is user specific. METHOD The scoring system is a web based questionnaire that captures and subsequently calculates the relative risk for the presence of Glaucoma at the time of screening. Three categories of patient are described: Unlikely to have Glaucoma; Glaucoma Suspect and Glaucoma. A case review methodology of patients with known diagnosis is employed to validate the calculator risk assessment. RESULTS Data from the patient records of 400 patients with an established diagnosis has been captured and used to validate the screening tool. The website reports that the calculated diagnosis correlates with the actual diagnosis 82% of the time. Biostatistics analysis showed: Sensitivity = 88%; Positive predictive value = 97%; Specificity = 75%. CONCLUSION Analysis of the first 400 patients validates the web based screening tool as being a good method of screening for the at risk population. The validation is ongoing. The web based format will allow a more widespread recruitment for different geographic, population and personnel variables. PMID:23550097

Predictive validity of cannabis consumption measures: Results from a national longitudinal study.

PubMed

Buu, Anne; Hu, Yi-Han; Pampati, Sanjana; Arterberry, Brooke J; Lin, Hsien-Chang

2017-10-01

Validating the utility of cannabis consumption measures for predicting later cannabis related symptomatology or progression to cannabis use disorder (CUD) is crucial for prevention and intervention work that may use consumption measures for quick screening. This study examined whether cannabis use quantity and frequency predicted CUD symptom counts, progression to onset of CUD, and persistence of CUD. Data from the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) at Wave 1 (2001-2002) and Wave 2 (2004-2005) were used to identify three risk samples: (1) current cannabis users at Wave 1 who were at risk for having CUD symptoms at Wave 2; (2) current users without lifetime CUD who were at risk for incident CUD; and (3) current users with past-year CUD who were at risk for persistent CUD. Logistic regression and zero-inflated Poisson models were used to examine the longitudinal effect of cannabis consumption on CUD outcomes. Higher frequency of cannabis use predicted lower likelihood of being symptom-free but it did not predict the severity of CUD symptomatology. Higher frequency of cannabis use also predicted higher likelihood of progression to onset of CUD and persistence of CUD. Cannabis use quantity, however, did not predict any of the developmental stages of CUD symptomatology examined in this study. This study has provided a new piece of evidence to support the predictive validity of cannabis use frequency based on national longitudinal data. The result supports the common practice of including frequency items in cannabis screening tools. Copyright © 2017 Elsevier Ltd. All rights reserved.

Measurement of predictive validity in violence risk assessment studies: a second-order systematic review.

PubMed

Singh, Jay P; Desmarais, Sarah L; Van Dorn, Richard A

2013-01-01

The objective of the present review was to examine how predictive validity is analyzed and reported in studies of instruments used to assess violence risk. We reviewed 47 predictive validity studies published between 1990 and 2011 of 25 instruments that were included in two recent systematic reviews. Although all studies reported receiver operating characteristic curve analyses and the area under the curve (AUC) performance indicator, this methodology was defined inconsistently and findings often were misinterpreted. In addition, there was between-study variation in benchmarks used to determine whether AUCs were small, moderate, or large in magnitude. Though virtually all of the included instruments were designed to produce categorical estimates of risk - through the use of either actuarial risk bins or structured professional judgments - only a minority of studies calculated performance indicators for these categorical estimates. In addition to AUCs, other performance indicators, such as correlation coefficients, were reported in 60% of studies, but were infrequently defined or interpreted. An investigation of sources of heterogeneity did not reveal significant variation in reporting practices as a function of risk assessment approach (actuarial vs. structured professional judgment), study authorship, geographic location, type of journal (general vs. specialized audience), sample size, or year of publication. Findings suggest a need for standardization of predictive validity reporting to improve comparison across studies and instruments. Copyright © 2013 John Wiley & Sons, Ltd.

Predictive and Incremental Validity of Global and Domain-Based Adolescent Life Satisfaction Reports

ERIC Educational Resources Information Center

Haranin, Emily C.; Huebner, E. Scott; Suldo, Shannon M.

2007-01-01

Concurrent, predictive, and incremental validity of global and domain-based adolescent life satisfaction reports are examined with respect to internalizing and externalizing behavior problems. The Students' Life Satisfaction Scale (SLSS), Multidimensional Students' Life Satisfaction Scale (MSLSS), and measures of internalizing and externalizing…

Prediction of prostate cancer in unscreened men: external validation of a risk calculator.

PubMed

van Vugt, Heidi A; Roobol, Monique J; Kranse, Ries; Määttänen, Liisa; Finne, Patrik; Hugosson, Jonas; Bangma, Chris H; Schröder, Fritz H; Steyerberg, Ewout W

2011-04-01

Prediction models need external validation to assess their value beyond the setting where the model was derived from. To assess the external validity of the European Randomized study of Screening for Prostate Cancer (ERSPC) risk calculator (www.prostatecancer-riskcalculator.com) for the probability of having a positive prostate biopsy (P(posb)). The ERSPC risk calculator was based on data of the initial screening round of the ERSPC section Rotterdam and validated in 1825 and 531 men biopsied at the initial screening round in the Finnish and Swedish sections of the ERSPC respectively. P(posb) was calculated using serum prostate specific antigen (PSA), outcome of digital rectal examination (DRE), transrectal ultrasound and ultrasound assessed prostate volume. The external validity was assessed for the presence of cancer at biopsy by calibration (agreement between observed and predicted outcomes), discrimination (separation of those with and without cancer), and decision curves (for clinical usefulness). Prostate cancer was detected in 469 men (26%) of the Finnish cohort and in 124 men (23%) of the Swedish cohort. Systematic miscalibration was present in both cohorts (mean predicted probability 34% versus 26% observed, and 29% versus 23% observed, both p<0.001). The areas under the curves were 0.76 and 0.78, and substantially lower for the model with PSA only (0.64 and 0.68 respectively). The model proved clinically useful for any decision threshold compared with a model with PSA only, PSA and DRE, or biopsying all men. A limitation is that the model is based on sextant biopsies results. The ERSPC risk calculator discriminated well between those with and without prostate cancer among initially screened men, but overestimated the risk of a positive biopsy. Further research is necessary to assess the performance and applicability of the ERSPC risk calculator when a clinical setting is considered rather than a screening setting. Copyright © 2010 Elsevier Ltd. All rights

Developing and Validating a Survival Prediction Model for NSCLC Patients Through Distributed Learning Across 3 Countries.

PubMed

Jochems, Arthur; Deist, Timo M; El Naqa, Issam; Kessler, Marc; Mayo, Chuck; Reeves, Jackson; Jolly, Shruti; Matuszak, Martha; Ten Haken, Randall; van Soest, Johan; Oberije, Cary; Faivre-Finn, Corinne; Price, Gareth; de Ruysscher, Dirk; Lambin, Philippe; Dekker, Andre

2017-10-01

Tools for survival prediction for non-small cell lung cancer (NSCLC) patients treated with chemoradiation or radiation therapy are of limited quality. In this work, we developed a predictive model of survival at 2 years. The model is based on a large volume of historical patient data and serves as a proof of concept to demonstrate the distributed learning approach. Clinical data from 698 lung cancer patients, treated with curative intent with chemoradiation or radiation therapy alone, were collected and stored at 2 different cancer institutes (559 patients at Maastro clinic (Netherlands) and 139 at Michigan university [United States]). The model was further validated on 196 patients originating from The Christie (United Kingdon). A Bayesian network model was adapted for distributed learning (the animation can be viewed at https://www.youtube.com/watch?v=ZDJFOxpwqEA). Two-year posttreatment survival was chosen as the endpoint. The Maastro clinic cohort data are publicly available at https://www.cancerdata.org/publication/developing-and-validating-survival-prediction-model-nsclc-patients-through-distributed, and the developed models can be found at www.predictcancer.org. Variables included in the final model were T and N category, age, performance status, and total tumor dose. The model has an area under the curve (AUC) of 0.66 on the external validation set and an AUC of 0.62 on a 5-fold cross validation. A model based on the T and N category performed with an AUC of 0.47 on the validation set, significantly worse than our model (P<.001). Learning the model in a centralized or distributed fashion yields a minor difference on the probabilities of the conditional probability tables (0.6%); the discriminative performance of the models on the validation set is similar (P=.26). Distributed learning from federated databases allows learning of predictive models on data originating from multiple institutions while avoiding many of the data-sharing barriers. We believe

Demonstrating the validity of three general scores of PET in predicting higher education achievement in Israel.

PubMed

Oren, Carmel; Kennet-Cohen, Tamar; Turvall, Elliot; Allalouf, Avi

2014-01-01

The Psychometric Entrance Test (PET), used for admission to higher education in Israel together with the Matriculation (Bagrut), had in the past one general (total) score in which the weights for its domains: Verbal, Quantitative and English, were 2:2:1, respectively. In 2011, two additional total scores were introduced, with different weights for the Verbal and the Quantitative domains. This study compares the predictive validity of the three general scores of PET, and demonstrates validity in terms of utility. 100,863 freshmen students of all Israeli universities over the classes of 2005-2009. Regression weights and correlations of the predictors with FYGPA were computed. Simulations based on these results supplied the utility estimates. On average, PET is slightly more predictive than the Bagrut; using them both yields a better tool than either of them alone. Assigning differential weights to the components in the respective schools further improves the validity. The introduction of the new general scores of PET is validated by gathering and analyzing evidence based on relations of test scores to other variables. The utility of using the test can be demonstrated in ways different from correlations.

Validation of statistical predictive models meant to select melanoma patients for sentinel lymph node biopsy.

PubMed

Sabel, Michael S; Rice, John D; Griffith, Kent A; Lowe, Lori; Wong, Sandra L; Chang, Alfred E; Johnson, Timothy M; Taylor, Jeremy M G

2012-01-01

To identify melanoma patients at sufficiently low risk of nodal metastases who could avoid sentinel lymph node biopsy (SLNB), several statistical models have been proposed based upon patient/tumor characteristics, including logistic regression, classification trees, random forests, and support vector machines. We sought to validate recently published models meant to predict sentinel node status. We queried our comprehensive, prospectively collected melanoma database for consecutive melanoma patients undergoing SLNB. Prediction values were estimated based upon four published models, calculating the same reported metrics: negative predictive value (NPV), rate of negative predictions (RNP), and false-negative rate (FNR). Logistic regression performed comparably with our data when considering NPV (89.4 versus 93.6%); however, the model's specificity was not high enough to significantly reduce the rate of biopsies (SLN reduction rate of 2.9%). When applied to our data, the classification tree produced NPV and reduction in biopsy rates that were lower (87.7 versus 94.1 and 29.8 versus 14.3, respectively). Two published models could not be applied to our data due to model complexity and the use of proprietary software. Published models meant to reduce the SLNB rate among patients with melanoma either underperformed when applied to our larger dataset, or could not be validated. Differences in selection criteria and histopathologic interpretation likely resulted in underperformance. Statistical predictive models must be developed in a clinically applicable manner to allow for both validation and ultimately clinical utility.

Validation of Statistical Predictive Models Meant to Select Melanoma Patients for Sentinel Lymph Node Biopsy

PubMed Central

Sabel, Michael S.; Rice, John D.; Griffith, Kent A.; Lowe, Lori; Wong, Sandra L.; Chang, Alfred E.; Johnson, Timothy M.; Taylor, Jeremy M.G.

2013-01-01

Introduction To identify melanoma patients at sufficiently low risk of nodal metastases who could avoid SLN biopsy (SLNB). Several statistical models have been proposed based upon patient/tumor characteristics, including logistic regression, classification trees, random forests and support vector machines. We sought to validate recently published models meant to predict sentinel node status. Methods We queried our comprehensive, prospectively-collected melanoma database for consecutive melanoma patients undergoing SLNB. Prediction values were estimated based upon 4 published models, calculating the same reported metrics: negative predictive value (NPV), rate of negative predictions (RNP), and false negative rate (FNR). Results Logistic regression performed comparably with our data when considering NPV (89.4% vs. 93.6%); however the model’s specificity was not high enough to significantly reduce the rate of biopsies (SLN reduction rate of 2.9%). When applied to our data, the classification tree produced NPV and reduction in biopsies rates that were lower 87.7% vs. 94.1% and 29.8% vs. 14.3%, respectively. Two published models could not be applied to our data due to model complexity and the use of proprietary software. Conclusions Published models meant to reduce the SLNB rate among patients with melanoma either underperformed when applied to our larger dataset, or could not be validated. Differences in selection criteria and histopathologic interpretation likely resulted in underperformance. Development of statistical predictive models must be created in a clinically applicable manner to allow for both validation and ultimately clinical utility. PMID:21822550

A systematic review of validated sinus surgery simulators.

PubMed

Stew, B; Kao, S S-T; Dharmawardana, N; Ooi, E H

2018-06-01

Simulation provides a safe and effective opportunity to develop surgical skills. A variety of endoscopic sinus surgery (ESS) simulators has been described in the literature. Validation of these simulators allows for effective utilisation in training. To conduct a systematic review of the published literature to analyse the evidence for validated ESS simulation. Pubmed, Embase, Cochrane and Cinahl were searched from inception of the databases to 11 January 2017. Twelve thousand five hundred and sixteen articles were retrieved of which 10 112 were screened following the removal of duplicates. Thirty-eight full-text articles were reviewed after meeting search criteria. Evidence of face, content, construct, discriminant and predictive validity was extracted. Twenty articles were included in the analysis describing 12 ESS simulators. Eleven of these simulators had undergone validation: 3 virtual reality, 7 physical bench models and 1 cadaveric simulator. Seven of the simulators were shown to have face validity, 7 had construct validity and 1 had predictive validity. None of the simulators demonstrated discriminate validity. This systematic review demonstrates that a number of ESS simulators have been comprehensively validated. Many of the validation processes, however, lack standardisation in outcome reporting, thus limiting a meta-analysis comparison between simulators. © 2017 John Wiley & Sons Ltd.

Nomogram predicting response after chemoradiotherapy in rectal cancer using sequential PETCT imaging: a multicentric prospective study with external validation.

PubMed

van Stiphout, Ruud G P M; Valentini, Vincenzo; Buijsen, Jeroen; Lammering, Guido; Meldolesi, Elisa; van Soest, Johan; Leccisotti, Lucia; Giordano, Alessandro; Gambacorta, Maria A; Dekker, Andre; Lambin, Philippe

2014-11-01

To develop and externally validate a predictive model for pathologic complete response (pCR) for locally advanced rectal cancer (LARC) based on clinical features and early sequential (18)F-FDG PETCT imaging. Prospective data (i.a. THUNDER trial) were used to train (N=112, MAASTRO Clinic) and validate (N=78, Università Cattolica del S. Cuore) the model for pCR (ypT0N0). All patients received long-course chemoradiotherapy (CRT) and surgery. Clinical parameters were age, gender, clinical tumour (cT) stage and clinical nodal (cN) stage. PET parameters were SUVmax, SUVmean, metabolic tumour volume (MTV) and maximal tumour diameter, for which response indices between pre-treatment and intermediate scan were calculated. Using multivariate logistic regression, three probability groups for pCR were defined. The pCR rates were 21.4% (training) and 23.1% (validation). The selected predictive features for pCR were cT-stage, cN-stage, response index of SUVmean and maximal tumour diameter during treatment. The models' performances (AUC) were 0.78 (training) and 0.70 (validation). The high probability group for pCR resulted in 100% correct predictions for training and 67% for validation. The model is available on the website www.predictcancer.org. The developed predictive model for pCR is accurate and externally validated. This model may assist in treatment decisions during CRT to select complete responders for a wait-and-see policy, good responders for extra RT boost and bad responders for additional chemotherapy. Copyright © 2014 The Authors. Published by Elsevier Ireland Ltd.. All rights reserved.

Development and validation of a predictive score for perioperative transfusion in patients with hepatocellular carcinoma undergoing liver resection.

PubMed

Wang, Hai-Qing; Yang, Jian; Yang, Jia-Yin; Wang, Wen-Tao; Yan, Lu-Nan

2015-08-01

Liver resection is a major surgery requiring perioperative blood transfusion. Predicting the need for blood transfusion for patients undergoing liver resection is of great importance. The present study aimed to develop and validate a model for predicting transfusion requirement in HBV-related hepatocellular carcinoma patients undergoing liver resection. A total of 1543 consecutive liver resections were included in the study. Randomly selected sample set of 1080 cases (70% of the study cohort) were used to develop a predictive score for transfusion requirement and the remaining 30% (n=463) was used to validate the score. Based on the preoperative and predictable intraoperative parameters, logistic regression was used to identify risk factors and to create an integer score for the prediction of transfusion requirement. Extrahepatic procedure, major liver resection, hemoglobin level and platelets count were identified as independent predictors for transfusion requirement by logistic regression analysis. A score system integrating these 4 factors was stratified into three groups which could predict the risk of transfusion, with a rate of 11.4%, 24.7% and 57.4% for low, moderate and high risk, respectively. The prediction model appeared accurate with good discriminatory abilities, generating an area under the receiver operating characteristic curve of 0.736 in the development set and 0.709 in the validation set. We have developed and validated an integer-based risk score to predict perioperative transfusion for patients undergoing liver resection in a high-volume surgical center. This score allows identifying patients at a high risk and may alter transfusion practices.

Derivation, Validation and Application of a Pragmatic Risk Prediction Index for Benchmarking of Surgical Outcomes.

PubMed

Spence, Richard T; Chang, David C; Kaafarani, Haytham M A; Panieri, Eugenio; Anderson, Geoffrey A; Hutter, Matthew M

2018-02-01

Despite the existence of multiple validated risk assessment and quality benchmarking tools in surgery, their utility outside of high-income countries is limited. We sought to derive, validate and apply a scoring system that is both (1) feasible, and (2) reliably predicts mortality in a middle-income country (MIC) context. A 5-step methodology was used: (1) development of a de novo surgical outcomes database modeled around the American College of Surgeons' National Surgical Quality Improvement Program (ACS-NSQIP) in South Africa (SA dataset), (2) use of the resultant data to identify all predictors of in-hospital death with more than 90% capture indicating feasibility of collection, (3) use these predictors to derive and validate an integer-based score that reliably predicts in-hospital death in the 2012 ACS-NSQIP, (4) apply the score in the original SA dataset and demonstrate its performance, (5) identify threshold cutoffs of the score to prompt action and drive quality improvement. Following step one-three above, the 13 point Codman's score was derived and validated on 211,737 and 109,079 patients, respectively, and includes: age 65 (1), partially or completely dependent functional status (1), preoperative transfusions ≥4 units (1), emergency operation (2), sepsis or septic shock (2) American Society of Anesthesia score ≥3 (3) and operative procedure (1-3). Application of the score to 373 patients in the SA dataset showed good discrimination and calibration to predict an in-hospital death. A Codman Score of 8 is an optimal cutoff point for defining expected and unexpected deaths. We have designed a novel risk prediction score specific for a MIC context. The Codman Score can prove useful for both (1) preoperative decision-making and (2) benchmarking the quality of surgical care in MIC's.

Validation of predictive rules and indices of severity for community acquired pneumonia

PubMed Central

Ewig, S; de Roux, A; Bauer, T; Garcia, E; Mensa, J; Niederman, M; Torres, A

2004-01-01

Background: A study was undertaken to validate the modified American Thoracic Society (ATS) rule and two British Thoracic Society (BTS) rules for the prediction of ICU admission and mortality of community acquired pneumonia and to provide a validation of these predictions on the basis of the pneumonia severity index (PSI). Method: Six hundred and ninety six consecutive patients (457 men (66%), mean (SD) age 67.8 (17.1) years, range 18–101) admitted to a tertiary care hospital were studied prospectively. Of these, 116 (16.7%) were admitted to the ICU. Results: The modified ATS rule achieved a sensitivity of 69% (95% CI 50.7 to 77.2), specificity of 97% (95% CI 96.4 to 98.9), positive predictive value of 87% (95% CI 78.3 to 93.1), and negative predictive value of 94% (95% CI 91.8 to 95.8) in predicting admission to the ICU. The corresponding predictive indices for mortality were 94% (95% CI 82.5 to 98.7), 93% (95% CI 90.6 to 94.7), 49% (95% CI 38.2 to 59.7), and 99.5% (95% CI 98.5 to 99.9), respectively. These figures compared favourably with both the BTS rules. The BTS-CURB criteria achieved predictions of pneumonia severity and mortality comparable to the PSI. Conclusions: This study confirms the power of the modified ATS rule to predict severe pneumonia in individual patients. It may be incorporated into current guidelines for the assessment of pneumonia severity. The CURB criteria may be used as an alternative tool to PSI for the detection of low risk patients. PMID:15115872

The Predictive Validity of the ABFM's In-Training Examination.

PubMed

O'Neill, Thomas R; Li, Zijia; Peabody, Michael R; Lybarger, Melanie; Royal, Kenneth; Puffer, James C

2015-05-01

Our objective was to examine the predictive validity of the American Board of Family Medicine's (ABFM) In-Training Examination (ITE) with regard to predicting outcomes on the ABFM certification examination. This study used a repeated measures design across three levels of medical training (PGY1--PGY2, PGY2--PGY3, and PGY3--initial certification) with three different cohorts (2010--2011, 2011--2012, and 2012--2013) to examine: (1) how well the residents' ITE scores correlated with their test scores in the following year, (2) what the typical score increase was across training years, and (3) what was the sensitivity, specificity, positive predictive value, and negative predictive value of the PGY3 scores with regard to predicting future results on the MC-FP Examination. ITE scores generally correlate at about .7 with the following year's ITE or with the following year's certification examination. The mean growth from PGY1 to PGY2 was 52 points, from PGY2 to PGY3 was 34 points, and from PGY3 to initial certification was 27 points. The sensitivity, specificity, positive predictive value, and negative predictive value were .91, .47, .96, and .27, respectively. The ITE is a useful predictor of future ITE and initial certification examination performance.

Validation and uncertainty analysis of a pre-treatment 2D dose prediction model

NASA Astrophysics Data System (ADS)

Baeza, Jose A.; Wolfs, Cecile J. A.; Nijsten, Sebastiaan M. J. J. G.; Verhaegen, Frank

2018-02-01

Independent verification of complex treatment delivery with megavolt photon beam radiotherapy (RT) has been effectively used to detect and prevent errors. This work presents the validation and uncertainty analysis of a model that predicts 2D portal dose images (PDIs) without a patient or phantom in the beam. The prediction model is based on an exponential point dose model with separable primary and secondary photon fluence components. The model includes a scatter kernel, off-axis ratio map, transmission values and penumbra kernels for beam-delimiting components. These parameters were derived through a model fitting procedure supplied with point dose and dose profile measurements of radiation fields. The model was validated against a treatment planning system (TPS; Eclipse) and radiochromic film measurements for complex clinical scenarios, including volumetric modulated arc therapy (VMAT). Confidence limits on fitted model parameters were calculated based on simulated measurements. A sensitivity analysis was performed to evaluate the effect of the parameter uncertainties on the model output. For the maximum uncertainty, the maximum deviating measurement sets were propagated through the fitting procedure and the model. The overall uncertainty was assessed using all simulated measurements. The validation of the prediction model against the TPS and the film showed a good agreement, with on average 90.8% and 90.5% of pixels passing a (2%,2 mm) global gamma analysis respectively, with a low dose threshold of 10%. The maximum and overall uncertainty of the model is dependent on the type of clinical plan used as input. The results can be used to study the robustness of the model. A model for predicting accurate 2D pre-treatment PDIs in complex RT scenarios can be used clinically and its uncertainties can be taken into account.

Project Evaluation: Validation of a Scale and Analysis of Its Predictive Capacity

ERIC Educational Resources Information Center

Fernandes Malaquias, Rodrigo; de Oliveira Malaquias, Fernanda Francielle

2014-01-01

The objective of this study was to validate a scale for assessment of academic projects. As a complement, we examined its predictive ability by comparing the scores of advised/corrected projects based on the model and the final scores awarded to the work by an examining panel (approximately 10 months after the project design). Results of…

Design for validation: An approach to systems validation

NASA Technical Reports Server (NTRS)

Carter, William C.; Dunham, Janet R.; Laprie, Jean-Claude; Williams, Thomas; Howden, William; Smith, Brian; Lewis, Carl M. (Editor)

1989-01-01

Every complex system built is validated in some manner. Computer validation begins with review of the system design. As systems became too complicated for one person to review, validation began to rely on the application of adhoc methods by many individuals. As the cost of the changes mounted and the expense of failure increased, more organized procedures became essential. Attempts at devising and carrying out those procedures showed that validation is indeed a difficult technical problem. The successful transformation of the validation process into a systematic series of formally sound, integrated steps is necessary if the liability inherent in the future digita-system-based avionic and space systems is to be minimized. A suggested framework and timetable for the transformtion are presented. Basic working definitions of two pivotal ideas (validation and system life-cyle) are provided and show how the two concepts interact. Many examples are given of past and present validation activities by NASA and others. A conceptual framework is presented for the validation process. Finally, important areas are listed for ongoing development of the validation process at NASA Langley Research Center.

Validation and Use of a Predictive Modeling Tool: Employing Scientific Findings to Improve Responsible Conduct of Research Education.

PubMed

Mulhearn, Tyler J; Watts, Logan L; Todd, E Michelle; Medeiros, Kelsey E; Connelly, Shane; Mumford, Michael D

2017-01-01

Although recent evidence suggests ethics education can be effective, the nature of specific training programs, and their effectiveness, varies considerably. Building on a recent path modeling effort, the present study developed and validated a predictive modeling tool for responsible conduct of research education. The predictive modeling tool allows users to enter ratings in relation to a given ethics training program and receive instantaneous evaluative information for course refinement. Validation work suggests the tool's predicted outcomes correlate strongly (r = 0.46) with objective course outcomes. Implications for training program development and refinement are discussed.

Derivation and validation of a discharge disposition predicting model after acute stroke.

PubMed

Tseng, Hung-Pin; Lin, Feng-Jenq; Chen, Pi-Tzu; Mou, Chih-Hsin; Lee, Siu-Pak; Chang, Chun-Yuan; Chen, An-Chih; Liu, Chung-Hsiang; Yeh, Chung-Hsin; Tsai, Song-Yen; Hsiao, Yu-Jen; Lin, Ching-Huang; Hsu, Shih-Pin; Yu, Shih-Chieh; Hsu, Chung-Y; Sung, Fung-Chang

2015-06-01

Discharge disposition planning is vital for poststroke patients. We investigated clinical factors associated with discharging patients to nursing homes, using the Taiwan Stroke Registry data collected from 39 major hospitals. We randomly assigned 21,575 stroke inpatients registered from 2006 to 2008 into derivation and validation groups at a 3-to-1 ratio. We used the derivation group to develop a prediction model by measuring cumulative risk scores associated with potential predictors: age, sex, hypertension, diabetes mellitus, heart diseases, stroke history, snoring, main caregivers, stroke types, and National Institutes of Health Stroke Scale (NIHSS). Probability of nursing home care and odds ratio (OR) of nursing home care relative to home care by cumulative risk scores were measured for the prediction. The area under the receiver operating characteristic curve (AUROC) was used to assess the model discrimination against the validation group. Except for hypertension, all remaining potential predictors were significant independent predictors associated with stroke patient disposition to nursing home care after discharge from hospitals. The risk sharply increased with age and NIHSS. Patients with a cumulative risk score of 15 or more had an OR of 86.4 for the nursing home disposition. The AUROC plots showed similar areas under curves for the derivation group (.86, 95% confidence interval [CI], .85-.87) and for the validation group (.84, 95% CI, .83-.86). The cumulative risk score is an easy-to-estimate tool for preparing stroke patients and their family for disposition on discharge. Copyright © 2015 National Stroke Association. Published by Elsevier Inc. All rights reserved.

Validity of the SAT® for Predicting First-Year Grades: 2010 SAT Validity Sample. Statistical Report 2013-2

ERIC Educational Resources Information Center

Patterson, Brian F.; Mattern, Krista D.

2013-01-01

The continued accumulation of validity evidence for the core uses of educational assessments is critical to ensure that proper inferences will be made for those core purposes. To that end, the College Board has continued to follow previous cohorts of college students and this report provides updated validity evidence for using the SAT to predict…

External Validation Study of First Trimester Obstetric Prediction Models (Expect Study I): Research Protocol and Population Characteristics.

PubMed

Meertens, Linda Jacqueline Elisabeth; Scheepers, Hubertina Cj; De Vries, Raymond G; Dirksen, Carmen D; Korstjens, Irene; Mulder, Antonius Lm; Nieuwenhuijze, Marianne J; Nijhuis, Jan G; Spaanderman, Marc Ea; Smits, Luc Jm

2017-10-26

A number of first-trimester prediction models addressing important obstetric outcomes have been published. However, most models have not been externally validated. External validation is essential before implementing a prediction model in clinical practice. The objective of this paper is to describe the design of a study to externally validate existing first trimester obstetric prediction models, based upon maternal characteristics and standard measurements (eg, blood pressure), for the risk of pre-eclampsia (PE), gestational diabetes mellitus (GDM), spontaneous preterm birth (PTB), small-for-gestational-age (SGA) infants, and large-for-gestational-age (LGA) infants among Dutch pregnant women (Expect Study I). The results of a pilot study on the feasibility and acceptability of the recruitment process and the comprehensibility of the Pregnancy Questionnaire 1 are also reported. A multicenter prospective cohort study was performed in The Netherlands between July 1, 2013 and December 31, 2015. First trimester obstetric prediction models were systematically selected from the literature. Predictor variables were measured by the Web-based Pregnancy Questionnaire 1 and pregnancy outcomes were established using the Postpartum Questionnaire 1 and medical records. Information about maternal health-related quality of life, costs, and satisfaction with Dutch obstetric care was collected from a subsample of women. A pilot study was carried out before the official start of inclusion. External validity of the models will be evaluated by assessing discrimination and calibration. Based on the pilot study, minor improvements were made to the recruitment process and online Pregnancy Questionnaire 1. The validation cohort consists of 2614 women. Data analysis of the external validation study is in progress. This study will offer insight into the generalizability of existing, non-invasive first trimester prediction models for various obstetric outcomes in a Dutch obstetric population

A prospectively validated nomogram for predicting the risk of chemotherapy-induced febrile neutropenia: a multicenter study.

PubMed

Bozcuk, H; Yıldız, M; Artaç, M; Kocer, M; Kaya, Ç; Ulukal, E; Ay, S; Kılıç, M P; Şimşek, E H; Kılıçkaya, P; Uçar, S; Coskun, H S; Savas, B

2015-06-01

There is clinical need to predict risk of febrile neutropenia before a specific cycle of chemotherapy in cancer patients. Data on 3882 chemotherapy cycles in 1089 consecutive patients with lung, breast, and colon cancer from four teaching hospitals were used to construct a predictive model for febrile neutropenia. A final nomogram derived from the multivariate predictive model was prospectively confirmed in a second cohort of 960 consecutive cases and 1444 cycles. The following factors were used to construct the nomogram: previous history of febrile neutropenia, pre-cycle lymphocyte count, type of cancer, cycle of current chemotherapy, and patient age. The predictive model had a concordance index of 0.95 (95 % confidence interval (CI) = 0.91-0.99) in the derivation cohort and 0.85 (95 % CI = 0.80-0.91) in the external validation cohort. A threshold of 15 % for the risk of febrile neutropenia in the derivation cohort was associated with a sensitivity of 0.76 and specificity of 0.98. These figures were 1.00 and 0.49 in the validation cohort if a risk threshold of 50 % was chosen. This nomogram is helpful in the prediction of febrile neutropenia after chemotherapy in patients with lung, breast, and colon cancer. Usage of this nomogram may help decrease the morbidity and mortality associated with febrile neutropenia and deserves further validation.

Examining construct and predictive validity of the Health-IT Usability Evaluation Scale: confirmatory factor analysis and structural equation modeling results

PubMed Central

Yen, Po-Yin; Sousa, Karen H; Bakken, Suzanne

2014-01-01

Background In a previous study, we developed the Health Information Technology Usability Evaluation Scale (Health-ITUES), which is designed to support customization at the item level. Such customization matches the specific tasks/expectations of a health IT system while retaining comparability at the construct level, and provides evidence of its factorial validity and internal consistency reliability through exploratory factor analysis. Objective In this study, we advanced the development of Health-ITUES to examine its construct validity and predictive validity. Methods The health IT system studied was a web-based communication system that supported nurse staffing and scheduling. Using Health-ITUES, we conducted a cross-sectional study to evaluate users’ perception toward the web-based communication system after system implementation. We examined Health-ITUES's construct validity through first and second order confirmatory factor analysis (CFA), and its predictive validity via structural equation modeling (SEM). Results The sample comprised 541 staff nurses in two healthcare organizations. The CFA (n=165) showed that a general usability factor accounted for 78.1%, 93.4%, 51.0%, and 39.9% of the explained variance in ‘Quality of Work Life’, ‘Perceived Usefulness’, ‘Perceived Ease of Use’, and ‘User Control’, respectively. The SEM (n=541) supported the predictive validity of Health-ITUES, explaining 64% of the variance in intention for system use. Conclusions The results of CFA and SEM provide additional evidence for the construct and predictive validity of Health-ITUES. The customizability of Health-ITUES has the potential to support comparisons at the construct level, while allowing variation at the item level. We also illustrate application of Health-ITUES across stages of system development. PMID:24567081

On various metrics used for validation of predictive QSAR models with applications in virtual screening and focused library design.

PubMed

Roy, Kunal; Mitra, Indrani

2011-07-01

Quantitative structure-activity relationships (QSARs) have important applications in drug discovery research, environmental fate modeling, property prediction, etc. Validation has been recognized as a very important step for QSAR model development. As one of the important objectives of QSAR modeling is to predict activity/property/toxicity of new chemicals falling within the domain of applicability of the developed models and QSARs are being used for regulatory decisions, checking reliability of the models and confidence of their predictions is a very important aspect, which can be judged during the validation process. One prime application of a statistically significant QSAR model is virtual screening for molecules with improved potency based on the pharmacophoric features and the descriptors appearing in the QSAR model. Validated QSAR models may also be utilized for design of focused libraries which may be subsequently screened for the selection of hits. The present review focuses on various metrics used for validation of predictive QSAR models together with an overview of the application of QSAR models in the fields of virtual screening and focused library design for diverse series of compounds with citation of some recent examples.

How to test validity in orthodontic research: a mixed dentition analysis example.

PubMed

Donatelli, Richard E; Lee, Shin-Jae

2015-02-01

The data used to test the validity of a prediction method should be different from the data used to generate the prediction model. In this study, we explored whether an independent data set is mandatory for testing the validity of a new prediction method and how validity can be tested without independent new data. Several validation methods were compared in an example using the data from a mixed dentition analysis with a regression model. The validation errors of real mixed dentition analysis data and simulation data were analyzed for increasingly large data sets. The validation results of both the real and the simulation studies demonstrated that the leave-1-out cross-validation method had the smallest errors. The largest errors occurred in the traditional simple validation method. The differences between the validation methods diminished as the sample size increased. The leave-1-out cross-validation method seems to be an optimal validation method for improving the prediction accuracy in a data set with limited sample sizes. Copyright © 2015 American Association of Orthodontists. Published by Elsevier Inc. All rights reserved.

Characterization and validation of an in silico toxicology model to predict the mutagenic potential of drug impurities*

DOE Office of Scientific and Technical Information (OSTI.GOV)

Valerio, Luis G., E-mail: luis.valerio@fda.hhs.gov; Cross, Kevin P.

Control and minimization of human exposure to potential genotoxic impurities found in drug substances and products is an important part of preclinical safety assessments of new drug products. The FDA's 2008 draft guidance on genotoxic and carcinogenic impurities in drug substances and products allows use of computational quantitative structure–activity relationships (QSAR) to identify structural alerts for known and expected impurities present at levels below qualified thresholds. This study provides the information necessary to establish the practical use of a new in silico toxicology model for predicting Salmonella t. mutagenicity (Ames assay outcome) of drug impurities and other chemicals. We describemore » the model's chemical content and toxicity fingerprint in terms of compound space, molecular and structural toxicophores, and have rigorously tested its predictive power using both cross-validation and external validation experiments, as well as case studies. Consistent with desired regulatory use, the model performs with high sensitivity (81%) and high negative predictivity (81%) based on external validation with 2368 compounds foreign to the model and having known mutagenicity. A database of drug impurities was created from proprietary FDA submissions and the public literature which found significant overlap between the structural features of drug impurities and training set chemicals in the QSAR model. Overall, the model's predictive performance was found to be acceptable for screening drug impurities for Salmonella mutagenicity. -- Highlights: ► We characterize a new in silico model to predict mutagenicity of drug impurities. ► The model predicts Salmonella mutagenicity and will be useful for safety assessment. ► We examine toxicity fingerprints and toxicophores of this Ames assay model. ► We compare these attributes to those found in drug impurities known to FDA/CDER. ► We validate the model and find it has a desired predictive performance.« less

Reliability and validity in a nutshell.

PubMed

Bannigan, Katrina; Watson, Roger

2009-12-01

To explore and explain the different concepts of reliability and validity as they are related to measurement instruments in social science and health care. There are different concepts contained in the terms reliability and validity and these are often explained poorly and there is often confusion between them. To develop some clarity about reliability and validity a conceptual framework was built based on the existing literature. The concepts of reliability, validity and utility are explored and explained. Reliability contains the concepts of internal consistency and stability and equivalence. Validity contains the concepts of content, face, criterion, concurrent, predictive, construct, convergent (and divergent), factorial and discriminant. In addition, for clinical practice and research, it is essential to establish the utility of a measurement instrument. To use measurement instruments appropriately in clinical practice, the extent to which they are reliable, valid and usable must be established.

Validity of a manual soft tissue profile prediction method following mandibular setback osteotomy.

PubMed

Kolokitha, Olga-Elpis

2007-10-01

The aim of this study was to determine the validity of a manual cephalometric method used for predicting the post-operative soft tissue profiles of patients who underwent mandibular setback surgery and compare it to a computerized cephalometric prediction method (Dentofacial Planner). Lateral cephalograms of 18 adults with mandibular prognathism taken at the end of pre-surgical orthodontics and approximately one year after surgery were used. To test the validity of the manual method the prediction tracings were compared to the actual post-operative tracings. The Dentofacial Planner software was used to develop the computerized post-surgical prediction tracings. Both manual and computerized prediction printouts were analyzed by using the cephalometric system PORDIOS. Statistical analysis was performed by means of t-test. Comparison between manual prediction tracings and the actual post-operative profile showed that the manual method results in more convex soft tissue profiles; the upper lip was found in a more prominent position, upper lip thickness was increased and, the mandible and lower lip were found in a less posterior position than that of the actual profiles. Comparison between computerized and manual prediction methods showed that in the manual method upper lip thickness was increased, the upper lip was found in a more anterior position and the lower anterior facial height was increased as compared to the computerized prediction method. Cephalometric simulation of post-operative soft tissue profile following orthodontic-surgical management of mandibular prognathism imposes certain limitations related to the methods implied. However, both manual and computerized prediction methods remain a useful tool for patient communication.

External validity of two nomograms for predicting distant brain failure after radiosurgery for brain metastases in a bi-institutional independent patient cohort.

PubMed

Prabhu, Roshan S; Press, Robert H; Boselli, Danielle M; Miller, Katherine R; Lankford, Scott P; McCammon, Robert J; Moeller, Benjamin J; Heinzerling, John H; Fasola, Carolina E; Patel, Kirtesh R; Asher, Anthony L; Sumrall, Ashley L; Curran, Walter J; Shu, Hui-Kuo G; Burri, Stuart H

2018-03-01

Patients treated with stereotactic radiosurgery (SRS) for brain metastases (BM) are at increased risk of distant brain failure (DBF). Two nomograms have been recently published to predict individualized risk of DBF after SRS. The goal of this study was to assess the external validity of these nomograms in an independent patient cohort. The records of consecutive patients with BM treated with SRS at Levine Cancer Institute and Emory University between 2005 and 2013 were reviewed. Three validation cohorts were generated based on the specific nomogram or recursive partitioning analysis (RPA) entry criteria: Wake Forest nomogram (n = 281), Canadian nomogram (n = 282), and Canadian RPA (n = 303) validation cohorts. Freedom from DBF at 1-year in the Wake Forest study was 30% compared with 50% in the validation cohort. The validation c-index for both the 6-month and 9-month freedom from DBF Wake Forest nomograms was 0.55, indicating poor discrimination ability, and the goodness-of-fit test for both nomograms was highly significant (p < 0.001), indicating poor calibration. The 1-year actuarial DBF in the Canadian nomogram study was 43.9% compared with 50.9% in the validation cohort. The validation c-index for the Canadian 1-year DBF nomogram was 0.56, and the goodness-of-fit test was also highly significant (p < 0.001). The validation accuracy and c-index of the Canadian RPA classification was 53% and 0.61, respectively. The Wake Forest and Canadian nomograms for predicting risk of DBF after SRS were found to have limited predictive ability in an independent bi-institutional validation cohort. These results reinforce the importance of validating predictive models in independent patient cohorts.

Modification and Validation of Conceptual Design Aerodynamic Prediction Method HASC95 With VTXCHN

NASA Technical Reports Server (NTRS)

Albright, Alan E.; Dixon, Charles J.; Hegedus, Martin C.

1996-01-01

A conceptual/preliminary design level subsonic aerodynamic prediction code HASC (High Angle of Attack Stability and Control) has been improved in several areas, validated, and documented. The improved code includes improved methodologies for increased accuracy and robustness, and simplified input/output files. An engineering method called VTXCHN (Vortex Chine) for prediciting nose vortex shedding from circular and non-circular forebodies with sharp chine edges has been improved and integrated into the HASC code. This report contains a summary of modifications, description of the code, user's guide, and validation of HASC. Appendices include discussion of a new HASC utility code, listings of sample input and output files, and a discussion of the application of HASC to buffet analysis.

Predicting survival of de novo metastatic breast cancer in Asian women: systematic review and validation study.

PubMed

Miao, Hui; Hartman, Mikael; Bhoo-Pathy, Nirmala; Lee, Soo-Chin; Taib, Nur Aishah; Tan, Ern-Yu; Chan, Patrick; Moons, Karel G M; Wong, Hoong-Seam; Goh, Jeremy; Rahim, Siti Mastura; Yip, Cheng-Har; Verkooijen, Helena M

2014-01-01

In Asia, up to 25% of breast cancer patients present with distant metastases at diagnosis. Given the heterogeneous survival probabilities of de novo metastatic breast cancer, individual outcome prediction is challenging. The aim of the study is to identify existing prognostic models for patients with de novo metastatic breast cancer and validate them in Asia. We performed a systematic review to identify prediction models for metastatic breast cancer. Models were validated in 642 women with de novo metastatic breast cancer registered between 2000 and 2010 in the Singapore Malaysia Hospital Based Breast Cancer Registry. Survival curves for low, intermediate and high-risk groups according to each prognostic score were compared by log-rank test and discrimination of the models was assessed by concordance statistic (C-statistic). We identified 16 prediction models, seven of which were for patients with brain metastases only. Performance status, estrogen receptor status, metastatic site(s) and disease-free interval were the most common predictors. We were able to validate nine prediction models. The capacity of the models to discriminate between poor and good survivors varied from poor to fair with C-statistics ranging from 0.50 (95% CI, 0.48-0.53) to 0.63 (95% CI, 0.60-0.66). The discriminatory performance of existing prediction models for de novo metastatic breast cancer in Asia is modest. Development of an Asian-specific prediction model is needed to improve prognostication and guide decision making.

Predictive Validity of National Basketball Association Draft Combine on Future Performance.

PubMed

Teramoto, Masaru; Cross, Chad L; Rieger, Randall H; Maak, Travis G; Willick, Stuart E

2018-02-01

Teramoto, M, Cross, CL, Rieger, RH, Maak, TG, and Willick, SE. Predictive validity of national basketball association draft combine on future performance. J Strength Cond Res 32(2): 396-408, 2018-The National Basketball Association (NBA) Draft Combine is an annual event where prospective players are evaluated in terms of their athletic abilities and basketball skills. Data collected at the Combine should help NBA teams select right the players for the upcoming NBA draft; however, its value for predicting future performance of players has not been examined. This study investigated predictive validity of the NBA Draft Combine on future performance of basketball players. We performed a principal component analysis (PCA) on the 2010-2015 Combine data to reduce correlated variables (N = 234), a correlation analysis on the Combine data and future on-court performance to examine relationships (maximum pairwise N = 217), and a robust principal component regression (PCR) analysis to predict first-year and 3-year on-court performance from the Combine measures (N = 148 and 127, respectively). Three components were identified within the Combine data through PCA (= Combine subscales): length-size, power-quickness, and upper-body strength. As per the correlation analysis, the individual Combine items for anthropometrics, including height without shoes, standing reach, weight, wingspan, and hand length, as well as the Combine subscale of length-size, had positive, medium-to-large-sized correlations (r = 0.313-0.545) with defensive performance quantified by Defensive Box Plus/Minus. The robust PCR analysis showed that the Combine subscale of length-size was a predictor most significantly associated with future on-court performance (p ≤ 0.05), including Win Shares, Box Plus/Minus, and Value Over Replacement Player, followed by upper-body strength. In conclusion, the NBA Draft Combine has value for predicting future performance of players.

Validation of High Frequency (HF) Propagation Prediction Models in the Arctic region

NASA Astrophysics Data System (ADS)

Athieno, R.; Jayachandran, P. T.

2014-12-01

Despite the emergence of modern techniques for long distance communication, Ionospheric communication in the high frequency (HF) band (3-30 MHz) remains significant to both civilian and military users. However, the efficient use of the ever-varying ionosphere as a propagation medium is dependent on the reliability of ionospheric and HF propagation prediction models. Most available models are empirical implying that data collection has to be sufficiently large to provide good intended results. The models we present were developed with little data from the high latitudes which necessitates their validation. This paper presents the validation of three long term High Frequency (HF) propagation prediction models over a path within the Arctic region. Measurements of the Maximum Usable Frequency for a 3000 km range (MUF (3000) F2) for Resolute, Canada (74.75° N, 265.00° E), are obtained from hand-scaled ionograms generated by the Canadian Advanced Digital Ionosonde (CADI). The observations have been compared with predictions obtained from the Ionospheric Communication Enhanced Profile Analysis Program (ICEPAC), Voice of America Coverage Analysis Program (VOACAP) and International Telecommunication Union Recommendation 533 (ITU-REC533) for 2009, 2011, 2012 and 2013. A statistical analysis shows that the monthly predictions seem to reproduce the general features of the observations throughout the year though it is more evident in the winter and equinox months. Both predictions and observations show a diurnal and seasonal variation. The analysed models did not show large differences in their performances. However, there are noticeable differences across seasons for the entire period analysed: REC533 gives a better performance in winter months while VOACAP has a better performance for both equinox and summer months. VOACAP gives a better performance in the daily predictions compared to ICEPAC though, in general, the monthly predictions seem to agree more with the

Assessing Predictive Validity of Pressure Ulcer Risk Scales- A Systematic Review and Meta-Analysis

PubMed Central

PARK, Seong-Hi; LEE, Hea Shoon

2016-01-01

Background: The purpose of this study was to present a scientific reason for pressure ulcer risk scales: Cubbin& Jackson modified Braden, Norton, and Waterlow, as a nursing diagnosis tool by utilizing predictive validity of pressure sores. Methods: Articles published between 1966 and 2013 from periodicals indexed in the Ovid Medline, Embase, CINAHL, KoreaMed, NDSL, and other databases were selected using the key word “pressure ulcer”. QUADAS-II was applied for assessment for internal validity of the diagnostic studies. Selected studies were analyzed using meta-analysis with MetaDisc 1.4. Results: Seventeen diagnostic studies with high methodological quality, involving 5,185 patients, were included. In the results of the meta-analysis, sROC AUC of Braden, Norton, and Waterflow scale was over 0.7, showing moderate predictive validity, but they have limited interpretation due to significant differences between studies. In addition, Waterlow scale is insufficient as a screening tool owing to low sensitivity compared with other scales. Conclusion: The contemporary pressure ulcer risk scale is not suitable for uninform practice on patients under standardized criteria. Therefore, in order to provide more effective nursing care for bedsores, a new or modified pressure ulcer risk scale should be developed upon strength and weaknesses of existing tools. PMID:27114977

Predictive Validity of Measures of the Pathfinder Scaling Algorithm on Programming Performance: Alternative Assessment Strategy for Programming Education

ERIC Educational Resources Information Center

Lau, Wilfred W. F.; Yuen, Allan H. K.

2009-01-01

Recent years have seen a shift in focus from assessment of learning to assessment for learning and the emergence of alternative assessment methods. However, the reliability and validity of these methods as assessment tools are still questionable. In this article, we investigated the predictive validity of measures of the Pathfinder Scaling…

Validation of a Deterministic Vibroacoustic Response Prediction Model

NASA Technical Reports Server (NTRS)

Caimi, Raoul E.; Margasahayam, Ravi

1997-01-01

This report documents the recently completed effort involving validation of a deterministic theory for the random vibration problem of predicting the response of launch pad structures in the low-frequency range (0 to 50 hertz). Use of the Statistical Energy Analysis (SEA) methods is not suitable in this range. Measurements of launch-induced acoustic loads and subsequent structural response were made on a cantilever beam structure placed in close proximity (200 feet) to the launch pad. Innovative ways of characterizing random, nonstationary, non-Gaussian acoustics are used for the development of a structure's excitation model. Extremely good correlation was obtained between analytically computed responses and those measured on the cantilever beam. Additional tests are recommended to bound the problem to account for variations in launch trajectory and inclination.

Resolving Contradictions of Predictive Validity of University Matriculation Examinations in Nigeria: A Meta-Analysis Approach

ERIC Educational Resources Information Center

Modupe, Ale Veronica; Babafemi, Kolawole Emmanuel

2015-01-01

The study examined the various means of solving contradictions of predictive studies of University Matriculation Examination in Nigeria. The study used a sample size of 35 studies on predictive validity of University Matriculation Examination in Nigeria, which was purposively selected to have met the criteria for meta-analysis. Two null hypotheses…

Shift Verification and Validation

DOE Office of Scientific and Technical Information (OSTI.GOV)

Pandya, Tara M.; Evans, Thomas M.; Davidson, Gregory G

2016-09-07

This documentation outlines the verification and validation of Shift for the Consortium for Advanced Simulation of Light Water Reactors (CASL). Five main types of problems were used for validation: small criticality benchmark problems; full-core reactor benchmarks for light water reactors; fixed-source coupled neutron-photon dosimetry benchmarks; depletion/burnup benchmarks; and full-core reactor performance benchmarks. We compared Shift results to measured data and other simulated Monte Carlo radiation transport code results, and found very good agreement in a variety of comparison measures. These include prediction of critical eigenvalue, radial and axial pin power distributions, rod worth, leakage spectra, and nuclide inventories over amore » burn cycle. Based on this validation of Shift, we are confident in Shift to provide reference results for CASL benchmarking.« less

Ruling out coronary artery disease in primary care: development and validation of a simple prediction rule.

PubMed

Bösner, Stefan; Haasenritter, Jörg; Becker, Annette; Karatolios, Konstantinos; Vaucher, Paul; Gencer, Baris; Herzig, Lilli; Heinzel-Gutenbrunner, Monika; Schaefer, Juergen R; Abu Hani, Maren; Keller, Heidi; Sönnichsen, Andreas C; Baum, Erika; Donner-Banzhoff, Norbert

2010-09-07

Chest pain can be caused by various conditions, with life-threatening cardiac disease being of greatest concern. Prediction scores to rule out coronary artery disease have been developed for use in emergency settings. We developed and validated a simple prediction rule for use in primary care. We conducted a cross-sectional diagnostic study in 74 primary care practices in Germany. Primary care physicians recruited all consecutive patients who presented with chest pain (n = 1249) and recorded symptoms and findings for each patient (derivation cohort). An independent expert panel reviewed follow-up data obtained at six weeks and six months on symptoms, investigations, hospital admissions and medications to determine the presence or absence of coronary artery disease. Adjusted odds ratios of relevant variables were used to develop a prediction rule. We calculated measures of diagnostic accuracy for different cut-off values for the prediction scores using data derived from another prospective primary care study (validation cohort). The prediction rule contained five determinants (age/sex, known vascular disease, patient assumes pain is of cardiac origin, pain is worse during exercise, and pain is not reproducible by palpation), with the score ranging from 0 to 5 points. The area under the curve (receiver operating characteristic curve) was 0.87 (95% confidence interval [CI] 0.83-0.91) for the derivation cohort and 0.90 (95% CI 0.87-0.93) for the validation cohort. The best overall discrimination was with a cut-off value of 3 (positive result 3-5 points; negative result prediction rule for coronary artery disease in primary care proved to be robust in the validation cohort. It can help to rule out coronary artery disease in patients presenting with chest pain in primary care.

Examining construct and predictive validity of the Health-IT Usability Evaluation Scale: confirmatory factor analysis and structural equation modeling results.

PubMed

Yen, Po-Yin; Sousa, Karen H; Bakken, Suzanne

2014-10-01

In a previous study, we developed the Health Information Technology Usability Evaluation Scale (Health-ITUES), which is designed to support customization at the item level. Such customization matches the specific tasks/expectations of a health IT system while retaining comparability at the construct level, and provides evidence of its factorial validity and internal consistency reliability through exploratory factor analysis. In this study, we advanced the development of Health-ITUES to examine its construct validity and predictive validity. The health IT system studied was a web-based communication system that supported nurse staffing and scheduling. Using Health-ITUES, we conducted a cross-sectional study to evaluate users' perception toward the web-based communication system after system implementation. We examined Health-ITUES's construct validity through first and second order confirmatory factor analysis (CFA), and its predictive validity via structural equation modeling (SEM). The sample comprised 541 staff nurses in two healthcare organizations. The CFA (n=165) showed that a general usability factor accounted for 78.1%, 93.4%, 51.0%, and 39.9% of the explained variance in 'Quality of Work Life', 'Perceived Usefulness', 'Perceived Ease of Use', and 'User Control', respectively. The SEM (n=541) supported the predictive validity of Health-ITUES, explaining 64% of the variance in intention for system use. The results of CFA and SEM provide additional evidence for the construct and predictive validity of Health-ITUES. The customizability of Health-ITUES has the potential to support comparisons at the construct level, while allowing variation at the item level. We also illustrate application of Health-ITUES across stages of system development. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Validity and reliability of bioelectrical impedance analysis and skinfold thickness in predicting body fat in military personnel.

PubMed

Aandstad, Anders; Holtberget, Kristian; Hageberg, Rune; Holme, Ingar; Anderssen, Sigmund A

2014-02-01

Previous studies show that body composition is related to injury risk and physical performance in soldiers. Thus, valid methods for measuring body composition in military personnel are needed. The frequently used body mass index method is not a valid measure of body composition in soldiers, but reliability and validity of alternative field methods are less investigated in military personnel. Thus, we carried out test and retest of skinfold (SKF), single frequency bioelectrical impedance analysis (SF-BIA), and multifrequency bioelectrical impedance analysis measurements in 65 male and female soldiers. Several validated equations were used to predict percent body fat from these methods. Dual-energy X-ray absorptiometry was also measured, and acted as the criterion method. Results showed that SF-BIA was the most reliable method in both genders. In women, SF-BIA was also the most valid method, whereas SKF or a combination of SKF and SF-BIA produced the highest validity in men. Reliability and validity varied substantially among the equations examined. The best methods and equations produced test-retest 95% limits of agreement below ±1% points, whereas the corresponding validity figures were ±3.5% points. Each investigator and practitioner must consider whether such measurement errors are acceptable for its specific use. Reprint & Copyright © 2014 Association of Military Surgeons of the U.S.

Personalized Prediction of Psychosis: External validation of the NAPLS2 Psychosis Risk Calculator with the EDIPPP project

PubMed Central

Carrión, Ricardo E.; Cornblatt, Barbara A.; Burton, Cynthia Z.; Tso, Ivy F; Auther, Andrea; Adelsheim, Steven; Calkins, Roderick; Carter, Cameron S.; Niendam, Tara; Taylor, Stephan F.; McFarlane, William R.

2016-01-01

Objective In the current issue, Cannon and colleagues, as part of the second phase of the North American Prodrome Longitudinal Study (NAPLS2), report on a risk calculator for the individualized prediction of developing a psychotic disorder in a 2-year period. The present study represents an external validation of the NAPLS2 psychosis risk calculator using an independent sample of subjects at clinical high risk for psychosis collected as part of the Early Detection, Intervention, and Prevention of Psychosis Program (EDIPPP). Methods 176 subjects with follow-up (from the total EDIPPP sample of 210) rated as clinical high-risk (CHR) based on the Structured Interview for Prodromal Syndromes were used to construct a new prediction model with the 6 significant predictor variables in the NAPLS2 psychosis risk calculator (unusual thoughts, suspiciousness, Symbol Coding, verbal learning, social functioning decline, baseline age, and family history). Discrimination performance was assessed with the area under the receiver operating curve (AUC). The NAPLS2 risk calculator was then used to generate a psychosis risk estimate for each case in the external validation sample. Results The external validation model showed good discrimination, with an AUC of 79% (95% CI 0.644–0.937). In addition, the personalized risk generated by the NAPLS calculator provided a solid estimation of the actual conversion outcome in the validation sample. Conclusions In the companion papers in this issue, two independent samples of CHR subjects converge to validate the NAPLS2 psychosis risk calculator. This prediction calculator represents a meaningful step towards early intervention and personalized treatment of psychotic disorders. PMID:27363511

Observed Parenting Behavior with Teens: Measurement Invariance and Predictive Validity Across Race

PubMed Central

Skinner, Martie L.; MacKenzie, Elizabeth P.; Haggerty, Kevin P.; Hill, Karl G.; Roberson, Kendra C.

2011-01-01

Previous reports supporting measurement equality between European American and African American families have often focused on self-reported risk factors or observed parent behavior with young children. This study examines equality of measurement of observer ratings of parenting behavior with adolescents during structured tasks; mean levels of observed parenting; and predictive validity of teen self-reports of antisocial behaviors and beliefs using a sample of 163 African American and 168 European American families. Multiple-group confirmatory factor analyses supported measurement invariance across ethnic groups for 4 measures of observed parenting behavior: prosocial rewards, psychological costs, antisocial rewards, and problem solving. Some mean-level differences were found: African American parents exhibited lower levels of prosocial rewards, higher levels of psychological costs, and lower problem solving when compared to European Americans. No significant mean difference was found in rewards for antisocial behavior. Multigroup structural equation models suggested comparable relationships across race (predictive validity) between parenting constructs and youth antisocial constructs (i.e., drug initiation, positive drug attitudes, antisocial attitudes, problem behaviors) in all but one of the tested relationships. This study adds to existing evidence that family-based interventions targeting parenting behaviors can be generalized to African American families. PMID:21787057

Derivation and Validation of a Clostridium difficile Infection Recurrence Prediction Rule in a National Cohort of Veterans.

PubMed

Reveles, Kelly R; Mortensen, Eric M; Koeller, Jim M; Lawson, Kenneth A; Pugh, Mary Jo V; Rumbellow, Sarah A; Argamany, Jacqueline R; Frei, Christopher R

2018-03-01

Prior studies have identified risk factors for recurrent Clostridium difficile infection (CDI), but few studies have integrated these factors into a clinical prediction rule that can aid clinical decision-making. The objectives of this study were to derive and validate a CDI recurrence prediction rule to identify patients at risk for first recurrence in a national cohort of veterans. Retrospective cohort study. Veterans Affairs Informatics and Computing Infrastructure. A total of 22,615 adult Veterans Health Administration beneficiaries with first-episode CDI between October 1, 2002, and September 30, 2014; of these patients, 7538 were assigned to the derivation cohort and 15,077 to the validation cohort. A 60-day CDI recurrence prediction rule was created in a derivation cohort using backward logistic regression. Those variables significant at p<0.01 were assigned an integer score proportional to the regression coefficient. The model was then validated in the derivation cohort and a separate validation cohort. Patients were then split into three risk categories, and rates of recurrence were described for each category. The CDI recurrence prediction rule included the following predictor variables with their respective point values: prior third- and fourth-generation cephalosporins (1 point), prior proton pump inhibitors (1 point), prior antidiarrheals (1 point), nonsevere CDI (2 points), and community-onset CDI (3 points). In the derivation cohort, the 60-day CDI recurrence risk for each score ranged from 7.5% (0 points) to 57.9% (8 points). The risk score was strongly correlated with recurrence (R 2  = 0.94). Patients were split into low-risk (0-2 points), medium-risk (3-5 points), and high-risk (6-8 points) classes and had the following recurrence rates: 8.9%, 20.2%, and 35.0%, respectively. Findings were similar in the validation cohort. Several CDI and patient-specific factors were independently associated with 60-day CDI recurrence risk. When integrated into

Development and external validation of a prostate health index-based nomogram for predicting prostate cancer

PubMed Central

Zhu, Yao; Han, Cheng-Tao; Zhang, Gui-Ming; Liu, Fang; Ding, Qiang; Xu, Jian-Feng; Vidal, Adriana C.; Freedland, Stephen J.; Ng, Chi-Fai; Ye, Ding-Wei

2015-01-01

To develop and externally validate a prostate health index (PHI)-based nomogram for predicting the presence of prostate cancer (PCa) at biopsy in Chinese men with prostate-specific antigen 4–10 ng/mL and normal digital rectal examination (DRE). 347 men were recruited from two hospitals between 2012 and 2014 to develop a PHI-based nomogram to predict PCa. To validate these results, we used a separate cohort of 230 men recruited at another center between 2008 and 2013. Receiver operator curves (ROC) were used to assess the ability to predict PCa. A nomogram was derived from the multivariable logistic regression model and its accuracy was assessed by the area under the ROC (AUC). PHI achieved the highest AUC of 0.839 in the development cohort compared to the other predictors (p < 0.001). Including age and prostate volume, a PHI-based nomogram was constructed and rendered an AUC of 0.877 (95% CI 0.813–0.938). The AUC of the nomogram in the validation cohort was 0.786 (95% CI 0.678–0.894). In clinical effectiveness analyses, the PHI-based nomogram reduced unnecessary biopsies from 42.6% to 27% using a 5% threshold risk of PCa to avoid biopsy with no increase in the number of missed cases relative to conventional biopsy decision. PMID:26471350

Development and validation of a risk-prediction algorithm for the recurrence of panic disorder.

PubMed

Liu, Yan; Sareen, Jitender; Bolton, James; Wang, JianLi

2015-05-01

To develop and validate a risk prediction algorithm for the recurrence of panic disorder. Three-year longitudinal data were taken from the National Epidemiologic Survey on Alcohol and Related Conditions (2001/2002-2004/2005). One thousand six hundred and eighty one participants with a lifetime panic disorder and who had not had panic attacks for at least 2 months at baseline were included. The development cohort included 949 participants; 732 from different census regions were in the validation cohort. Recurrence of panic disorder over the follow-up period was assessed using the Alcohol Use Disorder and Associated Disabilities Interview Schedule, based on the DSM-IV criteria. Logistic regression was used for deriving the algorithm. Discrimination and calibration were assessed in the development and the validation cohorts. The developed algorithm consisted of 11 predictors: age, sex, panic disorder in the past 12 months, nicotine dependence, rapid heartbeat/tachycardia, taking medication for panic attacks, feelings of choking and persistent worry about having another panic attack, two personality traits, and childhood trauma. The algorithm had good discriminative power (C statistic = 0.7863, 95% CI: 0.7487, 0.8240). The C statistic was 0.7283 (95% CI: 0.6889, 0.7764) in the external validation data set. The developed risk algorithm for predicting the recurrence of panic disorder has good discrimination and excellent calibration. Data related to the predictors can be easily attainable in routine clinical practice. It can be used by clinicians to calculate the probability of recurrence of panic disorder in the next 3 years for individual patients, communicate with patients regarding personal risks, and thus improve personalized treatment approaches. © 2015 Wiley Periodicals, Inc.

External prognostic validations and comparisons of age- and gender-adjusted exercise capacity predictions.

PubMed

Kim, Esther S H; Ishwaran, Hemant; Blackstone, Eugene; Lauer, Michael S

2007-11-06

The purpose of this study was to externally validate the prognostic value of age- and gender-based nomograms and categorical definitions of impaired exercise capacity (EC). Exercise capacity predicts death, but its use in routine clinical practice is hampered by its close correlation with age and gender. For a median of 5 years, we followed 22,275 patients without known heart disease who underwent symptom-limited stress testing. Models for predicted or impaired EC were identified by literature search. Gender-specific multivariable proportional hazards models were constructed. Four methods were used to assess validity: Akaike Information Criterion (AIC), right-censored c-index in 100 out-of-bootstrap samples, the Nagelkerke Index R2, and calculation of calibration error in 100 bootstrap samples. There were 646 and 430 deaths in 13,098 men and 9,177 women, respectively. Of the 7 models tested in men, a model based on a Veterans Affairs cohort (predicted metabolic equivalents [METs] = 18 - [0.15 x age]) had the highest AIC and R2. In women, a model based on the St. James Take Heart Project (predicted METs = 14.7 - [0.13 x age]) performed best. Categorical definitions of fitness performed less well. Even after accounting for age and gender, there was still an important interaction with age, whereby predicted EC was a weaker predictor in older subjects (p for interaction <0.001 in men and 0.003 in women). Several methods describe EC accounting for age and gender-related differences, but their ability to predict mortality differ. Simple cutoff values fail to fully describe EC's strong predictive value.

Chemotherapy effectiveness and mortality prediction in surgically treated osteosarcoma dogs: A validation study.

PubMed

Schmidt, A F; Nielen, M; Withrow, S J; Selmic, L E; Burton, J H; Klungel, O H; Groenwold, R H H; Kirpensteijn, J

2016-03-01

Canine osteosarcoma is the most common bone cancer, and an important cause of mortality and morbidity, in large purebred dogs. Previously we constructed two multivariable models to predict a dog's 5-month or 1-year mortality risk after surgical treatment for osteosarcoma. According to the 5-month model, dogs with a relatively low risk of 5-month mortality benefited most from additional chemotherapy treatment. In the present study, we externally validated these results using an independent cohort study of 794 dogs. External performance of our prediction models showed some disagreement between observed and predicted risk, mean difference: -0.11 (95% confidence interval [95% CI]-0.29; 0.08) for 5-month risk and 0.25 (95%CI 0.10; 0.40) for 1-year mortality risk. After updating the intercept, agreement improved: -0.0004 (95%CI-0.16; 0.16) and -0.002 (95%CI-0.15; 0.15). The chemotherapy by predicted mortality risk interaction (P-value=0.01) showed that the chemotherapy compared to no chemotherapy effectiveness was modified by 5-month mortality risk: dogs with a relatively lower risk of mortality benefited most from additional chemotherapy. Chemotherapy effectiveness on 1-year mortality was not significantly modified by predicted risk (P-value=0.28). In conclusion, this external validation study confirmed that our multivariable risk prediction models can predict a patient's mortality risk and that dogs with a relatively lower risk of 5-month mortality seem to benefit most from chemotherapy. Copyright © 2016 Elsevier B.V. All rights reserved.

Validity of a Manual Soft Tissue Profile Prediction Method Following Mandibular Setback Osteotomy

PubMed Central

Kolokitha, Olga-Elpis

2007-01-01

Objectives The aim of this study was to determine the validity of a manual cephalometric method used for predicting the post-operative soft tissue profiles of patients who underwent mandibular setback surgery and compare it to a computerized cephalometric prediction method (Dentofacial Planner). Lateral cephalograms of 18 adults with mandibular prognathism taken at the end of pre-surgical orthodontics and approximately one year after surgery were used. Methods To test the validity of the manual method the prediction tracings were compared to the actual post-operative tracings. The Dentofacial Planner software was used to develop the computerized post-surgical prediction tracings. Both manual and computerized prediction printouts were analyzed by using the cephalometric system PORDIOS. Statistical analysis was performed by means of t-test. Results Comparison between manual prediction tracings and the actual post-operative profile showed that the manual method results in more convex soft tissue profiles; the upper lip was found in a more prominent position, upper lip thickness was increased and, the mandible and lower lip were found in a less posterior position than that of the actual profiles. Comparison between computerized and manual prediction methods showed that in the manual method upper lip thickness was increased, the upper lip was found in a more anterior position and the lower anterior facial height was increased as compared to the computerized prediction method. Conclusions Cephalometric simulation of post-operative soft tissue profile following orthodontic-surgical management of mandibular prognathism imposes certain limitations related to the methods implied. However, both manual and computerized prediction methods remain a useful tool for patient communication. PMID:19212468

Statistical validation of predictive TRANSP simulations of baseline discharges in preparation for extrapolation to JET D-T

NASA Astrophysics Data System (ADS)

Kim, Hyun-Tae; Romanelli, M.; Yuan, X.; Kaye, S.; Sips, A. C. C.; Frassinetti, L.; Buchanan, J.; Contributors, JET

2017-06-01

This paper presents for the first time a statistical validation of predictive TRANSP simulations of plasma temperature using two transport models, GLF23 and TGLF, over a database of 80 baseline H-mode discharges in JET-ILW. While the accuracy of the predicted T e with TRANSP-GLF23 is affected by plasma collisionality, the dependency of predictions on collisionality is less significant when using TRANSP-TGLF, indicating that the latter model has a broader applicability across plasma regimes. TRANSP-TGLF also shows a good matching of predicted T i with experimental measurements allowing for a more accurate prediction of the neutron yields. The impact of input data and assumptions prescribed in the simulations are also investigated in this paper. The statistical validation and the assessment of uncertainty level in predictive TRANSP simulations for JET-ILW-DD will constitute the basis for the extrapolation to JET-ILW-DT experiments.

Joint use of over- and under-sampling techniques and cross-validation for the development and assessment of prediction models.

PubMed

Blagus, Rok; Lusa, Lara

2015-11-04

Prediction models are used in clinical research to develop rules that can be used to accurately predict the outcome of the patients based on some of their characteristics. They represent a valuable tool in the decision making process of clinicians and health policy makers, as they enable them to estimate the probability that patients have or will develop a disease, will respond to a treatment, or that their disease will recur. The interest devoted to prediction models in the biomedical community has been growing in the last few years. Often the data used to develop the prediction models are class-imbalanced as only few patients experience the event (and therefore belong to minority class). Prediction models developed using class-imbalanced data tend to achieve sub-optimal predictive accuracy in the minority class. This problem can be diminished by using sampling techniques aimed at balancing the class distribution. These techniques include under- and oversampling, where a fraction of the majority class samples are retained in the analysis or new samples from the minority class are generated. The correct assessment of how the prediction model is likely to perform on independent data is of crucial importance; in the absence of an independent data set, cross-validation is normally used. While the importance of correct cross-validation is well documented in the biomedical literature, the challenges posed by the joint use of sampling techniques and cross-validation have not been addressed. We show that care must be taken to ensure that cross-validation is performed correctly on sampled data, and that the risk of overestimating the predictive accuracy is greater when oversampling techniques are used. Examples based on the re-analysis of real datasets and simulation studies are provided. We identify some results from the biomedical literature where the incorrect cross-validation was performed, where we expect that the performance of oversampling techniques was heavily

Predicting Survival of De Novo Metastatic Breast Cancer in Asian Women: Systematic Review and Validation Study

PubMed Central

Miao, Hui; Hartman, Mikael; Bhoo-Pathy, Nirmala; Lee, Soo-Chin; Taib, Nur Aishah; Tan, Ern-Yu; Chan, Patrick; Moons, Karel G. M.; Wong, Hoong-Seam; Goh, Jeremy; Rahim, Siti Mastura; Yip, Cheng-Har; Verkooijen, Helena M.

2014-01-01

Background In Asia, up to 25% of breast cancer patients present with distant metastases at diagnosis. Given the heterogeneous survival probabilities of de novo metastatic breast cancer, individual outcome prediction is challenging. The aim of the study is to identify existing prognostic models for patients with de novo metastatic breast cancer and validate them in Asia. Materials and Methods We performed a systematic review to identify prediction models for metastatic breast cancer. Models were validated in 642 women with de novo metastatic breast cancer registered between 2000 and 2010 in the Singapore Malaysia Hospital Based Breast Cancer Registry. Survival curves for low, intermediate and high-risk groups according to each prognostic score were compared by log-rank test and discrimination of the models was assessed by concordance statistic (C-statistic). Results We identified 16 prediction models, seven of which were for patients with brain metastases only. Performance status, estrogen receptor status, metastatic site(s) and disease-free interval were the most common predictors. We were able to validate nine prediction models. The capacity of the models to discriminate between poor and good survivors varied from poor to fair with C-statistics ranging from 0.50 (95% CI, 0.48–0.53) to 0.63 (95% CI, 0.60–0.66). Conclusion The discriminatory performance of existing prediction models for de novo metastatic breast cancer in Asia is modest. Development of an Asian-specific prediction model is needed to improve prognostication and guide decision making. PMID:24695692

An examination of the predictive validity of the risk matrix 2000 in England and wales.

PubMed

Barnett, Georgia D; Wakeling, Helen C; Howard, Philip D

2010-12-01

This study examined the predictive validity of an actuarial risk-assessment tool with convicted sexual offenders in England and Wales. A modified version of the RM2000/s scale and the RM2000 v and c scales (Thornton et al., 2003) were examined for accuracy in predicting proven sexual violent, nonsexual violent, and combined sexual and/or nonsexual violent reoffending in a sample of sexual offenders who had either started a community sentence or been released from prison into the community by March 2007. Rates of proven reoffending were examined at 2 years for the majority of the sample (n = 4,946), and 4 years ( n = 578) for those for whom these data were available. The predictive validity of the RM2000 scales was also explored for different subgroups of sexual offenders to assess the robustness of the tool. Both the modified RM2000/s and the complete v and c scales effectively classified offenders into distinct risk categories that differed significantly in rates of proven sexual and/or nonsexual violent reoffending. Survival analyses on the RM2000/s and v scales (N = 9,284) indicated that the higher risk groups offended more quickly and at a higher rate than lower risk groups. The relative predictive validity of the RM2000/s, v, and c, as calculated using Receiver Operating Characteristics (ROC) analyses, were moderate (.68) for RM2000/s and large for both the RM2000/c (.73) and RM2000/v (.80), at the 2-year follow-up. RM2000/s was moderately accurate in predicting relative risk of proven sexual reoffending for a variety of subgroups of sexual offenders.

Derivation and External Validation of Prediction Models for Advanced Chronic Kidney Disease Following Acute Kidney Injury.

PubMed

James, Matthew T; Pannu, Neesh; Hemmelgarn, Brenda R; Austin, Peter C; Tan, Zhi; McArthur, Eric; Manns, Braden J; Tonelli, Marcello; Wald, Ron; Quinn, Robert R; Ravani, Pietro; Garg, Amit X

2017-11-14

Some patients will develop chronic kidney disease after a hospitalization with acute kidney injury; however, no risk-prediction tools have been developed to identify high-risk patients requiring follow-up. To derive and validate predictive models for progression of acute kidney injury to advanced chronic kidney disease. Data from 2 population-based cohorts of patients with a prehospitalization estimated glomerular filtration rate (eGFR) of more than 45 mL/min/1.73 m2 and who had survived hospitalization with acute kidney injury (defined by a serum creatinine increase during hospitalization > 0.3 mg/dL or > 50% of their prehospitalization baseline), were used to derive and validate multivariable prediction models. The risk models were derived from 9973 patients hospitalized in Alberta, Canada (April 2004-March 2014, with follow-up to March 2015). The risk models were externally validated with data from a cohort of 2761 patients hospitalized in Ontario, Canada (June 2004-March 2012, with follow-up to March 2013). Demographic, laboratory, and comorbidity variables measured prior to discharge. Advanced chronic kidney disease was defined by a sustained reduction in eGFR less than 30 mL/min/1.73 m2 for at least 3 months during the year after discharge. All participants were followed up for up to 1 year. The participants (mean [SD] age, 66 [15] years in the derivation and internal validation cohorts and 69 [11] years in the external validation cohort; 40%-43% women per cohort) had a mean (SD) baseline serum creatinine level of 1.0 (0.2) mg/dL and more than 20% had stage 2 or 3 acute kidney injury. Advanced chronic kidney disease developed in 408 (2.7%) of 9973 patients in the derivation cohort and 62 (2.2%) of 2761 patients in the external validation cohort. In the derivation cohort, 6 variables were independently associated with the outcome: older age, female sex, higher baseline serum creatinine value, albuminuria, greater severity of acute kidney injury, and higher

External Validation and Evaluation of Reliability and Validity of the Modified Seoul National University Renal Stone Complexity Scoring System to Predict Stone-Free Status After Retrograde Intrarenal Surgery.

PubMed

Park, Juhyun; Kang, Minyong; Jeong, Chang Wook; Oh, Sohee; Lee, Jeong Woo; Lee, Seung Bae; Son, Hwancheol; Jeong, Hyeon; Cho, Sung Yong

2015-08-01

The modified Seoul National University Renal Stone Complexity scoring system (S-ReSC-R) for retrograde intrarenal surgery (RIRS) was developed as a tool to predict stone-free rate (SFR) after RIRS. We externally validated the S-ReSC-R. We retrospectively reviewed 159 patients who underwent RIRS. The S-ReSC-R was assigned from 1 to 12 according to the location and number of sites involved. The stone-free status was defined as no evidence of a stone or with clinically insignificant residual fragment stones less than 2 mm. Interobserver and test-retest reliabilities were evaluated. Statistical performance of the prediction model was assessed by its predictive accuracy, predictive probability, and clinical usefulness. Overall SFR was 73.0%. The SFRs were 86.7%, 70.2%, and 48.6% in low-score (1-2), intermediate-score (3-4), and high-score (5-12) groups, respectively (p<0.001). External validation of S-ReSC-R revealed an area under the curve (AUC) of 0.731 (95% CI 0.650-0.813). The AUC of the three-titered S-ReSC-R was 0.701 (95% CI 0.609-0.794). The calibration plot showed that the predicted probability of SFR had a concordance comparable to that of observed frequency. The Hosmer-Lemeshow goodness of fit test revealed a p-value of 0.01 for the S-ReSC-R and 0.90 for the three-titered S-ReSC-R. Interobserver and test-retest reliabilities revealed an almost perfect level of agreement. The present study proved the predictive value of S-ReSC-R to predict SFR following RIRS in an independent cohort. Interobserver and test-retest reliabilities confirmed that S-ReSC-R was reliable and valid.

The Predictive Validity of the Tilburg Frailty Indicator: Disability, Health Care Utilization, and Quality of Life in a Population at Risk

ERIC Educational Resources Information Center

Gobbens, Robbert J. J.; van Assen, Marcel A. L. M.; Luijkx, Katrien G.; Schols, Jos M. G. A.

2012-01-01

Purpose: To assess the predictive validity of frailty and its domains (physical, psychological, and social), as measured by the Tilburg Frailty Indicator (TFI), for the adverse outcomes disability, health care utilization, and quality of life. Design and Methods: The predictive validity of the TFI was tested in a representative sample of 484…

Predictive and Treatment Validity of Life Satisfaction and the Quality of Life Inventory

ERIC Educational Resources Information Center

Frisch, Michael B.; Clark, Michelle P.; Rouse, Steven V.; Rudd, M. David; Paweleck, Jennifer K.; Greenstone, Andrew; Kopplin, David A.

2005-01-01

The clinical and positive psychology usefulness of quality of life, well-being, and life satisfaction assessments depends on their ability to predict important outcomes and to detect intervention-related change. These issues were explored in the context of a program of instrument validation for the Quality of Life Inventory (QOLI) involving 3,927…

Development and validation of a predictive model for 90-day readmission following elective spine surgery.

PubMed

Parker, Scott L; Sivaganesan, Ahilan; Chotai, Silky; McGirt, Matthew J; Asher, Anthony L; Devin, Clinton J

2018-06-15

OBJECTIVE Hospital readmissions lead to a significant increase in the total cost of care in patients undergoing elective spine surgery. Understanding factors associated with an increased risk of postoperative readmission could facilitate a reduction in such occurrences. The aims of this study were to develop and validate a predictive model for 90-day hospital readmission following elective spine surgery. METHODS All patients undergoing elective spine surgery for degenerative disease were enrolled in a prospective longitudinal registry. All 90-day readmissions were prospectively recorded. For predictive modeling, all covariates were selected by choosing those variables that were significantly associated with readmission and by incorporating other relevant variables based on clinical intuition and the Akaike information criterion. Eighty percent of the sample was randomly selected for model development and 20% for model validation. Multiple logistic regression analysis was performed with Bayesian model averaging (BMA) to model the odds of 90-day readmission. Goodness of fit was assessed via the C-statistic, that is, the area under the receiver operating characteristic curve (AUC), using the training data set. Discrimination (predictive performance) was assessed using the C-statistic, as applied to the 20% validation data set. RESULTS A total of 2803 consecutive patients were enrolled in the registry, and their data were analyzed for this study. Of this cohort, 227 (8.1%) patients were readmitted to the hospital (for any cause) within 90 days postoperatively. Variables significantly associated with an increased risk of readmission were as follows (OR [95% CI]): lumbar surgery 1.8 [1.1-2.8], government-issued insurance 2.0 [1.4-3.0], hypertension 2.1 [1.4-3.3], prior myocardial infarction 2.2 [1.2-3.8], diabetes 2.5 [1.7-3.7], and coagulation disorder 3.1 [1.6-5.8]. These variables, in addition to others determined a priori to be clinically relevant, comprised 32

Implicit Sex Guilt Predicts Sexual Behaviors: Evidence for the Validity of the Sex Guilt Implicit Association Test.

PubMed

Totonchi, Delaram A; Derlega, Valerian J; Janda, Louis H

2018-05-14

Self-report measures of sexuality may be influenced by people's conscious concerns about confidentiality and social desirability. Alternatively, non-conscious measures (e.g., implicit association tests; IATs) are designed to minimize these validity concerns. We constructed an IAT measure of sex guilt using 154 male and female university students. The sex guilt IAT demonstrated convergent validity as it correlated with various sexual behaviors and incremental validity as it improved the prediction of several sexual behaviors beyond that provided by the Mosher sex guilt scale. We conclude that a non-conscious measure of sex guilt may complement the use of self-reports in studying sexual behaviors.

Derivation and Internal Validation of a Clinical Prediction Tool for 30-Day Mortality in Lower Gastrointestinal Bleeding.

PubMed

Sengupta, Neil; Tapper, Elliot B

2017-05-01

There are limited data to predict which patients with lower gastrointestinal bleeding are at risk for adverse outcomes. We aimed to develop a clinical tool based on admission variables to predict 30-day mortality in lower gastrointestinal bleeding. We used a validated machine learning algorithm to identify adult patients hospitalized with lower gastrointestinal bleeding at an academic medical center between 2008 and 2015. The cohort was split randomly into derivation and validation cohorts. In the derivation cohort, we used multiple logistic regression on all candidate admission variables to create a prediction model for 30-day mortality, using area under the receiving operator characteristic curve and misclassification rate to estimate prediction accuracy. Regression coefficients were used to derive an integer score, and mortality risk associated with point totals was assessed. In the derivation cohort (n = 4044), 8 variables were most associated with 30-day mortality: age, dementia, metastatic cancer, chronic kidney disease, chronic pulmonary disease, anticoagulant use, admission hematocrit, and albumin. The model yielded a misclassification rate of 0.06 and area under the curve of 0.81. The integer score ranged from -10 to 26 in the derivation cohort, with a misclassification rate of 0.11 and area under the curve of 0.74. In the validation cohort (n = 2060), the score had an area under the curve of 0.72 with a misclassification rate of 0.12. After dividing the score into 4 quartiles of risk, 30-day mortality in the derivation and validation sets was 3.6% and 4.4% in quartile 1, 4.9% and 7.3% in quartile 2, 9.9% and 9.1% in quartile 3, and 24% and 26% in quartile 4, respectively. A clinical tool can be used to predict 30-day mortality in patients hospitalized with lower gastrointestinal bleeding. Copyright © 2017 Elsevier Inc. All rights reserved.

Incremental Validity of Useful Field of View Subtests for the Prediction of Instrumental Activities of Daily Living

PubMed Central

Aust, Frederik; Edwards, Jerri D.

2015-01-01

Introduction The Useful Field of View Test (UFOV®) is a cognitive measure that predicts older adults’ ability to perform a range of everyday activities. However, little is known about the individual contribution of each subtest to these predictions and the underlying constructs of UFOV performance remain a topic of debate. Method We investigated the incremental validity of UFOV subtests for the prediction of Instrumental Activities of Daily Living (IADL) performance in two independent datasets, the SKILL (n = 828) and ACTIVE (n = 2426) studies. We, then, explored the cognitive and visual abilities assessed by UFOV using a range of neuropsychological and vision tests administered in the SKILL study. Results In the four subtest variant of UFOV, only subtests 2 and 3 consistently made independent contributions to the prediction of IADL performance across three different behavioral measures. In all cases, the incremental validity of UFOV subtests 1 and 4 was negligible. Furthermore, we found that UFOV was related to processing speed, general non-speeded cognition, and visual function; the omission of subtests 1 and 4 from the test score did not affect these associations. Conclusions UFOV subtests 1 and 4 appear to be of limited use to predict IADL and possibly other everyday activities. Future experimental research should investigate if shortening the UFOV by omitting these subtests is a reliable and valid assessment approach. PMID:26782018

Validation of a new formula for predicting body weight in a Mexican population with overweight and obesity.

PubMed

Quiroz-Olguín, Gabriela; Serralde-Zúñiga, Aurora Elizabeth; Saldaña-Morales, Vianey; Guevara-Cruz, Martha

2013-01-01

Body weight measurement is of critical importance when evaluating the nutritional status of patients entering a hospital. In some situations, such as the case of patients who are bedridden or in wheelchairs, these measurements cannot be obtained using standardized methods. We have designed and validated a formula for predicting body weight. To design and validate a formula for predicting body weight using circumference-based equations. The following anthropometric measurements were taken for a sample of 76 patients: weight (kg), calf circumference, average arm circumference, waist circumference, hip circumference, wrist circumference and demispan. All circumferences were taken in centimetres (cm), and gender and age were taken into account. This equation was validated in 85 individuals from a different population. The correlation with the new equation was analyzed and compared to a previously validated method. The equation for weight prediction was the following: Weight = 0.524 (WC) - 0.176 (age) + 0.484 (HC) + 0.613 (DS) + 0.704 (CC) + 2.75 (WrC) - 3.330 (if female) - 140.87. The correlation coefficient was 0.96 for the total group of patients, 0.971 for men and 0.961 for women (p < 0.0001 for all measurements). The equation we developed is accurate and can be used to estimate body weight in overweight and/or obese patients with mobility problems, such as bedridden patients or patients in wheelchairs. Copyright © AULA MEDICA EDICIONES 2013. Published by AULA MEDICA. All rights reserved.

Evaluation of the Predictive Validity of Thermography in Identifying Extravasation With Intravenous Chemotherapy Infusions.

PubMed

Matsui, Yuko; Murayama, Ryoko; Tanabe, Hidenori; Oe, Makoto; Motoo, Yoshiharu; Wagatsuma, Takanori; Michibuchi, Michiko; Kinoshita, Sachiko; Sakai, Keiko; Konya, Chizuko; Sugama, Junko; Sanada, Hiromi

Early detection of extravasation is important, but conventional methods of detection lack objectivity and reliability. This study evaluated the predictive validity of thermography for identifying extravasation during intravenous antineoplastic therapy. Of 257 patients who received chemotherapy through peripheral veins, extravasation was identified in 26. Thermography was performed every 15 to 30 minutes during the infusions. Sensitivity, specificity, positive predictive value, and negative predictive value using thermography were 84.6%, 94.8%, 64.7%, and 98.2%, respectively. This study showed that thermography offers an accurate prediction of extravasation.

Evaluation of the Predictive Validity of Thermography in Identifying Extravasation With Intravenous Chemotherapy Infusions

PubMed Central

Murayama, Ryoko; Tanabe, Hidenori; Oe, Makoto; Motoo, Yoshiharu; Wagatsuma, Takanori; Michibuchi, Michiko; Kinoshita, Sachiko; Sakai, Keiko; Konya, Chizuko; Sugama, Junko; Sanada, Hiromi

2017-01-01

Early detection of extravasation is important, but conventional methods of detection lack objectivity and reliability. This study evaluated the predictive validity of thermography for identifying extravasation during intravenous antineoplastic therapy. Of 257 patients who received chemotherapy through peripheral veins, extravasation was identified in 26. Thermography was performed every 15 to 30 minutes during the infusions. Sensitivity, specificity, positive predictive value, and negative predictive value using thermography were 84.6%, 94.8%, 64.7%, and 98.2%, respectively. This study showed that thermography offers an accurate prediction of extravasation. PMID:29112585

A cross-validation package driving Netica with python

USGS Publications Warehouse

Fienen, Michael N.; Plant, Nathaniel G.

2014-01-01

Bayesian networks (BNs) are powerful tools for probabilistically simulating natural systems and emulating process models. Cross validation is a technique to avoid overfitting resulting from overly complex BNs. Overfitting reduces predictive skill. Cross-validation for BNs is known but rarely implemented due partly to a lack of software tools designed to work with available BN packages. CVNetica is open-source, written in Python, and extends the Netica software package to perform cross-validation and read, rebuild, and learn BNs from data. Insights gained from cross-validation and implications on prediction versus description are illustrated with: a data-driven oceanographic application; and a model-emulation application. These examples show that overfitting occurs when BNs become more complex than allowed by supporting data and overfitting incurs computational costs as well as causing a reduction in prediction skill. CVNetica evaluates overfitting using several complexity metrics (we used level of discretization) and its impact on performance metrics (we used skill).

An updated PREDICT breast cancer prognostication and treatment benefit prediction model with independent validation.

PubMed

Candido Dos Reis, Francisco J; Wishart, Gordon C; Dicks, Ed M; Greenberg, David; Rashbass, Jem; Schmidt, Marjanka K; van den Broek, Alexandra J; Ellis, Ian O; Green, Andrew; Rakha, Emad; Maishman, Tom; Eccles, Diana M; Pharoah, Paul D P

2017-05-22

PREDICT is a breast cancer prognostic and treatment benefit model implemented online. The overall fit of the model has been good in multiple independent case series, but PREDICT has been shown to underestimate breast cancer specific mortality in women diagnosed under the age of 40. Another limitation is the use of discrete categories for tumour size and node status resulting in 'step' changes in risk estimates on moving between categories. We have refitted the PREDICT prognostic model using the original cohort of cases from East Anglia with updated survival time in order to take into account age at diagnosis and to smooth out the survival function for tumour size and node status. Multivariable Cox regression models were used to fit separate models for ER negative and ER positive disease. Continuous variables were fitted using fractional polynomials and a smoothed baseline hazard was obtained by regressing the baseline cumulative hazard for each patients against time using fractional polynomials. The fit of the prognostic models were then tested in three independent data sets that had also been used to validate the original version of PREDICT. In the model fitting data, after adjusting for other prognostic variables, there is an increase in risk of breast cancer specific mortality in younger and older patients with ER positive disease, with a substantial increase in risk for women diagnosed before the age of 35. In ER negative disease the risk increases slightly with age. The association between breast cancer specific mortality and both tumour size and number of positive nodes was non-linear with a more marked increase in risk with increasing size and increasing number of nodes in ER positive disease. The overall calibration and discrimination of the new version of PREDICT (v2) was good and comparable to that of the previous version in both model development and validation data sets. However, the calibration of v2 improved over v1 in patients diagnosed under the age

On the incremental validity of irrational beliefs to predict subjective well-being while controlling for personality factors.

PubMed

Spörrle, Matthias; Strobel, Maria; Tumasjan, Andranik

2010-11-01

This research examines the incremental validity of irrational thinking as conceptualized by Albert Ellis to predict diverse aspects of subjective well-being while controlling for the influence of personality factors. Rational-emotive behavior therapy (REBT) argues that irrational beliefs result in maladaptive emotions leading to reduced well-being. Although there is some early scientific evidence for this relation, it has never been investigated whether this connection would still persist when statistically controlling for the Big Five personality factors, which were consistently found to be important determinants of well-being. Regression analyses revealed significant incremental validity of irrationality over personality factors when predicting life satisfaction, but not when predicting subjective happiness. Results are discussed with respect to conceptual differences between these two aspects of subjective well-being.

The reliability, validity, sensitivity, specificity and predictive values of the Chinese version of the Rowland Universal Dementia Assessment Scale.

PubMed

Chen, Chia-Wei; Chu, Hsin; Tsai, Chia-Fen; Yang, Hui-Ling; Tsai, Jui-Chen; Chung, Min-Huey; Liao, Yuan-Mei; Chi, Mei-Ju; Chou, Kuei-Ru

2015-11-01

The purpose of this study was to translate the Rowland Universal Dementia Assessment Scale into Chinese and to evaluate the psychometric properties (reliability and validity) and the diagnostic properties (sensitivity, specificity and predictive values) of the Chinese version of the Rowland Universal Dementia Assessment Scale. The accurate detection of early dementia requires screening tools with favourable cross-cultural linguistic and appropriate sensitivity, specificity, and predictive values, particularly for Chinese-speaking populations. This was a cross-sectional, descriptive study. Overall, 130 participants suspected to have cognitive impairment were enrolled in the study. A test-retest for determining reliability was scheduled four weeks after the initial test. Content validity was determined by five experts, whereas construct validity was established by using contrasted group technique. The participants' clinical diagnoses were used as the standard in calculating the sensitivity, specificity, positive predictive value and negative predictive value. The study revealed that the Chinese version of the Rowland Universal Dementia Assessment Scale exhibited a test-retest reliability of 0.90, an internal consistency reliability of 0.71, an inter-rater reliability (kappa value) of 0.88 and a content validity index of 0.97. Both the patients and healthy contrast group exhibited significant differences in their cognitive ability. The optimal cut-off points for the Chinese version of the Rowland Universal Dementia Assessment Scale in the test for mild cognitive impairment and dementia were 24 and 22, respectively; moreover, for these two conditions, the sensitivities of the scale were 0.79 and 0.76, the specificities were 0.91 and 0.81, the areas under the curve were 0.85 and 0.78, the positive predictive values were 0.99 and 0.83 and the negative predictive values were 0.96 and 0.91 respectively. The Chinese version of the Rowland Universal Dementia Assessment Scale

External validation of a prediction model for surgical site infection after thoracolumbar spine surgery in a Western European cohort.

PubMed

Janssen, Daniël M C; van Kuijk, Sander M J; d'Aumerie, Boudewijn B; Willems, Paul C

2018-05-16

A prediction model for surgical site infection (SSI) after spine surgery was developed in 2014 by Lee et al. This model was developed to compute an individual estimate of the probability of SSI after spine surgery based on the patient's comorbidity profile and invasiveness of surgery. Before any prediction model can be validly implemented in daily medical practice, it should be externally validated to assess how the prediction model performs in patients sampled independently from the derivation cohort. We included 898 consecutive patients who underwent instrumented thoracolumbar spine surgery. To quantify overall performance using Nagelkerke's R 2 statistic, the discriminative ability was quantified as the area under the receiver operating characteristic curve (AUC). We computed the calibration slope of the calibration plot, to judge prediction accuracy. Sixty patients developed an SSI. The overall performance of the prediction model in our population was poor: Nagelkerke's R 2 was 0.01. The AUC was 0.61 (95% confidence interval (CI) 0.54-0.68). The estimated slope of the calibration plot was 0.52. The previously published prediction model showed poor performance in our academic external validation cohort. To predict SSI after instrumented thoracolumbar spine surgery for the present population, a better fitting prediction model should be developed.

Dynamic Assessment of Reading Difficulties: Predictive and Incremental Validity on Attitude toward Reading and the Use of Dialogue/Participation Strategies in Classroom Activities.

PubMed

Navarro, Juan-José; Lara, Laura

2017-01-01

Dynamic Assessment (DA) has been shown to have more predictive value than conventional tests for academic performance. However, in relation to reading difficulties, further research is needed to determine the predictive validity of DA for specific aspects of the different processes involved in reading and the differential validity of DA for different subgroups of students with an academic disadvantage. This paper analyzes the implementation of a DA device that evaluates processes involved in reading (EDPL) among 60 students with reading comprehension difficulties between 9 and 16 years of age, of whom 20 have intellectual disabilities, 24 have reading-related learning disabilities, and 16 have socio-cultural disadvantages. We specifically analyze the predictive validity of the EDPL device over attitude toward reading, and the use of dialogue/participation strategies in reading activities in the classroom during the implementation stage. We also analyze if the EDPL device provides additional information to that obtained with a conventionally applied personal-social adjustment scale (APSL). Results showed that dynamic scores, obtained from the implementation of the EDPL device, significantly predict the studied variables. Moreover, dynamic scores showed a significant incremental validity in relation to predictions based on an APSL scale. In relation to differential validity, the results indicated the superior predictive validity for DA for students with intellectual disabilities and reading disabilities than for students with socio-cultural disadvantages. Furthermore, the role of metacognition and its relation to the processes of personal-social adjustment in explaining the results is discussed.

Dynamic Assessment of Reading Difficulties: Predictive and Incremental Validity on Attitude toward Reading and the Use of Dialogue/Participation Strategies in Classroom Activities

PubMed Central

Navarro, Juan-José; Lara, Laura

2017-01-01

Dynamic Assessment (DA) has been shown to have more predictive value than conventional tests for academic performance. However, in relation to reading difficulties, further research is needed to determine the predictive validity of DA for specific aspects of the different processes involved in reading and the differential validity of DA for different subgroups of students with an academic disadvantage. This paper analyzes the implementation of a DA device that evaluates processes involved in reading (EDPL) among 60 students with reading comprehension difficulties between 9 and 16 years of age, of whom 20 have intellectual disabilities, 24 have reading-related learning disabilities, and 16 have socio-cultural disadvantages. We specifically analyze the predictive validity of the EDPL device over attitude toward reading, and the use of dialogue/participation strategies in reading activities in the classroom during the implementation stage. We also analyze if the EDPL device provides additional information to that obtained with a conventionally applied personal-social adjustment scale (APSL). Results showed that dynamic scores, obtained from the implementation of the EDPL device, significantly predict the studied variables. Moreover, dynamic scores showed a significant incremental validity in relation to predictions based on an APSL scale. In relation to differential validity, the results indicated the superior predictive validity for DA for students with intellectual disabilities and reading disabilities than for students with socio-cultural disadvantages. Furthermore, the role of metacognition and its relation to the processes of personal-social adjustment in explaining the results is discussed. PMID:28243215

Explicating Validity

ERIC Educational Resources Information Center

Kane, Michael T.

2016-01-01

How we choose to use a term depends on what we want to do with it. If "validity" is to be used to support a score interpretation, validation would require an analysis of the plausibility of that interpretation. If validity is to be used to support score uses, validation would require an analysis of the appropriateness of the proposed…

Validating proposed migration equation and parameters' values as a tool to reproduce and predict 137Cs vertical migration activity in Spanish soils.

PubMed

Olondo, C; Legarda, F; Herranz, M; Idoeta, R

2017-04-01

This paper shows the procedure performed to validate the migration equation and the migration parameters' values presented in a previous paper (Legarda et al., 2011) regarding the migration of 137 Cs in Spanish mainland soils. In this paper, this model validation has been carried out checking experimentally obtained activity concentration values against those predicted by the model. This experimental data come from the measured vertical activity profiles of 8 new sampling points which are located in northern Spain. Before testing predicted values of the model, the uncertainty of those values has been assessed with the appropriate uncertainty analysis. Once establishing the uncertainty of the model, both activity concentration values, experimental versus model predicted ones, have been compared. Model validation has been performed analyzing its accuracy, studying it as a whole and also at different depth intervals. As a result, this model has been validated as a tool to predict 137 Cs behaviour in a Mediterranean environment. Copyright © 2017 Elsevier Ltd. All rights reserved.

Prediction and validation of residual feed intake and dry matter intake in Danish lactating dairy cows using mid-infrared spectroscopy of milk.

PubMed

Shetty, N; Løvendahl, P; Lund, M S; Buitenhuis, A J

2017-01-01

The present study explored the effectiveness of Fourier transform mid-infrared (FT-IR) spectral profiles as a predictor for dry matter intake (DMI) and residual feed intake (RFI). The partial least squares regression method was used to develop the prediction models. The models were validated using different external test sets, one randomly leaving out 20% of the records (validation A), the second randomly leaving out 20% of cows (validation B), and a third (for DMI prediction models) randomly leaving out one cow (validation C). The data included 1,044 records from 140 cows; 97 were Danish Holstein and 43 Danish Jersey. Results showed better accuracies for validation A compared with other validation methods. Milk yield (MY) contributed largely to DMI prediction; MY explained 59% of the variation and the validated model error root mean square error of prediction (RMSEP) was 2.24kg. The model was improved by adding live weight (LW) as an additional predictor trait, where the accuracy R 2 increased from 0.59 to 0.72 and error RMSEP decreased from 2.24 to 1.83kg. When only the milk FT-IR spectral profile was used in DMI prediction, a lower prediction ability was obtained, with R 2 =0.30 and RMSEP=2.91kg. However, once the spectral information was added, along with MY and LW as predictors, model accuracy improved and R 2 increased to 0.81 and RMSEP decreased to 1.49kg. Prediction accuracies of RFI changed throughout lactation. The RFI prediction model for the early-lactation stage was better compared with across lactation or mid- and late-lactation stages, with R 2 =0.46 and RMSEP=1.70. The most important spectral wavenumbers that contributed to DMI and RFI prediction models included fat, protein, and lactose peaks. Comparable prediction results were obtained when using infrared-predicted fat, protein, and lactose instead of full spectra, indicating that FT-IR spectral data do not add significant new information to improve DMI and RFI prediction models. Therefore, in

Predictive Validity of DSM-IV and ICD-10 Criteria for ADHD and Hyperkinetic Disorder

ERIC Educational Resources Information Center

Lee, Soyoung I.; Schachar, Russell J.; Chen, Shirley X.; Ornstein, Tisha J.; Charach, Alice; Barr, Cathy; Ickowicz, Abel

2008-01-01

Background: The goal of this study was to compare the predictive validity of the two main diagnostic schemata for childhood hyperactivity--attention-deficit hyperactivity disorder (ADHD; "Diagnostic and Statistical Manual"-IV) and hyperkinetic disorder (HKD; "International Classification of Diseases"-10th Edition). Methods: Diagnostic criteria for…

Predicting the success of IVF: external validation of the van Loendersloot's model.

PubMed

Sarais, Veronica; Reschini, Marco; Busnelli, Andrea; Biancardi, Rossella; Paffoni, Alessio; Somigliana, Edgardo

2016-06-01

Is the predictive model for IVF success proposed by van Loendersloot et al. valid in a different geographical and cultural context? The model discriminates well but was less accurate than in the original context where it was developed. Several independent groups have developed models that combine different variables with the aim of estimating the chance of pregnancy with IVF but only four of them have been externally validated. One of these four, the van Loendersloot's model, deserves particular attention and further investigation for at least three reasons; (i) the reported area under the receiver operating characteristics curve (c-statistics) in the temporal validation setting was the highest reported to date (0.68), (ii) the perspective of the model is clinically wise since it includes variables obtained from previous failed cycles, if any, so it can be applied to any women entering an IVF cycle, (iii) the model lacks external validation in a geographically different center. Retrospective cohort study of women undergoing oocyte retrieval for IVF between January 2013 and December 2013 at the infertility unit of the Fondazione Ca' Granda, Ospedale Maggiore Policlinico of Milan, Italy. Only the first oocyte retrieval cycle performed during the study period was included in the study. Women with previous IVF cycles were excluded if the last one before the study cycle was in another center. The main outcome was the cumulative live birth rate per oocytes retrieval. Seven hundred seventy-two women were selected. Variables included in the van Loendersloot's model and the relative weights (beta) were used. The variable resulting from this combination (Y) was transformed into a probability. The discriminatory capacity was assessed using the c-statistics. Calibration was made using a logistic regression that included Y as the unique variable and live birth as the outcome. Data are presented using both the original and the calibrated models. Performance was evaluated

Screening for frailty in community-dwelling elderly subjects: Predictive validity of the modified SEGA instrument.

PubMed

Oubaya, N; Dramé, M; Novella, J-L; Quignard, E; Cunin, C; Jolly, D; Mahmoudi, R

2017-11-01

To study the capacity of the SEGAm instrument to predict loss of independence among elderly community-dwelling subjects. The study was performed in four French departments (Ardennes, Marne, Meurthe-et-Moselle, Meuse). Subjects aged 65 years or more, living at home, who could read and understand French, with a degree of autonomy corresponding to groups 5 or 6 in the AGGIR autonomy evaluation scale were included. Assessment included demographic characteristics, comprehensive geriatric assessment, and the SEGAm instrument at baseline. Subjects had follow-up visits at home at 6 and 12 months. During follow-up, vital status and level of independence were recorded. Logistic regression was used to study predictive validity of the SEGAm instrument. Among the 116 subjects with complete follow-up, 84 (72.4%) were classed as not very frail at baseline, 23 (19.8%) as frail, and 9 (7.8%) as very frail; 63 (54.3%) suffered loss of at least one ADL or IADL at 12 months. By multivariable analysis, frailty status at baseline was significantly associated with loss of independence during the 12 months of follow-up (OR=4.52, 95% CI=1.40-14.68; p=0.01). We previously validated the SEGAm instrument in terms of feasibility, acceptability, internal structure validity, reliability, and discriminant validity. This instrument appears to be a suitable tool for screening frailty among community-dwelling elderly subjects, and could be used as a basis to plan early targeted interventions for subjects at risk of adverse outcome. Copyright © 2017 Elsevier B.V. All rights reserved.

Validation metrics for turbulent plasma transport

DOE PAGES

Holland, C.

2016-06-22

Developing accurate models of plasma dynamics is essential for confident predictive modeling of current and future fusion devices. In modern computer science and engineering, formal verification and validation processes are used to assess model accuracy and establish confidence in the predictive capabilities of a given model. This paper provides an overview of the key guiding principles and best practices for the development of validation metrics, illustrated using examples from investigations of turbulent transport in magnetically confined plasmas. Particular emphasis is given to the importance of uncertainty quantification and its inclusion within the metrics, and the need for utilizing synthetic diagnosticsmore » to enable quantitatively meaningful comparisons between simulation and experiment. As a starting point, the structure of commonly used global transport model metrics and their limitations is reviewed. An alternate approach is then presented, which focuses upon comparisons of predicted local fluxes, fluctuations, and equilibrium gradients against observation. Furthermore, the utility of metrics based upon these comparisons is demonstrated by applying them to gyrokinetic predictions of turbulent transport in a variety of discharges performed on the DIII-D tokamak, as part of a multi-year transport model validation activity.« less

Validation metrics for turbulent plasma transport

DOE Office of Scientific and Technical Information (OSTI.GOV)

Holland, C.

Developing accurate models of plasma dynamics is essential for confident predictive modeling of current and future fusion devices. In modern computer science and engineering, formal verification and validation processes are used to assess model accuracy and establish confidence in the predictive capabilities of a given model. This paper provides an overview of the key guiding principles and best practices for the development of validation metrics, illustrated using examples from investigations of turbulent transport in magnetically confined plasmas. Particular emphasis is given to the importance of uncertainty quantification and its inclusion within the metrics, and the need for utilizing synthetic diagnosticsmore » to enable quantitatively meaningful comparisons between simulation and experiment. As a starting point, the structure of commonly used global transport model metrics and their limitations is reviewed. An alternate approach is then presented, which focuses upon comparisons of predicted local fluxes, fluctuations, and equilibrium gradients against observation. Furthermore, the utility of metrics based upon these comparisons is demonstrated by applying them to gyrokinetic predictions of turbulent transport in a variety of discharges performed on the DIII-D tokamak, as part of a multi-year transport model validation activity.« less

Development and validation of the Neonatal Mortality Score-9 Mexico to predict mortality in critically ill neonates.

PubMed

Márquez-González, Horacio; Jiménez-Báez, María Valeria; Muñoz-Ramírez, C Mireya; Yáñez-Gutiérrez, Lucelli; Huelgas-Plaza, Ana C; Almeida-Gutiérrez, Eduardo; Villa-Romero, Antonio Rafael

2015-06-01

Prognostic scales or scores are useful for physicians who work in neonatal intensive care units. There are several validated neonatal scores but they are mostly applicable to low birth weight infants. The aim of this study was to develop and validate a mortality prognostic score in newborn infants, that would include new prognostic outcome measures. The study was conducted in a mother and child hospital in the city of Mexico, part of the Instituto Mexicano del Seguro Social (Mexican Institute of Social Security). In the first phase of the study, a nested case-control study was designed (newborn infants admitted on the basis of severity criteria during the first day of life), in which a scale was identified and developed with gradual parameters of cumulative score consisting of nine independent outcome measures to predict death, as follows: weight, metabolic acidemia, lactate, PaO2/FiO2, p(A-a) O2, A/a, platelets and serum glucose.Validation was performed in a matched prospective cohort, using 7-day mortality as an endpoint. The initial cohort consisted of 424 newborn infants. Twenty-two cases and 132 controls were selected; and 9 outcome measures were identified, making up the scale named neonatal mortality score-9 Mexico. The validation cohort consisted of 227 newborn infants. Forty-four (19%) deaths were recorded, with an area under the curve (AUC) of 0.92. With a score between 16 and 18, an 85 (11-102) hazard ratio, 99% specificity, 71% positive predictive value and 90% negative predictive value were reported. Conclusions .The proposed scale is a reliable tool to predict severity in newborn infants.

Validity of the Inbody 520™ to predict metabolic rate in apparently healthy adults.

PubMed

Salacinski, Amanda J; Howell, Steven M; Hill, Danielle L

2017-05-30

The present study seeks to assess the validity of the InBody 520™ device to predict RMR in apparently healthy adults relative to a metabolic cart (the standard, yet time intensive, method for determining resting metabolic rate). Twenty-six apparently healthy adults participated in the study. Predicted RMR (pRMR) was calculated by the InBody 520™ and measured RMR (mRMR) was determined by 30-minute gas analysis and ventilated hood system. Of the 78 measurement trials conducted, 64 yielded acceptable measurement trials. A Pearson product-moment correlation was used to determine the relationship between pRMR and mRMR (r = .87, P < .001). No significant difference existed between the pRMR (1650.89 ± 295.96 kcal) and mRMR (1675.36 ± 278.69 kcal) values (P =.19). Study findings suggest that the InBody520™ provides valid measurements of RMR in apparently healthy adults and can be an effective and efficient method for collecting data in a clinical setting.

Predicting surgical site infection after spine surgery: a validated model using a prospective surgical registry.

PubMed

Lee, Michael J; Cizik, Amy M; Hamilton, Deven; Chapman, Jens R

2014-09-01

The impact of surgical site infection (SSI) is substantial. Although previous study has determined relative risk and odds ratio (OR) values to quantify risk factors, these values may be difficult to translate to the patient during counseling of surgical options. Ideally, a model that predicts absolute risk of SSI, rather than relative risk or OR values, would greatly enhance the discussion of safety of spine surgery. To date, there is no risk stratification model that specifically predicts the risk of medical complication. The purpose of this study was to create and validate a predictive model for the risk of SSI after spine surgery. This study performs a multivariate analysis of SSI after spine surgery using a large prospective surgical registry. Using the results of this analysis, this study will then create and validate a predictive model for SSI after spine surgery. The patient sample is from a high-quality surgical registry from our two institutions with prospectively collected, detailed demographic, comorbidity, and complication data. An SSI that required return to the operating room for surgical debridement. Using a prospectively collected surgical registry of more than 1,532 patients with extensive demographic, comorbidity, surgical, and complication details recorded for 2 years after the surgery, we identified several risk factors for SSI after multivariate analysis. Using the beta coefficients from those regression analyses, we created a model to predict the occurrence of SSI after spine surgery. We split our data into two subsets for internal and cross-validation of our model. We created a predictive model based on our beta coefficients from our multivariate analysis. The final predictive model for SSI had a receiver-operator curve characteristic of 0.72, considered to be a fair measure. The final model has been uploaded for use on SpineSage.com. We present a validated model for predicting SSI after spine surgery. The value in this model is that it gives

An Approach for Validating Actinide and Fission Product Burnup Credit Criticality Safety Analyses-Isotopic Composition Predictions

DOE Office of Scientific and Technical Information (OSTI.GOV)

Radulescu, Georgeta; Gauld, Ian C; Ilas, Germina

2011-01-01

The expanded use of burnup credit in the United States (U.S.) for storage and transport casks, particularly in the acceptance of credit for fission products, has been constrained by the availability of experimental fission product data to support code validation. The U.S. Nuclear Regulatory Commission (NRC) staff has noted that the rationale for restricting the Interim Staff Guidance on burnup credit for storage and transportation casks (ISG-8) to actinide-only is based largely on the lack of clear, definitive experiments that can be used to estimate the bias and uncertainty for computational analyses associated with using burnup credit. To address themore » issues of burnup credit criticality validation, the NRC initiated a project with the Oak Ridge National Laboratory to (1) develop and establish a technically sound validation approach for commercial spent nuclear fuel (SNF) criticality safety evaluations based on best-available data and methods and (2) apply the approach for representative SNF storage and transport configurations/conditions to demonstrate its usage and applicability, as well as to provide reference bias results. The purpose of this paper is to describe the isotopic composition (depletion) validation approach and resulting observations and recommendations. Validation of the criticality calculations is addressed in a companion paper at this conference. For isotopic composition validation, the approach is to determine burnup-dependent bias and uncertainty in the effective neutron multiplication factor (keff) due to bias and uncertainty in isotopic predictions, via comparisons of isotopic composition predictions (calculated) and measured isotopic compositions from destructive radiochemical assay utilizing as much assay data as is available, and a best-estimate Monte Carlo based method. This paper (1) provides a detailed description of the burnup credit isotopic validation approach and its technical bases, (2) describes the application of the

Prospective validation of pathologic complete response models in rectal cancer: Transferability and reproducibility.

PubMed

van Soest, Johan; Meldolesi, Elisa; van Stiphout, Ruud; Gatta, Roberto; Damiani, Andrea; Valentini, Vincenzo; Lambin, Philippe; Dekker, Andre

2017-09-01

Multiple models have been developed to predict pathologic complete response (pCR) in locally advanced rectal cancer patients. Unfortunately, validation of these models normally omit the implications of cohort differences on prediction model performance. In this work, we will perform a prospective validation of three pCR models, including information whether this validation will target transferability or reproducibility (cohort differences) of the given models. We applied a novel methodology, the cohort differences model, to predict whether a patient belongs to the training or to the validation cohort. If the cohort differences model performs well, it would suggest a large difference in cohort characteristics meaning we would validate the transferability of the model rather than reproducibility. We tested our method in a prospective validation of three existing models for pCR prediction in 154 patients. Our results showed a large difference between training and validation cohort for one of the three tested models [Area under the Receiver Operating Curve (AUC) cohort differences model: 0.85], signaling the validation leans towards transferability. Two out of three models had a lower AUC for validation (0.66 and 0.58), one model showed a higher AUC in the validation cohort (0.70). We have successfully applied a new methodology in the validation of three prediction models, which allows us to indicate if a validation targeted transferability (large differences between training/validation cohort) or reproducibility (small cohort differences). © 2017 American Association of Physicists in Medicine.

The Smoking Consequences Questionnaire: Factor structure and predictive validity among Spanish-speaking Latino smokers in the United States.

PubMed

Vidrine, Jennifer Irvin; Vidrine, Damon J; Costello, Tracy J; Mazas, Carlos; Cofta-Woerpel, Ludmila; Mejia, Luz Maria; Wetter, David W

2009-11-01

Much of the existing research on smoking outcome expectancies has been guided by the Smoking Consequences Questionnaire (SCQ ). Although the original version of the SCQ has been modified over time for use in different populations, none of the existing versions have been evaluated for use among Spanish-speaking Latino smokers in the United States. The present study evaluated the factor structure and predictive validity of the 3 previously validated versions of the SCQ--the original, the SCQ-Adult, and the SCQ-Spanish, which was developed with Spanish-speaking smokers in Spain--among Spanish-speaking Latino smokers in Texas. The SCQ-Spanish represented the least complex solution. Each of the SCQ-Spanish scales had good internal consistency, and the predictive validity of the SCQ-Spanish was partially supported. Nearly all the SCQ-Spanish scales predicted withdrawal severity even after controlling for demographics and dependence. Boredom Reduction predicted smoking relapse across the 5- and 12-week follow-up assessments in a multivariate model that also controlled for demographics and dependence. Our results support use of the SCQ-Spanish with Spanish-speaking Latino smokers in the United States.

Validity of one-repetition maximum predictive equations in men with spinal cord injury.

PubMed

Ribeiro Neto, F; Guanais, P; Dornelas, E; Coutinho, A C B; Costa, R R G

2017-10-01

Cross-sectional study. The study aimed (a) to test the cross-validation of current one-repetition maximum (1RM) predictive equations in men with spinal cord injury (SCI); (b) to compare the current 1RM predictive equations to a newly developed equation based on the 4- to 12-repetition maximum test (4-12RM). SARAH Rehabilitation Hospital Network, Brasilia, Brazil. Forty-five men aged 28.0 years with SCI between C6 and L2 causing complete motor impairment were enrolled in the study. Volunteers were tested, in a random order, in 1RM test or 4-12RM with 2-3 interval days. Multiple regression analysis was used to generate an equation for predicting 1RM. There were no significant differences between 1RM test and the current predictive equations. ICC values were significant and were classified as excellent for all current predictive equations. The predictive equation of Lombardi presented the best Bland-Altman results (0.5 kg and 12.8 kg for mean difference and interval range around the differences, respectively). The two created equation models for 1RM demonstrated the same and a high adjusted R 2 (0.971, P<0.01), but different SEE of measured 1RM (2.88 kg or 5.4% and 2.90 kg or 5.5%). All 1RM predictive equations are accurate to assess individuals with SCI at the bench press exercise. However, the predictive equation of Lombardi presented the best associated cross-validity results. A specific 1RM prediction equation was also elaborated for individuals with SCI. The created equation should be tested in order to verify whether it presents better accuracy than the current ones.

External validation of a nomogram for prediction of side-specific extracapsular extension at robotic radical prostatectomy.

PubMed

Zorn, Kevin C; Gallina, Andrea; Hutterer, Georg C; Walz, Jochen; Shalhav, Arieh L; Zagaja, Gregory P; Valiquette, Luc; Gofrit, Ofer N; Orvieto, Marcelo A; Taxy, Jerome B; Karakiewicz, Pierre I

2007-11-01

Several staging tools have been developed for open radical prostatectomy (ORP) patients. However, the validity of these tools has never been formally tested in patients treated with robot-assisted laparoscopic radical prostatectomy (RALP). We tested the accuracy of an ORP-derived nomogram in predicting the rate of extracapsular extension (ECE) in a large RALP cohort. Serum prostate specific antigen (PSA) and side-specific clinical stage and biopsy Gleason sum information were used in a previously validated nomogram predicting side-specific ECE. The nomogram-derived predictions were compared with the observed rate of ECE, and the accuracy of the predictions was quantified. Each prostate lobe was analyzed independently. As complete data were available for 576 patients, the analyses targeted 1152 prostate lobes. Median age and serum PSA concentration at radical prostatectomy were 60 years and 5.4 ng/mL, respectively. The majority of side-specific clinical stages were T(1c) (993; 86.2%). Most side-specific biopsy Gleason sums were 6 (572; 49.7%). The median side-specific percentages of positive cores and of cancer were, respectively, 20.0% and 5.0%. At final pathologic review, 107 patients (18.6%) had ECE, and side-specific ECE was present in 117 patients (20.3%). The nomogram was 89% accurate in the RALP cohort v 84% in the previously reported ORP validation. The ORP side-specific ECE nomogram is highly accurate in the RALP population, suggesting that predictive and possibly prognostic tools developed in ORP patients may be equally accurate in their RALP counterparts.

Validity and Measurement

ERIC Educational Resources Information Center

Maraun, Michael D.

2012-01-01

As illuminated forcefully by Professor Newton's provocative analytical and historical excursion, as long as tests are employed to practical ends (prediction, selection, etc.) there is little cause for the metatheoretic angst that occasions rounds of papers on the topic of validity. But then, also, there seems little need, within this context of…

Prediction of Outcome after Moderate and Severe Traumatic Brain Injury: External Validation of the IMPACT and CRASH Prognostic Models

PubMed Central

Roozenbeek, Bob; Lingsma, Hester F.; Lecky, Fiona E.; Lu, Juan; Weir, James; Butcher, Isabella; McHugh, Gillian S.; Murray, Gordon D.; Perel, Pablo; Maas, Andrew I.R.; Steyerberg, Ewout W.

2012-01-01

Objective The International Mission on Prognosis and Analysis of Clinical Trials (IMPACT) and Corticoid Randomisation After Significant Head injury (CRASH) prognostic models predict outcome after traumatic brain injury (TBI) but have not been compared in large datasets. The objective of this is study is to validate externally and compare the IMPACT and CRASH prognostic models for prediction of outcome after moderate or severe TBI. Design External validation study. Patients We considered 5 new datasets with a total of 9036 patients, comprising three randomized trials and two observational series, containing prospectively collected individual TBI patient data. Measurements Outcomes were mortality and unfavourable outcome, based on the Glasgow Outcome Score (GOS) at six months after injury. To assess performance, we studied the discrimination of the models (by AUCs), and calibration (by comparison of the mean observed to predicted outcomes and calibration slopes). Main Results The highest discrimination was found in the TARN trauma registry (AUCs between 0.83 and 0.87), and the lowest discrimination in the Pharmos trial (AUCs between 0.65 and 0.71). Although differences in predictor effects between development and validation populations were found (calibration slopes varying between 0.58 and 1.53), the differences in discrimination were largely explained by differences in case-mix in the validation studies. Calibration was good, the fraction of observed outcomes generally agreed well with the mean predicted outcome. No meaningful differences were noted in performance between the IMPACT and CRASH models. More complex models discriminated slightly better than simpler variants. Conclusions Since both the IMPACT and the CRASH prognostic models show good generalizability to more recent data, they are valid instruments to quantify prognosis in TBI. PMID:22511138

A Quantitative Structure Activity Relationship for acute oral toxicity of pesticides on rats: Validation, domain of application and prediction.

PubMed

Hamadache, Mabrouk; Benkortbi, Othmane; Hanini, Salah; Amrane, Abdeltif; Khaouane, Latifa; Si Moussa, Cherif

2016-02-13

Quantitative Structure Activity Relationship (QSAR) models are expected to play an important role in the risk assessment of chemicals on humans and the environment. In this study, we developed a validated QSAR model to predict acute oral toxicity of 329 pesticides to rats because a few QSAR models have been devoted to predict the Lethal Dose 50 (LD50) of pesticides on rats. This QSAR model is based on 17 molecular descriptors, and is robust, externally predictive and characterized by a good applicability domain. The best results were obtained with a 17/9/1 Artificial Neural Network model trained with the Quasi Newton back propagation (BFGS) algorithm. The prediction accuracy for the external validation set was estimated by the Q(2)ext and the root mean square error (RMS) which are equal to 0.948 and 0.201, respectively. 98.6% of external validation set is correctly predicted and the present model proved to be superior to models previously published. Accordingly, the model developed in this study provides excellent predictions and can be used to predict the acute oral toxicity of pesticides, particularly for those that have not been tested as well as new pesticides. Copyright © 2015 Elsevier B.V. All rights reserved.

[Reliability and validity of the Braden Scale for predicting pressure sore risk].

PubMed

Boes, C

2000-12-01

For more accurate and objective pressure sore risk assessment various risk assessment tools were developed mainly in the USA and Great Britain. The Braden Scale for Predicting Pressure Sore Risk is one such example. By means of a literature analysis of German and English texts referring to the Braden Scale the scientific control criteria reliability and validity will be traced and consequences for application of the scale in Germany will be demonstrated. Analysis of 4 reliability studies shows an exclusive focus on interrater reliability. Further, even though examination of 19 validity studies occurs in many different settings, such examination is limited to the criteria sensitivity and specificity (accuracy). The range of sensitivity and specificity level is 35-100%. The recommended cut off points rank in the field of 10 to 19 points. The studies prove to be not comparable with each other. Furthermore, distortions in these studies can be found which affect accuracy of the scale. The results of the here presented analysis show an insufficient proof for reliability and validity in the American studies. In Germany, the Braden scale has not yet been tested under scientific criteria. Such testing is needed before using the scale in different German settings. During the course of such testing, construction and study procedures of the American studies can be used as a basis as can the problems be identified in the analysis presented below.

Validation of the Retinal Detachment after Open Globe Injury (RD-OGI) Score as an Effective Tool for Predicting Retinal Detachment.

PubMed

Brodowska, Katarzyna; Stryjewski, Tomasz P; Papavasileiou, Evangelia; Chee, Yewlin E; Eliott, Dean

2017-05-01

The Retinal Detachment after Open Globe Injury (RD-OGI) Score is a clinical prediction model that was developed at the Massachusetts Eye and Ear Infirmary to predict the risk of retinal detachment (RD) after open globe injury (OGI). This study sought to validate the RD-OGI Score in an independent cohort of patients. Retrospective cohort study. The predictive value of the RD-OGI Score was evaluated by comparing the original RD-OGI Scores of 893 eyes with OGI that presented between 1999 and 2011 (the derivation cohort) with 184 eyes with OGI that presented from January 1, 2012, to January 31, 2014 (the validation cohort). Three risk classes (low, moderate, and high) were created and logistic regression was undertaken to evaluate the optimal predictive value of the RD-OGI Score. A Kaplan-Meier survival analysis evaluated survival experience between the risk classes. Time to RD. At 1 year after OGI, 255 eyes (29%) in the derivation cohort and 66 eyes (36%) in the validation cohort were diagnosed with an RD. At 1 year, the low risk class (RD-OGI Scores 0-2) had a 3% detachment rate in the derivation cohort and a 0% detachment rate in the validation cohort, the moderate risk class (RD-OGI Scores 2.5-4.5) had a 29% detachment rate in the derivation cohort and a 35% detachment rate in the validation cohort, and the high risk class (RD-OGI scores 5-7.5) had a 73% detachment rate in the derivation cohort and an 86% detachment rate in the validation cohort. Regression modeling revealed the RD-OGI to be highly discriminative, especially 30 days after injury, with an area under the receiver operating characteristic curve of 0.939 in the validation cohort. Survival experience was significantly different depending upon the risk class (P < 0.0001, log-rank chi-square). The RD-OGI Score can reliably predict the future risk of developing an RD based on clinical variables that are present at the time of the initial evaluation after OGI. Copyright © 2017 American Academy of

Development and external validation of a risk-prediction model to predict 5-year overall survival in advanced larynx cancer.

PubMed

Petersen, Japke F; Stuiver, Martijn M; Timmermans, Adriana J; Chen, Amy; Zhang, Hongzhen; O'Neill, James P; Deady, Sandra; Vander Poorten, Vincent; Meulemans, Jeroen; Wennerberg, Johan; Skroder, Carl; Day, Andrew T; Koch, Wayne; van den Brekel, Michiel W M

2018-05-01

TNM-classification inadequately estimates patient-specific overall survival (OS). We aimed to improve this by developing a risk-prediction model for patients with advanced larynx cancer. Cohort study. We developed a risk prediction model to estimate the 5-year OS rate based on a cohort of 3,442 patients with T3T4N0N+M0 larynx cancer. The model was internally validated using bootstrapping samples and externally validated on patient data from five external centers (n = 770). The main outcome was performance of the model as tested by discrimination, calibration, and the ability to distinguish risk groups based on tertiles from the derivation dataset. The model performance was compared to a model based on T and N classification only. We included age, gender, T and N classification, and subsite as prognostic variables in the standard model. After external validation, the standard model had a significantly better fit than a model based on T and N classification alone (C statistic, 0.59 vs. 0.55, P < .001). The model was able to distinguish well among three risk groups based on tertiles of the risk score. Adding treatment modality to the model did not decrease the predictive power. As a post hoc analysis, we tested the added value of comorbidity as scored by American Society of Anesthesiologists score in a subsample, which increased the C statistic to 0.68. A risk prediction model for patients with advanced larynx cancer, consisting of readily available clinical variables, gives more accurate estimations of the estimated 5-year survival rate when compared to a model based on T and N classification alone. 2c. Laryngoscope, 128:1140-1145, 2018. © 2017 The American Laryngological, Rhinological and Otological Society, Inc.

Development and Validation of a Practical Two-Step Prediction Model and Clinical Risk Score for Post-Thrombotic Syndrome.

PubMed

Amin, Elham E; van Kuijk, Sander M J; Joore, Manuela A; Prandoni, Paolo; Cate, Hugo Ten; Cate-Hoek, Arina J Ten

2018-06-04

 Post-thrombotic syndrome (PTS) is a common chronic consequence of deep vein thrombosis that affects the quality of life and is associated with substantial costs. In clinical practice, it is not possible to predict the individual patient risk. We develop and validate a practical two-step prediction tool for PTS in the acute and sub-acute phase of deep vein thrombosis.  Multivariable regression modelling with data from two prospective cohorts in which 479 (derivation) and 1,107 (validation) consecutive patients with objectively confirmed deep vein thrombosis of the leg, from thrombosis outpatient clinic of Maastricht University Medical Centre, the Netherlands (derivation) and Padua University hospital in Italy (validation), were included. PTS was defined as a Villalta score of ≥ 5 at least 6 months after acute thrombosis.  Variables in the baseline model in the acute phase were: age, body mass index, sex, varicose veins, history of venous thrombosis, smoking status, provoked thrombosis and thrombus location. For the secondary model, the additional variable was residual vein obstruction. Optimism-corrected area under the receiver operating characteristic curves (AUCs) were 0.71 for the baseline model and 0.60 for the secondary model. Calibration plots showed well-calibrated predictions. External validation of the derived clinical risk scores was successful: AUC, 0.66 (95% confidence interval [CI], 0.63-0.70) and 0.64 (95% CI, 0.60-0.69).  Individual risk for PTS in the acute phase of deep vein thrombosis can be predicted based on readily accessible baseline clinical and demographic characteristics. The individual risk in the sub-acute phase can be predicted with limited additional clinical characteristics. Schattauer GmbH Stuttgart.

Validity of Measured Interest for Decided and Undecided Students.

ERIC Educational Resources Information Center

Bartling, Herbert C.; Hood, Albert B.

The usefulness of vocational interest measures has been questioned by those who have studied the predictive validity of expressed choice. The predictive validities of measured interest for decided and undecided students, expressed choice and measured interest, and expressed choice and measured interest when they are congruent and incongruent were…

A prediction algorithm for first onset of major depression in the general population: development and validation.

PubMed

Wang, JianLi; Sareen, Jitender; Patten, Scott; Bolton, James; Schmitz, Norbert; Birney, Arden

2014-05-01

Prediction algorithms are useful for making clinical decisions and for population health planning. However, such prediction algorithms for first onset of major depression do not exist. The objective of this study was to develop and validate a prediction algorithm for first onset of major depression in the general population. Longitudinal study design with approximate 3-year follow-up. The study was based on data from a nationally representative sample of the US general population. A total of 28 059 individuals who participated in Waves 1 and 2 of the US National Epidemiologic Survey on Alcohol and Related Conditions and who had not had major depression at Wave 1 were included. The prediction algorithm was developed using logistic regression modelling in 21 813 participants from three census regions. The algorithm was validated in participants from the 4th census region (n=6246). Major depression occurred since Wave 1 of the National Epidemiologic Survey on Alcohol and Related Conditions, assessed by the Alcohol Use Disorder and Associated Disabilities Interview Schedule-diagnostic and statistical manual for mental disorders IV. A prediction algorithm containing 17 unique risk factors was developed. The algorithm had good discriminative power (C statistics=0.7538, 95% CI 0.7378 to 0.7699) and excellent calibration (F-adjusted test=1.00, p=0.448) with the weighted data. In the validation sample, the algorithm had a C statistic of 0.7259 and excellent calibration (Hosmer-Lemeshow χ(2)=3.41, p=0.906). The developed prediction algorithm has good discrimination and calibration capacity. It can be used by clinicians, mental health policy-makers and service planners and the general public to predict future risk of having major depression. The application of the algorithm may lead to increased personalisation of treatment, better clinical decisions and more optimal mental health service planning.

Derivation and External Validation of Prediction Models for Advanced Chronic Kidney Disease Following Acute Kidney Injury

PubMed Central

Pannu, Neesh; Hemmelgarn, Brenda R.; Austin, Peter C.; Tan, Zhi; McArthur, Eric; Manns, Braden J.; Tonelli, Marcello; Wald, Ron; Quinn, Robert R.; Ravani, Pietro; Garg, Amit X.

2017-01-01

Importance Some patients will develop chronic kidney disease after a hospitalization with acute kidney injury; however, no risk-prediction tools have been developed to identify high-risk patients requiring follow-up. Objective To derive and validate predictive models for progression of acute kidney injury to advanced chronic kidney disease. Design, Setting, and Participants Data from 2 population-based cohorts of patients with a prehospitalization estimated glomerular filtration rate (eGFR) of more than 45 mL/min/1.73 m2 and who had survived hospitalization with acute kidney injury (defined by a serum creatinine increase during hospitalization > 0.3 mg/dL or > 50% of their prehospitalization baseline), were used to derive and validate multivariable prediction models. The risk models were derived from 9973 patients hospitalized in Alberta, Canada (April 2004-March 2014, with follow-up to March 2015). The risk models were externally validated with data from a cohort of 2761 patients hospitalized in Ontario, Canada (June 2004-March 2012, with follow-up to March 2013). Exposures Demographic, laboratory, and comorbidity variables measured prior to discharge. Main Outcomes and Measures Advanced chronic kidney disease was defined by a sustained reduction in eGFR less than 30 mL/min/1.73 m2 for at least 3 months during the year after discharge. All participants were followed up for up to 1 year. Results The participants (mean [SD] age, 66 [15] years in the derivation and internal validation cohorts and 69 [11] years in the external validation cohort; 40%-43% women per cohort) had a mean (SD) baseline serum creatinine level of 1.0 (0.2) mg/dL and more than 20% had stage 2 or 3 acute kidney injury. Advanced chronic kidney disease developed in 408 (2.7%) of 9973 patients in the derivation cohort and 62 (2.2%) of 2761 patients in the external validation cohort. In the derivation cohort, 6 variables were independently associated with the outcome: older age, female sex, higher

A Unified Model of Performance: Validation of its Predictions across Different Sleep/Wake Schedules.

PubMed

Ramakrishnan, Sridhar; Wesensten, Nancy J; Balkin, Thomas J; Reifman, Jaques

2016-01-01

Historically, mathematical models of human neurobehavioral performance developed on data from one sleep study were limited to predicting performance in similar studies, restricting their practical utility. We recently developed a unified model of performance (UMP) to predict the effects of the continuum of sleep loss-from chronic sleep restriction (CSR) to total sleep deprivation (TSD) challenges-and validated it using data from two studies of one laboratory. Here, we significantly extended this effort by validating the UMP predictions across a wide range of sleep/wake schedules from different studies and laboratories. We developed the UMP on psychomotor vigilance task (PVT) lapse data from one study encompassing four different CSR conditions (7 d of 3, 5, 7, and 9 h of sleep/night), and predicted performance in five other studies (from four laboratories), including different combinations of TSD (40 to 88 h), CSR (2 to 6 h of sleep/night), control (8 to 10 h of sleep/night), and nap (nocturnal and diurnal) schedules. The UMP accurately predicted PVT performance trends across 14 different sleep/wake conditions, yielding average prediction errors between 7% and 36%, with the predictions lying within 2 standard errors of the measured data 87% of the time. In addition, the UMP accurately predicted performance impairment (average error of 15%) for schedules (TSD and naps) not used in model development. The unified model of performance can be used as a tool to help design sleep/wake schedules to optimize the extent and duration of neurobehavioral performance and to accelerate recovery after sleep loss. © 2016 Associated Professional Sleep Societies, LLC.

Development, Testing, and Validation of a Model-Based Tool to Predict Operator Responses in Unexpected Workload Transitions

NASA Technical Reports Server (NTRS)

Sebok, Angelia; Wickens, Christopher; Sargent, Robert

2015-01-01

One human factors challenge is predicting operator performance in novel situations. Approaches such as drawing on relevant previous experience, and developing computational models to predict operator performance in complex situations, offer potential methods to address this challenge. A few concerns with modeling operator performance are that models need to realistic, and they need to be tested empirically and validated. In addition, many existing human performance modeling tools are complex and require that an analyst gain significant experience to be able to develop models for meaningful data collection. This paper describes an effort to address these challenges by developing an easy to use model-based tool, using models that were developed from a review of existing human performance literature and targeted experimental studies, and performing an empirical validation of key model predictions.

Testing and validating environmental models

USGS Publications Warehouse

Kirchner, J.W.; Hooper, R.P.; Kendall, C.; Neal, C.; Leavesley, G.

1996-01-01

Generally accepted standards for testing and validating ecosystem models would benefit both modellers and model users. Universally applicable test procedures are difficult to prescribe, given the diversity of modelling approaches and the many uses for models. However, the generally accepted scientific principles of documentation and disclosure provide a useful framework for devising general standards for model evaluation. Adequately documenting model tests requires explicit performance criteria, and explicit benchmarks against which model performance is compared. A model's validity, reliability, and accuracy can be most meaningfully judged by explicit comparison against the available alternatives. In contrast, current practice is often characterized by vague, subjective claims that model predictions show 'acceptable' agreement with data; such claims provide little basis for choosing among alternative models. Strict model tests (those that invalid models are unlikely to pass) are the only ones capable of convincing rational skeptics that a model is probably valid. However, 'false positive' rates as low as 10% can substantially erode the power of validation tests, making them insufficiently strict to convince rational skeptics. Validation tests are often undermined by excessive parameter calibration and overuse of ad hoc model features. Tests are often also divorced from the conditions under which a model will be used, particularly when it is designed to forecast beyond the range of historical experience. In such situations, data from laboratory and field manipulation experiments can provide particularly effective tests, because one can create experimental conditions quite different from historical data, and because experimental data can provide a more precisely defined 'target' for the model to hit. We present a simple demonstration showing that the two most common methods for comparing model predictions to environmental time series (plotting model time series

Three DIBELS Tasks vs. Three Informal Reading/Spelling Tasks: A Comparison of Predictive Validity

ERIC Educational Resources Information Center

Morris, Darrell; Trathen, Woodrow; Perney, Jan; Gill, Tom; Schlagal, Robert; Ward, Devery; Frye, Elizabeth M.

2017-01-01

Within a developmental framework, this study compared the predictive validity of three DIBELS tasks (phoneme segmentation fluency [PSF], nonsense word fluency [NWF], and oral reading fluency [ORF]) with that of three alternative tasks drawn from the field of reading (phonemic spelling [phSPEL], word recognition-timed [WR-t], and graded passage…

Post-bronchoscopy pneumonia in patients suffering from lung cancer: Development and validation of a risk prediction score.

PubMed

Takiguchi, Hiroto; Hayama, Naoki; Oguma, Tsuyoshi; Harada, Kazuki; Sato, Masako; Horio, Yukihiro; Tanaka, Jun; Tomomatsu, Hiromi; Tomomatsu, Katsuyoshi; Takihara, Takahisa; Niimi, Kyoko; Nakagawa, Tomoki; Masuda, Ryota; Aoki, Takuya; Urano, Tetsuya; Iwazaki, Masayuki; Asano, Koichiro

2017-05-01

The incidence, risk factors, and consequences of pneumonia after flexible bronchoscopy in patients with lung cancer have not been studied in detail. We retrospectively analyzed the data from 237 patients with lung cancer who underwent diagnostic bronchoscopy between April 2012 and July 2013 (derivation sample) and 241 patients diagnosed between August 2013 and July 2014 (validation sample) in a tertiary referral hospital in Japan. A score predictive of post-bronchoscopy pneumonia was developed in the derivation sample and tested in the validation sample. Pneumonia developed after bronchoscopy in 6.3% and 4.1% of patients in the derivation and validation samples, respectively. Patients who developed post-bronchoscopy pneumonia needed to change or cancel their planned cancer therapy more frequently than those without pneumonia (56% vs. 6%, p<0.001). Age ≥70 years, current smoking, and central location of the tumor were independent predictors of pneumonia, which we added to develop our predictive score. The incidence of pneumonia associated with scores=0, 1, and ≥2 was 0, 3.7, and 13.4% respectively in the derivation sample (p=0.003), and 0, 2.9, and 9.7% respectively in the validation sample (p=0.016). The incidence of post-bronchoscopy pneumonia in patients with lung cancer was not rare and associated with adverse effects on the clinical course. A simple 3-point predictive score identified patients with lung cancer at high risk of post-bronchoscopy pneumonia prior to the procedure. Copyright © 2017 The Japanese Respiratory Society. Published by Elsevier B.V. All rights reserved.

Propeller aircraft interior noise model utilization study and validation

NASA Technical Reports Server (NTRS)

Pope, L. D.

1984-01-01

Utilization and validation of a computer program designed for aircraft interior noise prediction is considered. The program, entitled PAIN (an acronym for Propeller Aircraft Interior Noise), permits (in theory) predictions of sound levels inside propeller driven aircraft arising from sidewall transmission. The objective of the work reported was to determine the practicality of making predictions for various airplanes and the extent of the program's capabilities. The ultimate purpose was to discern the quality of predictions for tonal levels inside an aircraft occurring at the propeller blade passage frequency and its harmonics. The effort involved three tasks: (1) program validation through comparisons of predictions with scale-model test results; (2) development of utilization schemes for large (full scale) fuselages; and (3) validation through comparisons of predictions with measurements taken in flight tests on a turboprop aircraft. Findings should enable future users of the program to efficiently undertake and correctly interpret predictions.

Using Dynamic Risk and Protective Factors to Predict Inpatient Aggression: Reliability and Validity of START Assessments

PubMed Central

Desmarais, Sarah L.; Nicholls, Tonia L.; Wilson, Catherine M.; Brink, Johann

2012-01-01

The Short-Term Assessment of Risk and Treatability (START) is a relatively new structured professional judgment guide for the assessment and management of short-term risks associated with mental, substance use, and personality disorders. The scheme may be distinguished from other violence risk instruments because of its inclusion of 20 dynamic factors that are rated in terms of both vulnerability and strength. This study examined the reliability and validity of START assessments in predicting inpatient aggression. Research assistants completed START assessments for 120 male forensic psychiatric patients through review of hospital files. They additionally completed Historical-Clinical-Risk Management – 20 (HCR-20) and the Hare Psychopathy Checklist: Screening Version (PCL:SV) assessments. Outcome data was coded from hospital files for a 12-month follow-up period using the Overt Aggression Scale (OAS). START assessments evidenced excellent interrater reliability and demonstrated both predictive and incremental validity over the HCR-20 Historical subscale scores and PCL:SV total scores. Overall, results support the reliability and validity of START assessments, and use of the structured professional judgment approach more broadly, as well as the value of using dynamic risk and protective factors to assess violence risk. PMID:22250595

Validation of a new measure of availability and accommodation of health care that is valid for rural and urban contexts.

PubMed

Haggerty, Jeannie L; Levesque, Jean-Frédéric

2017-04-01

Patients are the most valid source for evaluating the accessibility of services, but a previous study observed differential psychometric performance of instruments in rural and urban respondents. To validate a measure of organizational accessibility free of differential rural-urban performance that predicts consequences of difficult access for patient-initiated care. Sequential qualitative-quantitative study. Qualitative findings used to adapt or develop evaluative and reporting items. Quantitative validation study. Primary data by telephone from 750 urban, rural and remote respondents in Quebec, Canada; follow-up mailed questionnaire to a subset of 316. Items were developed for barriers along the care trajectory. We used common factor and confirmatory factor analysis to identify constructs and compare models. We used item response theory analysis to test for differential rural-urban performance; examine individual item performance; adjust response options; and exclude redundant or non-discriminatory items. We used logistic regression to examine predictive validity of the subscale on access difficulty (outcome). Initial factor resolution suggested geographic and organizational dimensions, plus consequences of access difficulty. After second administration, organizational accommodation and geographic indicators were integrated into a 6-item subscale of Effective Availability and Accommodation, which demonstrates good variability and internal consistency (α = 0.84) and no differential functioning by geographic area. Each unit increase predicts decreased likelihood of consequences of access difficulties (unmet need and problem aggravation). The new subscale is a practical, valid and reliable measure for patients to evaluate first-contact health services accessibility, yielding valid comparisons between urban and rural contexts. © 2016 The Authors. Health Expectations published by John Wiley & Sons Ltd.

Number of organ dysfunctions predicts mortality in emergency department patients with suspected infection: a multicenter validation study.

PubMed

Jessen, Marie K; Skibsted, Simon; Shapiro, Nathan I

2017-06-01

The aim of this study was to validate the association between number of organ dysfunctions and mortality in emergency department (ED) patients with suspected infection. This study was conducted at two medical care center EDs. The internal validation set was a prospective cohort study conducted in Boston, USA. The external validation set was a retrospective case-control study conducted in Aarhus, Denmark. The study included adult patients (>18 years) with clinically suspected infection. Laboratory results and clinical data were used to assess organ dysfunctions. Inhospital mortality was the outcome measure. Multivariate logistic regression was used to determine the independent mortality odds for number and types of organ dysfunctions. We enrolled 4952 (internal) and 483 (external) patients. The mortality rate significantly increased with increasing number of organ dysfunctions: internal validation: 0 organ dysfunctions: 0.5% mortality, 1: 3.6%, 2: 9.5%, 3: 17%, and 4 or more: 37%; external validation: 2.2, 6.7, 17, 41, and 57% mortality (both P<0.001 for trend). Age-adjusted and comorbidity-adjusted number of organ dysfunctions remained an independent predictor. The effect of specific types of organ dysfunction on mortality was most pronounced for hematologic [odds ratio (OR) 3.3 (95% confidence interval (CI) 2.0-5.4)], metabolic [OR 3.3 (95% CI 2.4-4.6); internal validation], and cardiovascular dysfunctions [OR 14 (95% CI 3.7-50); external validation]. The number of organ dysfunctions predicts sepsis mortality.

Building and validating a prediction model for paediatric type 1 diabetes risk using next generation targeted sequencing of class II HLA genes.

PubMed

Zhao, Lue Ping; Carlsson, Annelie; Larsson, Helena Elding; Forsander, Gun; Ivarsson, Sten A; Kockum, Ingrid; Ludvigsson, Johnny; Marcus, Claude; Persson, Martina; Samuelsson, Ulf; Örtqvist, Eva; Pyo, Chul-Woo; Bolouri, Hamid; Zhao, Michael; Nelson, Wyatt C; Geraghty, Daniel E; Lernmark, Åke

2017-11-01

It is of interest to predict possible lifetime risk of type 1 diabetes (T1D) in young children for recruiting high-risk subjects into longitudinal studies of effective prevention strategies. Utilizing a case-control study in Sweden, we applied a recently developed next generation targeted sequencing technology to genotype class II genes and applied an object-oriented regression to build and validate a prediction model for T1D. In the training set, estimated risk scores were significantly different between patients and controls (P = 8.12 × 10 -92 ), and the area under the curve (AUC) from the receiver operating characteristic (ROC) analysis was 0.917. Using the validation data set, we validated the result with AUC of 0.886. Combining both training and validation data resulted in a predictive model with AUC of 0.903. Further, we performed a "biological validation" by correlating risk scores with 6 islet autoantibodies, and found that the risk score was significantly correlated with IA-2A (Z-score = 3.628, P < 0.001). When applying this prediction model to the Swedish population, where the lifetime T1D risk ranges from 0.5% to 2%, we anticipate identifying approximately 20 000 high-risk subjects after testing all newborns, and this calculation would identify approximately 80% of all patients expected to develop T1D in their lifetime. Through both empirical and biological validation, we have established a prediction model for estimating lifetime T1D risk, using class II HLA. This prediction model should prove useful for future investigations to identify high-risk subjects for prevention research in high-risk populations. Copyright © 2017 John Wiley & Sons, Ltd.

On Validity Theory and Test Validation

ERIC Educational Resources Information Center

Sireci, Stephen G.

2007-01-01

Lissitz and Samuelsen (2007) propose a new framework for conceptualizing test validity that separates analysis of test properties from analysis of the construct measured. In response, the author of this article reviews fundamental characteristics of test validity, drawing largely from seminal writings as well as from the accepted standards. He…

Validation of a New Skinfold Prediction Equation Based on Dual-Energy X-Ray Absorptiometry

ERIC Educational Resources Information Center

Ball, Stephen; Cowan, Celsi; Thyfault, John; LaFontaine, Tom

2014-01-01

Skinfold prediction equations recommended by the American College of Sports Medicine underestimate body fat percentage. The purpose of this research was to validate an alternative equation for men created from dual energy x-ray absorptiometry. Two hundred ninety-seven males, aged 18-65, completed a skinfold assessment and dual energy x-ray…

Validation of model predictions of pore-scale fluid distributions during two-phase flow

NASA Astrophysics Data System (ADS)

Bultreys, Tom; Lin, Qingyang; Gao, Ying; Raeini, Ali Q.; AlRatrout, Ahmed; Bijeljic, Branko; Blunt, Martin J.

2018-05-01

Pore-scale two-phase flow modeling is an important technology to study a rock's relative permeability behavior. To investigate if these models are predictive, the calculated pore-scale fluid distributions which determine the relative permeability need to be validated. In this work, we introduce a methodology to quantitatively compare models to experimental fluid distributions in flow experiments visualized with microcomputed tomography. First, we analyzed five repeated drainage-imbibition experiments on a single sample. In these experiments, the exact fluid distributions were not fully repeatable on a pore-by-pore basis, while the global properties of the fluid distribution were. Then two fractional flow experiments were used to validate a quasistatic pore network model. The model correctly predicted the fluid present in more than 75% of pores and throats in drainage and imbibition. To quantify what this means for the relevant global properties of the fluid distribution, we compare the main flow paths and the connectivity across the different pore sizes in the modeled and experimental fluid distributions. These essential topology characteristics matched well for drainage simulations, but not for imbibition. This suggests that the pore-filling rules in the network model we used need to be improved to make reliable predictions of imbibition. The presented analysis illustrates the potential of our methodology to systematically and robustly test two-phase flow models to aid in model development and calibration.

Validation metrics for turbulent plasma transport

DOE Office of Scientific and Technical Information (OSTI.GOV)

Holland, C., E-mail: chholland@ucsd.edu

Developing accurate models of plasma dynamics is essential for confident predictive modeling of current and future fusion devices. In modern computer science and engineering, formal verification and validation processes are used to assess model accuracy and establish confidence in the predictive capabilities of a given model. This paper provides an overview of the key guiding principles and best practices for the development of validation metrics, illustrated using examples from investigations of turbulent transport in magnetically confined plasmas. Particular emphasis is given to the importance of uncertainty quantification and its inclusion within the metrics, and the need for utilizing synthetic diagnosticsmore » to enable quantitatively meaningful comparisons between simulation and experiment. As a starting point, the structure of commonly used global transport model metrics and their limitations is reviewed. An alternate approach is then presented, which focuses upon comparisons of predicted local fluxes, fluctuations, and equilibrium gradients against observation. The utility of metrics based upon these comparisons is demonstrated by applying them to gyrokinetic predictions of turbulent transport in a variety of discharges performed on the DIII-D tokamak [J. L. Luxon, Nucl. Fusion 42, 614 (2002)], as part of a multi-year transport model validation activity.« less

Validating the Predicted Effect of Astemizole and Ketoconazole Using a Drosophila Model of Parkinson's Disease.

PubMed

Styczyńska-Soczka, Katarzyna; Zechini, Luigi; Zografos, Lysimachos

2017-04-01

Parkinson's disease is a growing threat to an ever-ageing population. Despite progress in our understanding of the molecular and cellular mechanisms underlying the disease, all therapeutics currently available only act to improve symptoms and do not stop the disease process. It is therefore imperative that more effective drug discovery methods and approaches are developed, validated, and used for the discovery of disease-modifying treatments for Parkinson's. Drug repurposing has been recognized as being equally as promising as de novo drug discovery in the field of neurodegeneration and Parkinson's disease specifically. In this work, we utilize a transgenic Drosophila model of Parkinson's disease, made by expressing human alpha-synuclein in the Drosophila brain, to validate two repurposed compounds: astemizole and ketoconazole. Both have been computationally predicted to have an ameliorative effect on Parkinson's disease, but neither had been tested using an in vivo model of the disease. After treating the flies in parallel, results showed that both drugs rescue the motor phenotype that is developed by the Drosophila model with age, but only ketoconazole treatment reversed the increased dopaminergic neuron death also observed in these models, which is a hallmark of Parkinson's disease. In addition to validating the predicted improvement in Parkinson's disease symptoms for both drugs and revealing the potential neuroprotective activity of ketoconazole, these results highlight the value of Drosophila models of Parkinson's disease as key tools in the context of in vivo drug discovery, drug repurposing, and prioritization of hits, especially when coupled with computational predictions.

Predicting Relapse among Young Adults: Psychometric Validation of the Advanced Warning of Relapse (AWARE) Scale

PubMed Central

Kelly, John F.; Hoeppner, Bettina B.; Urbanoski, Karen A.; Slaymaker, Valerie

2011-01-01

Objective Failure to maintain abstinence despite incurring severe harm is perhaps the key defining feature of addiction. Relapse prevention strategies have been developed to attenuate this propensity to relapse, but predicting who will, and who will not, relapse has stymied attempts to more efficiently tailor treatments according to relapse risk profile. Here we examine the psychometric properties of a promising relapse risk measure - the Advance WArning of RElapse scale (AWARE) scale (Miller and Harris, 2000) in an understudied but clinically important sample of young adults. Method Inpatient youth (N=303; Age 18-24; 26% female) completed the AWARE scale and the Brief Symptom Inventory-18 (BSI) at the end of residential treatment, and at 1-, 3-, and 6-months following discharge. Internal and convergent validity was tested for each of these four timepoints using confirmatory factor analysis and correlations (with BSI scores). Predictive validity was tested for relapse 1, 3, and 6 months following discharge, as was incremental utility, where AWARE scores were used as predictors of any substance use while controlling for treatment entry substance use severity and having spent time in a controlled environment following treatment. Results Confirmatory factor analysis revealed a single, internally consistent, 25-item factor that demonstrated convergent validity and predicted subsequent relapse alone and when controlling for other important relapse risk predictors. Conclusions The AWARE scale may be a useful and efficient clinical tool for assessing short-term relapse risk among young people and, thus, could serve to enhance the effectiveness of relapse prevention efforts. PMID:21700396

Predicting relapse among young adults: psychometric validation of the Advanced WArning of RElapse (AWARE) scale.

PubMed

Kelly, John F; Hoeppner, Bettina B; Urbanoski, Karen A; Slaymaker, Valerie

2011-10-01

Failure to maintain abstinence despite incurring severe harm is perhaps the key defining feature of addiction. Relapse prevention strategies have been developed to attenuate this propensity to relapse, but predicting who will, and who will not, relapse has stymied attempts to more efficiently tailor treatments according to relapse risk profile. Here we examine the psychometric properties of a promising relapse risk measure-the Advance WArning of RElapse (AWARE) scale (Miller & Harris, 2000) in an understudied but clinically important sample of young adults. Inpatient youth (N=303; Ages 18-24; 26% female) completed the AWARE scale and the Brief Symptom Inventory-18 (BSI) at the end of residential treatment, and at 1-, 3-, and 6-months following discharge. Internal and convergent validity was tested for each of these four timepoints using confirmatory factor analysis and correlations (with BSI scores). Predictive validity was tested for relapse 1, 3, and 6 months following discharge, as was incremental utility, where AWARE scores were used as predictors of any substance use while controlling for treatment entry substance use severity and having spent time in a controlled environment following treatment. Confirmatory factor analysis revealed a single, internally consistent, 25-item factor that demonstrated convergent validity and predicted subsequent relapse alone and when controlling for other important relapse risk predictors. The AWARE scale may be a useful and efficient clinical tool for assessing short-term relapse risk among young people and, thus, could serve to enhance the effectiveness of relapse prevention efforts. Copyright © 2011 Elsevier Ltd. All rights reserved.

Output-Adaptive Tetrahedral Cut-Cell Validation for Sonic Boom Prediction

NASA Technical Reports Server (NTRS)

Park, Michael A.; Darmofal, David L.

2008-01-01

A cut-cell approach to Computational Fluid Dynamics (CFD) that utilizes the median dual of a tetrahedral background grid is described. The discrete adjoint is also calculated, which permits adaptation based on improving the calculation of a specified output (off-body pressure signature) in supersonic inviscid flow. These predicted signatures are compared to wind tunnel measurements on and off the configuration centerline 10 body lengths below the model to validate the method for sonic boom prediction. Accurate mid-field sonic boom pressure signatures are calculated with the Euler equations without the use of hybrid grid or signature propagation methods. Highly-refined, shock-aligned anisotropic grids were produced by this method from coarse isotropic grids created without prior knowledge of shock locations. A heuristic reconstruction limiter provided stable flow and adjoint solution schemes while producing similar signatures to Barth-Jespersen and Venkatakrishnan limiters. The use of cut-cells with an output-based adaptive scheme completely automated this accurate prediction capability after a triangular mesh is generated for the cut surface. This automation drastically reduces the manual intervention required by existing methods.

Lightweight ZERODUR: Validation of Mirror Performance and Mirror Modeling Predictions

NASA Technical Reports Server (NTRS)

Hull, Tony; Stahl, H. Philip; Westerhoff, Thomas; Valente, Martin; Brooks, Thomas; Eng, Ron

2017-01-01

Upcoming spaceborne missions, both moderate and large in scale, require extreme dimensional stability while relying both upon established lightweight mirror materials, and also upon accurate modeling methods to predict performance under varying boundary conditions. We describe tests, recently performed at NASA's XRCF chambers and laboratories in Huntsville Alabama, during which a 1.2 m diameter, f/1.2988% lightweighted SCHOTT lightweighted ZERODUR(TradeMark) mirror was tested for thermal stability under static loads in steps down to 230K. Test results are compared to model predictions, based upon recently published data on ZERODUR(TradeMark). In addition to monitoring the mirror surface for thermal perturbations in XRCF Thermal Vacuum tests, static load gravity deformations have been measured and compared to model predictions. Also the Modal Response(dynamic disturbance) was measured and compared to model. We will discuss the fabrication approach and optomechanical design of the ZERODUR(TradeMark) mirror substrate by SCHOTT, its optical preparation for test by Arizona Optical Systems (AOS). Summarize the outcome of NASA's XRCF tests and model validations

Lightweight ZERODUR®: Validation of mirror performance and mirror modeling predictions

NASA Astrophysics Data System (ADS)

Hull, Anthony B.; Stahl, H. Philip; Westerhoff, Thomas; Valente, Martin; Brooks, Thomas; Eng, Ron

2017-01-01

Upcoming spaceborne missions, both moderate and large in scale, require extreme dimensional stability while relying both upon established lightweight mirror materials, and also upon accurate modeling methods to predict performance under varying boundary conditions. We describe tests, recently performed at NASA’s XRCF chambers and laboratories in Huntsville Alabama, during which a 1.2m diameter, f/1.29 88% lightweighted SCHOTT lightweighted ZERODUR® mirror was tested for thermal stability under static loads in steps down to 230K. Test results are compared to model predictions, based upon recently published data on ZERODUR®. In addition to monitoring the mirror surface for thermal perturbations in XRCF Thermal Vacuum tests, static load gravity deformations have been measured and compared to model predictions. Also the Modal Response (dynamic disturbance) was measured and compared to model. We will discuss the fabrication approach and optomechanical design of the ZERODUR® mirror substrate by SCHOTT, its optical preparation for test by Arizona Optical Systems (AOS), and summarize the outcome of NASA’s XRCF tests and model validations.

The HIrisPlex-S system for eye, hair and skin colour prediction from DNA: Introduction and forensic developmental validation.

PubMed

Chaitanya, Lakshmi; Breslin, Krystal; Zuñiga, Sofia; Wirken, Laura; Pośpiech, Ewelina; Kukla-Bartoszek, Magdalena; Sijen, Titia; Knijff, Peter de; Liu, Fan; Branicki, Wojciech; Kayser, Manfred; Walsh, Susan

2018-07-01

Forensic DNA Phenotyping (FDP), i.e. the prediction of human externally visible traits from DNA, has become a fast growing subfield within forensic genetics due to the intelligence information it can provide from DNA traces. FDP outcomes can help focus police investigations in search of unknown perpetrators, who are generally unidentifiable with standard DNA profiling. Therefore, we previously developed and forensically validated the IrisPlex DNA test system for eye colour prediction and the HIrisPlex system for combined eye and hair colour prediction from DNA traces. Here we introduce and forensically validate the HIrisPlex-S DNA test system (S for skin) for the simultaneous prediction of eye, hair, and skin colour from trace DNA. This FDP system consists of two SNaPshot-based multiplex assays targeting a total of 41 SNPs via a novel multiplex assay for 17 skin colour predictive SNPs and the previous HIrisPlex assay for 24 eye and hair colour predictive SNPs, 19 of which also contribute to skin colour prediction. The HIrisPlex-S system further comprises three statistical prediction models, the previously developed IrisPlex model for eye colour prediction based on 6 SNPs, the previous HIrisPlex model for hair colour prediction based on 22 SNPs, and the recently introduced HIrisPlex-S model for skin colour prediction based on 36 SNPs. In the forensic developmental validation testing, the novel 17-plex assay performed in full agreement with the Scientific Working Group on DNA Analysis Methods (SWGDAM) guidelines, as previously shown for the 24-plex assay. Sensitivity testing of the 17-plex assay revealed complete SNP profiles from as little as 63 pg of input DNA, equalling the previously demonstrated sensitivity threshold of the 24-plex HIrisPlex assay. Testing of simulated forensic casework samples such as blood, semen, saliva stains, of inhibited DNA samples, of low quantity touch (trace) DNA samples, and of artificially degraded DNA samples as well as

Development and external validation of a prediction rule for an unfavorable course of late-life depression: A multicenter cohort study.

PubMed

Maarsingh, O R; Heymans, M W; Verhaak, P F; Penninx, B W J H; Comijs, H C

2018-08-01

Given the poor prognosis of late-life depression, it is crucial to identify those at risk. Our objective was to construct and validate a prediction rule for an unfavourable course of late-life depression. For development and internal validation of the model, we used The Netherlands Study of Depression in Older Persons (NESDO) data. We included participants with a major depressive disorder (MDD) at baseline (n = 270; 60-90 years), assessed with the Composite International Diagnostic Interview (CIDI). For external validation of the model, we used The Netherlands Study of Depression and Anxiety (NESDA) data (n = 197; 50-66 years). The outcome was MDD after 2 years of follow-up, assessed with the CIDI. Candidate predictors concerned sociodemographics, psychopathology, physical symptoms, medication, psychological determinants, and healthcare setting. Model performance was assessed by calculating calibration and discrimination. 111 subjects (41.1%) had MDD after 2 years of follow-up. Independent predictors of MDD after 2 years were (older) age, (early) onset of depression, severity of depression, anxiety symptoms, comorbid anxiety disorder, fatigue, and loneliness. The final model showed good calibration and reasonable discrimination (AUC of 0.75; 0.70 after external validation). The strongest individual predictor was severity of depression (AUC of 0.69; 0.68 after external validation). The model was developed and validated in The Netherlands, which could affect the cross-country generalizability. Based on rather simple clinical indicators, it is possible to predict the 2-year course of MDD. The prediction rule can be used for monitoring MDD patients and identifying those at risk of an unfavourable outcome. Copyright © 2018 Elsevier B.V. All rights reserved.

Validation of pathological grading systems for predicting metastatic potential in pheochromocytoma and paraganglioma

PubMed Central

Koh, Jung-Min; Ahn, Seong Hee; Kim, Hyeonmok; Kim, Beom-Jun; Sung, Tae-Yon; Kim, Young Hoon; Hong, Suck Joon; Song, Dong Eun

2017-01-01

Purpose The Grading system for Adrenal Pheochromocytoma and Paraganglioma (GAPP) was proposed for predicting the metastatic potential of pheochromocytoma and paraganglioma to overcome the limitations of the Pheochromocytoma of the Adrenal Scaled Score (PASS). However, to date, no study validating the GAPP has been conducted, and previous studies did not include mutations in the succinate dehydrogenase type B (SDHB) gene in the score calculation. In this retrospective cohort study, we validated the prediction ability of GAPP and assessed whether it would be improved by inclusion of the loss of SDHB immunohistochemical staining. Methods We divided the tumors into non-metastatic and metastatic groups based on the presence of synchronous or metachronous metastases. The GAPP score and PASS at the initial operation were measured. Moreover, we combined some GAPP parameters with the immunohistochemical staining of SDHB to obtain a modified GAPP (M-GAPP) score. Results Metastasis occurred in 15/72 (20.8%) patients, with a mean follow-up of 43.5 months. Loss of SDHB staining was more frequent (P = 0.044) in the metastatic group. The GAPP score (P = 0.006), PASS (P = 0.003), and M-GAPP score (P<0.001) were all higher in the metastatic group. Twelve of 40 (30.0%) moderately or poorly differentiated tumors, as defined by the GAPP score, and 12/34 (35.3%) tumors with a PASS ≥4 were metastatic. Conversely, 10/19 (52.6%) tumors with an M-GAPP score ≥3 were metastatic. The area under the curve of the M-GAPP score (0.822) was significantly higher than that of the GAPP (0.728) (P = 0.012), but similar to that of the PASS (0.753) (P = 0.411). The GAPP (P = 0.032) and M-GAPP scores (P = 0.040), but not PASS (P = 0.200), negatively correlated with metastasis-free survival. Conclusion The GAPP was validated, and M-GAPP, a combination of some GAPP parameters and loss of SDHB staining, might be useful for the prediction of the metastatic potential of pheochromocytoma and paraganglioma

Prediction of insufficient serum vitamin D status in older women: a validated model.

PubMed

Merlijn, T; Swart, K M A; Lips, P; Heymans, M W; Sohl, E; Van Schoor, N M; Netelenbos, C J; Elders, P J M

2018-05-28

We developed an externally validated simple prediction model to predict serum 25(OH)D levels < 30, < 40, < 50 and 60 nmol/L in older women with risk factors for fractures. The benefit of the model reduces when a higher 25(OH)D threshold is chosen. Vitamin D deficiency is associated with increased fracture risk in older persons. General supplementation of all older women with vitamin D could cause medicalization and costs. We developed a clinical model to identify insufficient serum 25-hydroxyvitamin D (25(OH)D) status in older women at risk for fractures. In a sample of 2689 women ≥ 65 years selected from general practices, with at least one risk factor for fractures, a questionnaire was administered and serum 25(OH)D was measured. Multivariable logistic regression models with backward selection were developed to select predictors for insufficient serum 25(OH)D status, using separate thresholds 30, 40, 50 and 60 nmol/L. Internal and external model validations were performed. Predictors in the models were as follows: age, BMI, vitamin D supplementation, multivitamin supplementation, calcium supplementation, daily use of margarine, fatty fish ≥ 2×/week, ≥ 1 hours/day outdoors in summer, season of blood sampling, the use of a walking aid and smoking. The AUC was 0.77 for the model using a 30 nmol/L threshold and decreased in the models with higher thresholds to 0.72 for 60 nmol/L. We demonstrate that the model can help to distinguish patients with or without insufficient serum 25(OH)D levels at thresholds of 30 and 40 nmol/L, but not when a threshold of 50 nmol/L is demanded. This externally validated model can predict the presence of vitamin D insufficiency in women at risk for fractures. The potential clinical benefit of this tool is highly dependent of the chosen 25(OH)D threshold and decreases when a higher threshold is used.

Validating a model that predicts daily growth and feed quality of New Zealand dairy pastures.

PubMed

Woodward, S J

2001-09-01

The Pasture Quality (PQ) model is a simple, mechanistic, dynamical system model that was designed to capture the essential biological processes in grazed grass-clover pasture, and to be optimised to derive improved grazing strategies for New Zealand dairy farms. While the individual processes represented in the model (photosynthesis, tissue growth, flowering, leaf death, decomposition, worms) were based on experimental data, this did not guarantee that the assembled model would accurately predict the behaviour of the system as a whole (i.e., pasture growth and quality). Validation of the whole model was thus a priority, since any strategy derived from the model could impact a farm business in the order of thousands of dollars per annum if adopted. This paper describes the process of defining performance criteria for the model, obtaining suitable data to test the model, and carrying out the validation analysis. The validation process highlighted a number of weaknesses in the model, which will lead to the model being improved. As a result, the model's utility will be enhanced. Furthermore, validation was found to have an unexpected additional benefit, in that despite the model's poor initial performance, support was generated for the model among field scientists involved in the wider project.

How Nonrecidivism Affects Predictive Accuracy: Evidence from a Cross-Validation of the Ontario Domestic Assault Risk Assessment (ODARA)

ERIC Educational Resources Information Center

Hilton, N. Zoe; Harris, Grant T.

2009-01-01

Prediction effect sizes such as ROC area are important for demonstrating a risk assessment's generalizability and utility. How a study defines recidivism might affect predictive accuracy. Nonrecidivism is problematic when predicting specialized violence (e.g., domestic violence). The present study cross-validates the ability of the Ontario…

The predictive validity of quality of evidence grades for the stability of effect estimates was low: a meta-epidemiological study.

PubMed

Gartlehner, Gerald; Dobrescu, Andreea; Evans, Tammeka Swinson; Bann, Carla; Robinson, Karen A; Reston, James; Thaler, Kylie; Skelly, Andrea; Glechner, Anna; Peterson, Kimberly; Kien, Christina; Lohr, Kathleen N

2016-02-01

To determine the predictive validity of the U.S. Evidence-based Practice Center (EPC) approach to GRADE (Grading of Recommendations Assessment, Development and Evaluation). Based on Cochrane reports with outcomes graded as high quality of evidence (QOE), we prepared 160 documents which represented different levels of QOE. Professional systematic reviewers dually graded the QOE. For each document, we determined whether estimates were concordant with high QOE estimates of the Cochrane reports. We compared the observed proportion of concordant estimates with the expected proportion from an international survey. To determine the predictive validity, we used the Hosmer-Lemeshow test to assess calibration and the C (concordance) index to assess discrimination. The predictive validity of the EPC approach to GRADE was limited. Estimates graded as high QOE were less likely, estimates graded as low or insufficient QOE more likely to remain stable than expected. The EPC approach to GRADE could not reliably predict the likelihood that individual bodies of evidence remain stable as new evidence becomes available. C-indices ranged between 0.56 (95% CI, 0.47 to 0.66) and 0.58 (95% CI, 0.50 to 0.67) indicating a low discriminatory ability. The limited predictive validity of the EPC approach to GRADE seems to reflect a mismatch between expected and observed changes in treatment effects as bodies of evidence advance from insufficient to high QOE. Copyright © 2016 Elsevier Inc. All rights reserved.

Comparison of the predictive validity of diagnosis-based risk adjusters for clinical outcomes.

PubMed

Petersen, Laura A; Pietz, Kenneth; Woodard, LeChauncy D; Byrne, Margaret

2005-01-01

Many possible methods of risk adjustment exist, but there is a dearth of comparative data on their performance. We compared the predictive validity of 2 widely used methods (Diagnostic Cost Groups [DCGs] and Adjusted Clinical Groups [ACGs]) for 2 clinical outcomes using a large national sample of patients. We studied all patients who used Veterans Health Administration (VA) medical services in fiscal year (FY) 2001 (n = 3,069,168) and assigned both a DCG and an ACG to each. We used logistic regression analyses to compare predictive ability for death or long-term care (LTC) hospitalization for age/gender models, DCG models, and ACG models. We also assessed the effect of adding age to the DCG and ACG models. Patients in the highest DCG categories, indicating higher severity of illness, were more likely to die or to require LTC hospitalization. Surprisingly, the age/gender model predicted death slightly more accurately than the ACG model (c-statistic of 0.710 versus 0.700, respectively). The addition of age to the ACG model improved the c-statistic to 0.768. The highest c-statistic for prediction of death was obtained with a DCG/age model (0.830). The lowest c-statistics were obtained for age/gender models for LTC hospitalization (c-statistic 0.593). The c-statistic for use of ACGs to predict LTC hospitalization was 0.783, and improved to 0.792 with the addition of age. The c-statistics for use of DCGs and DCG/age to predict LTC hospitalization were 0.885 and 0.890, respectively, indicating the best prediction. We found that risk adjusters based upon diagnoses predicted an increased likelihood of death or LTC hospitalization, exhibiting good predictive validity. In this comparative analysis using VA data, DCG models were generally superior to ACG models in predicting clinical outcomes, although ACG model performance was enhanced by the addition of age.

Food for Thought ... Mechanistic Validation

PubMed Central

Hartung, Thomas; Hoffmann, Sebastian; Stephens, Martin

2013-01-01

Summary Validation of new approaches in regulatory toxicology is commonly defined as the independent assessment of the reproducibility and relevance (the scientific basis and predictive capacity) of a test for a particular purpose. In large ring trials, the emphasis to date has been mainly on reproducibility and predictive capacity (comparison to the traditional test) with less attention given to the scientific or mechanistic basis. Assessing predictive capacity is difficult for novel approaches (which are based on mechanism), such as pathways of toxicity or the complex networks within the organism (systems toxicology). This is highly relevant for implementing Toxicology for the 21st Century, either by high-throughput testing in the ToxCast/ Tox21 project or omics-based testing in the Human Toxome Project. This article explores the mostly neglected assessment of a test's scientific basis, which moves mechanism and causality to the foreground when validating/qualifying tests. Such mechanistic validation faces the problem of establishing causality in complex systems. However, pragmatic adaptations of the Bradford Hill criteria, as well as bioinformatic tools, are emerging. As critical infrastructures of the organism are perturbed by a toxic mechanism we argue that by focusing on the target of toxicity and its vulnerability, in addition to the way it is perturbed, we can anchor the identification of the mechanism and its verification. PMID:23665802

Prediction of new brain metastases after radiosurgery: validation and analysis of performance of a multi-institutional nomogram.

PubMed

Ayala-Peacock, Diandra N; Attia, Albert; Braunstein, Steve E; Ahluwalia, Manmeet S; Hepel, Jaroslaw; Chung, Caroline; Contessa, Joseph; McTyre, Emory; Peiffer, Ann M; Lucas, John T; Isom, Scott; Pajewski, Nicholas M; Kotecha, Rupesh; Stavas, Mark J; Page, Brandi R; Kleinberg, Lawrence; Shen, Colette; Taylor, Robert B; Onyeuku, Nasarachi E; Hyde, Andrew T; Gorovets, Daniel; Chao, Samuel T; Corso, Christopher; Ruiz, Jimmy; Watabe, Kounosuke; Tatter, Stephen B; Zadeh, Gelareh; Chiang, Veronica L S; Fiveash, John B; Chan, Michael D

2017-11-01

Stereotactic radiosurgery (SRS) without whole brain radiotherapy (WBRT) for brain metastases can avoid WBRT toxicities, but with risk of subsequent distant brain failure (DBF). Sole use of number of metastases to triage patients may be an unrefined method. Data on 1354 patients treated with SRS monotherapy from 2000 to 2013 for new brain metastases was collected across eight academic centers. The cohort was divided into training and validation datasets and a prognostic model was developed for time to DBF. We then evaluated the discrimination and calibration of the model within the validation dataset, and confirmed its performance with an independent contemporary cohort. Number of metastases (≥8, HR 3.53 p = 0.0001), minimum margin dose (HR 1.07 p = 0.0033), and melanoma histology (HR 1.45, p = 0.0187) were associated with DBF. A prognostic index derived from the training dataset exhibited ability to discriminate patients' DBF risk within the validation dataset (c-index = 0.631) and Heller's explained relative risk (HERR) = 0.173 (SE = 0.048). Absolute number of metastases was evaluated for its ability to predict DBF in the derivation and validation datasets, and was inferior to the nomogram. A nomogram high-risk threshold yielding a 2.1-fold increased need for early WBRT was identified. Nomogram values also correlated to number of brain metastases at time of failure (r = 0.38, p < 0.0001). We present a multi-institutionally validated prognostic model and nomogram to predict risk of DBF and guide risk-stratification of patients who are appropriate candidates for radiosurgery versus upfront WBRT.

External validation of the ability of the DRAGON score to predict outcome after thrombolysis treatment.

PubMed

Ovesen, C; Christensen, A; Nielsen, J K; Christensen, H

2013-11-01

Easy-to-perform and valid assessment scales for the effect of thrombolysis are essential in hyperacute stroke settings. Because of this we performed an external validation of the DRAGON scale proposed by Strbian et al. in a Danish cohort. All patients treated with intravenous recombinant plasminogen activator between 2009 and 2011 were included. Upon admission all patients underwent physical and neurological examination using the National Institutes of Health Stroke Scale along with non-contrast CT scans and CT angiography. Patients were followed up through the Outpatient Clinic and their modified Rankin Scale (mRS) was assessed after 3 months. Three hundred and three patients were included in the analysis. The DRAGON scale proved to have a good discriminative ability for predicting highly unfavourable outcome (mRS 5-6) (area under the curve-receiver operating characteristic [AUC-ROC]: 0.89; 95% confidence interval [CI] 0.81-0.96; p<0.001) and good outcome (mRS 0-2) (AUC-ROC: 0.79; 95% CI 0.73-0.85; p<0.001). When only patients with M1 occlusions were selected the DRAGON scale provided good discriminative capability (AUC-ROC: 0.89; 95% CI 0.78-1.0; p=0.003) for highly unfavourable outcome. We confirmed the validity of the DRAGON scale in predicting outcome after thrombolysis treatment. Copyright © 2013 Elsevier Ltd. All rights reserved.

Predictive validity of a selection centre testing non-technical skills for recruitment to training in anaesthesia.

PubMed

Gale, T C E; Roberts, M J; Sice, P J; Langton, J A; Patterson, F C; Carr, A S; Anderson, I R; Lam, W H; Davies, P R F

2010-11-01

Assessment centres are an accepted method of recruitment in industry and are gaining popularity within medicine. We describe the development and validation of a selection centre for recruitment to speciality training in anaesthesia based on an assessment centre model incorporating the rating of candidate's non-technical skills. Expert consensus identified non-technical skills suitable for assessment at the point of selection. Four stations-structured interview, portfolio review, presentation, and simulation-were developed, the latter two being realistic scenarios of work-related tasks. Evaluation of the selection centre focused on applicant and assessor feedback ratings, inter-rater agreement, and internal consistency reliability coefficients. Predictive validity was sought via correlations of selection centre scores with subsequent workplace-based ratings of appointed trainees. Two hundred and twenty-four candidates were assessed over two consecutive annual recruitment rounds; 68 were appointed and followed up during training. Candidates and assessors demonstrated strong approval of the selection centre with more than 70% of ratings 'good' or 'excellent'. Mean inter-rater agreement coefficients ranged from 0.62 to 0.77 and internal consistency reliability of the selection centre score was high (Cronbach's α=0.88-0.91). The overall selection centre score was a good predictor of workplace performance during the first year of appointment. An assessment centre model based on the rating of non-technical skills can produce a reliable and valid selection tool for recruitment to speciality training in anaesthesia. Early results on predictive validity are encouraging and justify further development and evaluation.

Validating the TeleStroke Mimic Score: A Prediction Rule for Identifying Stroke Mimics Evaluated Over Telestroke Networks.

PubMed

Ali, Syed F; Hubert, Gordian J; Switzer, Jeffrey A; Majersik, Jennifer J; Backhaus, Roland; Shepard, L Wylie; Vedala, Kishore; Schwamm, Lee H

2018-03-01

Up to 30% of acute stroke evaluations are deemed stroke mimics, and these are common in telestroke as well. We recently published a risk prediction score for use during telestroke encounters to differentiate stroke mimics from ischemic cerebrovascular disease derived and validated in the Partners TeleStroke Network. Using data from 3 distinct US and European telestroke networks, we sought to externally validate the TeleStroke Mimic (TM) score in a broader population. We evaluated the TM score in 1930 telestroke consults from the University of Utah, Georgia Regents University, and the German TeleMedical Project for Integrative Stroke Care Network. We report the area under the curve in receiver-operating characteristic curve analysis with 95% confidence interval for our previously derived TM score in which lower TM scores correspond with a higher likelihood of being a stroke mimic. Based on final diagnosis at the end of the telestroke consultation, there were 630 of 1930 (32.6%) stroke mimics in the external validation cohort. All 6 variables included in the score were significantly different between patients with ischemic cerebrovascular disease versus stroke mimics. The TM score performed well (area under curve, 0.72; 95% confidence interval, 0.70-0.73; P <0.001), similar to our prior external validation in the Partners National Telestroke Network. The TM score's ability to predict the presence of a stroke mimic during telestroke consultation in these diverse cohorts was similar to its performance in our original cohort. Predictive decision-support tools like the TM score may help highlight key clinical differences between mimics and patients with stroke during complex, time-critical telestroke evaluations. © 2018 American Heart Association, Inc.

Design and Validation of a Prehospital Scale to Predict Stroke Severity: The Cincinnati Prehospital Stroke Severity Scale

PubMed Central

Katz, Brian S.; McMullan, Jason T.; Sucharew, Heidi; Adeoye, Opeolu; Broderick, Joseph P.

2015-01-01

Background and Purpose We derived and validated the Cincinnati Prehospital Stroke Severity Scale (CPSSS) to identify patients with severe strokes and large vessel occlusion (LVO). Methods CPSSS was developed with regression tree analysis, objectivity, anticipated ease in administration by EMS personnel, and the presence of cortical signs. We derived and validated the tool using the two NINDS t-PA Stroke Study trials and IMS III Trial cohorts, respectively, to predict severe stroke [NIH stroke scale (NIHSS) ≥15] and LVO. Standard test characteristics were determined and receiver operator curves were generated and summarized by the area under the curve (AUC). Results CPSSS score ranges from 0-4; composed and scored by individual NIHSS items: 2 points for presence of conjugate gaze (NIHSS ≥1); 1 point for presence of arm weakness (NIHSS ≥2); and 1 point for presence abnormal level of consciousness (LOC) commands and questions (NIHSS LOC ≥1 each). In the derivation set, CPSSS had an AUC of 0.89; score ≥2 was 89% sensitive and 73% specific in identifying NIHSS ≥15. Validation results were similar with an AUC of 0.83; score ≥2 was 92% sensitive, 51% specific, a positive likelihood ratio (PLR) of 3.3 and a negative likelihood ratio (NLR) of 0.15 in predicting severe stroke. For 222/303 IMS III subjects with LVO, CPSSS had an AUC of 0.67; a score ≥2 was 83% sensitive, 40% specific, PLR of 1.4, and NLR of 0.4 in predicting LVO. Conclusions CPSSS can identify stroke patients with NIHSS ≥15 and LVO. Prospective prehospital validation is warranted. PMID:25899242

Predicting the chance of vaginal delivery after one cesarean section: validation and elaboration of a published prediction model.

PubMed

Fagerberg, Marie C; Maršál, Karel; Källén, Karin

2015-05-01

We aimed to validate a widely used US prediction model for vaginal birth after cesarean (Grobman et al. [8]) and modify it to suit Swedish conditions. Women having experienced one cesarean section and at least one subsequent delivery (n=49,472) in the Swedish Medical Birth Registry 1992-2011 were randomly divided into two data sets. In the development data set, variables associated with successful trial of labor were identified using multiple logistic regression. The predictive ability of the estimates previously published by Grobman et al., and of our modified and new estimates, respectively, was then evaluated using the validation data set. The accuracy of the models for prediction of vaginal birth after cesarean was measured by area under the receiver operating characteristics curve. For maternal age, body mass index, prior vaginal delivery, and prior labor arrest, the odds ratio estimates for vaginal birth after cesarean were similar to those previously published. The prediction accuracy increased when information on indication for the previous cesarean section was added (from area under the receiver operating characteristics curve=0.69-0.71), and increased further when maternal height and delivery unit cesarean section rates were included (area under the receiver operating characteristics curve=0.74). The correlation between the individual predicted vaginal birth after cesarean probability and the observed trial of labor success rate was high in all the respective predicted probability decentiles. Customization of prediction models for vaginal birth after cesarean is of considerable value. Choosing relevant indicators for a Swedish setting made it possible to achieve excellent prediction accuracy for success in trial of labor after cesarean. During the delicate process of counseling about preferred delivery mode after one cesarean section, considering the results of our study may facilitate the choice between a trial of labor or an elective repeat cesarean

The development and validation of different decision-making tools to predict urine culture growth out of urine flow cytometry parameter.

PubMed

Müller, Martin; Seidenberg, Ruth; Schuh, Sabine K; Exadaktylos, Aristomenis K; Schechter, Clyde B; Leichtle, Alexander B; Hautz, Wolf E

2018-01-01

Patients presenting with suspected urinary tract infection are common in every day emergency practice. Urine flow cytometry has replaced microscopic urine evaluation in many emergency departments, but interpretation of the results remains challenging. The aim of this study was to develop and validate tools that predict urine culture growth out of urine flow cytometry parameter. This retrospective study included all adult patients that presented in a large emergency department between January and July 2017 with a suspected urinary tract infection and had a urine flow cytometry as well as a urine culture obtained. The objective was to identify urine flow cytometry parameters that reliably predict urine culture growth and mixed flora growth. The data set was split into a training (70%) and a validation set (30%) and different decision-making approaches were developed and validated. Relevant urine culture growth (respectively mixed flora growth) was found in 40.2% (7.2% respectively) of the 613 patients included. The number of leukocytes and bacteria in flow cytometry were highly associated with urine culture growth, but mixed flora growth could not be sufficiently predicted from the urine flow cytometry parameters. A decision tree, predictive value figures, a nomogram, and a cut-off table to predict urine culture growth from bacteria and leukocyte count were developed, validated and compared. Urine flow cytometry parameters are insufficient to predict mixed flora growth. However, the prediction of urine culture growth based on bacteria and leukocyte count is highly accurate and the developed tools should be used as part of the decision-making process of ordering a urine culture or starting an antibiotic therapy if a urogenital infection is suspected.

The development and validation of different decision-making tools to predict urine culture growth out of urine flow cytometry parameter

PubMed Central

Seidenberg, Ruth; Schuh, Sabine K.; Exadaktylos, Aristomenis K.; Schechter, Clyde B.; Leichtle, Alexander B.; Hautz, Wolf E.

2018-01-01

Objective Patients presenting with suspected urinary tract infection are common in every day emergency practice. Urine flow cytometry has replaced microscopic urine evaluation in many emergency departments, but interpretation of the results remains challenging. The aim of this study was to develop and validate tools that predict urine culture growth out of urine flow cytometry parameter. Methods This retrospective study included all adult patients that presented in a large emergency department between January and July 2017 with a suspected urinary tract infection and had a urine flow cytometry as well as a urine culture obtained. The objective was to identify urine flow cytometry parameters that reliably predict urine culture growth and mixed flora growth. The data set was split into a training (70%) and a validation set (30%) and different decision-making approaches were developed and validated. Results Relevant urine culture growth (respectively mixed flora growth) was found in 40.2% (7.2% respectively) of the 613 patients included. The number of leukocytes and bacteria in flow cytometry were highly associated with urine culture growth, but mixed flora growth could not be sufficiently predicted from the urine flow cytometry parameters. A decision tree, predictive value figures, a nomogram, and a cut-off table to predict urine culture growth from bacteria and leukocyte count were developed, validated and compared. Conclusions Urine flow cytometry parameters are insufficient to predict mixed flora growth. However, the prediction of urine culture growth based on bacteria and leukocyte count is highly accurate and the developed tools should be used as part of the decision-making process of ordering a urine culture or starting an antibiotic therapy if a urogenital infection is suspected. PMID:29474463

Lean body mass: the development and validation of prediction equations in healthy adults

PubMed Central

2013-01-01

Background There is a loss of lean body mass (LBM) with increasing age. A low LBM has been associated with increased adverse effects from prescribed medications such as chemotherapy. Accurate assessment of LBM may allow for more accurate drug prescribing. The aims of this study were to develop new prediction equations (PEs) for LBM with anthropometric and biochemical variables from a development cohort and then validate the best performing PEs in validation cohorts. Methods PEs were developed in a cohort of 188 healthy subjects and then validated in a convenience cohort of 52 healthy subjects. The best performing anthropometric PE was then compared to published anthropometric PEs in an older (age ≥ 50 years) cohort of 2287 people. Best subset regression analysis was used to derive PEs. Correlation, Bland-Altman and Sheiner & Beal methods were used to validate and compare the PEs against dual X-ray absorptiometry (DXA)-derived LBM. Results The PE which included biochemistry variables performed only marginally better than the anthropometric PE. The anthropometric PE on average over-estimated LBM by 0.74 kg in the combined cohort. Across gender (male vs. female), body mass index (< 22, 22- < 27, 27- < 30 and ≥30 kg/m2) and age groups (50–64, 65–79 and ≥80 years), the maximum mean over-estimation of the anthropometric PE was 1.36 kg. Conclusions A new anthropometric PE has been developed that offers an alternative for clinicians when access to DXA is limited. Further research is required to determine the clinical utility and if it will improve the safety of medication use. PMID:24499708

Independent data validation of an in vitro method for ...

EPA Pesticide Factsheets

In vitro bioaccessibility assays (IVBA) estimate arsenic (As) relative bioavailability (RBA) in contaminated soils to improve the accuracy of site-specific human exposure assessments and risk calculations. For an IVBA assay to gain acceptance for use in risk assessment, it must be shown to reliably predict in vivo RBA that is determined in an established animal model. Previous studies correlating soil As IVBA with RBA have been limited by the use of few soil types as the source of As. Furthermore, the predictive value of As IVBA assays has not been validated using an independent set of As-contaminated soils. Therefore, the current study was undertaken to develop a robust linear model to predict As RBA in mice using an IVBA assay and to independently validate the predictive capability of this assay using a unique set of As-contaminated soils. Thirty-six As-contaminated soils varying in soil type, As contaminant source, and As concentration were included in this study, with 27 soils used for initial model development and nine soils used for independent model validation. The initial model reliably predicted As RBA values in the independent data set, with a mean As RBA prediction error of 5.3% (range 2.4 to 8.4%). Following validation, all 36 soils were used for final model development, resulting in a linear model with the equation: RBA = 0.59 * IVBA + 9.8 and R2 of 0.78. The in vivo-in vitro correlation and independent data validation presented here provide

Cross-validation of oxygen uptake prediction during walking in ambulatory persons with multiple sclerosis.

PubMed

Agiovlasitis, Stamatis; Motl, Robert W

2016-01-01

An equation for predicting the gross oxygen uptake (gross-VO2) during walking for persons with multiple sclerosis (MS) has been developed. Predictors included walking speed and total score from the 12-Item Multiple Sclerosis Walking Scale (MSWS-12). This study examined the validity of this prediction equation in another sample of persons with MS. Participants were 18 persons with MS with limited mobility problems (42 ± 13 years; 14 women). Participants completed the MSWS-12. Gross-VO2 was measured with open-circuit spirometry during treadmill walking at 2.0, 3.0, and 4.0 mph (0.89, 1.34, and 1.79 m·s(-1)). Absolute percent error was small: 8.3 ± 6.1% , 8.0 ± 5.6% , and 12.2 ± 9.0% at 2.0, 3.0, and 4.0 mph, respectively. Actual gross-VO2 did not differ significantly from predicted gross-VO2 at 2.0 and 3.0 mph, but was significantly higher than predicted gross-VO2 at 4.0 mph (p <  0.001). Bland-Altman plots indicated nearly zero mean difference between actual and predicted gross-VO2 with modest 95% confidence intervals at 2.0 and 3.0 mph, but there was some underestimation at 4.0 mph. Speed and MSWS-12 score provide valid prediction of gross-VO2 during treadmill walking at slow and moderate speeds in ambulatory persons with MS. However, there is a possibility of small underestimation for walking at 4.0 mph.

Validity of the Medical College Admission Test for Predicting MD-PhD Student Outcomes

ERIC Educational Resources Information Center

Bills, James L.; VanHouten, Jacob; Grundy, Michelle M.; Chalkley, Roger; Dermody, Terence S.

2016-01-01

The Medical College Admission Test (MCAT) is a quantitative metric used by MD and MD-PhD programs to evaluate applicants for admission. This study assessed the validity of the MCAT in predicting training performance measures and career outcomes for MD-PhD students at a single institution. The study population consisted of 153 graduates of the…

Incremental Validity of the WJ III COG: Limited Predictive Effects beyond the GIA-E

ERIC Educational Resources Information Center

McGill, Ryan J.; Busse, R. T.

2015-01-01

This study is an examination of the incremental validity of Cattell-Horn-Carroll (CHC) broad clusters from the Woodcock-Johnson III Tests of Cognitive Abilities (WJ III COG) for predicting scores on the Woodcock-Johnson III Tests of Achievement (WJ III ACH). The participants were children and adolescents, ages 6-18 (n = 4,722), drawn from the WJ…

Development and Validation of Decision Forest Model for Estrogen Receptor Binding Prediction of Chemicals Using Large Data Sets.

PubMed

Ng, Hui Wen; Doughty, Stephen W; Luo, Heng; Ye, Hao; Ge, Weigong; Tong, Weida; Hong, Huixiao

2015-12-21

Some chemicals in the environment possess the potential to interact with the endocrine system in the human body. Multiple receptors are involved in the endocrine system; estrogen receptor α (ERα) plays very important roles in endocrine activity and is the most studied receptor. Understanding and predicting estrogenic activity of chemicals facilitates the evaluation of their endocrine activity. Hence, we have developed a decision forest classification model to predict chemical binding to ERα using a large training data set of 3308 chemicals obtained from the U.S. Food and Drug Administration's Estrogenic Activity Database. We tested the model using cross validations and external data sets of 1641 chemicals obtained from the U.S. Environmental Protection Agency's ToxCast project. The model showed good performance in both internal (92% accuracy) and external validations (∼ 70-89% relative balanced accuracies), where the latter involved the validations of the model across different ER pathway-related assays in ToxCast. The important features that contribute to the prediction ability of the model were identified through informative descriptor analysis and were related to current knowledge of ER binding. Prediction confidence analysis revealed that the model had both high prediction confidence and accuracy for most predicted chemicals. The results demonstrated that the model constructed based on the large training data set is more accurate and robust for predicting ER binding of chemicals than the published models that have been developed using much smaller data sets. The model could be useful for the evaluation of ERα-mediated endocrine activity potential of environmental chemicals.

Establishment and validation of the scoring system for preoperative prediction of central lymph node metastasis in papillary thyroid carcinoma.

PubMed

Liu, Wen; Cheng, Ruochuan; Ma, Yunhai; Wang, Dan; Su, Yanjun; Diao, Chang; Zhang, Jianming; Qian, Jun; Liu, Jin

2018-05-03

Early preoperative diagnosis of central lymph node metastasis (CNM) is crucial to improve survival rates among patients with papillary thyroid carcinoma (PTC). Here, we analyzed clinical data from 2862 PTC patients and developed a scoring system using multivariable logistic regression and testified by the validation group. The predictive diagnostic effectiveness of the scoring system was evaluated based on consistency, discrimination ability, and accuracy. The scoring system considered seven variables: gender, age, tumor size, microcalcification, resistance index >0.7, multiple nodular lesions, and extrathyroid extension. The area under the receiver operating characteristic curve (AUC) was 0.742, indicating a good discrimination. Using 5 points as a diagnostic threshold, the validation results for validation group had an AUC of 0.758, indicating good discrimination and consistency in the scoring system. The sensitivity of this predictive model for preoperative diagnosis of CNM was 4 times higher than a direct ultrasound diagnosis. These data indicate that the CNM prediction model would improve preoperative diagnostic sensitivity for CNM in patients with papillary thyroid carcinoma.

Tone Noise Predictions for a Spacecraft Cabin Ventilation Fan Ingesting Distorted Inflow and the Challenges of Validation

NASA Technical Reports Server (NTRS)

Koch, L. Danielle; Shook, Tony D.; Astler, Douglas T.; Bittinger, Samantha A.

2011-01-01

A fan tone noise prediction code has been developed at NASA Glenn Research Center that is capable of estimating duct mode sound power levels for a fan ingesting distorted inflow. This code was used to predict the circumferential and radial mode sound power levels in the inlet and exhaust duct of an axial spacecraft cabin ventilation fan. Noise predictions at fan design rotational speed were generated. Three fan inflow conditions were studied: an undistorted inflow, a circumferentially symmetric inflow distortion pattern (cylindrical rods inserted radially into the flowpath at 15deg, 135deg, and 255deg), and a circumferentially asymmetric inflow distortion pattern (rods located at 15deg, 52deg and 173deg). Noise predictions indicate that tones are produced for the distorted inflow cases that are not present when the fan operates with an undistorted inflow. Experimental data are needed to validate these acoustic predictions, as well as the aerodynamic performance predictions. Given the aerodynamic design of the spacecraft cabin ventilation fan, a mechanical and electrical conceptual design study was conducted. Design features of a fan suitable for obtaining detailed acoustic and aerodynamic measurements needed to validate predictions are discussed.

Tone Noise Predictions for a Spacecraft Cabin Ventilation Fan Ingesting Distorted Inflow and the Challenges of Validation

NASA Technical Reports Server (NTRS)

Koch, L. Danielle; Shook, Tony D.; Astler, Douglas T.; Bittinger, Samantha A.

2012-01-01

A fan tone noise prediction code has been developed at NASA Glenn Research Center that is capable of estimating duct mode sound power levels for a fan ingesting distorted inflow. This code was used to predict the circumferential and radial mode sound power levels in the inlet and exhaust duct of an axial spacecraft cabin ventilation fan. Noise predictions at fan design rotational speed were generated. Three fan inflow conditions were studied: an undistorted inflow, a circumferentially symmetric inflow distortion pattern (cylindrical rods inserted radially into the flowpath at 15deg, 135deg, and 255deg), and a circumferentially asymmetric inflow distortion pattern (rods located at 15deg, 52deg and 173deg). Noise predictions indicate that tones are produced for the distorted inflow cases that are not present when the fan operates with an undistorted inflow. Experimental data are needed to validate these acoustic predictions, as well as the aerodynamic performance predictions. Given the aerodynamic design of the spacecraft cabin ventilation fan, a mechanical and electrical conceptual design study was conducted. Design features of a fan suitable for obtaining detailed acoustic and aerodynamic measurements needed to validate predictions are discussed.

Validation of the procedures. [integrated multidisciplinary optimization of rotorcraft

NASA Technical Reports Server (NTRS)

Mantay, Wayne R.

1989-01-01

Validation strategies are described for procedures aimed at improving the rotor blade design process through a multidisciplinary optimization approach. Validation of the basic rotor environment prediction tools and the overall rotor design are discussed.

Assessing the reliability, predictive and construct validity of historical, clinical and risk management-20 (HCR-20) in Mexican psychiatric inpatients.

PubMed

Sada, Andrea; Robles-García, Rebeca; Martínez-López, Nicolás; Hernández-Ramírez, Rafael; Tovilla-Zarate, Carlos-Alfonso; López-Munguía, Fernando; Suárez-Alvarez, Enrique; Ayala, Xochitl; Fresán, Ana

2016-08-01

Assessing dangerousness to gauge the likelihood of future violent behaviour has become an integral part of clinical mental health practice in forensic and non-forensic psychiatric settings, one of the most effective instruments for this being the Historical, Clinical and Risk Management-20 (HCR-20). To examine the HCR-20 factor structure in Mexican psychiatric inpatients and to obtain its predictive validity and reliability for use in this population. In total, 225 patients diagnosed with psychotic, affective or personality disorders were included. The HCR-20 was applied at hospital admission and violent behaviours were assessed during psychiatric hospitalization using the Overt Aggression Scale (OAS). Construct validity, predictive validity and internal consistency were determined. Violent behaviour remains more severe in patients classified in the high-risk group during hospitalization. Fifteen items displayed adequate communalities in the original designated domains of the HCR-20 and internal consistency of the instruments was high. The HCR-20 is a suitable instrument for predicting violence risk in Mexican psychiatric inpatients.

Development and validation of a clinical prediction rule to identify suspected breast cancer: a prospective cohort study.

PubMed

Galvin, Rose; Joyce, Doireann; Downey, Eithne; Boland, Fiona; Fahey, Tom; Hill, Arnold K

2014-10-03

The number of primary care referrals of women with breast symptoms to symptomatic breast units (SBUs) has increased exponentially in the past decade in Ireland. The aim of this study is to develop and validate a clinical prediction rule (CPR) to identify women with breast cancer so that a more evidence based approach to referral from primary care to these SBUs can be developed. We analysed routine data from a prospective cohort of consecutive women reviewed at a SBU with breast symptoms. The dataset was split into a derivation and validation cohort. Regression analysis was used to derive a CPR from the patient's history and clinical findings. Validation of the CPR consisted of estimating the number of breast cancers predicted to occur compared with the actual number of observed breast cancers across deciles of risk. A total of 6,590 patients were included in the derivation study and 4.9% were diagnosed with breast cancer. Independent clinical predictors for breast cancer were: increasing age by year (adjusted odds ratio 1.08, 95% CI 1.07-1.09); presence of a lump (5.63, 95% CI 4.2-7.56); nipple change (2.77, 95% CI 1.68-4.58) and nipple discharge (2.09, 95% CI 1.1-3.97). Validation of the rule (n = 911) demonstrated that the probability of breast cancer was higher with an increasing number of these independent variables. The Hosmer-Lemeshow goodness of fit showed no overall significant difference between the expected and the observed numbers of breast cancer (χ(2)HL: 6.74, p-value: 0.56). This study derived and validated a CPR for breast cancer in women attending an Irish national SBU. We found that increasing age, presence of a lump, nipple discharge and nipple change are all associated with increased risk of breast cancer. Further validation of the rule is necessary as well as an assessment of its impact on referral practice.

OVERFLOW Validation for Predicting Plume Impingement of Underexpanded Axisymmetric Jets onto Angled Flat Plates

NASA Technical Reports Server (NTRS)

Lee, Henry C.; Klopfer, Goetz

2011-01-01

This report documents how OVERFLOW, a computational fluid dynamics code, predicts plume impingement of underexpanded axisymmetric jets onto both perpendicular and inclined flat plates. The effects of the plume impinging on a range of plate inclinations varying from 90deg to 30deg are investigated and compared to the experimental results in Reference 1 and 2. The flow fields are extremely complex due to the interaction between the shock waves from the free jet and those deflected by the plate. Additionally, complex mixing effects create very intricate structures in the flow. The experimental data is very limited, so these validation studies will focus only on cold plume impingement on flat and inclined plates. This validation study will help quantify the error in the OVERFLOW simulation when applied to stage separation scenarios.

Adolescent Domain Screening Inventory-Short Form: Development and Initial Validation

ERIC Educational Resources Information Center

Corrigan, Matthew J.

2017-01-01

This study sought to develop a short version of the ADSI, and investigate its psychometric properties. Methods: This is a secondary analysis. Analysis to determine the Cronbach's Alpha, correlations to determine concurrent criterion validity and known instrument validity and a logistic regression to determine predictive validity were conducted.…

Initial Reliability and Validity of the Perceived Social Competence Scale

ERIC Educational Resources Information Center

Anderson-Butcher, Dawn; Iachini, Aidyn L.; Amorose, Anthony J.

2008-01-01

Objective: This study describes the development and validation of a perceived social competence scale that social workers can easily use to assess children's and youth's social competence. Method: Exploratory and confirmatory factor analyses were conducted on a calibration and a cross-validation sample of youth. Predictive validity was also…

Predictive Validity of Curriculum-Embedded Measures on Outcomes of Kindergarteners Identified as At Risk for Reading Difficulty

ERIC Educational Resources Information Center

Oslund, Eric L.; Hagan-Burke, Shanna; Simmons, Deborah C.; Clemens, Nathan H.; Simmons, Leslie E.; Taylor, Aaron B.; Kwok, Oi-man; Coyne, Michael D.

2017-01-01

This study examined the predictive validity of formative assessments embedded in a Tier 2 intervention curriculum for kindergarten students identified as at risk for reading difficulty. We examined when (i.e., months during the school year) measures could predict reading outcomes gathered at the end of kindergarten and whether the predictive…

Predictive Validity of Early Literacy Measures for Korean English Language Learners in the United States

ERIC Educational Resources Information Center

Han, Jeanie Nam; Vanderwood, Michael L.; Lee, Catherine Y.

2015-01-01

This study examined the predictive validity of early literacy measures with first-grade Korean English language learners (ELLs) in the United States at varying levels of English proficiency. Participants were screened using Dynamic Indicators of Basic Early Literacy Skills (DIBELS) Phoneme Segmentation Fluency (PSF), DIBELS Nonsense Word Fluency…

Predictive Validity of a National Examination for Medical Graduates in the People's Republic of China.

ERIC Educational Resources Information Center

Bingxiun, Liu; And Others

1990-01-01

To estimate the predictive validity of the Chinese National Medical Examination, scores of a sample (n=1,717) of participating examinees were compared with program directors' ratings on nine aspects of clinical competence. Test scores were consistent with competence measures and overall, correlated significantly with ratings, while varying for…

Validity of the Optometry Admission Test in Predicting Performance in Schools and Colleges of Optometry.

ERIC Educational Resources Information Center

Kramer, Gene A.; Johnston, JoElle

1997-01-01

A study examined the relationship between Optometry Admission Test scores and pre-optometry or undergraduate grade point average (GPA) with first and second year performance in optometry schools. The test's predictive validity was limited but significant, and comparable to those reported for other admission tests. In addition, the scores…

Validity of the MMPI Personality Disorder scales (MMPI-PD).

PubMed

Schuler, C E; Snibbe, J R; Buckwalter, J G

1994-03-01

The MMPI Personality Disorder scales, developed by Morey, Waugh, and Blashfield (1985), were validated on an inpatient population by comparing 104 patients' MMPI-PD scores with the MCMI and with DSM-III-R diagnosis. Conservative significance levels were used to ensure more valid conclusions. Schizoid, Avoidant, Dependent, Histrionic, and Narcissistic scales were correlated significantly. Passive-Aggressive, Schizotypal, and Borderline scales did not correlate with corresponding MCMI scales. The MMPI-PD nonoverlapping scales were most effective in predicting diagnosis, specifically the Personality Disorder NOS, Eccentric and Borderline groups. The overlapping scales were not as effective in predicting diagnosis, but best predicted the Eccentric and Borderline groups. This study provides support for the validity of specific scales and circumscribed diagnostic utility for both measures.

Multisite external validation of a risk prediction model for the diagnosis of blood stream infections in febrile pediatric oncology patients without severe neutropenia.

PubMed

Esbenshade, Adam J; Zhao, Zhiguo; Aftandilian, Catherine; Saab, Raya; Wattier, Rachel L; Beauchemin, Melissa; Miller, Tamara P; Wilkes, Jennifer J; Kelly, Michael J; Fernbach, Alison; Jeng, Michael; Schwartz, Cindy L; Dvorak, Christopher C; Shyr, Yu; Moons, Karl G M; Sulis, Maria-Luisa; Friedman, Debra L

2017-10-01

Pediatric oncology patients are at an increased risk of invasive bacterial infection due to immunosuppression. The risk of such infection in the absence of severe neutropenia (absolute neutrophil count ≥ 500/μL) is not well established and a validated prediction model for blood stream infection (BSI) risk offers clinical usefulness. A 6-site retrospective external validation was conducted using a previously published risk prediction model for BSI in febrile pediatric oncology patients without severe neutropenia: the Esbenshade/Vanderbilt (EsVan) model. A reduced model (EsVan2) excluding 2 less clinically reliable variables also was created using the initial EsVan model derivative cohort, and was validated using all 5 external validation cohorts. One data set was used only in sensitivity analyses due to missing some variables. From the 5 primary data sets, there were a total of 1197 febrile episodes and 76 episodes of bacteremia. The overall C statistic for predicting bacteremia was 0.695, with a calibration slope of 0.50 for the original model and a calibration slope of 1.0 when recalibration was applied to the model. The model performed better in predicting high-risk bacteremia (gram-negative or Staphylococcus aureus infection) versus BSI alone, with a C statistic of 0.801 and a calibration slope of 0.65. The EsVan2 model outperformed the EsVan model across data sets with a C statistic of 0.733 for predicting BSI and a C statistic of 0.841 for high-risk BSI. The results of this external validation demonstrated that the EsVan and EsVan2 models are able to predict BSI across multiple performance sites and, once validated and implemented prospectively, could assist in decision making in clinical practice. Cancer 2017;123:3781-3790. © 2017 American Cancer Society. © 2017 American Cancer Society.

Self-perceived Coparenting of Nonresident Fathers: Scale Development and Validation.

PubMed

Dyer, W Justin; Fagan, Jay; Kaufman, Rebecca; Pearson, Jessica; Cabrera, Natasha

2017-11-16

This study reports on the development and validation of the Fatherhood Research and Practice Network coparenting perceptions scale for nonresident fathers. Although other measures of coparenting have been developed, this is the first measure developed specifically for low-income, nonresident fathers. Focus groups were conducted to determine various aspects of coparenting. Based on this, a scale was created and administered to 542 nonresident fathers. Participants also responded to items used to examine convergent and predictive validity (i.e., parental responsibility, contact with the mother, father self-efficacy and satisfaction, child behavior problems, and contact and engagement with the child). Factor analyses and reliability tests revealed three distinct and reliable perceived coparenting factors: undermining, alliance, and gatekeeping. Validity tests suggest substantial overlap between the undermining and alliance factors, though undermining was uniquely related to child behavior problems. The alliance and gatekeeping factors showed strong convergent validity and evidence for predictive validity. Taken together, results suggest this relatively short measure (11 items) taps into three coparenting dimensions significantly predictive of aspects of individual and family life. © 2017 Family Process Institute.

Flow unit modeling and fine-scale predicted permeability validation in Atokan sandstones: Norcan East Kansas

USGS Publications Warehouse

Bhattacharya, S.; Byrnes, A.P.; Watney, W.L.; Doveton, J.H.

2008-01-01

Characterizing the reservoir interval into flow units is an effective way to subdivide the net-pay zone into layers for reservoir simulation. Commonly used flow unit identification techniques require a reliable estimate of permeability in the net pay on a foot-by-foot basis. Most of the wells do not have cores, and the literature is replete with different kinds of correlations, transforms, and prediction methods for profiling permeability in pay. However, for robust flow unit determination, predicted permeability at noncored wells requires validation and, if necessary, refinement. This study outlines the use o f a spreadsheet-based permeability validation technique to characterize flow units in wells from the Norcan East field, Clark County, Kansas, that produce from Atokan aged fine- to very fine-grained quartzarenite sandstones interpreted to have been deposited in brackish-water, tidally dominated restricted tidal-flat, tidal-channel, tidal-bar, and estuary bay environments within a small incised-valley-fill system. The methodology outlined enables the identification of fieldwide free-water level and validates and refines predicted permeability at 0.5-ft (0.15-m) intervals by iteratively reconciling differences in water saturation calculated from wire-line log and a capillary-pressure formulation that models fine- to very fine-grained sandstone with diagenetic clay and silt or shale laminae. The effectiveness of this methodology was confirmed by successfully matching primary and secondary production histories using a flow unit-based reservoir model of the Norcan East field without permeability modifications. The methodologies discussed should prove useful for robust flow unit characterization of different kinds of reservoirs. Copyright ?? 2008. The American Association of Petroleum Geologists. All rights reserved.

Developing and validating a model to predict the success of an IHCS implementation: the Readiness for Implementation Model.

PubMed

Wen, Kuang-Yi; Gustafson, David H; Hawkins, Robert P; Brennan, Patricia F; Dinauer, Susan; Johnson, Pauley R; Siegler, Tracy

2010-01-01

To develop and validate the Readiness for Implementation Model (RIM). This model predicts a healthcare organization's potential for success in implementing an interactive health communication system (IHCS). The model consists of seven weighted factors, with each factor containing five to seven elements. Two decision-analytic approaches, self-explicated and conjoint analysis, were used to measure the weights of the RIM with a sample of 410 experts. The RIM model with weights was then validated in a prospective study of 25 IHCS implementation cases. Orthogonal main effects design was used to develop 700 conjoint-analysis profiles, which varied on seven factors. Each of the 410 experts rated the importance and desirability of the factors and their levels, as well as a set of 10 different profiles. For the prospective 25-case validation, three time-repeated measures of the RIM scores were collected for comparison with the implementation outcomes. Two of the seven factors, 'organizational motivation' and 'meeting user needs,' were found to be most important in predicting implementation readiness. No statistically significant difference was found in the predictive validity of the two approaches (self-explicated and conjoint analysis). The RIM was a better predictor for the 1-year implementation outcome than the half-year outcome. The expert sample, the order of the survey tasks, the additive model, and basing the RIM cut-off score on experience are possible limitations of the study. The RIM needs to be empirically evaluated in institutions adopting IHCS and sustaining the system in the long term.

External validation of the diffuse intrinsic pontine glioma survival prediction model: a collaborative report from the International DIPG Registry and the SIOPE DIPG Registry.

PubMed

Veldhuijzen van Zanten, Sophie E M; Lane, Adam; Heymans, Martijn W; Baugh, Joshua; Chaney, Brooklyn; Hoffman, Lindsey M; Doughman, Renee; Jansen, Marc H A; Sanchez, Esther; Vandertop, William P; Kaspers, Gertjan J L; van Vuurden, Dannis G; Fouladi, Maryam; Jones, Blaise V; Leach, James

2017-08-01

We aimed to perform external validation of the recently developed survival prediction model for diffuse intrinsic pontine glioma (DIPG), and discuss its utility. The DIPG survival prediction model was developed in a cohort of patients from the Netherlands, United Kingdom and Germany, registered in the SIOPE DIPG Registry, and includes age <3 years, longer symptom duration and receipt of chemotherapy as favorable predictors, and presence of ring-enhancement on MRI as unfavorable predictor. Model performance was evaluated by analyzing the discrimination and calibration abilities. External validation was performed using an unselected cohort from the International DIPG Registry, including patients from United States, Canada, Australia and New Zealand. Basic comparison with the results of the original study was performed using descriptive statistics, and univariate- and multivariable regression analyses in the validation cohort. External validation was assessed following a variety of analyses described previously. Baseline patient characteristics and results from the regression analyses were largely comparable. Kaplan-Meier curves of the validation cohort reproduced separated groups of standard (n = 39), intermediate (n = 125), and high-risk (n = 78) patients. This discriminative ability was confirmed by similar values for the hazard ratios across these risk groups. The calibration curve in the validation cohort showed a symmetric underestimation of the predicted survival probabilities. In this external validation study, we demonstrate that the DIPG survival prediction model has acceptable cross-cohort calibration and is able to discriminate patients with short, average, and increased survival. We discuss how this clinico-radiological model may serve a useful role in current clinical practice.

Development and validation of a preoperative prediction model for colorectal cancer T-staging based on MDCT images and clinical information.

PubMed

Sa, Sha; Li, Jing; Li, Xiaodong; Li, Yongrui; Liu, Xiaoming; Wang, Defeng; Zhang, Huimao; Fu, Yu

2017-08-15

This study aimed to establish and evaluate the efficacy of a prediction model for colorectal cancer T-staging. T-staging was positively correlated with the level of carcinoembryonic antigen (CEA), expression of carbohydrate antigen 19-9 (CA19-9), wall deformity, blurred outer edges, fat infiltration, infiltration into the surrounding tissue, tumor size and wall thickness. Age, location, enhancement rate and enhancement homogeneity were negatively correlated with T-staging. The predictive results of the model were consistent with the pathological gold standard, and the kappa value was 0.805. The total accuracy of staging improved from 51.04% to 86.98% with the proposed model. The clinical, imaging and pathological data of 611 patients with colorectal cancer (419 patients in the training group and 192 patients in the validation group) were collected. A spearman correlation analysis was used to validate the relationship among these factors and pathological T-staging. A prediction model was trained with the random forest algorithm. T staging of the patients in the validation group was predicted by both prediction model and traditional method. The consistency, accuracy, sensitivity, specificity and area under the curve (AUC) were used to compare the efficacy of the two methods. The newly established comprehensive model can improve the predictive efficiency of preoperative colorectal cancer T-staging.

STR-validator: an open source platform for validation and process control.

PubMed

Hansson, Oskar; Gill, Peter; Egeland, Thore

2014-11-01

This paper addresses two problems faced when short tandem repeat (STR) systems are validated for forensic purposes: (1) validation is extremely time consuming and expensive, and (2) there is strong consensus about what to validate but not how. The first problem is solved by powerful data processing functions to automate calculations. Utilising an easy-to-use graphical user interface, strvalidator (hereafter referred to as STR-validator) can greatly increase the speed of validation. The second problem is exemplified by a series of analyses, and subsequent comparison with published material, highlighting the need for a common validation platform. If adopted by the forensic community STR-validator has the potential to standardise the analysis of validation data. This would not only facilitate information exchange but also increase the pace at which laboratories are able to switch to new technology. Copyright © 2014 Elsevier Ireland Ltd. All rights reserved.

Development and Validation of a Multidisciplinary Tool for Accurate and Efficient Rotorcraft Noise Prediction (MUTE)

NASA Technical Reports Server (NTRS)

Liu, Yi; Anusonti-Inthra, Phuriwat; Diskin, Boris

2011-01-01

A physics-based, systematically coupled, multidisciplinary prediction tool (MUTE) for rotorcraft noise was developed and validated with a wide range of flight configurations and conditions. MUTE is an aggregation of multidisciplinary computational tools that accurately and efficiently model the physics of the source of rotorcraft noise, and predict the noise at far-field observer locations. It uses systematic coupling approaches among multiple disciplines including Computational Fluid Dynamics (CFD), Computational Structural Dynamics (CSD), and high fidelity acoustics. Within MUTE, advanced high-order CFD tools are used around the rotor blade to predict the transonic flow (shock wave) effects, which generate the high-speed impulsive noise. Predictions of the blade-vortex interaction noise in low speed flight are also improved by using the Particle Vortex Transport Method (PVTM), which preserves the wake flow details required for blade/wake and fuselage/wake interactions. The accuracy of the source noise prediction is further improved by utilizing a coupling approach between CFD and CSD, so that the effects of key structural dynamics, elastic blade deformations, and trim solutions are correctly represented in the analysis. The blade loading information and/or the flow field parameters around the rotor blade predicted by the CFD/CSD coupling approach are used to predict the acoustic signatures at far-field observer locations with a high-fidelity noise propagation code (WOPWOP3). The predicted results from the MUTE tool for rotor blade aerodynamic loading and far-field acoustic signatures are compared and validated with a variation of experimental data sets, such as UH60-A data, DNW test data and HART II test data.

Behavioral Indicators on a Mobile Sensing Platform Predict Clinically Validated Psychiatric Symptoms of Mood and Anxiety Disorders

PubMed Central

Place, Skyler; Rubin, Channah; Gorrostieta, Cristina; Mead, Caroline; Kane, John; Marx, Brian P; Feast, Joshua; Deckersbach, Thilo; Pentland, Alex “Sandy”; Nierenberg, Andrew; Azarbayejani, Ali

2017-01-01

Background There is a critical need for real-time tracking of behavioral indicators of mental disorders. Mobile sensing platforms that objectively and noninvasively collect, store, and analyze behavioral indicators have not yet been clinically validated or scalable. Objective The aim of our study was to report on models of clinical symptoms for post-traumatic stress disorder (PTSD) and depression derived from a scalable mobile sensing platform. Methods A total of 73 participants (67% [49/73] male, 48% [35/73] non-Hispanic white, 33% [24/73] veteran status) who reported at least one symptom of PTSD or depression completed a 12-week field trial. Behavioral indicators were collected through the noninvasive mobile sensing platform on participants’ mobile phones. Clinical symptoms were measured through validated clinical interviews with a licensed clinical social worker. A combination hypothesis and data-driven approach was used to derive key features for modeling symptoms, including the sum of outgoing calls, count of unique numbers texted, absolute distance traveled, dynamic variation of the voice, speaking rate, and voice quality. Participants also reported ease of use and data sharing concerns. Results Behavioral indicators predicted clinically assessed symptoms of depression and PTSD (cross-validated area under the curve [AUC] for depressed mood=.74, fatigue=.56, interest in activities=.75, and social connectedness=.83). Participants reported comfort sharing individual data with physicians (Mean 3.08, SD 1.22), mental health providers (Mean 3.25, SD 1.39), and medical researchers (Mean 3.03, SD 1.36). Conclusions Behavioral indicators passively collected through a mobile sensing platform predicted symptoms of depression and PTSD. The use of mobile sensing platforms can provide clinically validated behavioral indicators in real time; however, further validation of these models and this platform in large clinical samples is needed. PMID:28302595

Development and validation of a prognostic score to predict mortality in patients with acute-on-chronic liver failure.

PubMed

Jalan, Rajiv; Saliba, Faouzi; Pavesi, Marco; Amoros, Alex; Moreau, Richard; Ginès, Pere; Levesque, Eric; Durand, Francois; Angeli, Paolo; Caraceni, Paolo; Hopf, Corinna; Alessandria, Carlo; Rodriguez, Ezequiel; Solis-Muñoz, Pablo; Laleman, Wim; Trebicka, Jonel; Zeuzem, Stefan; Gustot, Thierry; Mookerjee, Rajeshwar; Elkrief, Laure; Soriano, German; Cordoba, Joan; Morando, Filippo; Gerbes, Alexander; Agarwal, Banwari; Samuel, Didier; Bernardi, Mauro; Arroyo, Vicente

2014-11-01

Acute-on-chronic liver failure (ACLF) is a frequent syndrome (30% prevalence), characterized by acute decompensation of cirrhosis, organ failure(s) and high short-term mortality. This study develops and validates a specific prognostic score for ACLF patients. Data from 1349 patients included in the CANONIC study were used. First, a simplified organ function scoring system (CLIF Consortium Organ Failure score, CLIF-C OFs) was developed to diagnose ACLF using data from all patients. Subsequently, in 275 patients with ACLF, CLIF-C OFs and two other independent predictors of mortality (age and white blood cell count) were combined to develop a specific prognostic score for ACLF (CLIF Consortium ACLF score [CLIF-C ACLFs]). A concordance index (C-index) was used to compare the discrimination abilities of CLIF-C ACLF, MELD, MELD-sodium (MELD-Na), and Child-Pugh (CPs) scores. The CLIF-C ACLFs was validated in an external cohort and assessed for sequential use. The CLIF-C ACLFs showed a significantly higher predictive accuracy than MELDs, MELD-Nas, and CPs, reducing (19-28%) the corresponding prediction error rates at all main time points after ACLF diagnosis (28, 90, 180, and 365 days) in both the CANONIC and the external validation cohort. CLIF-C ACLFs computed at 48 h, 3-7 days, and 8-15 days after ACLF diagnosis predicted the 28-day mortality significantly better than at diagnosis. The CLIF-C ACLFs at ACLF diagnosis is superior to the MELDs and MELD-Nas in predicting mortality. The CLIF-C ACLFs is a clinically relevant, validated scoring system that can be used sequentially to stratify the risk of mortality in ACLF patients. Copyright © 2014 European Association for the Study of the Liver. Published by Elsevier B.V. All rights reserved.

Construction, internal validation and implementation in a mobile application of a scoring system to predict nonadherence to proton pump inhibitors.

PubMed

Mares-García, Emma; Palazón-Bru, Antonio; Folgado-de la Rosa, David Manuel; Pereira-Expósito, Avelino; Martínez-Martín, Álvaro; Cortés-Castell, Ernesto; Gil-Guillén, Vicente Francisco

2017-01-01

Other studies have assessed nonadherence to proton pump inhibitors (PPIs), but none has developed a screening test for its detection. To construct and internally validate a predictive model for nonadherence to PPIs. This prospective observational study with a one-month follow-up was carried out in 2013 in Spain, and included 302 patients with a prescription for PPIs. The primary variable was nonadherence to PPIs (pill count). Secondary variables were gender, age, antidepressants, type of PPI, non-guideline-recommended prescription (NGRP) of PPIs, and total number of drugs. With the secondary variables, a binary logistic regression model to predict nonadherence was constructed and adapted to a points system. The ROC curve, with its area (AUC), was calculated and the optimal cut-off point was established. The points system was internally validated through 1,000 bootstrap samples and implemented in a mobile application (Android). The points system had three prognostic variables: total number of drugs, NGRP of PPIs, and antidepressants. The AUC was 0.87 (95% CI [0.83-0.91], p  < 0.001). The test yielded a sensitivity of 0.80 (95% CI [0.70-0.87]) and a specificity of 0.82 (95% CI [0.76-0.87]). The three parameters were very similar in the bootstrap validation. A points system to predict nonadherence to PPIs has been constructed, internally validated and implemented in a mobile application. Provided similar results are obtained in external validation studies, we will have a screening tool to detect nonadherence to PPIs.

Extending the validity of the Feeding Practices and Structure Questionnaire.

PubMed

Jansen, Elena; Mallan, Kimberley M; Daniels, Lynne A

2015-06-30

Feeding practices are commonly examined as potentially modifiable determinants of children's eating behaviours and weight status. Although a variety of questionnaires exist to assess different feeding aspects, many lack thorough reliability and validity testing. The Feeding Practices and Structure Questionnaire (FPSQ) is a tool designed to measure early feeding practices related to non-responsive feeding and structure of the meal environment. Face validity, factorial validity, internal reliability and cross-sectional correlations with children's eating behaviours have been established in mothers with 2-year-old children. The aim of the present study was to further extend the validity of the FPSQ by examining factorial, construct and predictive validity, and stability. Participants were from the NOURISH randomised controlled trial which evaluated an intervention with first-time mothers designed to promote protective feeding practices. Maternal feeding practices (FP) and child eating behaviours were assessed when children were aged 2 years and 3.7 years (n = 388). Confirmatory Factor analysis, group differences, predictive relationships, and stability were tested. The original 9-factor structure was confirmed when children were aged 3.7 ± 0.3 years. Cronbach's alpha was above the recommended 0.70 cut-off for all factors except Structured Meal Timing, Over Restriction and Distrust in Appetite which were 0.58, 0.67 and 0.66 respectively. Allocated group differences reflected behaviour consistent with intervention content and all feeding practices were stable across both time points (range of r = 0.45-0.70). There was some evidence for the predictive validity of factors with 2 FP showing expected relationships, 2 FP showing expected and unexpected relationships and 5 FP showing no relationship. Reliability and validity was demonstrated for most subscales of the FPSQ. Future validation is warranted with culturally diverse samples and with fathers and

Factors associated with therapeutic inertia in hypertension: validation of a predictive model.

PubMed

Redón, Josep; Coca, Antonio; Lázaro, Pablo; Aguilar, Ma Dolores; Cabañas, Mercedes; Gil, Natividad; Sánchez-Zamorano, Miguel Angel; Aranda, Pedro

2010-08-01

To study factors associated with therapeutic inertia in treating hypertension and to develop a predictive model to estimate the probability of therapeutic inertia in a given medical consultation, based on variables related to the consultation, patient, physician, clinical characteristics, and level of care. National, multicentre, observational, cross-sectional study in primary care and specialist (hospital) physicians who each completed a questionnaire on therapeutic inertia, provided professional data and collected clinical data on four patients. Therapeutic inertia was defined as a consultation in which treatment change was indicated (i.e., SBP >or= 140 or DBP >or= 90 mmHg in all patients; SBP >or= 130 or DBP >or= 80 in patients with diabetes or stroke), but did not occur. A predictive model was constructed and validated according to the factors associated with therapeutic inertia. Data were collected on 2595 patients and 13,792 visits. Therapeutic inertia occurred in 7546 (75%) of the 10,041 consultations in which treatment change was indicated. Factors associated with therapeutic inertia were primary care setting, male sex, older age, SPB and/or DBP values close to normal, treatment with more than one antihypertensive drug, treatment with an ARB II, and more than six visits/year. Physician characteristics did not weigh heavily in the association. The predictive model was valid internally and externally, with acceptable calibration, discrimination and reproducibility, and explained one-third of the variability in therapeutic inertia. Although therapeutic inertia is frequent in the management of hypertension, the factors explaining it are not completely clear. Whereas some aspects of the consultations were associated with therapeutic inertia, physician characteristics were not a decisive factor.

The Predictive Validity of a Computer-Adaptive Assessment of Kindergarten and First-Grade Reading Skills

ERIC Educational Resources Information Center

Clemens, Nathan H.; Hagan-Burke, Shanna; Luo, Wen; Cerda, Carissa; Blakely, Alane; Frosch, Jennifer; Gamez-Patience, Brenda; Jones, Meredith

2015-01-01

This study examined the predictive validity of a computer-adaptive assessment for measuring kindergarten reading skills using the STAR Early Literacy (SEL) test. The findings showed that the results of SEL assessments administered during the fall, winter, and spring of kindergarten were moderate and statistically significant predictors of year-end…

Growth of finiteness in the third year of life: replication and predictive validity.

PubMed

Hadley, Pamela A; Rispoli, Matthew; Holt, Janet K; Fitzgerald, Colleen; Bahnsen, Alison

2014-06-01

The authors of this study investigated the validity of tense and agreement productivity (TAP) scoring in diverse sentence frames obtained during conversational language sampling as an alternative measure of finiteness for use with young children. Longitudinal language samples were used to model TAP growth from 21 to 30 months of age for 37 typically developing toddlers. Empirical Bayes (EB) linear and quadratic growth coefficients and child sex were then used to predict elicited grammar composite scores on the Test of Early Grammatical Impairment (TEGI; Rice & Wexler, 2001) at 36 months. A random-effects quadratic model with no intercept best characterized TAP growth, replicating the findings of Rispoli, Hadley, and Holt (2009). The combined regression model was significant, with the 3 variables accounting for 55.5% of the variance in the TEGI composite scores. These findings establish TAP growth as a valid metric of finiteness in the 3rd year of life. Developmental and theoretical implications are discussed.

Predicting umbilical artery pH during labour: Development and validation of a nomogram using fetal heart rate patterns.

PubMed

Ramanah, Rajeev; Omar, Sikiyah; Guillien, Alicia; Pugin, Aurore; Martin, Alain; Riethmuller, Didier; Mottet, Nicolas

2018-06-01

Nomograms are statistical models that combine variables to obtain the most accurate and reliable prediction for a particular risk. Fetal heart rate (FHR) interpretation alone has been found to be poorly predictive for fetal acidosis while other clinical risk factors exist. The aim of this study was to create and validate a nomogram based on FHR patterns and relevant clinical parameters to provide a non-invasive individualized prediction of umbilical artery pH during labour. A retrospective observational study was conducted on 4071 patients in labour presenting singleton pregnancies at >34 gestational weeks and delivering vaginally. Clinical characteristics, FHR patterns and umbilical cord gas of 1913 patients were used to construct a nomogram predicting an umbilical artery (Ua) pH <7.18 (10th centile of the study population) after an univariate and multivariate stepwise logistic regression analysis. External validation was obtained from an independent cohort of 2158 patients. Area under the receiver operating characteristics (ROC) curve, sensitivity, specificity, positive and negative predictive values of the nomogram were determined. Upon multivariate analysis, parity (p < 0.01), induction of labour (p = 0.01), a prior uterine scar (p = 0.02), maternal fever (p = 0.02) and the type of FHR (p < 0.01) were significantly associated with an Ua pH <7.18 (p < 0.05). Apgar score at 1, 5 and 10 min were significantly lower in the group with an Ua pH <7.18 (p < 0.01). The nomogram constructed had a Concordance Index of 0.75 (area under the curve) with a sensitivity of 57%, a specificity of 91%, a negative predictive value of 5% and a positive predictive value of 99%. Calibration found no difference between the predicted probabilities and the observed rate of Ua pH <7.18 (p = 0.63). The validation set had a Concordance Index of 0.72 and calibration with a p < 0.77. We successfully developed and validated a nomogram to predict Ua pH by

Design and validation of a model to predict early mortality in haemodialysis patients.

PubMed

Mauri, Joan M; Clèries, Montse; Vela, Emili

2008-05-01

Mortality and morbidity rates are higher in patients receiving haemodialysis therapy than in the general population. Detection of risk factors related to early death in these patients could be of aid for clinical and administrative decision making. Objectives. The aims of this study were (1) to identify risk factors (comorbidity and variables specific to haemodialysis) associated with death in the first year following the start of haemodialysis and (2) to design and validate a prognostic model to quantify the probability of death for each patient. An analysis was carried out on all patients starting haemodialysis treatment in Catalonia during the period 1997-2003 (n = 5738). The data source was the Renal Registry of Catalonia, a mandatory population registry. Patients were randomly divided into two samples: 60% (n = 3455) of the total were used to develop the prognostic model and the remaining 40% (n = 2283) to validate the model. Logistic regression analysis was used to construct the model. One-year mortality in the total study population was 16.5%. The predictive model included the following variables: age, sex, primary renal disease, grade of functional autonomy, chronic obstructive pulmonary disease, malignant processes, chronic liver disease, cardiovascular disease, initial vascular access and malnutrition. The analyses showed adequate calibration for both the sample to develop the model and the validation sample (Hosmer-Lemeshow statistic 0.97 and P = 0.49, respectively) as well as adequate discrimination (ROC curve 0.78 in both cases). Risk factors implicated in mortality at one year following the start of haemodialysis have been determined and a prognostic model designed. The validated, easy-to-apply model quantifies individual patient risk attributable to various factors, some of them amenable to correction by directed interventions.

The SIST-M: Predictive validity of a brief structured Clinical Dementia Rating interview

PubMed Central

Okereke, Olivia I.; Pantoja-Galicia, Norberto; Copeland, Maura; Hyman, Bradley T.; Wanggaard, Taylor; Albert, Marilyn S.; Betensky, Rebecca A.; Blacker, Deborah

2011-01-01

Background We previously established reliability and cross-sectional validity of the SIST-M (Structured Interview and Scoring Tool–Massachusetts Alzheimer's Disease Research Center), a shortened version of an instrument shown to predict progression to Alzheimer disease (AD), even among persons with very mild cognitive impairment (vMCI). Objective To test predictive validity of the SIST-M. Methods Participants were 342 community-dwelling, non-demented older adults in a longitudinal study. Baseline Clinical Dementia Rating (CDR) ratings were determined by either: 1) clinician interviews or 2) a previously developed computer algorithm based on 60 questions (of a possible 131) extracted from clinician interviews. We developed age+gender+education-adjusted Cox proportional hazards models using CDR-sum-of-boxes (CDR-SB) as the predictor, where CDR-SB was determined by either clinician interview or algorithm; models were run for the full sample (n=342) and among those jointly classified as vMCI using clinician- and algorithm-based CDR ratings (n=156). We directly compared predictive accuracy using time-dependent Receiver Operating Characteristic (ROC) curves. Results AD hazard ratios (HRs) were similar for clinician-based and algorithm-based CDR-SB: for a 1-point increment in CDR-SB, respective HRs (95% CI)=3.1 (2.5,3.9) and 2.8 (2.2,3.5); among those with vMCI, respective HRs (95% CI) were 2.2 (1.6,3.2) and 2.1 (1.5,3.0). Similarly high predictive accuracy was achieved: the concordance probability (weighted average of the area-under-the-ROC curves) over follow-up was 0.78 vs. 0.76 using clinician-based vs. algorithm-based CDR-SB. Conclusion CDR scores based on items from this shortened interview had high predictive ability for AD – comparable to that using a lengthy clinical interview. PMID:21986342

Reconceptualising the external validity of discrete choice experiments.

PubMed

Lancsar, Emily; Swait, Joffre

2014-10-01

External validity is a crucial but under-researched topic when considering using discrete choice experiment (DCE) results to inform decision making in clinical, commercial or policy contexts. We present the theory and tests traditionally used to explore external validity that focus on a comparison of final outcomes and review how this traditional definition has been empirically tested in health economics and other sectors (such as transport, environment and marketing) in which DCE methods are applied. While an important component, we argue that the investigation of external validity should be much broader than a comparison of final outcomes. In doing so, we introduce a new and more comprehensive conceptualisation of external validity, closely linked to process validity, that moves us from the simple characterisation of a model as being or not being externally valid on the basis of predictive performance, to the concept that external validity should be an objective pursued from the initial conceptualisation and design of any DCE. We discuss how such a broader definition of external validity can be fruitfully used and suggest innovative ways in which it can be explored in practice.

Predictive validity of the UK clinical aptitude test in the final years of medical school: a prospective cohort study.

PubMed

Husbands, Adrian; Mathieson, Alistair; Dowell, Jonathan; Cleland, Jennifer; MacKenzie, Rhoda

2014-04-23

The UK Clinical Aptitude Test (UKCAT) was designed to address issues identified with traditional methods of selection. This study aims to examine the predictive validity of the UKCAT and compare this to traditional selection methods in the senior years of medical school. This was a follow-up study of two cohorts of students from two medical schools who had previously taken part in a study examining the predictive validity of the UKCAT in first year. The sample consisted of 4th and 5th Year students who commenced their studies at the University of Aberdeen or University of Dundee medical schools in 2007. Data collected were: demographics (gender and age group), UKCAT scores; Universities and Colleges Admissions Service (UCAS) form scores; admission interview scores; Year 4 and 5 degree examination scores. Pearson's correlations were used to examine the relationships between admissions variables, examination scores, gender and age group, and to select variables for multiple linear regression analysis to predict examination scores. Ninety-nine and 89 students at Aberdeen medical school from Years 4 and 5 respectively, and 51 Year 4 students in Dundee, were included in the analysis. Neither UCAS form nor interview scores were statistically significant predictors of examination performance. Conversely, the UKCAT yielded statistically significant validity coefficients between .24 and .36 in four of five assessments investigated. Multiple regression analysis showed the UKCAT made a statistically significant unique contribution to variance in examination performance in the senior years. Results suggest the UKCAT appears to predict performance better in the later years of medical school compared to earlier years and provides modest supportive evidence for the UKCAT's role in student selection within these institutions. Further research is needed to assess the predictive validity of the UKCAT against professional and behavioural outcomes as the cohort commences working life.

Predictive validity of the UK clinical aptitude test in the final years of medical school: a prospective cohort study

PubMed Central

2014-01-01

Background The UK Clinical Aptitude Test (UKCAT) was designed to address issues identified with traditional methods of selection. This study aims to examine the predictive validity of the UKCAT and compare this to traditional selection methods in the senior years of medical school. This was a follow-up study of two cohorts of students from two medical schools who had previously taken part in a study examining the predictive validity of the UKCAT in first year. Methods The sample consisted of 4th and 5th Year students who commenced their studies at the University of Aberdeen or University of Dundee medical schools in 2007. Data collected were: demographics (gender and age group), UKCAT scores; Universities and Colleges Admissions Service (UCAS) form scores; admission interview scores; Year 4 and 5 degree examination scores. Pearson’s correlations were used to examine the relationships between admissions variables, examination scores, gender and age group, and to select variables for multiple linear regression analysis to predict examination scores. Results Ninety-nine and 89 students at Aberdeen medical school from Years 4 and 5 respectively, and 51 Year 4 students in Dundee, were included in the analysis. Neither UCAS form nor interview scores were statistically significant predictors of examination performance. Conversely, the UKCAT yielded statistically significant validity coefficients between .24 and .36 in four of five assessments investigated. Multiple regression analysis showed the UKCAT made a statistically significant unique contribution to variance in examination performance in the senior years. Conclusions Results suggest the UKCAT appears to predict performance better in the later years of medical school compared to earlier years and provides modest supportive evidence for the UKCAT’s role in student selection within these institutions. Further research is needed to assess the predictive validity of the UKCAT against professional and behavioural

Validation of a single-stage fixed-rate step test for the prediction of maximal oxygen uptake in healthy adults.

PubMed

Hansen, Dominique; Jacobs, Nele; Thijs, Herbert; Dendale, Paul; Claes, Neree

2016-09-01

Healthcare professionals with limited access to ergospirometry remain in need of valid and simple submaximal exercise tests to predict maximal oxygen uptake (VO2max ). Despite previous validation studies concerning fixed-rate step tests, accurate equations for the estimation of VO2max remain to be formulated from a large sample of healthy adults between age 18-75 years (n > 100). The aim of this study was to develop a valid equation to estimate VO2max from a fixed-rate step test in a larger sample of healthy adults. A maximal ergospirometry test, with assessment of cardiopulmonary parameters and VO2max , and a 5-min fixed-rate single-stage step test were executed in 112 healthy adults (age 18-75 years). During the step test and subsequent recovery, heart rate was monitored continuously. By linear regression analysis, an equation to predict VO2max from the step test was formulated. This equation was assessed for level of agreement by displaying Bland-Altman plots and calculation of intraclass correlations with measured VO2max . Validity further was assessed by employing a Jackknife procedure. The linear regression analysis generated the following equation to predict VO2max (l min(-1) ) from the step test: 0·054(BMI)+0·612(gender)+3·359(body height in m)+0·019(fitness index)-0·012(HRmax)-0·011(age)-3·475. This equation explained 78% of the variance in measured VO2max (F = 66·15, P<0·001). The level of agreement and intraclass correlation was high (ICC = 0·94, P<0·001) between measured and predicted VO2max . From this study, a valid fixed-rate single-stage step test equation has been developed to estimate VO2max in healthy adults. This tool could be employed by healthcare professionals with limited access to ergospirometry. © 2015 Scandinavian Society of Clinical Physiology and Nuclear Medicine. Published by John Wiley & Sons Ltd.

Development and Validation of a Predictive Model for Functional Outcome After Stroke Rehabilitation: The Maugeri Model.

PubMed

Scrutinio, Domenico; Lanzillo, Bernardo; Guida, Pietro; Mastropasqua, Filippo; Monitillo, Vincenzo; Pusineri, Monica; Formica, Roberto; Russo, Giovanna; Guarnaschelli, Caterina; Ferretti, Chiara; Calabrese, Gianluigi

2017-12-01

Prediction of outcome after stroke rehabilitation may help clinicians in decision-making and planning rehabilitation care. We developed and validated a predictive tool to estimate the probability of achieving improvement in physical functioning (model 1) and a level of independence requiring no more than supervision (model 2) after stroke rehabilitation. The models were derived from 717 patients admitted for stroke rehabilitation. We used multivariable logistic regression analysis to build each model. Then, each model was prospectively validated in 875 patients. Model 1 included age, time from stroke occurrence to rehabilitation admission, admission motor and cognitive Functional Independence Measure scores, and neglect. Model 2 included age, male gender, time since stroke onset, and admission motor and cognitive Functional Independence Measure score. Both models demonstrated excellent discrimination. In the derivation cohort, the area under the curve was 0.883 (95% confidence intervals, 0.858-0.910) for model 1 and 0.913 (95% confidence intervals, 0.884-0.942) for model 2. The Hosmer-Lemeshow χ 2 was 4.12 ( P =0.249) and 1.20 ( P =0.754), respectively. In the validation cohort, the area under the curve was 0.866 (95% confidence intervals, 0.840-0.892) for model 1 and 0.850 (95% confidence intervals, 0.815-0.885) for model 2. The Hosmer-Lemeshow χ 2 was 8.86 ( P =0.115) and 34.50 ( P =0.001), respectively. Both improvement in physical functioning (hazard ratios, 0.43; 0.25-0.71; P =0.001) and a level of independence requiring no more than supervision (hazard ratios, 0.32; 0.14-0.68; P =0.004) were independently associated with improved 4-year survival. A calculator is freely available for download at https://goo.gl/fEAp81. This study provides researchers and clinicians with an easy-to-use, accurate, and validated predictive tool for potential application in rehabilitation research and stroke management. © 2017 American Heart Association, Inc.

Systematic review and retrospective validation of prediction models for weight loss after bariatric surgery.

PubMed

Sharples, Alistair J; Mahawar, Kamal; Cheruvu, Chandra V N

2017-11-01

Patients often have less than realistic expectations of the weight loss they are likely to achieve after bariatric surgery. It would be useful to have a well-validated prediction tool that could give patients a realistic estimate of their expected weight loss. To perform a systematic review of the literature to identify existing prediction models and attempt to validate these models. University hospital, United Kingdom. A systematic review was performed. All English language studies were included if they used data to create a prediction model for postoperative weight loss after bariatric surgery. These models were then tested on patients undergoing bariatric surgery between January 1, 2013 and December 31, 2014 within our unit. An initial literature search produced 446 results, of which only 4 were included in the final review. Our study population included 317 patients. Mean preoperative body mass index was 46.1 ± 7.1. For 257 (81.1%) patients, 12-month follow-up was available, and mean body mass index and percentage excess weight loss at 12 months was 33.0 ± 6.7 and 66.1% ± 23.7%, respectively. All 4 of the prediction models significantly overestimated the amount of weight loss achieved by patients. The best performing prediction model in our series produced a correlation coefficient (R 2 ) of .61 and an area under the curve of .71 on receiver operating curve analysis. All prediction models overestimated weight loss after bariatric surgery in our cohort. There is a need to develop better procedures and patient-specific models for better patient counselling. Copyright © 2017 American Society for Bariatric Surgery. Published by Elsevier Inc. All rights reserved.

In silico target prediction for elucidating the mode of action of herbicides including prospective validation.

PubMed

Chiddarwar, Rucha K; Rohrer, Sebastian G; Wolf, Antje; Tresch, Stefan; Wollenhaupt, Sabrina; Bender, Andreas

2017-01-01

The rapid emergence of pesticide resistance has given rise to a demand for herbicides with new mode of action (MoA). In the agrochemical sector, with the availability of experimental high throughput screening (HTS) data, it is now possible to utilize in silico target prediction methods in the early discovery phase to suggest the MoA of a compound via data mining of bioactivity data. While having been established in the pharmaceutical context, in the agrochemical area this approach poses rather different challenges, as we have found in this work, partially due to different chemistry, but even more so due to different (usually smaller) amounts of data, and different ways of conducting HTS. With the aim to apply computational methods for facilitating herbicide target identification, 48,000 bioactivity data against 16 herbicide targets were processed to train Laplacian modified Naïve Bayesian (NB) classification models. The herbicide target prediction model ("HerbiMod") is an ensemble of 16 binary classification models which are evaluated by internal, external and prospective validation sets. In addition to the experimental inactives, 10,000 random agrochemical inactives were included in the training process, which showed to improve the overall balanced accuracy of our models up to 40%. For all the models, performance in terms of balanced accuracy of≥80% was achieved in five-fold cross validation. Ranking target predictions was addressed by means of z-scores which improved predictivity over using raw scores alone. An external testset of 247 compounds from ChEMBL and a prospective testset of 394 compounds from BASF SE tested against five well studied herbicide targets (ACC, ALS, HPPD, PDS and PROTOX) were used for further validation. Only 4% of the compounds in the external testset lied in the applicability domain and extrapolation (and correct prediction) was hence impossible, which on one hand was surprising, and on the other hand illustrated the utilization of

Toward Supersonic Retropropulsion CFD Validation

NASA Technical Reports Server (NTRS)

Kleb, Bil; Schauerhamer, D. Guy; Trumble, Kerry; Sozer, Emre; Barnhardt, Michael; Carlson, Jan-Renee; Edquist, Karl

2011-01-01

This paper begins the process of verifying and validating computational fluid dynamics (CFD) codes for supersonic retropropulsive flows. Four CFD codes (DPLR, FUN3D, OVERFLOW, and US3D) are used to perform various numerical and physical modeling studies toward the goal of comparing predictions with a wind tunnel experiment specifically designed to support CFD validation. Numerical studies run the gamut in rigor from code-to-code comparisons to observed order-of-accuracy tests. Results indicate that this complex flowfield, involving time-dependent shocks and vortex shedding, design order of accuracy is not clearly evident. Also explored is the extent of physical modeling necessary to predict the salient flowfield features found in high-speed Schlieren images and surface pressure measurements taken during the validation experiment. Physical modeling studies include geometric items such as wind tunnel wall and sting mount interference, as well as turbulence modeling that ranges from a RANS (Reynolds-Averaged Navier-Stokes) 2-equation model to DES (Detached Eddy Simulation) models. These studies indicate that tunnel wall interference is minimal for the cases investigated; model mounting hardware effects are confined to the aft end of the model; and sparse grid resolution and turbulence modeling can damp or entirely dissipate the unsteadiness of this self-excited flow.

External Validation of a Tool Predicting 7-Year Risk of Developing Cardiovascular Disease, Type 2 Diabetes or Chronic Kidney Disease.

PubMed

Rauh, Simone P; Rutters, Femke; van der Heijden, Amber A W A; Luimes, Thomas; Alssema, Marjan; Heymans, Martijn W; Magliano, Dianna J; Shaw, Jonathan E; Beulens, Joline W; Dekker, Jacqueline M

2018-02-01

Chronic cardiometabolic diseases, including cardiovascular disease (CVD), type 2 diabetes (T2D) and chronic kidney disease (CKD), share many modifiable risk factors and can be prevented using combined prevention programs. Valid risk prediction tools are needed to accurately identify individuals at risk. We aimed to validate a previously developed non-invasive risk prediction tool for predicting the combined 7-year-risk for chronic cardiometabolic diseases. The previously developed tool is stratified for sex and contains the predictors age, BMI, waist circumference, use of antihypertensives, smoking, family history of myocardial infarction/stroke, and family history of diabetes. This tool was externally validated, evaluating model performance using area under the receiver operating characteristic curve (AUC)-assessing discrimination-and Hosmer-Lemeshow goodness-of-fit (HL) statistics-assessing calibration. The intercept was recalibrated to improve calibration performance. The risk prediction tool was validated in 3544 participants from the Australian Diabetes, Obesity and Lifestyle Study (AusDiab). Discrimination was acceptable, with an AUC of 0.78 (95% CI 0.75-0.81) in men and 0.78 (95% CI 0.74-0.81) in women. Calibration was poor (HL statistic: p < 0.001), but improved considerably after intercept recalibration. Examination of individual outcomes showed that in men, AUC was highest for CKD (0.85 [95% CI 0.78-0.91]) and lowest for T2D (0.69 [95% CI 0.65-0.74]). In women, AUC was highest for CVD (0.88 [95% CI 0.83-0.94)]) and lowest for T2D (0.71 [95% CI 0.66-0.75]). Validation of our previously developed tool showed robust discriminative performance across populations. Model recalibration is recommended to account for different disease rates. Our risk prediction tool can be useful in large-scale prevention programs for identifying those in need of further risk profiling because of their increased risk for chronic cardiometabolic diseases.

Predicting medical complications after spine surgery: a validated model using a prospective surgical registry.

PubMed

Lee, Michael J; Cizik, Amy M; Hamilton, Deven; Chapman, Jens R

2014-02-01

The possibility and likelihood of a postoperative medical complication after spine surgery undoubtedly play a major role in the decision making of the surgeon and patient alike. Although prior study has determined relative risk and odds ratio values to quantify risk factors, these values may be difficult to translate to the patient during counseling of surgical options. Ideally, a model that predicts absolute risk of medical complication, rather than relative risk or odds ratio values, would greatly enhance the discussion of safety of spine surgery. To date, there is no risk stratification model that specifically predicts the risk of medical complication. The purpose of this study was to create and validate a predictive model for the risk of medical complication during and after spine surgery. Statistical analysis using a prospective surgical spine registry that recorded extensive demographic, surgical, and complication data. Outcomes examined are medical complications that were specifically defined a priori. This analysis is a continuation of statistical analysis of our previously published report. Using a prospectively collected surgical registry of more than 1,476 patients with extensive demographic, comorbidity, surgical, and complication detail recorded for 2 years after surgery, we previously identified several risk factor for medical complications. Using the beta coefficients from those log binomial regression analyses, we created a model to predict the occurrence of medical complication after spine surgery. We split our data into two subsets for internal and cross-validation of our model. We created two predictive models: one predicting the occurrence of any medical complication and the other predicting the occurrence of a major medical complication. The final predictive model for any medical complications had a receiver operator curve characteristic of 0.76, considered to be a fair measure. The final predictive model for any major medical complications had

Predictive Validity of the HKT-R Risk Assessment Tool: Two and 5-Year Violent Recidivism in a Nationwide Sample of Dutch Forensic Psychiatric Patients.

PubMed

Bogaerts, Stefan; Spreen, Marinus; Ter Horst, Paul; Gerlsma, Coby

2018-06-01

This study has examined the predictive validity of the Historical Clinical Future [ Historisch Klinisch Toekomst] Revised risk assessment scheme in a cohort of 347 forensic psychiatric patients, which were discharged between 2004 and 2008 from any of 12 highly secure forensic centers in the Netherlands. Predictive validity was measured 2 and 5 years after release. Official reconviction data obtained from the Dutch Ministry of Security and Justice were used as outcome measures. Violent reoffending within 2 and 5 years after discharge was assessed. With regard to violent reoffending, results indicated that the predictive validity of the Historical domain was modest for 2 (area under the curve [AUC] = .75) and 5 (AUC = .74) years. The predictive validity of the Clinical domain was marginal for 2 (admission: AUC = .62; discharge: AUC = .63) and 5 (admission: AUC = .69; discharge: AUC = .62) years after release. The predictive validity of the Future domain was modest (AUC = .71) for 2 years and low for 5 (AUC = .58) years. The total score of the instrument was modest for 2 years (AUC = .78) and marginal for 5 (AUC = .68) years. Finally, the Final Risk Judgment was modest for 2 years (AUC = .78) and marginal for 5 (AUC = .63) years time at risk. It is concluded that this risk assessment instrument appears to be a satisfactory instrument for risk assessment.

Predicting hospital stay, mortality and readmission in people admitted for hypoglycaemia: prognostic models derivation and validation.

PubMed

Zaccardi, Francesco; Webb, David R; Davies, Melanie J; Dhalwani, Nafeesa N; Gray, Laura J; Chatterjee, Sudesna; Housley, Gemma; Shaw, Dominick; Hatton, James W; Khunti, Kamlesh

2017-06-01

Hospital admissions for hypoglycaemia represent a significant burden on individuals with diabetes and have a substantial economic impact on healthcare systems. To date, no prognostic models have been developed to predict outcomes following admission for hypoglycaemia. We aimed to develop and validate prediction models to estimate risk of inpatient death, 24 h discharge and one month readmission in people admitted to hospital for hypoglycaemia. We used the Hospital Episode Statistics database, which includes data on all hospital admission to National Health Service hospital trusts in England, to extract admissions for hypoglycaemia between 2010 and 2014. We developed, internally and temporally validated, and compared two prognostic risk models for each outcome. The first model included age, sex, ethnicity, region, social deprivation and Charlson score ('base' model). In the second model, we added to the 'base' model the 20 most common medical conditions and applied a stepwise backward selection of variables ('disease' model). We used C-index and calibration plots to assess model performance and developed a calculator to estimate probabilities of outcomes according to individual characteristics. In derivation samples, 296 out of 11,136 admissions resulted in inpatient death, 1789/33,825 in one month readmission and 8396/33,803 in 24 h discharge. Corresponding values for validation samples were: 296/10,976, 1207/22,112 and 5363/22,107. The two models had similar discrimination. In derivation samples, C-indices for the base and disease models, respectively, were: 0.77 (95% CI 0.75, 0.80) and 0.78 (0.75, 0.80) for death, 0.57 (0.56, 0.59) and 0.57 (0.56, 0.58) for one month readmission, and 0.68 (0.67, 0.69) and 0.69 (0.68, 0.69) for 24 h discharge. Corresponding values in validation samples were: 0.74 (0.71, 0.76) and 0.74 (0.72, 0.77), 0.55 (0.54, 0.57) and 0.55 (0.53, 0.56), and 0.66 (0.65, 0.67) and 0.67 (0.66, 0.68). In both derivation and validation samples

Validation of a multifactorial risk factor model used for predicting future caries risk with Nevada adolescents.

PubMed

Ditmyer, Marcia M; Dounis, Georgia; Howard, Katherine M; Mobley, Connie; Cappelli, David

2011-05-20

The objective of this study was to measure the validity and reliability of a multifactorial Risk Factor Model developed for use in predicting future caries risk in Nevada adolescents in a public health setting. This study examined retrospective data from an oral health surveillance initiative that screened over 51,000 students 13-18 years of age, attending public/private schools in Nevada across six academic years (2002/2003-2007/2008). The Risk Factor Model included ten demographic variables: exposure to fluoridation in the municipal water supply, environmental smoke exposure, race, age, locale (metropolitan vs. rural), tobacco use, Body Mass Index, insurance status, sex, and sealant application. Multiple regression was used in a previous study to establish which significantly contributed to caries risk. Follow-up logistic regression ascertained the weight of contribution and odds ratios of the ten variables. Researchers in this study computed sensitivity, specificity, positive predictive value (PVP), negative predictive value (PVN), and prevalence across all six years of screening to assess the validity of the Risk Factor Model. Subjects' overall mean caries prevalence across all six years was 66%. Average sensitivity across all six years was 79%; average specificity was 81%; average PVP was 89% and average PVN was 67%. Overall, the Risk Factor Model provided a relatively constant, valid measure of caries that could be used in conjunction with a comprehensive risk assessment in population-based screenings by school nurses/nurse practitioners, health educators, and physicians to guide them in assessing potential future caries risk for use in prevention and referral practices.

Derivation and validation of simple anthropometric equations to predict adipose tissue mass and total fat mass with MRI as the reference method

PubMed Central

Al-Gindan, Yasmin Y.; Hankey, Catherine R.; Govan, Lindsay; Gallagher, Dympna; Heymsfield, Steven B.; Lean, Michael E. J.

2017-01-01

The reference organ-level body composition measurement method is MRI. Practical estimations of total adipose tissue mass (TATM), total adipose tissue fat mass (TATFM) and total body fat are valuable for epidemiology, but validated prediction equations based on MRI are not currently available. We aimed to derive and validate new anthropometric equations to estimate MRI-measured TATM/TATFM/total body fat and compare them with existing prediction equations using older methods. The derivation sample included 416 participants (222 women), aged between 18 and 88 years with BMI between 15·9 and 40·8 (kg/m2). The validation sample included 204 participants (110 women), aged between 18 and 86 years with BMI between 15·7 and 36·4 (kg/m2). Both samples included mixed ethnic/racial groups. All the participants underwent whole-body MRI to quantify TATM (dependent variable) and anthropometry (independent variables). Prediction equations developed using stepwise multiple regression were further investigated for agreement and bias before validation in separate data sets. Simplest equations with optimal R2 and Bland–Altman plots demonstrated good agreement without bias in the validation analyses: men: TATM (kg) = 0·198 weight (kg) + 0·478 waist (cm) − 0·147 height (cm) − 12·8 (validation: R2 0·79, CV = 20 %, standard error of the estimate (SEE)=3·8 kg) and women: TATM (kg)=0·789 weight (kg) + 0·0786 age (years) − 0·342 height (cm) + 24·5 (validation: R2 0·84, CV = 13 %, SEE = 3·0 kg). Published anthropometric prediction equations, based on MRI and computed tomographic scans, correlated strongly with MRI-measured TATM: (R2 0·70 – 0·82). Estimated TATFM correlated well with published prediction equations for total body fat based on underwater weighing (R2 0·70–0·80), with mean bias of 2·5–4·9 kg, correctable with log-transformation in most equations. In conclusion, new equations, using simple anthropometric measurements, estimated MRI-measured TATM

Evaluating the predictive accuracy and the clinical benefit of a nomogram aimed to predict survival in node-positive prostate cancer patients: External validation on a multi-institutional database.

PubMed

Bianchi, Lorenzo; Schiavina, Riccardo; Borghesi, Marco; Bianchi, Federico Mineo; Briganti, Alberto; Carini, Marco; Terrone, Carlo; Mottrie, Alex; Gacci, Mauro; Gontero, Paolo; Imbimbo, Ciro; Marchioro, Giansilvio; Milanese, Giulio; Mirone, Vincenzo; Montorsi, Francesco; Morgia, Giuseppe; Novara, Giacomo; Porreca, Angelo; Volpe, Alessandro; Brunocilla, Eugenio

2018-04-06

To assess the predictive accuracy and the clinical value of a recent nomogram predicting cancer-specific mortality-free survival after surgery in pN1 prostate cancer patients through an external validation. We evaluated 518 prostate cancer patients treated with radical prostatectomy and pelvic lymph node dissection with evidence of nodal metastases at final pathology, at 10 tertiary centers. External validation was carried out using regression coefficients of the previously published nomogram. The performance characteristics of the model were assessed by quantifying predictive accuracy, according to the area under the curve in the receiver operating characteristic curve and model calibration. Furthermore, we systematically analyzed the specificity, sensitivity, positive predictive value and negative predictive value for each nomogram-derived probability cut-off. Finally, we implemented decision curve analysis, in order to quantify the nomogram's clinical value in routine practice. External validation showed inferior predictive accuracy as referred to in the internal validation (65.8% vs 83.3%, respectively). The discrimination (area under the curve) of the multivariable model was 66.7% (95% CI 60.1-73.0%) by testing with receiver operating characteristic curve analysis. The calibration plot showed an overestimation throughout the range of predicted cancer-specific mortality-free survival rates probabilities. However, in decision curve analysis, the nomogram's use showed a net benefit when compared with the scenarios of treating all patients or none. In an external setting, the nomogram showed inferior predictive accuracy and suboptimal calibration characteristics as compared to that reported in the original population. However, decision curve analysis showed a clinical net benefit, suggesting a clinical implication to correctly manage pN1 prostate cancer patients after surgery. © 2018 The Japanese Urological Association.

Predicting Hemorrhagic Transformation of Acute Ischemic Stroke: Prospective Validation of the HeRS Score.

PubMed

Marsh, Elisabeth B; Llinas, Rafael H; Schneider, Andrea L C; Hillis, Argye E; Lawrence, Erin; Dziedzic, Peter; Gottesman, Rebecca F

2016-01-01

Hemorrhagic transformation (HT) increases the morbidity and mortality of ischemic stroke. Anticoagulation is often indicated in patients with atrial fibrillation, low ejection fraction, or mechanical valves who are hospitalized with acute stroke, but increases the risk of HT. Risk quantification would be useful. Prior studies have investigated risk of systemic hemorrhage in anticoagulated patients, but none looked specifically at HT. In our previously published work, age, infarct volume, and estimated glomerular filtration rate (eGFR) significantly predicted HT. We created the hemorrhage risk stratification (HeRS) score based on regression coefficients in multivariable modeling and now determine its validity in a prospectively followed inpatient cohort.A total of 241 consecutive patients presenting to 2 academic stroke centers with acute ischemic stroke and an indication for anticoagulation over a 2.75-year period were included. Neuroimaging was evaluated for infarct volume and HT. Hemorrhages were classified as symptomatic versus asymptomatic, and by severity. HeRS scores were calculated for each patient and compared to actual hemorrhage status using receiver operating curve analysis.Area under the curve (AUC) comparing predicted odds of hemorrhage (HeRS score) to actual hemorrhage status was 0.701. Serum glucose (P < 0.001), white blood cell count (P < 0.001), and warfarin use prior to admission (P = 0.002) were also associated with HT in the validation cohort. With these variables, AUC improved to 0.854. Anticoagulation did not significantly increase HT; but with higher intensity anticoagulation, hemorrhages were more likely to be symptomatic and more severe.The HeRS score is a valid predictor of HT in patients with ischemic stroke and indication for anticoagulation.

Real external predictivity of QSAR models: how to evaluate it? Comparison of different validation criteria and proposal of using the concordance correlation coefficient.

PubMed

Chirico, Nicola; Gramatica, Paola

2011-09-26

The main utility of QSAR models is their ability to predict activities/properties for new chemicals, and this external prediction ability is evaluated by means of various validation criteria. As a measure for such evaluation the OECD guidelines have proposed the predictive squared correlation coefficient Q(2)(F1) (Shi et al.). However, other validation criteria have been proposed by other authors: the Golbraikh-Tropsha method, r(2)(m) (Roy), Q(2)(F2) (Schüürmann et al.), Q(2)(F3) (Consonni et al.). In QSAR studies these measures are usually in accordance, though this is not always the case, thus doubts can arise when contradictory results are obtained. It is likely that none of the aforementioned criteria is the best in every situation, so a comparative study using simulated data sets is proposed here, using threshold values suggested by the proponents or those widely used in QSAR modeling. In addition, a different and simple external validation measure, the concordance correlation coefficient (CCC), is proposed and compared with other criteria. Huge data sets were used to study the general behavior of validation measures, and the concordance correlation coefficient was shown to be the most restrictive. On using simulated data sets of a more realistic size, it was found that CCC was broadly in agreement, about 96% of the time, with other validation measures in accepting models as predictive, and in almost all the examples it was the most precautionary. The proposed concordance correlation coefficient also works well on real data sets, where it seems to be more stable, and helps in making decisions when the validation measures are in conflict. Since it is conceptually simple, and given its stability and restrictiveness, we propose the concordance correlation coefficient as a complementary, or alternative, more prudent measure of a QSAR model to be externally predictive.

A Formal Approach to Empirical Dynamic Model Optimization and Validation

NASA Technical Reports Server (NTRS)

Crespo, Luis G; Morelli, Eugene A.; Kenny, Sean P.; Giesy, Daniel P.

2014-01-01

A framework was developed for the optimization and validation of empirical dynamic models subject to an arbitrary set of validation criteria. The validation requirements imposed upon the model, which may involve several sets of input-output data and arbitrary specifications in time and frequency domains, are used to determine if model predictions are within admissible error limits. The parameters of the empirical model are estimated by finding the parameter realization for which the smallest of the margins of requirement compliance is as large as possible. The uncertainty in the value of this estimate is characterized by studying the set of model parameters yielding predictions that comply with all the requirements. Strategies are presented for bounding this set, studying its dependence on admissible prediction error set by the analyst, and evaluating the sensitivity of the model predictions to parameter variations. This information is instrumental in characterizing uncertainty models used for evaluating the dynamic model at operating conditions differing from those used for its identification and validation. A practical example based on the short period dynamics of the F-16 is used for illustration.

Predicting return to work after low back injury using the Psychosocial Risk for Occupational Disability Instrument: a validation study.

PubMed

Schultz, I Z; Crook, J; Berkowitz, J; Milner, R; Meloche, G R

2005-09-01

This paper reports on the predictive validity of a Psychosocial Risk for Occupational Disability Scale in the workers' compensation environment using a paper and pencil version of a previously validated multimethod instrument on a new, subacute sample of workers with low back pain. A cohort longitudinal study design with a randomly selected cohort off work for 4-6 weeks was applied. The questionnaire was completed by 111 eligible workers at 4-6 weeks following injury. Return to work status data at three months was obtained from 100 workers. Sixty-four workers had returned to work (RTW) and 36 had not (NRTW). Stepwise backward elimination resulted in a model with these predictors: Expectations of Recovery, SF-36 Vitality, SF-36 Mental Health, and Waddell Symptoms. The correct classification of RTW/NRTW was 79%, with sensitivity (NRTW) of 61% and specificity (RTW) of 89%. The area under the ROC curve was 84%. New evidence for predictive validity for the Psychosocial Risk-for-Disability Instrument was provided. The instrument can be useful and practical for prediction of return to work outcomes in the subacute stage after low back injury in the workers' compensation context.

Prediction of violent reoffending on release from prison: derivation and external validation of a scalable tool.

PubMed

Fazel, Seena; Chang, Zheng; Fanshawe, Thomas; Långström, Niklas; Lichtenstein, Paul; Larsson, Henrik; Mallett, Susan

2016-06-01

More than 30 million people are released from prison worldwide every year, who include a group at high risk of perpetrating interpersonal violence. Because there is considerable inconsistency and inefficiency in identifying those who would benefit from interventions to reduce this risk, we developed and validated a clinical prediction rule to determine the risk of violent offending in released prisoners. We did a cohort study of a population of released prisoners in Sweden. Through linkage of population-based registers, we developed predictive models for violent reoffending for the cohort. First, we developed a derivation model to determine the strength of prespecified, routinely obtained criminal history, sociodemographic, and clinical risk factors using multivariable Cox proportional hazard regression, and then tested them in an external validation. We measured discrimination and calibration for prediction of our primary outcome of violent reoffending at 1 and 2 years using cutoffs of 10% for 1-year risk and 20% for 2-year risk. We identified a cohort of 47 326 prisoners released in Sweden between 2001 and 2009, with 11 263 incidents of violent reoffending during this period. We developed a 14-item derivation model to predict violent reoffending and tested it in an external validation (assigning 37 100 individuals to the derivation sample and 10 226 to the validation sample). The model showed good measures of discrimination (Harrell's c-index 0·74) and calibration. For risk of violent reoffending at 1 year, sensitivity was 76% (95% CI 73-79) and specificity was 61% (95% CI 60-62). Positive and negative predictive values were 21% (95% CI 19-22) and 95% (95% CI 94-96), respectively. At 2 years, sensitivity was 67% (95% CI 64-69) and specificity was 70% (95% CI 69-72). Positive and negative predictive values were 37% (95% CI 35-39) and 89% (95% CI 88-90), respectively. Of individuals with a predicted risk of violent reoffending of 50% or more, 88% had drug

Journal Reviewer Ratings: Issues of Particularistic Bias, Agreement, and Predictive Validity within the Manuscript Review Process

ERIC Educational Resources Information Center

Vecchio, Robert P.

2006-01-01

Reviewer evaluations and recommendations for 853 manuscript submissions, over a span of 4 years, are analyzed for evidence of particularistic bias, reviewer agreement, and predictive validity for forecasting a published manuscript's citation impact. Attributes of the submitters, their affiliated institutions, and the reviewers have little…

Incremental Validity of Thinking Styles in Predicting Academic Achievements: An Experimental Study in Hypermedia Learning Environments

ERIC Educational Resources Information Center

Fan, Weiqiao; Zhang, Li-Fang; Watkins, David

2010-01-01

The study examined the incremental validity of thinking styles in predicting academic achievement after controlling for personality and achievement motivation in the hypermedia-based learning environment. Seventy-two Chinese college students from Shanghai, the People's Republic of China, took part in this instructional experiment. The…

A model of prediction and cross-validation of fat-free mass in men with motor complete spinal cord injury.

PubMed

Gorgey, Ashraf S; Dolbow, David R; Gater, David R

2012-07-01

To establish and validate prediction equations by using body weight to predict legs, trunk, and whole-body fat-free mass (FFM) in men with chronic complete spinal cord injury (SCI). Cross-sectional design. Research setting in a large medical center. Individuals with SCI (N=63) divided into prediction (n=42) and cross-validation (n=21) groups. Not applicable. Whole-body FFM and regional FFM were determined by using dual-energy x-ray absorptiometry. Body weight was measured by using a wheelchair weighing scale after subtracting the weight of the chair. Body weight predicted legs FFM (legs FFM=.09×body weight+6.1; R(2)=.25, standard error of the estimate [SEE]=3.1kg, P<.01), trunk FFM (trunk FFM=.21×body weight+8.6; R(2)=.56, SEE=3.6kg, P<.0001), and whole-body FFM (whole-body FFM=.288×body weight+26.3; R(2)=.53, SEE=5.3kg, P<.0001). The whole-body FFM(predicted) (FFM predicted from the derived equations) shared 86% of the variance in whole-body FFM(measured) (FFM measured using dual-energy x-ray absorptiometry scan) (R(2)=.86, SEE=1.8kg, P<.0001), 69% of trunk FFM(measured), and 66% of legs FFM(measured). The trunk FFM(predicted) shared 69% of the variance in trunk FFM(measured) (R(2)=.69, SEE=2.7kg, P<.0001), and legs FFM(predicted) shared 67% of the variance in legs FFM(measured) (R(2)=.67, SEE=2.8kg, P<.0001). Values of FFM did not differ between the prediction and validation groups. Body weight can be used to predict whole-body FFM and regional FFM. The predicted whole-body FFM improved the prediction of trunk FFM and legs FFM. Copyright © 2012 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.

Development and validation of a clinical risk score for predicting drug-resistant bacterial pneumonia in older Chinese patients.

PubMed

Ma, Hon Ming; Ip, Margaret; Woo, Jean; Hui, David S C

2014-05-01

Health care-associated pneumonia (HCAP) and drug-resistant bacterial pneumonia may not share identical risk factors. We have shown that bronchiectasis, recent hospitalization and severe pneumonia (confusion, blood urea level, respiratory rate, low blood pressure and 65 year old (CURB-65) score ≥ 3) were independent predictors of pneumonia caused by potentially drug-resistant (PDR) pathogens. This study aimed to develop and validate a clinical risk score for predicting drug-resistant bacterial pneumonia in older patients. We derived a risk score by assigning a weighting to each of these risk factors as follows: 14, bronchiectasis; 5, recent hospitalization; 2, severe pneumonia. A 0.5 point was defined for the presence of other risk factors for HCAP. We compared the areas under the receiver-operating characteristics curve (AUROC) of our risk score and the HCAP definition in predicting PDR pathogens in two cohorts of older patients hospitalized with non-nosocomial pneumonia. The derivation and validation cohorts consisted of 354 and 96 patients with bacterial pneumonia, respectively. PDR pathogens were isolated in 48 and 21 patients in the derivation and validation cohorts, respectively. The AUROCs of our risk score and the HCAP definition were 0.751 and 0.650, respectively, in the derivation cohort, and were 0.782 and 0.671, respectively, in the validation cohort. The differences between our risk score and the HCAP definition reached statistical significance. A score ≥ 2.5 had the best balance between sensitivity and specificity. Our risk score outperformed the HCAP definition to predict pneumonia caused by PDR pathogens. A history of bronchiectasis or recent hospitalization is the major indication of starting empirical broad-spectrum antibiotics. © 2014 Asian Pacific Society of Respirology.

Development and validation of a nomogram predicting recurrence risk in women with symptomatic urinary tract infection.

PubMed

Cai, Tommaso; Mazzoli, Sandra; Migno, Serena; Malossini, Gianni; Lanzafame, Paolo; Mereu, Liliana; Tateo, Saverio; Wagenlehner, Florian M E; Pickard, Robert S; Bartoletti, Riccardo

2014-09-01

To develop and externally validate a novel nomogram predicting recurrence risk probability at 12 months in women after an episode of urinary tract infection. The study included 768 women from Santa Maria Annunziata Hospital, Florence, Italy, affected by urinary tract infections from January 2005 to December 2009. Another 373 women with the same criteria enrolled at Santa Chiara Hospital, Trento, Italy, from January 2010 to June 2012 were used to externally validate and calibrate the nomogram. Univariate and multivariate Cox regression models tested the relationship between urinary tract infection recurrence risk, and patient clinical and laboratory characteristics. The nomogram was evaluated by calculating concordance probabilities, as well as testing calibration of predicted urinary tract infection recurrence with observed urinary tract infections. Nomogram variables included: number of partners, bowel function, type of pathogens isolated (Gram-positive/negative), hormonal status, number of previous urinary tract infection recurrences and previous treatment of asymptomatic bacteriuria. Of the original development data, 261 out of 768 women presented at least one episode of recurrence of urinary tract infection (33.9%). The nomogram had a concordance index of 0.85. The nomogram predictions were well calibrated. This model showed high discrimination accuracy and favorable calibration characteristics. In the validation group (373 women), the overall c-index was 0.83 (P = 0.003, 95% confidence interval 0.51-0.99), whereas the area under the receiver operating characteristic curve was 0.85 (95% confidence interval 0.79-0.91). The present nomogram accurately predicts the recurrence risk of urinary tract infection at 12 months, and can assist in identifying women at high risk of symptomatic recurrence that can be suitable candidates for a prophylactic strategy. © 2014 The Japanese Urological Association.

Concurrent and Predictive Validity of the Raven Progressive Matrices and the Naglieri Nonverbal Ability Test

ERIC Educational Resources Information Center

Balboni, Giulia; Naglieri, Jack A.; Cubelli, Roberto

2010-01-01

The concurrent and predictive validities of the Naglieri Nonverbal Ability Test (NNAT) and Raven's Colored Progressive Matrices (CPM) were investigated in a large group of Italian third-and fifth-grade students with different sociocultural levels evaluated at the beginning and end of the school year. CPM and NNAT scores were related to math and…

Brazilian validation of the Alberta Infant Motor Scale.

PubMed

Valentini, Nadia Cristina; Saccani, Raquel

2012-03-01

The Alberta Infant Motor Scale (AIMS) is a well-known motor assessment tool used to identify potential delays in infants' motor development. Although Brazilian researchers and practitioners have used the AIMS in laboratories and clinical settings, its translation to Portuguese and validation for the Brazilian population is yet to be investigated. This study aimed to translate and validate all AIMS items with respect to internal consistency and content, criterion, and construct validity. A cross-sectional and longitudinal design was used. A cross-cultural translation was used to generate a Brazilian-Portuguese version of the AIMS. In addition, a validation process was conducted involving 22 professionals and 766 Brazilian infants (aged 0-18 months). The results demonstrated language clarity and internal consistency for the motor criteria (motor development score, α=.90; prone, α=.85; supine, α=.92; sitting, α=.84; and standing, α=.86). The analysis also revealed high discriminative power to identify typical and atypical development (motor development score, P<.001; percentile, P=.04; classification criterion, χ(2)=6.03; P=.05). Temporal stability (P=.07) (rho=.85, P<.001) was observed, and predictive power (P<.001) was limited to the group of infants aged from 3 months to 9 months. Limited predictive validity was observed, which may have been due to the restricted time that the groups were followed longitudinally. In sum, the translated version of AIMS presented adequate validity and reliability.

Validated predictive modelling of the environmental resistome

PubMed Central

Amos, Gregory CA; Gozzard, Emma; Carter, Charlotte E; Mead, Andrew; Bowes, Mike J; Hawkey, Peter M; Zhang, Lihong; Singer, Andrew C; Gaze, William H; Wellington, Elizabeth M H

2015-01-01

Multi-drug-resistant bacteria pose a significant threat to public health. The role of the environment in the overall rise in antibiotic-resistant infections and risk to humans is largely unknown. This study aimed to evaluate drivers of antibiotic-resistance levels across the River Thames catchment, model key biotic, spatial and chemical variables and produce predictive models for future risk assessment. Sediment samples from 13 sites across the River Thames basin were taken at four time points across 2011 and 2012. Samples were analysed for class 1 integron prevalence and enumeration of third-generation cephalosporin-resistant bacteria. Class 1 integron prevalence was validated as a molecular marker of antibiotic resistance; levels of resistance showed significant geospatial and temporal variation. The main explanatory variables of resistance levels at each sample site were the number, proximity, size and type of surrounding wastewater-treatment plants. Model 1 revealed treatment plants accounted for 49.5% of the variance in resistance levels. Other contributing factors were extent of different surrounding land cover types (for example, Neutral Grassland), temporal patterns and prior rainfall; when modelling all variables the resulting model (Model 2) could explain 82.9% of variations in resistance levels in the whole catchment. Chemical analyses correlated with key indicators of treatment plant effluent and a model (Model 3) was generated based on water quality parameters (contaminant and macro- and micro-nutrient levels). Model 2 was beta tested on independent sites and explained over 78% of the variation in integron prevalence showing a significant predictive ability. We believe all models in this study are highly useful tools for informing and prioritising mitigation strategies to reduce the environmental resistome. PMID:25679532

Validated predictive modelling of the environmental resistome.

PubMed

Amos, Gregory C A; Gozzard, Emma; Carter, Charlotte E; Mead, Andrew; Bowes, Mike J; Hawkey, Peter M; Zhang, Lihong; Singer, Andrew C; Gaze, William H; Wellington, Elizabeth M H

2015-06-01

Multi-drug-resistant bacteria pose a significant threat to public health. The role of the environment in the overall rise in antibiotic-resistant infections and risk to humans is largely unknown. This study aimed to evaluate drivers of antibiotic-resistance levels across the River Thames catchment, model key biotic, spatial and chemical variables and produce predictive models for future risk assessment. Sediment samples from 13 sites across the River Thames basin were taken at four time points across 2011 and 2012. Samples were analysed for class 1 integron prevalence and enumeration of third-generation cephalosporin-resistant bacteria. Class 1 integron prevalence was validated as a molecular marker of antibiotic resistance; levels of resistance showed significant geospatial and temporal variation. The main explanatory variables of resistance levels at each sample site were the number, proximity, size and type of surrounding wastewater-treatment plants. Model 1 revealed treatment plants accounted for 49.5% of the variance in resistance levels. Other contributing factors were extent of different surrounding land cover types (for example, Neutral Grassland), temporal patterns and prior rainfall; when modelling all variables the resulting model (Model 2) could explain 82.9% of variations in resistance levels in the whole catchment. Chemical analyses correlated with key indicators of treatment plant effluent and a model (Model 3) was generated based on water quality parameters (contaminant and macro- and micro-nutrient levels). Model 2 was beta tested on independent sites and explained over 78% of the variation in integron prevalence showing a significant predictive ability. We believe all models in this study are highly useful tools for informing and prioritising mitigation strategies to reduce the environmental resistome.

Validating genetic markers of response to recombinant human growth hormone in children with growth hormone deficiency and Turner syndrome: the PREDICT validation study

PubMed Central

Stevens, Adam; Murray, Philip; Wojcik, Jerome; Raelson, John; Koledova, Ekaterina; Chatelain, Pierre

2016-01-01

Objective Single-nucleotide polymorphisms (SNPs) associated with the response to recombinant human growth hormone (r-hGH) have previously been identified in growth hormone deficiency (GHD) and Turner syndrome (TS) children in the PREDICT long-term follow-up (LTFU) study (Nbib699855). Here, we describe the PREDICT validation (VAL) study (Nbib1419249), which aimed to confirm these genetic associations. Design and methods Children with GHD (n = 293) or TS (n = 132) were recruited retrospectively from 29 sites in nine countries. All children had completed 1 year of r-hGH therapy. 48 SNPs previously identified as associated with first year growth response to r-hGH were genotyped. Regression analysis was used to assess the association between genotype and growth response using clinical/auxological variables as covariates. Further analysis was undertaken using random forest classification. Results The children were younger, and the growth response was higher in VAL study. Direct genotype analysis did not replicate what was found in the LTFU study. However, using exploratory regression models with covariates, a consistent relationship with growth response in both VAL and LTFU was shown for four genes – SOS1 and INPPL1 in GHD and ESR1 and PTPN1 in TS. The random forest analysis demonstrated that only clinical covariates were important in the prediction of growth response in mild GHD (>4 to <10 μg/L on GH stimulation test), however, in severe GHD (≤4 μg/L) several SNPs contributed (in IGF2, GRB10, FOS, IGFBP3 and GHRHR). Conclusions The PREDICT validation study supports, in an independent cohort, the association of four of 48 genetic markers with growth response to r-hGH treatment in both pre-pubertal GHD and TS children after controlling for clinical/auxological covariates. However, the contribution of these SNPs in a prediction model of first-year response is not sufficient for routine clinical use. PMID:27651465

Validating genetic markers of response to recombinant human growth hormone in children with growth hormone deficiency and Turner syndrome: the PREDICT validation study.

PubMed

Stevens, Adam; Murray, Philip; Wojcik, Jerome; Raelson, John; Koledova, Ekaterina; Chatelain, Pierre; Clayton, Peter

2016-12-01

Single-nucleotide polymorphisms (SNPs) associated with the response to recombinant human growth hormone (r-hGH) have previously been identified in growth hormone deficiency (GHD) and Turner syndrome (TS) children in the PREDICT long-term follow-up (LTFU) study (Nbib699855). Here, we describe the PREDICT validation (VAL) study (Nbib1419249), which aimed to confirm these genetic associations. Children with GHD (n = 293) or TS (n = 132) were recruited retrospectively from 29 sites in nine countries. All children had completed 1 year of r-hGH therapy. 48 SNPs previously identified as associated with first year growth response to r-hGH were genotyped. Regression analysis was used to assess the association between genotype and growth response using clinical/auxological variables as covariates. Further analysis was undertaken using random forest classification. The children were younger, and the growth response was higher in VAL study. Direct genotype analysis did not replicate what was found in the LTFU study. However, using exploratory regression models with covariates, a consistent relationship with growth response in both VAL and LTFU was shown for four genes - SOS1 and INPPL1 in GHD and ESR1 and PTPN1 in TS. The random forest analysis demonstrated that only clinical covariates were important in the prediction of growth response in mild GHD (>4 to <10 μg/L on GH stimulation test), however, in severe GHD (≤4 μg/L) several SNPs contributed (in IGF2, GRB10, FOS, IGFBP3 and GHRHR). The PREDICT validation study supports, in an independent cohort, the association of four of 48 genetic markers with growth response to r-hGH treatment in both pre-pubertal GHD and TS children after controlling for clinical/auxological covariates. However, the contribution of these SNPs in a prediction model of first-year response is not sufficient for routine clinical use. © 2016 European Society of Endocrinology.

Molecular Evolution of the Tissue-nonspecific Alkaline Phosphatase Allows Prediction and Validation of Missense Mutations Responsible for Hypophosphatasia*

PubMed Central

Silvent, Jérémie; Gasse, Barbara; Mornet, Etienne; Sire, Jean-Yves

2014-01-01

ALPL encodes the tissue nonspecific alkaline phosphatase (TNSALP), which removes phosphate groups from various substrates. Its function is essential for bone and tooth mineralization. In humans, ALPL mutations lead to hypophosphatasia, a genetic disorder characterized by defective bone and/or tooth mineralization. To date, 275 ALPL mutations have been reported to cause hypophosphatasia, of which 204 were simple missense mutations. Molecular evolutionary analysis has proved to be an efficient method to highlight residues important for the protein function and to predict or validate sensitive positions for genetic disease. Here we analyzed 58 mammalian TNSALP to identify amino acids unchanged, or only substituted by residues sharing similar properties, through 220 millions years of mammalian evolution. We found 469 sensitive positions of the 524 residues of human TNSALP, which indicates a highly constrained protein. Any substitution occurring at one of these positions is predicted to lead to hypophosphatasia. We tested the 204 missense mutations resulting in hypophosphatasia against our predictive chart, and validated 99% of them. Most sensitive positions were located in functionally important regions of TNSALP (active site, homodimeric interface, crown domain, calcium site, …). However, some important positions are located in regions, the structure and/or biological function of which are still unknown. Our chart of sensitive positions in human TNSALP (i) enables to validate or invalidate at low cost any ALPL mutation, which would be suspected to be responsible for hypophosphatasia, by contrast with time consuming and expensive functional tests, and (ii) displays higher predictive power than in silico models of prediction. PMID:25023282

Applicability Analysis of Validation Evidence for Biomedical Computational Models

DOE Office of Scientific and Technical Information (OSTI.GOV)

Pathmanathan, Pras; Gray, Richard A.; Romero, Vicente J.

Computational modeling has the potential to revolutionize medicine the way it transformed engineering. However, despite decades of work, there has only been limited progress to successfully translate modeling research to patient care. One major difficulty which often occurs with biomedical computational models is an inability to perform validation in a setting that closely resembles how the model will be used. For example, for a biomedical model that makes in vivo clinically relevant predictions, direct validation of predictions may be impossible for ethical, technological, or financial reasons. Unavoidable limitations inherent to the validation process lead to challenges in evaluating the credibilitymore » of biomedical model predictions. Therefore, when evaluating biomedical models, it is critical to rigorously assess applicability, that is, the relevance of the computational model, and its validation evidence to the proposed context of use (COU). However, there are no well-established methods for assessing applicability. In this paper, we present a novel framework for performing applicability analysis and demonstrate its use with a medical device computational model. The framework provides a systematic, step-by-step method for breaking down the broad question of applicability into a series of focused questions, which may be addressed using supporting evidence and subject matter expertise. The framework can be used for model justification, model assessment, and validation planning. While motivated by biomedical models, it is relevant to a broad range of disciplines and underlying physics. Finally, the proposed applicability framework could help overcome some of the barriers inherent to validation of, and aid clinical implementation of, biomedical models.« less

Applicability Analysis of Validation Evidence for Biomedical Computational Models

DOE PAGES

Pathmanathan, Pras; Gray, Richard A.; Romero, Vicente J.; ...

2017-09-07

Computational modeling has the potential to revolutionize medicine the way it transformed engineering. However, despite decades of work, there has only been limited progress to successfully translate modeling research to patient care. One major difficulty which often occurs with biomedical computational models is an inability to perform validation in a setting that closely resembles how the model will be used. For example, for a biomedical model that makes in vivo clinically relevant predictions, direct validation of predictions may be impossible for ethical, technological, or financial reasons. Unavoidable limitations inherent to the validation process lead to challenges in evaluating the credibilitymore » of biomedical model predictions. Therefore, when evaluating biomedical models, it is critical to rigorously assess applicability, that is, the relevance of the computational model, and its validation evidence to the proposed context of use (COU). However, there are no well-established methods for assessing applicability. In this paper, we present a novel framework for performing applicability analysis and demonstrate its use with a medical device computational model. The framework provides a systematic, step-by-step method for breaking down the broad question of applicability into a series of focused questions, which may be addressed using supporting evidence and subject matter expertise. The framework can be used for model justification, model assessment, and validation planning. While motivated by biomedical models, it is relevant to a broad range of disciplines and underlying physics. Finally, the proposed applicability framework could help overcome some of the barriers inherent to validation of, and aid clinical implementation of, biomedical models.« less

The Optimal Screening for Prediction of Referral and Outcome (OSPRO) in patients with musculoskeletal pain conditions: a longitudinal validation cohort from the USA

PubMed Central

George, Steven Z; Beneciuk, Jason M; Lentz, Trevor A; Wu, Samuel S

2017-01-01

Purpose There is an increased need for determining which patients with musculoskeletal pain benefit from additional diagnostic testing or psychologically informed intervention. The Optimal Screening for Prediction of Referral and Outcome (OSPRO) cohort studies were designed to develop and validate standard assessment tools for review of systems and yellow flags. This cohort profile paper provides a description of and future plans for the validation cohort. Participants Patients (n=440) with primary complaint of spine, shoulder or knee pain were recruited into the OSPRO validation cohort via a national Orthopaedic Physical Therapy-Investigative Network. Patients were followed up at 4 weeks, 6 months and 12 months for pain, functional status and quality of life outcomes. Healthcare utilisation outcomes were also collected at 6 and 12 months. Findings to date There are no longitudinal findings reported to date from the ongoing OSPRO validation cohort. The previously completed cross-sectional OSPRO development cohort yielded two assessment tools that were investigated in the validation cohort. Future plans Follow-up data collection was completed in January 2017. Primary analyses will investigate how accurately the OSPRO review of systems and yellow flag tools predict 12-month pain, functional status, quality of life and healthcare utilisation outcomes. Planned secondary analyses include prediction of pain interference and/or development of chronic pain, investigation of treatment expectation on patient outcomes and analysis of patient satisfaction following an episode of physical therapy. Trial registration number The OSPRO validation cohort was not registered. PMID:28600371

Strong claims and weak evidence: reassessing the predictive validity of the IAT.

PubMed

Blanton, Hart; Jaccard, James; Klick, Jonathan; Mellers, Barbara; Mitchell, Gregory; Tetlock, Philip E

2009-05-01

The authors reanalyzed data from 2 influential studies-A. R. McConnell and J. M. Leibold and J. C. Ziegert and P. J. Hanges-that explore links between implicit bias and discriminatory behavior and that have been invoked to support strong claims about the predictive validity of the Implicit Association Test. In both of these studies, the inclusion of race Implicit Association Test scores in regression models reduced prediction errors by only tiny amounts, and Implicit Association Test scores did not permit prediction of individual-level behaviors. Furthermore, the results were not robust when the impact of rater reliability, statistical specifications, and/or outliers were taken into account, and reanalysis of A. R. McConnell & J. M. Leibold (2001) revealed a pattern of behavior consistent with a pro-Black behavioral bias, rather than the anti-Black bias suggested in the original study. (c) 2009 APA, all rights reserved.

Validity of the MicroDYN Approach: Complex Problem Solving Predicts School Grades beyond Working Memory Capacity

ERIC Educational Resources Information Center

Schweizer, Fabian; Wustenberg, Sascha; Greiff, Samuel

2013-01-01

This study examines the validity of the complex problem solving (CPS) test MicroDYN by investigating a) the relation between its dimensions--rule identification (exploration strategy), rule knowledge (acquired knowledge), rule application (control performance)--and working memory capacity (WMC), and b) whether CPS predicts school grades in…

MODELS FOR SUBMARINE OUTFALL - VALIDATION AND PREDICTION UNCERTAINTIES

EPA Science Inventory

This address reports on some efforts to verify and validate dilution models, including those found in Visual Plumes. This is done in the context of problem experience: a range of problems, including different pollutants such as bacteria; scales, including near-field and far-field...

Development, calibration, and validation of performance prediction models for the Texas M-E flexible pavement design system.

DOT National Transportation Integrated Search

2010-08-01

This study was intended to recommend future directions for the development of TxDOTs Mechanistic-Empirical : (TexME) design system. For stress predictions, a multi-layer linear elastic system was evaluated and its validity was : verified by compar...

External validation of a simple clinical tool used to predict falls in people with Parkinson disease

PubMed Central

Duncan, Ryan P.; Cavanaugh, James T.; Earhart, Gammon M.; Ellis, Terry D.; Ford, Matthew P.; Foreman, K. Bo; Leddy, Abigail L.; Paul, Serene S.; Canning, Colleen G.; Thackeray, Anne; Dibble, Leland E.

2015-01-01

Background Assessment of fall risk in an individual with Parkinson disease (PD) is a critical yet often time consuming component of patient care. Recently a simple clinical prediction tool based only on fall history in the previous year, freezing of gait in the past month, and gait velocity <1.1 m/s was developed and accurately predicted future falls in a sample of individuals with PD. METHODS We sought to externally validate the utility of the tool by administering it to a different cohort of 171 individuals with PD. Falls were monitored prospectively for 6 months following predictor assessment. RESULTS The tool accurately discriminated future fallers from non-fallers (area under the curve [AUC] = 0.83; 95% CI 0.76 –0.89), comparable to the developmental study. CONCLUSION The results validated the utility of the tool for allowing clinicians to quickly and accurately identify an individual’s risk of an impending fall. PMID:26003412

External validation of a simple clinical tool used to predict falls in people with Parkinson disease.

PubMed

Duncan, Ryan P; Cavanaugh, James T; Earhart, Gammon M; Ellis, Terry D; Ford, Matthew P; Foreman, K Bo; Leddy, Abigail L; Paul, Serene S; Canning, Colleen G; Thackeray, Anne; Dibble, Leland E

2015-08-01

Assessment of fall risk in an individual with Parkinson disease (PD) is a critical yet often time consuming component of patient care. Recently a simple clinical prediction tool based only on fall history in the previous year, freezing of gait in the past month, and gait velocity <1.1 m/s was developed and accurately predicted future falls in a sample of individuals with PD. We sought to externally validate the utility of the tool by administering it to a different cohort of 171 individuals with PD. Falls were monitored prospectively for 6 months following predictor assessment. The tool accurately discriminated future fallers from non-fallers (area under the curve [AUC] = 0.83; 95% CI 0.76-0.89), comparable to the developmental study. The results validated the utility of the tool for allowing clinicians to quickly and accurately identify an individual's risk of an impending fall. Copyright © 2015 Elsevier Ltd. All rights reserved.

Critical validation studies of neurofeedback.

PubMed

Gruzelier, John; Egner, Tobias

2005-01-01

The field of neurofeedback training has proceeded largely without validation. In this article the authors review studies directed at validating sensory motor rhythm, beta and alpha-theta protocols for improving attention, memory, and music performance in healthy participants. Importantly, benefits were demonstrable with cognitive and neurophysiologic measures that were predicted on the basis of regression models of learning to enhance sensory motor rhythm and beta activity. The first evidence of operant control over the alpha-theta ratio is provided, together with remarkable improvements in artistic aspects of music performance equivalent to two class grades in conservatory students. These are initial steps in providing a much needed scientific basis to neurofeedback.

Are There Sex Differences in the Predictive Validity of DSM-IV ADHD among Younger Children?

ERIC Educational Resources Information Center

Lahey, Benjamin B.; Hartung, Cynthia M.; Loney, Jan; Pelham, William E.; Chronis, Andrea M.; Lee, Steve S.

2007-01-01

We assessed the predictive validity of attention-deficit/hyperactivity disorder (ADHD) in 20 girls and 98 boys who met the Diagnostic and Statistical Manual for Mental Disorders (4th ed., American Psychiatric Association, 1994) criteria for ADHD at 4 to 6 years of age compared to 24 female and 102 male comparison children. Over the next 8 years,…

The Predictive Validity of Using Admissions Testing and Multiple Mini-Interviews in Undergraduate University Admissions

ERIC Educational Resources Information Center

Makransky, Guido; Havmose, Philip; Vang, Maria Louison; Andersen, Tonny Elmose; Nielsen, Tine

2017-01-01

The aim of this study was to evaluate the predictive validity of a two-step admissions procedure that included a cognitive ability test followed by multiple mini-interviews (MMIs) used to assess non-cognitive skills, compared to grade-based admissions relative to subsequent drop-out rates and academic achievement after one and two years of study.…

Aptitude Tests and Successful College Students: The Predictive Validity of the General Aptitude Test (GAT) in Saudi Arabia

ERIC Educational Resources Information Center

Alnahdi, Ghaleb Hamad

2015-01-01

Aptitude tests should predict student success at the university level. This study examined the predictive validity of the General Aptitude Test (GAT) in Saudi Arabia. Data for 27420 students enrolled at Prince Sattam bin Abdulaziz University were analyzed. Of these students, 17565 were male students, and 9855 were female students. Multiple…

AIRS Retrieval Validation During the EAQUATE

NASA Technical Reports Server (NTRS)

Zhou, Daniel K.; Smith, William L.; Cuomo, Vincenzo; Taylor, Jonathan P.; Barnet, Christopher D.; DiGirolamo, Paolo; Pappalardo, Gelsomina; Larar, Allen M.; Liu, Xu; Newman, Stuart M.

2006-01-01

Atmospheric and surface thermodynamic parameters retrieved with advanced hyperspectral remote sensors of Earth observing satellites are critical for weather prediction and scientific research. The retrieval algorithms and retrieved parameters from satellite sounders must be validated to demonstrate the capability and accuracy of both observation and data processing systems. The European AQUA Thermodynamic Experiment (EAQUATE) was conducted mainly for validation of the Atmospheric InfraRed Sounder (AIRS) on the AQUA satellite, but also for assessment of validation systems of both ground-based and aircraft-based instruments which will be used for other satellite systems such as the Infrared Atmospheric Sounding Interferometer (IASI) on the European MetOp satellite, the Cross-track Infrared Sounder (CrIS) from the NPOESS Preparatory Project and the following NPOESS series of satellites. Detailed inter-comparisons were conducted and presented using different retrieval methodologies: measurements from airborne ultraspectral Fourier transform spectrometers, aircraft in-situ instruments, dedicated dropsondes and radiosondes, and ground based Raman Lidar, as well as from the European Center for Medium range Weather Forecasting (ECMWF) modeled thermal structures. The results of this study not only illustrate the quality of the measurements and retrieval products but also demonstrate the capability of these validation systems which are put in place to validate current and future hyperspectral sounding instruments and their scientific products.

Development and validation of a risk-prediction nomogram for in-hospital mortality in adults poisoned with drugs and nonpharmaceutical agents

PubMed Central

Lionte, Catalina; Sorodoc, Victorita; Jaba, Elisabeta; Botezat, Alina

2017-01-01

Abstract Acute poisoning with drugs and nonpharmaceutical agents represents an important challenge in the emergency department (ED). The objective is to create and validate a risk-prediction nomogram for use in the ED to predict the risk of in-hospital mortality in adults from acute poisoning with drugs and nonpharmaceutical agents. This was a prospective cohort study involving adults with acute poisoning from drugs and nonpharmaceutical agents admitted to a tertiary referral center for toxicology between January and December 2015 (derivation cohort) and between January and June 2016 (validation cohort). We used a program to generate nomograms based on binary logistic regression predictive models. We included variables that had significant associations with death. Using regression coefficients, we calculated scores for each variable, and estimated the event probability. Model validation was performed using bootstrap to quantify our modeling strategy and using receiver operator characteristic (ROC) analysis. The nomogram was tested on a separate validation cohort using ROC analysis and goodness-of-fit tests. Data from 315 patients aged 18 to 91 years were analyzed (n = 180 in the derivation cohort; n = 135 in the validation cohort). In the final model, the following variables were significantly associated with mortality: age, laboratory test results (lactate, potassium, MB isoenzyme of creatine kinase), electrocardiogram parameters (QTc interval), and echocardiography findings (E wave velocity deceleration time). Sex was also included to use the same model for men and women. The resulting nomogram showed excellent survival/mortality discrimination (area under the curve [AUC] 0.976, 95% confidence interval [CI] 0.954–0.998, P < 0.0001 for the derivation cohort; AUC 0.957, 95% CI 0.892–1, P < 0.0001 for the validation cohort). This nomogram provides more precise, rapid, and simple risk-analysis information for individual patients acutely exposed to

The temporal stability and predictive validity of pupils' causal attributions for difficult classroom behaviour.

PubMed

Lambert, Nathan; Miller, Andy

2010-12-01

Recent studies have investigated the causal attributions for difficult pupil behaviour made by teachers, pupils, and parents but none have investigated the temporal stability or predictive validity of these attributions. This study examines the causal attributions made for difficult classroom behaviour by students on two occasions 30 months apart. The longitudinal stability of these attributions is considered as is the predictive validity of the first set of attributions in relation to teachers' later judgments about individual students' behaviour. Two hundred and seventeen secondary school age pupils (114 males, 103 females) provided data on the two occasions. Teachers also rated each student's behaviour at the two times. A questionnaire listing 63 possible causes of classroom misbehaviour was delivered to pupils firstly when they were in Year 7 (aged 11-12) and then again, 30 months later. Responses were analysed through exploratory factor analysis (EFA). Additionally, teachers were asked to rate the standard of behaviour of each of the students on the two occasions. EFA of the Years 7 and 10 data indicated that pupils' attributions yielded broadly similar five-factor models with the perceived relative importance of these factors remaining the same. Analysis also revealed a predictive relationship between pupils' attributions regarding the factor named culture of misbehaviour in Year 7, and teachers' judgments of their standard of behaviour in Year 10. The present study suggests that young adolescents' causal attributions for difficult classroom behaviour remain stable over time and are predictive of teachers' later judgments about their behaviour.

Validity Evidence in Scale Development: The Application of Cross Validation and Classification-Sequencing Validation

ERIC Educational Resources Information Center

Acar, Tu¨lin

2014-01-01

In literature, it has been observed that many enhanced criteria are limited by factor analysis techniques. Besides examinations of statistical structure and/or psychological structure, such validity studies as cross validation and classification-sequencing studies should be performed frequently. The purpose of this study is to examine cross…

Predicting plant uptake of cadmium: validated with long-term contaminated soils.

PubMed

Lamb, Dane T; Kader, Mohammed; Ming, Hui; Wang, Liang; Abbasi, Sedigheh; Megharaj, Mallavarapu; Naidu, Ravi

2016-10-01

Cadmium accumulates in plant tissues at low soil loadings and is a concern for human health. Yet at higher levels it is also of concern for ecological receptors. We determined Cd partitioning constants for 41 soils to examine the role of soil properties controlling Cd partitioning and plant uptake. From a series of sorption and dose response studies, transfer functions were developed for predicting Cd uptake in Cucumis sativa L. (cucumber). The parameter log K f was predicted with soil pH ca , logCEC and log OC. Transfer of soil pore-water Cd 2+ to shoots was described with a power function (R 2  = 0.73). The dataset was validated with 13 long-term contaminated soils (plus 2 control soils) ranging in Cd concentration from 0.2 to 300 mg kg -1 . The series of equations predicting Cd shoot from pore-water Cd 2+ were able to predict the measured data in the independent dataset (root mean square error = 2.2). The good relationship indicated that Cd uptake to cucumber shoots could be predicted with Cd pore and Cd 2+ without other pore-water parameters such as pH or Ca 2+ . The approach may be adapted to a range of plant species.

Validation of NASA Thermal Ice Protection Computer Codes. Part 1; Program Overview

NASA Technical Reports Server (NTRS)

Miller, Dean; Bond, Thomas; Sheldon, David; Wright, William; Langhals, Tammy; Al-Khalil, Kamel; Broughton, Howard

1996-01-01

The Icing Technology Branch at NASA Lewis has been involved in an effort to validate two thermal ice protection codes developed at the NASA Lewis Research Center. LEWICE/Thermal (electrothermal deicing & anti-icing), and ANTICE (hot-gas & electrothermal anti-icing). The Thermal Code Validation effort was designated as a priority during a 1994 'peer review' of the NASA Lewis Icing program, and was implemented as a cooperative effort with industry. During April 1996, the first of a series of experimental validation tests was conducted in the NASA Lewis Icing Research Tunnel(IRT). The purpose of the April 96 test was to validate the electrothermal predictive capabilities of both LEWICE/Thermal, and ANTICE. A heavily instrumented test article was designed and fabricated for this test, with the capability of simulating electrothermal de-icing and anti-icing modes of operation. Thermal measurements were then obtained over a range of test conditions, for comparison with analytical predictions. This paper will present an overview of the test, including a detailed description of: (1) the validation process; (2) test article design; (3) test matrix development; and (4) test procedures. Selected experimental results will be presented for de-icing and anti-icing modes of operation. Finally, the status of the validation effort at this point will be summarized. Detailed comparisons between analytical predictions and experimental results are contained in the following two papers: 'Validation of NASA Thermal Ice Protection Computer Codes: Part 2- The Validation of LEWICE/Thermal' and 'Validation of NASA Thermal Ice Protection Computer Codes: Part 3-The Validation of ANTICE'

Construct, Concurrent and Predictive Validity of the URICA: Data from Two Multi-site Clinical Trials

PubMed Central

Field, Craig A.; Adinoff, Bryon; Harris, T. Robert; Ball, Samuel A.; Carroll, Kathleen M.

2011-01-01

Background A better understanding of how to measure motivation to change and how it relates to behavior change in patients with drug and alcohol dependence would broaden our understanding of the role of motivation in addiction treatment. Methods Two multi-site, randomized clinical trials comparing brief motivational interventions with standard care were conducted in the National Institute on Drug Abuse Clinical Trials Network. Patients with primary drug dependence and alcohol dependence entering outpatient treatment participated in a study of either Motivational Enhancement Therapy (n=431) or Motivational Interviewing (n=423). The construct, concurrent, and predictive validity of two composite measures of motivation to change derived from the University of Rhode Island Change Assessment (URICA): Readiness to Change (RTC) and Committed Action (CA) were evaluated. Results Confirmatory factor analysis confirmed the a priori factor structure of the URICA. RTC was significantly associated with measures of addiction severity at baseline (r=.12-.52, p<.05). Although statistically significant (p<.01), the correlations between treatment outcomes and RTC were low (r=-.15 and -18). Additional analyses did not support a moderating or mediating effect of motivation on treatment retention or substance use. Conclusions The construct validity of the URICA was confirmed separately in a large sample of drug- and alcohol-dependent patients. However, evidence for the predictive validity of composite scores was very limited and there were no moderating or mediating effects of either measure on treatment outcome. Thus, increased motivation to change, as measured by the composite scores of motivation derived from the URICA, does not appear to influence treatment outcome. PMID:19157723

First Principles Predictions of the Structure and Function of G-Protein-Coupled Receptors: Validation for Bovine Rhodopsin

PubMed Central

Trabanino, Rene J.; Hall, Spencer E.; Vaidehi, Nagarajan; Floriano, Wely B.; Kam, Victor W. T.; Goddard, William A.

2004-01-01

G-protein-coupled receptors (GPCRs) are involved in cell communication processes and with mediating such senses as vision, smell, taste, and pain. They constitute a prominent superfamily of drug targets, but an atomic-level structure is available for only one GPCR, bovine rhodopsin, making it difficult to use structure-based methods to design receptor-specific drugs. We have developed the MembStruk first principles computational method for predicting the three-dimensional structure of GPCRs. In this article we validate the MembStruk procedure by comparing its predictions with the high-resolution crystal structure of bovine rhodopsin. The crystal structure of bovine rhodopsin has the second extracellular (EC-II) loop closed over the transmembrane regions by making a disulfide linkage between Cys-110 and Cys-187, but we speculate that opening this loop may play a role in the activation process of the receptor through the cysteine linkage with helix 3. Consequently we predicted two structures for bovine rhodopsin from the primary sequence (with no input from the crystal structure)—one with the EC-II loop closed as in the crystal structure, and the other with the EC-II loop open. The MembStruk-predicted structure of bovine rhodopsin with the closed EC-II loop deviates from the crystal by 2.84 Å coordinate root mean-square (CRMS) in the transmembrane region main-chain atoms. The predicted three-dimensional structures for other GPCRs can be validated only by predicting binding sites and energies for various ligands. For such predictions we developed the HierDock first principles computational method. We validate HierDock by predicting the binding site of 11-cis-retinal in the crystal structure of bovine rhodopsin. Scanning the whole protein without using any prior knowledge of the binding site, we find that the best scoring conformation in rhodopsin is 1.1 Å CRMS from the crystal structure for the ligand atoms. This predicted conformation has the carbonyl O only 2

Validation of catchment models for predicting land-use and climate change impacts. 2. Case study for a Mediterranean catchment

NASA Astrophysics Data System (ADS)

Parkin, G.; O'Donnell, G.; Ewen, J.; Bathurst, J. C.; O'Connell, P. E.; Lavabre, J.

1996-02-01

Validation methods commonly used to test catchment models are not capable of demonstrating a model's fitness for making predictions for catchments where the catchment response is not known (including hypothetical catchments, and future conditions of existing catchments which are subject to land-use or climate change). This paper describes the first use of a new method of validation (Ewen and Parkin, 1996. J. Hydrol., 175: 583-594) designed to address these types of application; the method involves making 'blind' predictions of selected hydrological responses which are considered important for a particular application. SHETRAN (a physically based, distributed catchment modelling system) is tested on a small Mediterranean catchment. The test involves quantification of the uncertainty in four predicted features of the catchment response (continuous hydrograph, peak discharge rates, monthly runoff, and total runoff), and comparison of observations with the predicted ranges for these features. The results of this test are considered encouraging.

Modified Maturity Offset Prediction Equations: Validation in Independent Longitudinal Samples of Boys and Girls.

PubMed

Kozieł, Sławomir M; Malina, Robert M

2018-01-01

Predicted maturity offset and age at peak height velocity are increasingly used with youth athletes, although validation studies of the equations indicated major limitations. The equations have since been modified and simplified. The objective of this study was to validate the new maturity offset prediction equations in independent longitudinal samples of boys and girls. Two new equations for boys with chronological age and sitting height and chronological age and stature as predictors, and one equation for girls with chronological age and stature as predictors were evaluated in serial data from the Wrocław Growth Study, 193 boys (aged 8-18 years) and 198 girls (aged 8-16 years). Observed age at peak height velocity for each youth was estimated with the Preece-Baines Model 1. The original prediction equations were included for comparison. Predicted age at peak height velocity was the difference between chronological age at prediction and maturity offset. Predicted ages at peak height velocity with the new equations approximated observed ages at peak height velocity in average maturing boys near the time of peak height velocity; a corresponding window for average maturing girls was not apparent. Compared with observed age at peak height velocity, predicted ages at peak height velocity with the new and original equations were consistently later in early maturing youth and earlier in late maturing youth of both sexes. Predicted ages at peak height velocity with the new equations had reduced variation compared with the original equations and especially observed ages at peak height velocity. Intra-individual variation in predicted ages at peak height velocity with all equations was considerable. The new equations are useful for average maturing boys close to the time of peak height velocity; there does not appear to be a clear window for average maturing girls. The new and original equations have major limitations with early and late maturing boys and girls.

Improving the Validity of Activity of Daily Living Dependency Risk Assessment

PubMed Central

Clark, Daniel O.; Stump, Timothy E.; Tu, Wanzhu; Miller, Douglas K.

2015-01-01

Objectives Efforts to prevent activity of daily living (ADL) dependency may be improved through models that assess older adults’ dependency risk. We evaluated whether cognition and gait speed measures improve the predictive validity of interview-based models. Method Participants were 8,095 self-respondents in the 2006 Health and Retirement Survey who were aged 65 years or over and independent in five ADLs. Incident ADL dependency was determined from the 2008 interview. Models were developed using random 2/3rd cohorts and validated in the remaining 1/3rd. Results Compared to a c-statistic of 0.79 in the best interview model, the model including cognitive measures had c-statistics of 0.82 and 0.80 while the best fitting gait speed model had c-statistics of 0.83 and 0.79 in the development and validation cohorts, respectively. Conclusion Two relatively brief models, one that requires an in-person assessment and one that does not, had excellent validity for predicting incident ADL dependency but did not significantly improve the predictive validity of the best fitting interview-based models. PMID:24652867

Random Qualitative Validation: A Mixed-Methods Approach to Survey Validation

ERIC Educational Resources Information Center

Van Duzer, Eric

2012-01-01

The purpose of this paper is to introduce the process and value of Random Qualitative Validation (RQV) in the development and interpretation of survey data. RQV is a method of gathering clarifying qualitative data that improves the validity of the quantitative analysis. This paper is concerned with validity in relation to the participants'…

Criteria of validity for animal models of psychiatric disorders: focus on anxiety disorders and depression

PubMed Central

2011-01-01

Animal models of psychiatric disorders are usually discussed with regard to three criteria first elaborated by Willner; face, predictive and construct validity. Here, we draw the history of these concepts and then try to redraw and refine these criteria, using the framework of the diathesis model of depression that has been proposed by several authors. We thus propose a set of five major criteria (with sub-categories for some of them); homological validity (including species validity and strain validity), pathogenic validity (including ontopathogenic validity and triggering validity), mechanistic validity, face validity (including ethological and biomarker validity) and predictive validity (including induction and remission validity). Homological validity requires that an adequate species and strain be chosen: considering species validity, primates will be considered to have a higher score than drosophila, and considering strains, a high stress reactivity in a strain scores higher than a low stress reactivity in another strain. Pathological validity corresponds to the fact that, in order to shape pathological characteristics, the organism has been manipulated both during the developmental period (for example, maternal separation: ontopathogenic validity) and during adulthood (for example, stress: triggering validity). Mechanistic validity corresponds to the fact that the cognitive (for example, cognitive bias) or biological mechanisms (such as dysfunction of the hormonal stress axis regulation) underlying the disorder are identical in both humans and animals. Face validity corresponds to the observable behavioral (ethological validity) or biological (biomarker validity) outcomes: for example anhedonic behavior (ethological validity) or elevated corticosterone (biomarker validity). Finally, predictive validity corresponds to the identity of the relationship between the triggering factor and the outcome (induction validity) and between the effects of the treatments

Predicting the need for institutional care shortly after admission to rehabilitation: Rasch analysis and predictive validity of the BRASS Index.

PubMed

Panella, L; La Porta, F; Caselli, S; Marchisio, S; Tennant, A

2012-09-01

Effective discharge planning is increasingly recognised as a critical component of hospital-based Rehabilitation. The BRASS index is a risk screening tool for identification, shortly after hospital admission, of patients who are at risk of post-discharge problems. To evaluate the internal construct validity and reliability of the Blaylock Risk Assessment Screening Score (BRASS) within the rehabilitation setting. Observational prospective study. Rehabilitation ward of an Italian district hospital. One hundred and four consecutively admitted patients. Using classical psychometric methods and Rasch analysis (RA), the internal construct validity and reliability of the BRASS were examined. Also, external and predictive validity of the Rasch-modified BRASS (RMB) score were determined. Reliability of the original BRASS was low (Cronbach's alpha=0.595) and factor analyses showed that it was clearly multidimensional. A RA, based on a reduced 7-BRASS item set (RMB), satisfied model's expectations. Reliability was 0.777. The RMB scores strongly correlated with the original BRASS (rho=0.952; P<0.000) and with FIM™ admission scores (rho=-0.853; P<0.000). A RMB score of 12 was associated with an increased risk of nursing home admission (RR=2.1, 95%CI=1.7-2.5), whereas a score of 17 was associated to a higher risk of length of stay >28 days (RR=7.6, 95%CI=1.8-31.9). This study demonstrated that the original BRASS was multidimensional and unreliable. However, the RMB holds adequate internal construct validity and is sufficiently reliable as a predictor of discharge problems for group, but not individual use. The application of tools and methods (such as the BRASS Index) developed under the biomedical paradigm in a Physical and Rehabilitation Medicine setting may have limitations. Further research is needed to develop, within the rehabilitation setting, a valid measuring tool of risk of post-discharge problems at the individual level.

Construct and Predictive Validity of the Core Phonics Survey: A Diagnostic Assessment for Students with Specific Learning Disabilities

ERIC Educational Resources Information Center

Park, Yujeong; Benedict, Amber E.; Brownell, Mary T.

2014-01-01

The factor structure of the CORE Phonics Survey was analyzed using a sample of 165 students in upper elementary school with specific learning disabilities. Confirmatory factor analysis was used to identify the hypothesized constructs of the CORE Phonics Survey and predictive validity of the CORE Phonics Survey to predict students' success in word…

Predictive validity of the classroom strategies scale-observer form on statewide testing scores: an initial investigation.

PubMed

Reddy, Linda A; Fabiano, Gregory A; Dudek, Christopher M; Hsu, Louis

2013-12-01

The present study examined the validity of a teacher observation measure, the Classroom Strategies Scale--Observer Form (CSS), as a predictor of student performance on statewide tests of mathematics and English language arts. The CSS is a teacher practice observational measure that assesses evidence-based instructional and behavioral management practices in elementary school. A series of two-level hierarchical generalized linear models were fitted to data of a sample of 662 third- through fifth-grade students to assess whether CSS Part 2 Instructional Strategy and Behavioral Management Strategy scale discrepancy scores (i.e., ∑ |recommended frequency--frequency ratings|) predicted statewide mathematics and English language arts proficiency scores when percentage of minority students in schools was controlled. Results indicated that the Instructional Strategy scale discrepancy scores significantly predicted mathematics and English language arts proficiency scores: Relatively larger discrepancies on observer ratings of what teachers did versus what should have been done were associated with lower proficiency scores. Results offer initial evidence of the predictive validity of the CSS Part 2 Instructional Strategy discrepancy scores on student academic outcomes. PsycINFO Database Record (c) 2013 APA, all rights reserved.

Anxiety measures validated in perinatal populations: a systematic review.

PubMed

Meades, Rose; Ayers, Susan

2011-09-01

Research and screening of anxiety in the perinatal period is hampered by a lack of psychometric data on self-report anxiety measures used in perinatal populations. This paper aimed to review self-report measures that have been validated with perinatal women. A systematic search was carried out of four electronic databases. Additional papers were obtained through searching identified articles. Thirty studies were identified that reported validation of an anxiety measure with perinatal women. Most commonly validated self-report measures were the General Health Questionnaire (GHQ), State-Trait Anxiety Inventory (STAI), and Hospital Anxiety and Depression Scales (HADS). Of the 30 studies included, 11 used a clinical interview to provide criterion validity. Remaining studies reported one or more other forms of validity (factorial, discriminant, concurrent and predictive) or reliability. The STAI shows criterion, discriminant and predictive validity and may be most useful for research purposes as a specific measure of anxiety. The Kessler 10 (K-10) may be the best short screening measure due to its ability to differentiate anxiety disorders. The Depression Anxiety Stress Scales 21 (DASS-21) measures multiple types of distress, shows appropriate content, and remains to be validated against clinical interview in perinatal populations. Nineteen studies did not report sensitivity or specificity data. The early stages of research into perinatal anxiety, the multitude of measures in use, and methodological differences restrict comparison of measures across studies. There is a need for further validation of self-report measures of anxiety in the perinatal period to enable accurate screening and detection of anxiety symptoms and disorders. Copyright © 2010 Elsevier B.V. All rights reserved.