Validation of the Oncentra Brachy Advanced Collapsed cone Engine for a commercial (192)Ir source using heterogeneous geometries.

PubMed

Ma, Yunzhi; Lacroix, Fréderic; Lavallée, Marie-Claude; Beaulieu, Luc

2015-01-01

To validate the Advanced Collapsed cone Engine (ACE) dose calculation engine of Oncentra Brachy (OcB) treatment planning system using an (192)Ir source. Two levels of validation were performed, conformant to the model-based dose calculation algorithm commissioning guidelines of American Association of Physicists in Medicine TG-186 report. Level 1 uses all-water phantoms, and the validation is against TG-43 methodology. Level 2 uses real-patient cases, and the validation is against Monte Carlo (MC) simulations. For each case, the ACE and TG-43 calculations were performed in the OcB treatment planning system. ALGEBRA MC system was used to perform MC simulations. In Level 1, the ray effect depends on both accuracy mode and the number of dwell positions. The volume fraction with dose error ≥2% quickly reduces from 23% (13%) for a single dwell to 3% (2%) for eight dwell positions in the standard (high) accuracy mode. In Level 2, the 10% and higher isodose lines were observed overlapping between ACE (both standard and high-resolution modes) and MC. Major clinical indices (V100, V150, V200, D90, D50, and D2cc) were investigated and validated by MC. For example, among the Level 2 cases, the maximum deviation in V100 of ACE from MC is 2.75% but up to ~10% for TG-43. Similarly, the maximum deviation in D90 is 0.14 Gy between ACE and MC but up to 0.24 Gy for TG-43. ACE demonstrated good agreement with MC in most clinically relevant regions in the cases tested. Departure from MC is significant for specific situations but limited to low-dose (<10% isodose) regions. Copyright © 2015 American Brachytherapy Society. Published by Elsevier Inc. All rights reserved.

Agility performance in high-level junior basketball players: the predictive value of anthropometrics and power qualities.

PubMed

Sisic, Nedim; Jelicic, Mario; Pehar, Miran; Spasic, Miodrag; Sekulic, Damir

2016-01-01

In basketball, anthropometric status is an important factor when identifying and selecting talents, while agility is one of the most vital motor performances. The aim of this investigation was to evaluate the influence of anthropometric variables and power capacities on different preplanned agility performances. The participants were 92 high-level, junior-age basketball players (16-17 years of age; 187.6±8.72 cm in body height, 78.40±12.26 kg in body mass), randomly divided into a validation and cross-validation subsample. The predictors set consisted of 16 anthropometric variables, three tests of power-capacities (Sargent-jump, broad-jump and medicine-ball-throw) as predictors. The criteria were three tests of agility: a T-Shape-Test; a Zig-Zag-Test, and a test of running with a 180-degree turn (T180). Forward stepwise multiple regressions were calculated for validation subsamples and then cross-validated. Cross validation included correlations between observed and predicted scores, dependent samples t-test between predicted and observed scores; and Bland Altman graphics. Analysis of the variance identified centres being advanced in most of the anthropometric indices, and medicine-ball-throw (all at P<0.05); with no significant between-position-differences for other studied motor performances. Multiple regression models originally calculated for the validation subsample were then cross-validated, and confirmed for Zig-zag-Test (R of 0.71 and 0.72 for the validation and cross-validation subsample, respectively). Anthropometrics were not strongly related to agility performance, but leg length is found to be negatively associated with performance in basketball-specific agility. Power capacities are confirmed to be an important factor in agility. The results highlighted the importance of sport-specific tests when studying pre-planned agility performance in basketball. The improvement in power capacities will probably result in an improvement in agility in basketball athletes, while anthropometric indices should be used in order to identify those athletes who can achieve superior agility performance.

Addressing criticisms of existing predictive bias research: cognitive ability test scores still overpredict African Americans' job performance.

PubMed

Berry, Christopher M; Zhao, Peng

2015-01-01

Predictive bias studies have generally suggested that cognitive ability test scores overpredict job performance of African Americans, meaning these tests are not predictively biased against African Americans. However, at least 2 issues call into question existing over-/underprediction evidence: (a) a bias identified by Aguinis, Culpepper, and Pierce (2010) in the intercept test typically used to assess over-/underprediction and (b) a focus on the level of observed validity instead of operational validity. The present study developed and utilized a method of assessing over-/underprediction that draws on the math of subgroup regression intercept differences, does not rely on the biased intercept test, allows for analysis at the level of operational validity, and can use meta-analytic estimates as input values. Therefore, existing meta-analytic estimates of key parameters, corrected for relevant statistical artifacts, were used to determine whether African American job performance remains overpredicted at the level of operational validity. African American job performance was typically overpredicted by cognitive ability tests across levels of job complexity and across conditions wherein African American and White regression slopes did and did not differ. Because the present study does not rely on the biased intercept test and because appropriate statistical artifact corrections were carried out, the present study's results are not affected by the 2 issues mentioned above. The present study represents strong evidence that cognitive ability tests generally overpredict job performance of African Americans. (c) 2015 APA, all rights reserved.

Validation of Sea levels from coastal altimetry waveform retracking expert system: a case study around the Prince William Sound in Alaska

NASA Astrophysics Data System (ADS)

Idris, N. H.; Deng, X.; Idris, N. H.

2017-05-01

This paper presents the validation of Coastal Altimetry Waveform Retracking Expert System (CAWRES), a novel method to optimize the Jason satellite altimetric sea levels from multiple retracking solutions. The validation is conducted over the region of Prince William Sound in Alaska, USA, where altimetric waveforms are perturbed by emerged land and sea states. Validation is performed in twofold. First, comparison with existing retrackers (i.e. MLE4 and Ice) from the Sensor Geophysical Data Records (SGDR), and second, comparison with in-situ tide gauge data. From the first validation assessment, in general, CAWRES outperforms the MLE4 and Ice retrackers. In 4 out of 6 cases, the value of improvement percentage (standard deviation of difference) is higher (lower) than those of the SGDR retrackers. CAWRES also presents the best performance in producing valid observations, and has the lowest noise when compared to the SGDR retrackers. From the second assessment with tide gauge, CAWRES retracked sea level anomalies (SLAs) are consistent with those of the tide gauge. The accuracy of CAWRES retracked SLAs is slightly better than those of the MLE4. However, the performance of Ice retracker is better than those of CAWRES and MLE4, suggesting the empirical-based retracker is more effective. The results demonstrate that the CAWRES would have potential to be applied to coastal regions elsewhere.

Validation of the Narrowing Beam Walking Test in Lower Limb Prosthesis Users.

PubMed

Sawers, Andrew; Hafner, Brian

2018-04-11

To evaluate the content, construct, and discriminant validity of the Narrowing Beam Walking Test (NBWT), a performance-based balance test for lower limb prosthesis users. Cross-sectional study. Research laboratory and prosthetics clinic. Unilateral transtibial and transfemoral prosthesis users (N=40). Not applicable. Content validity was examined by quantifying the percentage of participants receiving maximum or minimum scores (ie, ceiling and floor effects). Convergent construct validity was examined using correlations between participants' NBWT scores and scores or times on existing clinical balance tests regularly administered to lower limb prosthesis users. Known-groups construct validity was examined by comparing NBWT scores between groups of participants with different fall histories, amputation levels, amputation etiologies, and functional levels. Discriminant validity was evaluated by analyzing the area under each test's receiver operating characteristic (ROC) curve. No minimum or maximum scores were recorded on the NBWT. NBWT scores demonstrated strong correlations (ρ=.70‒.85) with scores/times on performance-based balance tests (timed Up and Go test, Four Square Step Test, and Berg Balance Scale) and a moderate correlation (ρ=.49) with the self-report Activities-specific Balance Confidence scale. NBWT performance was significantly lower among participants with a history of falls (P=.003), transfemoral amputation (P=.011), and a lower mobility level (P<.001). The NBWT also had the largest area under the ROC curve (.81) and was the only test to exhibit an area that was statistically significantly >.50 (ie, chance). The results provide strong evidence of content, construct, and discriminant validity for the NBWT as a performance-based test of balance ability. The evidence supports its use to assess balance impairments and fall risk in unilateral transtibial and transfemoral prosthesis users. Copyright © 2018 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.

Cultural Adaptation and Validation of the Cultural Self-Efficacy Scale for Colombian Nursing Professionals.

PubMed

Herrero-Hahn, Raquel; Rojas, Juan Guillermo; Ospina-Díaz, Juan Manuel; Montoya-Juárez, Rafael; Restrepo-Medrano, Juan Carlos; Hueso-Montoro, César

2017-03-01

The level of cultural self-efficacy indicates the degree of confidence nursing professionals possess for their ability to provide culturally competent care. Cultural adaptation and validation of the Cultural Self-Efficacy Scale was performed for nursing professionals in Colombia. A scale validation study was conducted. Cultural adaptation and validation of the Cultural Self-Efficacy Scale was performed using a sample of 190 nurses in Colombia, between September 2013 and April 2014. This sample was chosen via systematic random sampling from a finite population. The scale was culturally adapted. Cronbach's alpha for the revised scale was .978. Factor analysis revealed the existence of six factors grouped in three dimensions that explained 68% of the variance. The results demonstrated that the version of the Cultural Self-Efficacy Scale adapted to the Colombian context is a valid and reliable instrument for determining the level of cultural self-efficacy of nursing professionals.

The Virtual Shop: A new immersive virtual reality environment and scenario for the assessment of everyday memory.

PubMed

Ouellet, Émilie; Boller, Benjamin; Corriveau-Lecavalier, Nick; Cloutier, Simon; Belleville, Sylvie

2018-06-01

Assessing and predicting memory performance in everyday life is a common assignment for neuropsychologists. However, most traditional neuropsychological tasks are not conceived to capture everyday memory performance. The Virtual Shop is a fully immersive task developed to assess memory in a more ecological way than traditional neuropsychological assessments. Two studies were undertaken to assess the feasibility of the Virtual Shop and to appraise its ecological and construct validity. In study 1, 20 younger and 19 older adults completed the Virtual Shop task to evaluate its level of difficulty and the way the participants interacted with the VR material. The construct validity was examined with the contrasted-group method, by comparing the performance of younger and older adults. In study 2, 35 individuals with subjective cognitive decline completed the Virtual Shop task. Performance was correlated with an existing questionnaire evaluating everyday memory in order to appraise its ecological validity. To add further support to its construct validity, performance was correlated with traditional episodic memory and executive tasks. All participants successfully completed the Virtual Shop. The task had an appropriate level of difficulty that helped differentiate younger and older adults, supporting the feasibility and construct validity of the task. The performance on the Virtual Shop was significantly and moderately correlated with the performance on the questionnaire and on the traditional memory and executive tasks. Results support the feasibility and both the ecological and construct validity of the Virtual Shop. Copyright © 2018 The Authors. Published by Elsevier B.V. All rights reserved.

The English and Chinese versions of the five-level EuroQoL Group's five-dimension questionnaire (EQ-5D) were valid and reliable and provided comparable scores in Asian breast cancer patients.

PubMed

Lee, Chun Fan; Ng, Raymond; Luo, Nan; Wong, Nan Soon; Yap, Yoon Sim; Lo, Soo Kien; Chia, Whay Kuang; Yee, Alethea; Krishna, Lalit; Wong, Celest; Goh, Cynthia; Cheung, Yin Bun

2013-01-01

To examine the measurement properties of and comparability between the English and Chinese versions of the five-level EuroQoL Group's five-dimension questionnaire (EQ-5D) in breast cancer patients in Singapore. This is an observational study of 269 patients. Known-group validity and responsiveness of the EQ-5D utility index and visual analog scale (VAS) were assessed in relation to various clinical characteristics and longitudinal change in performance status, respectively. Convergent and divergent validity was examined by correlation coefficients between the EQ-5D and a breast cancer-specific instrument. Test-retest reliability was evaluated. The two language versions were compared by multiple regression analyses. For both English and Chinese versions, the EQ-5D utility index and VAS demonstrated known-group validity and convergent and divergent validity, and presented sufficient test-retest reliability (intraclass correlation = 0.72 to 0.83). The English version was responsive to changes in performance status. The Chinese version was responsive to decline in performance status, but there was no conclusive evidence about its responsiveness to improvement in performance status. In the comparison analyses of the utility index and VAS between the two language versions, borderline results were obtained, and equivalence cannot be definitely confirmed. The five-level EQ-5D is valid, responsive, and reliable in assessing health outcome of breast cancer patients. The English and Chinese versions provide comparable measurement results.

A machine learning approach to multi-level ECG signal quality classification.

PubMed

Li, Qiao; Rajagopalan, Cadathur; Clifford, Gari D

2014-12-01

Current electrocardiogram (ECG) signal quality assessment studies have aimed to provide a two-level classification: clean or noisy. However, clinical usage demands more specific noise level classification for varying applications. This work outlines a five-level ECG signal quality classification algorithm. A total of 13 signal quality metrics were derived from segments of ECG waveforms, which were labeled by experts. A support vector machine (SVM) was trained to perform the classification and tested on a simulated dataset and was validated using data from the MIT-BIH arrhythmia database (MITDB). The simulated training and test datasets were created by selecting clean segments of the ECG in the 2011 PhysioNet/Computing in Cardiology Challenge database, and adding three types of real ECG noise at different signal-to-noise ratio (SNR) levels from the MIT-BIH Noise Stress Test Database (NSTDB). The MITDB was re-annotated for five levels of signal quality. Different combinations of the 13 metrics were trained and tested on the simulated datasets and the best combination that produced the highest classification accuracy was selected and validated on the MITDB. Performance was assessed using classification accuracy (Ac), and a single class overlap accuracy (OAc), which assumes that an individual type classified into an adjacent class is acceptable. An Ac of 80.26% and an OAc of 98.60% on the test set were obtained by selecting 10 metrics while 57.26% (Ac) and 94.23% (OAc) were the numbers for the unseen MITDB validation data without retraining. By performing the fivefold cross validation, an Ac of 88.07±0.32% and OAc of 99.34±0.07% were gained on the validation fold of MITDB. Copyright © 2014 Elsevier Ireland Ltd. All rights reserved.

Model performance evaluation (validation and calibration) in model-based studies of therapeutic interventions for cardiovascular diseases : a review and suggested reporting framework.

PubMed

Haji Ali Afzali, Hossein; Gray, Jodi; Karnon, Jonathan

2013-04-01

Decision analytic models play an increasingly important role in the economic evaluation of health technologies. Given uncertainties around the assumptions used to develop such models, several guidelines have been published to identify and assess 'best practice' in the model development process, including general modelling approach (e.g., time horizon), model structure, input data and model performance evaluation. This paper focuses on model performance evaluation. In the absence of a sufficient level of detail around model performance evaluation, concerns regarding the accuracy of model outputs, and hence the credibility of such models, are frequently raised. Following presentation of its components, a review of the application and reporting of model performance evaluation is presented. Taking cardiovascular disease as an illustrative example, the review investigates the use of face validity, internal validity, external validity, and cross model validity. As a part of the performance evaluation process, model calibration is also discussed and its use in applied studies investigated. The review found that the application and reporting of model performance evaluation across 81 studies of treatment for cardiovascular disease was variable. Cross-model validation was reported in 55 % of the reviewed studies, though the level of detail provided varied considerably. We found that very few studies documented other types of validity, and only 6 % of the reviewed articles reported a calibration process. Considering the above findings, we propose a comprehensive model performance evaluation framework (checklist), informed by a review of best-practice guidelines. This framework provides a basis for more accurate and consistent documentation of model performance evaluation. This will improve the peer review process and the comparability of modelling studies. Recognising the fundamental role of decision analytic models in informing public funding decisions, the proposed framework should usefully inform guidelines for preparing submissions to reimbursement bodies.

Construct-level predictive validity of educational attainment and intellectual aptitude tests in medical student selection: meta-regression of six UK longitudinal studies.

PubMed

McManus, I C; Dewberry, Chris; Nicholson, Sandra; Dowell, Jonathan S; Woolf, Katherine; Potts, Henry W W

2013-11-14

Measures used for medical student selection should predict future performance during training. A problem for any selection study is that predictor-outcome correlations are known only in those who have been selected, whereas selectors need to know how measures would predict in the entire pool of applicants. That problem of interpretation can be solved by calculating construct-level predictive validity, an estimate of true predictor-outcome correlation across the range of applicant abilities. Construct-level predictive validities were calculated in six cohort studies of medical student selection and training (student entry, 1972 to 2009) for a range of predictors, including A-levels, General Certificates of Secondary Education (GCSEs)/O-levels, and aptitude tests (AH5 and UK Clinical Aptitude Test (UKCAT)). Outcomes included undergraduate basic medical science and finals assessments, as well as postgraduate measures of Membership of the Royal Colleges of Physicians of the United Kingdom (MRCP(UK)) performance and entry in the Specialist Register. Construct-level predictive validity was calculated with the method of Hunter, Schmidt and Le (2006), adapted to correct for right-censorship of examination results due to grade inflation. Meta-regression analyzed 57 separate predictor-outcome correlations (POCs) and construct-level predictive validities (CLPVs). Mean CLPVs are substantially higher (.450) than mean POCs (.171). Mean CLPVs for first-year examinations, were high for A-levels (.809; CI: .501 to .935), and lower for GCSEs/O-levels (.332; CI: .024 to .583) and UKCAT (mean = .245; CI: .207 to .276). A-levels had higher CLPVs for all undergraduate and postgraduate assessments than did GCSEs/O-levels and intellectual aptitude tests. CLPVs of educational attainment measures decline somewhat during training, but continue to predict postgraduate performance. Intellectual aptitude tests have lower CLPVs than A-levels or GCSEs/O-levels. Educational attainment has strong CLPVs for undergraduate and postgraduate performance, accounting for perhaps 65% of true variance in first year performance. Such CLPVs justify the use of educational attainment measure in selection, but also raise a key theoretical question concerning the remaining 35% of variance (and measurement error, range restriction and right-censorship have been taken into account). Just as in astrophysics, 'dark matter' and 'dark energy' are posited to balance various theoretical equations, so medical student selection must also have its 'dark variance', whose nature is not yet properly characterized, but explains a third of the variation in performance during training. Some variance probably relates to factors which are unpredictable at selection, such as illness or other life events, but some is probably also associated with factors such as personality, motivation or study skills.

Construct-level predictive validity of educational attainment and intellectual aptitude tests in medical student selection: meta-regression of six UK longitudinal studies

PubMed Central

2013-01-01

Background Measures used for medical student selection should predict future performance during training. A problem for any selection study is that predictor-outcome correlations are known only in those who have been selected, whereas selectors need to know how measures would predict in the entire pool of applicants. That problem of interpretation can be solved by calculating construct-level predictive validity, an estimate of true predictor-outcome correlation across the range of applicant abilities. Methods Construct-level predictive validities were calculated in six cohort studies of medical student selection and training (student entry, 1972 to 2009) for a range of predictors, including A-levels, General Certificates of Secondary Education (GCSEs)/O-levels, and aptitude tests (AH5 and UK Clinical Aptitude Test (UKCAT)). Outcomes included undergraduate basic medical science and finals assessments, as well as postgraduate measures of Membership of the Royal Colleges of Physicians of the United Kingdom (MRCP(UK)) performance and entry in the Specialist Register. Construct-level predictive validity was calculated with the method of Hunter, Schmidt and Le (2006), adapted to correct for right-censorship of examination results due to grade inflation. Results Meta-regression analyzed 57 separate predictor-outcome correlations (POCs) and construct-level predictive validities (CLPVs). Mean CLPVs are substantially higher (.450) than mean POCs (.171). Mean CLPVs for first-year examinations, were high for A-levels (.809; CI: .501 to .935), and lower for GCSEs/O-levels (.332; CI: .024 to .583) and UKCAT (mean = .245; CI: .207 to .276). A-levels had higher CLPVs for all undergraduate and postgraduate assessments than did GCSEs/O-levels and intellectual aptitude tests. CLPVs of educational attainment measures decline somewhat during training, but continue to predict postgraduate performance. Intellectual aptitude tests have lower CLPVs than A-levels or GCSEs/O-levels. Conclusions Educational attainment has strong CLPVs for undergraduate and postgraduate performance, accounting for perhaps 65% of true variance in first year performance. Such CLPVs justify the use of educational attainment measure in selection, but also raise a key theoretical question concerning the remaining 35% of variance (and measurement error, range restriction and right-censorship have been taken into account). Just as in astrophysics, ‘dark matter’ and ‘dark energy’ are posited to balance various theoretical equations, so medical student selection must also have its ‘dark variance’, whose nature is not yet properly characterized, but explains a third of the variation in performance during training. Some variance probably relates to factors which are unpredictable at selection, such as illness or other life events, but some is probably also associated with factors such as personality, motivation or study skills. PMID:24229353

A high power ion thruster for deep space missions

NASA Astrophysics Data System (ADS)

Polk, James E.; Goebel, Dan M.; Snyder, John S.; Schneider, Analyn C.; Johnson, Lee K.; Sengupta, Anita

2012-07-01

The Nuclear Electric Xenon Ion System ion thruster was developed for potential outer planet robotic missions using nuclear electric propulsion (NEP). This engine was designed to operate at power levels ranging from 13 to 28 kW at specific impulses of 6000-8500 s and for burn times of up to 10 years. State-of-the-art performance and life assessment tools were used to design the thruster, which featured 57-cm-diameter carbon-carbon composite grids operating at voltages of 3.5-6.5 kV. Preliminary validation of the thruster performance was accomplished with a laboratory model thruster, while in parallel, a flight-like development model (DM) thruster was completed and two DM thrusters fabricated. The first thruster completed full performance testing and a 2000-h wear test. The second successfully completed vibration tests at the full protoflight levels defined for this NEP program and then passed performance validation testing. The thruster design, performance, and the experimental validation of the design tools are discussed in this paper.

A high power ion thruster for deep space missions.

PubMed

Polk, James E; Goebel, Dan M; Snyder, John S; Schneider, Analyn C; Johnson, Lee K; Sengupta, Anita

2012-07-01

The Nuclear Electric Xenon Ion System ion thruster was developed for potential outer planet robotic missions using nuclear electric propulsion (NEP). This engine was designed to operate at power levels ranging from 13 to 28 kW at specific impulses of 6000-8500 s and for burn times of up to 10 years. State-of-the-art performance and life assessment tools were used to design the thruster, which featured 57-cm-diameter carbon-carbon composite grids operating at voltages of 3.5-6.5 kV. Preliminary validation of the thruster performance was accomplished with a laboratory model thruster, while in parallel, a flight-like development model (DM) thruster was completed and two DM thrusters fabricated. The first thruster completed full performance testing and a 2000-h wear test. The second successfully completed vibration tests at the full protoflight levels defined for this NEP program and then passed performance validation testing. The thruster design, performance, and the experimental validation of the design tools are discussed in this paper.

Validity of the two-level model for Viterbi decoder gap-cycle performance

NASA Technical Reports Server (NTRS)

Dolinar, S.; Arnold, S.

1990-01-01

A two-level model has previously been proposed for approximating the performance of a Viterbi decoder which encounters data received with periodically varying signal-to-noise ratio. Such cyclically gapped data is obtained from the Very Large Array (VLA), either operating as a stand-alone system or arrayed with Goldstone. This approximate model predicts that the decoder error rate will vary periodically between two discrete levels with the same period as the gap cycle. It further predicts that the length of the gapped portion of the decoder error cycle for a constraint length K decoder will be about K-1 bits shorter than the actual duration of the gap. The two-level model for Viterbi decoder performance with gapped data is subjected to detailed validation tests. Curves showing the cyclical behavior of the decoder error burst statistics are compared with the simple square-wave cycles predicted by the model. The validity of the model depends on a parameter often considered irrelevant in the analysis of Viterbi decoder performance, the overall scaling of the received signal or the decoder's branch-metrics. Three scaling alternatives are examined: optimum branch-metric scaling and constant branch-metric scaling combined with either constant noise-level scaling or constant signal-level scaling. The simulated decoder error cycle curves roughly verify the accuracy of the two-level model for both the case of optimum branch-metric scaling and the case of constant branch-metric scaling combined with constant noise-level scaling. However, the model is not accurate for the case of constant branch-metric scaling combined with constant signal-level scaling.

The Validity and Incremental Validity of Knowledge Tests, Low-Fidelity Simulations, and High-Fidelity Simulations for Predicting Job Performance in Advanced-Level High-Stakes Selection

ERIC Educational Resources Information Center

Lievens, Filip; Patterson, Fiona

2011-01-01

In high-stakes selection among candidates with considerable domain-specific knowledge and experience, investigations of whether high-fidelity simulations (assessment centers; ACs) have incremental validity over low-fidelity simulations (situational judgment tests; SJTs) are lacking. Therefore, this article integrates research on the validity of…

Development, Validation and Integration of the ATLAS Trigger System Software in Run 2

NASA Astrophysics Data System (ADS)

Keyes, Robert; ATLAS Collaboration

2017-10-01

The trigger system of the ATLAS detector at the LHC is a combination of hardware, firmware, and software, associated to various sub-detectors that must seamlessly cooperate in order to select one collision of interest out of every 40,000 delivered by the LHC every millisecond. These proceedings discuss the challenges, organization and work flow of the ongoing trigger software development, validation, and deployment. The goal of this development is to ensure that the most up-to-date algorithms are used to optimize the performance of the experiment. The goal of the validation is to ensure the reliability and predictability of the software performance. Integration tests are carried out to ensure that the software deployed to the online trigger farm during data-taking run as desired. Trigger software is validated by emulating online conditions using a benchmark run and mimicking the reconstruction that occurs during normal data-taking. This exercise is computationally demanding and thus runs on the ATLAS high performance computing grid with high priority. Performance metrics ranging from low-level memory and CPU requirements, to distributions and efficiencies of high-level physics quantities are visualized and validated by a range of experts. This is a multifaceted critical task that ties together many aspects of the experimental effort and thus directly influences the overall performance of the ATLAS experiment.

Validity threats: overcoming interference with proposed interpretations of assessment data.

PubMed

Downing, Steven M; Haladyna, Thomas M

2004-03-01

Factors that interfere with the ability to interpret assessment scores or ratings in the proposed manner threaten validity. To be interpreted in a meaningful manner, all assessments in medical education require sound, scientific evidence of validity. The purpose of this essay is to discuss 2 major threats to validity: construct under-representation (CU) and construct-irrelevant variance (CIV). Examples of each type of threat for written, performance and clinical performance examinations are provided. The CU threat to validity refers to undersampling the content domain. Using too few items, cases or clinical performance observations to adequately generalise to the domain represents CU. Variables that systematically (rather than randomly) interfere with the ability to meaningfully interpret scores or ratings represent CIV. Issues such as flawed test items written at inappropriate reading levels or statistically biased questions represent CIV in written tests. For performance examinations, such as standardised patient examinations, flawed cases or cases that are too difficult for student ability contribute CIV to the assessment. For clinical performance data, systematic rater error, such as halo or central tendency error, represents CIV. The term face validity is rejected as representative of any type of legitimate validity evidence, although the fact that the appearance of the assessment may be an important characteristic other than validity is acknowledged. There are multiple threats to validity in all types of assessment in medical education. Methods to eliminate or control validity threats are suggested.

Individualized prediction of perineural invasion in colorectal cancer: development and validation of a radiomics prediction model.

PubMed

Huang, Yanqi; He, Lan; Dong, Di; Yang, Caiyun; Liang, Cuishan; Chen, Xin; Ma, Zelan; Huang, Xiaomei; Yao, Su; Liang, Changhong; Tian, Jie; Liu, Zaiyi

2018-02-01

To develop and validate a radiomics prediction model for individualized prediction of perineural invasion (PNI) in colorectal cancer (CRC). After computed tomography (CT) radiomics features extraction, a radiomics signature was constructed in derivation cohort (346 CRC patients). A prediction model was developed to integrate the radiomics signature and clinical candidate predictors [age, sex, tumor location, and carcinoembryonic antigen (CEA) level]. Apparent prediction performance was assessed. After internal validation, independent temporal validation (separate from the cohort used to build the model) was then conducted in 217 CRC patients. The final model was converted to an easy-to-use nomogram. The developed radiomics nomogram that integrated the radiomics signature and CEA level showed good calibration and discrimination performance [Harrell's concordance index (c-index): 0.817; 95% confidence interval (95% CI): 0.811-0.823]. Application of the nomogram in validation cohort gave a comparable calibration and discrimination (c-index: 0.803; 95% CI: 0.794-0.812). Integrating the radiomics signature and CEA level into a radiomics prediction model enables easy and effective risk assessment of PNI in CRC. This stratification of patients according to their PNI status may provide a basis for individualized auxiliary treatment.

The validity of consumer-level, activity monitors in healthy adults worn in free-living conditions: a cross-sectional study.

PubMed

Ferguson, Ty; Rowlands, Alex V; Olds, Tim; Maher, Carol

2015-03-27

Technological advances have seen a burgeoning industry for accelerometer-based wearable activity monitors targeted at the consumer market. The purpose of this study was to determine the convergent validity of a selection of consumer-level accelerometer-based activity monitors. 21 healthy adults wore seven consumer-level activity monitors (Fitbit One, Fitbit Zip, Jawbone UP, Misfit Shine, Nike Fuelband, Striiv Smart Pedometer and Withings Pulse) and two research-grade accelerometers/multi-sensor devices (BodyMedia SenseWear, and ActiGraph GT3X+) for 48-hours. Participants went about their daily life in free-living conditions during data collection. The validity of the consumer-level activity monitors relative to the research devices for step count, moderate to vigorous physical activity (MVPA), sleep and total daily energy expenditure (TDEE) was quantified using Bland-Altman analysis, median absolute difference and Pearson's correlation. All consumer-level activity monitors correlated strongly (r > 0.8) with research-grade devices for step count and sleep time, but only moderately-to-strongly for TDEE (r = 0.74-0.81) and MVPA (r = 0.52-0.91). Median absolute differences were generally modest for sleep and steps (<10% of research device mean values for the majority of devices) moderate for TDEE (<30% of research device mean values), and large for MVPA (26-298%). Across the constructs examined, the Fitbit One, Fitbit Zip and Withings Pulse performed most strongly. In free-living conditions, the consumer-level activity monitors showed strong validity for the measurement of steps and sleep duration, and moderate valid for measurement of TDEE and MVPA. Validity for each construct ranged widely between devices, with the Fitbit One, Fitbit Zip and Withings Pulse being the strongest performers.

Validity and reliability of a video questionnaire to assess physical function in older adults.

PubMed

Balachandran, Anoop; N Verduin, Chelsea; Potiaumpai, Melanie; Ni, Meng; Signorile, Joseph F

2016-08-01

Self-report questionnaires are widely used to assess physical function in older adults. However, they often lack a clear frame of reference and hence interpreting and rating task difficulty levels can be problematic for the responder. Consequently, the usefulness of traditional self-report questionnaires for assessing higher-level functioning is limited. Video-based questionnaires can overcome some of these limitations by offering a clear and objective visual reference for the performance level against which the subject is to compare his or her perceived capacity. Hence the purpose of the study was to develop and validate a novel, video-based questionnaire to assess physical function in older adults independently living in the community. A total of 61 community-living adults, 60years or older, were recruited. To examine validity, 35 of the subjects completed the video questionnaire, two types of physical performance tests: a test of instrumental activity of daily living (IADL) included in the Short Physical Functional Performance battery (PFP-10), and a composite of 3 performance tests (30s chair stand, single-leg balance and usual gait speed). To ascertain reliability, two-week test-retest reliability was assessed in the remaining 26 subjects who did not participate in validity testing. The video questionnaire showed a moderate correlation with the IADLs (Spearman rho=0.64, p<0.001; 95% CI (0.4, 0.8)), and a lower correlation with the composite score of physical performance tests (Spearman rho=0.49, p<0.01; 95% CI (0.18, 0.7)). The test-retest assessment yielded an intra-class correlation (ICC) of 0.87 (p<0.001; 95% CI (0.70, 0.94)) and a Cronbach's alpha of 0.89 demonstrating good reliability and internal consistency. Our results show that the video questionnaire developed to evaluate physical function in community-living older adults is a valid and reliable assessment tool; however, further validation is needed for definitive conclusions. Copyright © 2016 Elsevier Inc. All rights reserved.

Validation and Inter-comparison Against Observations of GODAE Ocean View Ocean Prediction Systems

NASA Astrophysics Data System (ADS)

Xu, J.; Davidson, F. J. M.; Smith, G. C.; Lu, Y.; Hernandez, F.; Regnier, C.; Drevillon, M.; Ryan, A.; Martin, M.; Spindler, T. D.; Brassington, G. B.; Oke, P. R.

2016-02-01

For weather forecasts, validation of forecast performance is done at the end user level as well as by the meteorological forecast centers. In the development of Ocean Prediction Capacity, the same level of care for ocean forecast performance and validation is needed. Herein we present results from a validation against observations of 6 Global Ocean Forecast Systems under the GODAE OceanView International Collaboration Network. These systems include the Global Ocean Ice Forecast System (GIOPS) developed by the Government of Canada, two systems PSY3 and PSY4 from the French Mercator-Ocean Ocean Forecasting Group, the FOAM system from UK met office, HYCOM-RTOFS from NOAA/NCEP/NWA of USA, and the Australian Bluelink-OceanMAPS system from the CSIRO, the Australian Meteorological Bureau and the Australian Navy.The observation data used in the comparison are sea surface temperature, sub-surface temperature, sub-surface salinity, sea level anomaly, and sea ice total concentration data. Results of the inter-comparison demonstrate forecast performance limits, strengths and weaknesses of each of the six systems. This work establishes validation protocols and routines by which all new prediction systems developed under the CONCEPTS Collaborative Network will be benchmarked prior to approval for operations. This includes anticipated delivery of CONCEPTS regional prediction systems over the next two years including a pan Canadian 1/12th degree resolution ice ocean prediction system and limited area 1/36th degree resolution prediction systems. The validation approach of comparing forecasts to observations at the time and location of the observation is called Class 4 metrics. It has been adopted by major international ocean prediction centers, and will be recommended to JCOMM-WMO as routine validation approach for operational oceanography worldwide.

Development of a new instrument for determining the level of chewing function in children.

PubMed

Serel Arslan, S; Demir, N; Barak Dolgun, A; Karaduman, A A

2016-07-01

This study aimed to develop a chewing performance scale that classifies chewing from normal to severely impaired and to investigate its validity and reliability. The study included the developmental phase and reported the content, structural, criterion validity, interobserver and intra-observer reliability of the chewing performance scale, which was called the Karaduman Chewing Performance Scale (KCPS). A dysphagia literature review, other questionnaires and clinical experiences were used in the developmental phase. Seven experts assessed the steps for content validity over two Delphi rounds. To test structural, criterion validity, interobserver and intra-observer reliability, two swallowing therapists evaluated chewing videos of 144 children (Group I: 61 healthy children without chewing disorders, mean age of 42·38 ± 9·36 months; Group II: 83 children with cerebral palsy who have chewing disorders, mean age of 39·09 ± 22·95 months) using KCPS. The Behavioral Pediatrics Feeding Assessment Scale (BPFAS) was used for criterion validity. The KCPS steps arranged between 0-4 were found to be necessary. The content validity index was 0·885. The KCPS levels were found to be different between groups I and II (χ(2) = 123·286, P < 0·001). A moderately strong positive correlation was found between the KCPS and the subscales of the BPFAS (r = 0·444-0·773, P < 0·001). An excellent positive correlation was detected between two swallowing therapists and between two examinations of one swallowing therapist (r = 0·962, P < 0·001; r = 0·990, P < 0·001, respectively). The KCPS is a valid, reliable, quick and clinically easy-to-use functional instrument for determining the level of chewing function in children. © 2016 John Wiley & Sons Ltd.

Determination of anthelmintic drug residues in milk using ultra high performance liquid chromatography-tandem mass spectrometry with rapid polarity switching.

PubMed

Whelan, Michelle; Kinsella, Brian; Furey, Ambrose; Moloney, Mary; Cantwell, Helen; Lehotay, Steven J; Danaher, Martin

2010-07-02

A new UHPLC-MS/MS (ultra high performance liquid chromatography coupled to tandem mass spectrometry) method was developed and validated to detect 38 anthelmintic drug residues, consisting of benzimidazoles, avermectins and flukicides. A modified QuEChERS-type extraction method was developed with an added concentration step to detect most of the analytes at <1 microg kg(-1) levels in milk. Anthelmintic residues were extracted into acetonitrile using magnesium sulphate and sodium chloride to induce liquid-liquid partitioning followed by dispersive solid phase extraction for cleanup. The extract was concentrated into dimethyl sulphoxide, which was used as a keeper to ensure analytes remain in solution. Using rapid polarity switching in electrospray ionisation, a single injection was capable of detecting both positively and negatively charged ions in a 13 min run time. The method was validated at two levels: the unapproved use level and at the maximum residue level (MRL) according to Commission Decision (CD) 2002/657/EC criteria. The decision limit (CCalpha) of the method was in the range of 0.14-1.9 and 11-123 microg kg(-1) for drugs validated at unapproved and MRL levels, respectively. The performance of the method was successfully verified for benzimidazoles and levamisole by participating in a proficiency study.

Development and validation of rapid multiresidue and multi-class analysis for antibiotics and anthelmintics in feed by ultra-high-performance liquid chromatography coupled to tandem mass spectrometry.

PubMed

Robert, Christelle; Brasseur, Pierre-Yves; Dubois, Michel; Delahaut, Philippe; Gillard, Nathalie

2016-08-01

A new multi-residue method for the analysis of veterinary drugs, namely amoxicillin, chlortetracycline, colistins A and B, doxycycline, fenbendazole, flubendazole, ivermectin, lincomycin, oxytetracycline, sulfadiazine, tiamulin, tilmicosin and trimethoprim, was developed and validated for feed. After acidic extraction, the samples were centrifuged, purified by SPE and analysed by ultra-high-performance liquid chromatography coupled to tandem mass spectrometry. Quantitative validation was done in accordance with the guidelines laid down in European Commission Decision 2002/657/CE. Matrix-matched calibration with internal standards was used to reduce matrix effects. The target level was set at the authorised carryover level (1%) and validation levels were set at 0.5%, 1% and 1.5%. Method performances were evaluated by the following parameters: linearity (0.986 < R(2) < 0.999), precision (repeatability < 12.4% and reproducibility < 14.0%), accuracy (89% < recovery < 107%), sensitivity, decision limit (CCα), detection capability (CCβ), selectivity and expanded measurement uncertainty (k = 2).This method has been used successfully for three years for routine monitoring of antibiotic residues in feeds during which period 20% of samples were found to exceed the 1% authorised carryover limit and were deemed non-compliant.

Flight-Test Validation and Flying Qualities Evaluation of a Rotorcraft UAV Flight Control System

NASA Technical Reports Server (NTRS)

Mettler, Bernard; Tuschler, Mark B.; Kanade, Takeo

2000-01-01

This paper presents a process of design and flight-test validation and flying qualities evaluation of a flight control system for a rotorcraft-based unmanned aerial vehicle (RUAV). The keystone of this process is an accurate flight-dynamic model of the aircraft, derived by using system identification modeling. The model captures the most relevant dynamic features of our unmanned rotorcraft, and explicitly accounts for the presence of a stabilizer bar. Using the identified model we were able to determine the performance margins of our original control system and identify limiting factors. The performance limitations were addressed and the attitude control system was 0ptimize.d for different three performance levels: slow, medium, fast. The optimized control laws will be implemented in our RUAV. We will first determine the validity of our control design approach by flight test validating our optimized controllers. Subsequently, we will fly a series of maneuvers with the three optimized controllers to determine the level of flying qualities that can be attained. The outcome enable us to draw important conclusions on the flying qualities requirements for small-scale RUAVs.

The Role of Integrated Modeling in the Design and Verification of the James Webb Space Telescope

NASA Technical Reports Server (NTRS)

Mosier, Gary E.; Howard, Joseph M.; Johnston, John D.; Parrish, Keith A.; Hyde, T. Tupper; McGinnis, Mark A.; Bluth, Marcel; Kim, Kevin; Ha, Kong Q.

2004-01-01

The James Web Space Telescope (JWST) is a large, infrared-optimized space telescope scheduled for launch in 2011. System-level verification of critical optical performance requirements will rely on integrated modeling to a considerable degree. In turn, requirements for accuracy of the models are significant. The size of the lightweight observatory structure, coupled with the need to test at cryogenic temperatures, effectively precludes validation of the models and verification of optical performance with a single test in 1-g. Rather, a complex series of steps are planned by which the components of the end-to-end models are validated at various levels of subassembly, and the ultimate verification of optical performance is by analysis using the assembled models. This paper describes the critical optical performance requirements driving the integrated modeling activity, shows how the error budget is used to allocate and track contributions to total performance, and presents examples of integrated modeling methods and results that support the preliminary observatory design. Finally, the concepts for model validation and the role of integrated modeling in the ultimate verification of observatory are described.

Timed activity performance in persons with upper limb amputation: A preliminary study.

PubMed

Resnik, Linda; Borgia, Mathew; Acluche, Frantzy

55 subjects with upper limb amputation were administered the T-MAP twice within one week. To develop a timed measure of activity performance for persons with upper limb amputation (T-MAP); examine the measure's internal consistency, test-retest reliability and validity; and compare scores by prosthesis use. Measures of activity performance for persons with upper limb amputation are needed The time required to perform daily activities is a meaningful metric that implication for participation in life roles. Internal consistency and test-retest reliability were evaluated. Construct validity was examined by comparing scores by amputation level. Exploratory analyses compared sub-group scores, and examined correlations with other measures. Scale alpha was 0.77, ICC was 0.93. Timed scores differed by amputation level. Subjects using a prosthesis took longer to perform all tasks. T-MAP was not correlated with other measures of dexterity or activity, but was correlated with pain for non-prosthesis users. The timed scale had adequate internal consistency and excellent test-retest reliability. Analyses support reliability and construct validity of the T-MAP. 2c "outcomes" research. Published by Elsevier Inc.

Pulsed Inductive Thruster (PIT): Modeling and Validation Using the MACH2 Code

NASA Technical Reports Server (NTRS)

Schneider, Steven (Technical Monitor); Mikellides, Pavlos G.

2003-01-01

Numerical modeling of the Pulsed Inductive Thruster exercising the magnetohydrodynamics code, MACH2 aims to provide bilateral validation of the thruster's measured performance and the code's capability of capturing the pertinent physical processes. Computed impulse values for helium and argon propellants demonstrate excellent correlation to the experimental data for a range of energy levels and propellant-mass values. The effects of the vacuum tank wall and massinjection scheme were investigated to show trivial changes in the overall performance. An idealized model for these energy levels and propellants deduces that the energy expended to the internal energy modes and plasma dissipation processes is independent of the propellant type, mass, and energy level.

Validating a biometric authentication system: sample size requirements.

PubMed

Dass, Sarat C; Zhu, Yongfang; Jain, Anil K

2006-12-01

Authentication systems based on biometric features (e.g., fingerprint impressions, iris scans, human face images, etc.) are increasingly gaining widespread use and popularity. Often, vendors and owners of these commercial biometric systems claim impressive performance that is estimated based on some proprietary data. In such situations, there is a need to independently validate the claimed performance levels. System performance is typically evaluated by collecting biometric templates from n different subjects, and for convenience, acquiring multiple instances of the biometric for each of the n subjects. Very little work has been done in 1) constructing confidence regions based on the ROC curve for validating the claimed performance levels and 2) determining the required number of biometric samples needed to establish confidence regions of prespecified width for the ROC curve. To simplify the analysis that address these two problems, several previous studies have assumed that multiple acquisitions of the biometric entity are statistically independent. This assumption is too restrictive and is generally not valid. We have developed a validation technique based on multivariate copula models for correlated biometric acquisitions. Based on the same model, we also determine the minimum number of samples required to achieve confidence bands of desired width for the ROC curve. We illustrate the estimation of the confidence bands as well as the required number of biometric samples using a fingerprint matching system that is applied on samples collected from a small population.

Performance-based comparison of neonatal intubation training outcomes: simulator and live animal.

PubMed

Andreatta, Pamela B; Klotz, Jessica J; Dooley-Hash, Suzanne L; Hauptman, Joe G; Biddinger, Bea; House, Joseph B

2015-02-01

The purpose of this article was to establish psychometric validity evidence for competency assessment instruments and to evaluate the impact of 2 forms of training on the abilities of clinicians to perform neonatal intubation. To inform the development of assessment instruments, we conducted comprehensive task analyses including each performance domain associated with neonatal intubation. Expert review confirmed content validity. Construct validity was established using the instruments to differentiate between the intubation performance abilities of practitioners (N = 294) with variable experience (novice through expert). Training outcomes were evaluated using a quasi-experimental design to evaluate performance differences between 294 subjects randomly assigned to 1 of 2 training groups. The training intervention followed American Heart Association Pediatric Advanced Life Support and Neonatal Resuscitation Program protocols with hands-on practice using either (1) live feline or (2) simulated feline models. Performance assessment data were captured before and directly following the training. All data were analyzed using analysis of variance with repeated measures and statistical significance set at P < .05. Content validity, reliability, and consistency evidence were established for each assessment instrument. Construct validity for each assessment instrument was supported by significantly higher scores for subjects with greater levels of experience, as compared with those with less experience (P = .000). Overall, subjects performed significantly better in each assessment domain, following the training intervention (P = .000). After controlling for experience level, there were no significant differences among the cognitive, performance, and self-efficacy outcomes between clinicians trained with live animal model or simulator model. Analysis of retention scores showed that simulator trained subjects had significantly higher performance scores after 18 weeks (P = .01) and 52 weeks (P = .001) and cognitive scores after 52 weeks (P = .001). The results of this study demonstrate the feasibility of using valid, reliable assessment instruments to assess clinician competency and self-efficacy in the performance of neonatal intubation. We demonstrated the relative equivalency of live animal and simulation-based models as tools to support acquisition of neonatal intubation skills. Retention of performance abilities was greater for subjects trained using the simulator, likely because it afforded greater opportunity for repeated practice. Outcomes in each assessment area were influenced by the previous intubation experience of participants. This suggests that neonatal intubation training programs could be tailored to the level of provider experience to make efficient use of time and educational resources. Future research focusing on the uses of assessment in the applied clinical environment, as well as identification of optimal training cycles for performance retention, is merited.

Derivation and Cross-Validation of Cutoff Scores for Patients With Schizophrenia Spectrum Disorders on WAIS-IV Digit Span-Based Performance Validity Measures.

PubMed

Glassmire, David M; Toofanian Ross, Parnian; Kinney, Dominique I; Nitch, Stephen R

2016-06-01

Two studies were conducted to identify and cross-validate cutoff scores on the Wechsler Adult Intelligence Scale-Fourth Edition Digit Span-based embedded performance validity (PV) measures for individuals with schizophrenia spectrum disorders. In Study 1, normative scores were identified on Digit Span-embedded PV measures among a sample of patients (n = 84) with schizophrenia spectrum diagnoses who had no known incentive to perform poorly and who put forth valid effort on external PV tests. Previously identified cutoff scores resulted in unacceptable false positive rates and lower cutoff scores were adopted to maintain specificity levels ≥90%. In Study 2, the revised cutoff scores were cross-validated within a sample of schizophrenia spectrum patients (n = 96) committed as incompetent to stand trial. Performance on Digit Span PV measures was significantly related to Full Scale IQ in both studies, indicating the need to consider the intellectual functioning of examinees with psychotic spectrum disorders when interpreting scores on Digit Span PV measures. © The Author(s) 2015.

Development of student performance assessment based on scientific approach for a basic physics practicum in simple harmonic motion materials

NASA Astrophysics Data System (ADS)

Serevina, V.; Muliyati, D.

2018-05-01

This research aims to develop students’ performance assessment instrument based on scientific approach is valid and reliable in assessing the performance of students on basic physics lab of Simple Harmonic Motion (SHM). This study uses the ADDIE consisting of stages: Analyze, Design, Development, Implementation, and Evaluation. The student performance assessment developed can be used to measure students’ skills in observing, asking, conducting experiments, associating and communicate experimental results that are the ‘5M’ stages in a scientific approach. Each grain of assessment in the instrument is validated by the instrument expert and the evaluation with the result of all points of assessment shall be eligible to be used with a 100% eligibility percentage. The instrument is then tested for the quality of construction, material, and language by panel (lecturer) with the result: 85% or very good instrument construction aspect, material aspect 87.5% or very good, and language aspect 83% or very good. For small group trial obtained instrument reliability level of 0.878 or is in the high category, where r-table is 0.707. For large group trial obtained instrument reliability level of 0.889 or is in the high category, where r-table is 0.320. Instruments declared valid and reliable for 5% significance level. Based on the result of this research, it can be concluded that the student performance appraisal instrument based on the developed scientific approach is declared valid and reliable to be used in assessing student skill in SHM experimental activity.

Validity, Reliability, and Sensitivity of a Volleyball Intermittent Endurance Test.

PubMed

Rodríguez-Marroyo, Jose A; Medina-Carrillo, Javier; García-López, Juan; Morante, Juan C; Villa, José G; Foster, Carl

2017-03-01

To analyze the concurrent and construct validity of a volleyball intermittent endurance test (VIET). The VIET's test-retest reliability and sensitivity to assess seasonal changes was also studied. During the preseason, 71 volleyball players of different competitive levels took part in this study. All performed the VIET and a graded treadmill test with gas-exchange measurement (GXT). Thirty-one of the players performed an additional VIET to analyze the test-retest reliability. To test the VIET's sensitivity, 28 players repeated the VIET and GXT at the end of their season. Significant (P < .001) relationships between VIET distance and maximal oxygen uptake (r = .74) and GXT maximal speed (r = .78) were observed. There were no significant differences between the VIET performance test and retest (1542.1 ± 338.1 vs 1567.1 ± 358.2 m). Significant (P < .001) relationships and intraclass correlation coefficient (ICC) were found (r = .95, ICC = .96) for VIET performance. VIET performance increased significantly (P < .001) with player performance level and was sensitive to fitness changes across the season (1458.8 ± 343.5 vs 1581.1 ± 334.0 m, P < .01). The VIET may be considered a valid, reliable, and sensitive test to assess the aerobic endurance in volleyball players.

Calibration and Validation Plan for the L2A Processor and Products of the SENTINEL-2 Mission

NASA Astrophysics Data System (ADS)

Main-Knorn, M.; Pflug, B.; Debaecker, V.; Louis, J.

2015-04-01

The Copernicus programme, is a European initiative for the implementation of information services based on observation data received from Earth Observation (EO) satellites and ground based information. In the frame of this programme, ESA is developing the Sentinel-2 optical imaging mission that will deliver optical data products designed to feed downstream services mainly related to land monitoring, emergency management and security. To ensure the highest quality of service, ESA sets up the Sentinel-2 Mission Performance Centre (MPC) in charge of the overall performance monitoring of the Sentinel-2 mission. TPZ F and DLR have teamed up in order to provide the best added-value support to the MPC for calibration and validation of the Level-2A processor (Sen2Cor) and products. This paper gives an overview over the planned L2A calibration and validation activities. Level-2A processing is applied to Top-Of-Atmosphere (TOA) Level-1C ortho-image reflectance products. Level-2A main output is the Bottom-Of-Atmosphere (BOA) corrected reflectance product. Additional outputs are an Aerosol Optical Thickness (AOT) map, a Water Vapour (WV) map and a Scene Classification (SC) map with Quality Indicators for cloud and snow probabilities. Level-2A BOA, AOT and WV outputs are calibrated and validated using ground-based data of automatic operating stations and data of in-situ campaigns. Scene classification is validated by the visual inspection of test datasets and cross-sensor comparison, supplemented by meteorological data, if available. Contributions of external in-situ campaigns would enlarge the reference dataset and enable extended validation exercise. Therefore, we are highly interested in and welcome external contributors.

Validation approach for a fast and simple targeted screening method for 75 antibiotics in meat and aquaculture products using LC-MS/MS.

PubMed

Dubreil, Estelle; Gautier, Sophie; Fourmond, Marie-Pierre; Bessiral, Mélaine; Gaugain, Murielle; Verdon, Eric; Pessel, Dominique

2017-04-01

An approach is described to validate a fast and simple targeted screening method for antibiotic analysis in meat and aquaculture products by LC-MS/MS. The strategy of validation was applied for a panel of 75 antibiotics belonging to different families, i.e., penicillins, cephalosporins, sulfonamides, macrolides, quinolones and phenicols. The samples were extracted once with acetonitrile, concentrated by evaporation and injected into the LC-MS/MS system. The approach chosen for the validation was based on the Community Reference Laboratory (CRL) guidelines for the validation of screening qualitative methods. The aim of the validation was to prove sufficient sensitivity of the method to detect all the targeted antibiotics at the level of interest, generally the maximum residue limit (MRL). A robustness study was also performed to test the influence of different factors. The validation showed that the method is valid to detect and identify 73 antibiotics of the 75 antibiotics studied in meat and aquaculture products at the validation levels.

External validation of a Cox prognostic model: principles and methods

PubMed Central

2013-01-01

Background A prognostic model should not enter clinical practice unless it has been demonstrated that it performs a useful role. External validation denotes evaluation of model performance in a sample independent of that used to develop the model. Unlike for logistic regression models, external validation of Cox models is sparsely treated in the literature. Successful validation of a model means achieving satisfactory discrimination and calibration (prediction accuracy) in the validation sample. Validating Cox models is not straightforward because event probabilities are estimated relative to an unspecified baseline function. Methods We describe statistical approaches to external validation of a published Cox model according to the level of published information, specifically (1) the prognostic index only, (2) the prognostic index together with Kaplan-Meier curves for risk groups, and (3) the first two plus the baseline survival curve (the estimated survival function at the mean prognostic index across the sample). The most challenging task, requiring level 3 information, is assessing calibration, for which we suggest a method of approximating the baseline survival function. Results We apply the methods to two comparable datasets in primary breast cancer, treating one as derivation and the other as validation sample. Results are presented for discrimination and calibration. We demonstrate plots of survival probabilities that can assist model evaluation. Conclusions Our validation methods are applicable to a wide range of prognostic studies and provide researchers with a toolkit for external validation of a published Cox model. PMID:23496923

Propulsion Risk Reduction Activities for Non-Toxic Cryogenic Propulsion

NASA Technical Reports Server (NTRS)

Smith, Timothy D.; Klem, Mark D.; Fisher, Kenneth

2010-01-01

The Propulsion and Cryogenics Advanced Development (PCAD) Project s primary objective is to develop propulsion system technologies for non-toxic or "green" propellants. The PCAD project focuses on the development of non-toxic propulsion technologies needed to provide necessary data and relevant experience to support informed decisions on implementation of non-toxic propellants for space missions. Implementation of non-toxic propellants in high performance propulsion systems offers NASA an opportunity to consider other options than current hypergolic propellants. The PCAD Project is emphasizing technology efforts in reaction control system (RCS) thruster designs, ascent main engines (AME), and descent main engines (DME). PCAD has a series of tasks and contracts to conduct risk reduction and/or retirement activities to demonstrate that non-toxic cryogenic propellants can be a feasible option for space missions. Work has focused on 1) reducing the risk of liquid oxygen/liquid methane ignition, demonstrating the key enabling technologies, and validating performance levels for reaction control engines for use on descent and ascent stages; 2) demonstrating the key enabling technologies and validating performance levels for liquid oxygen/liquid methane ascent engines; and 3) demonstrating the key enabling technologies and validating performance levels for deep throttling liquid oxygen/liquid hydrogen descent engines. The progress of these risk reduction and/or retirement activities will be presented.

Propulsion Risk Reduction Activities for Nontoxic Cryogenic Propulsion

NASA Technical Reports Server (NTRS)

Smith, Timothy D.; Klem, Mark D.; Fisher, Kenneth L.

2010-01-01

The Propulsion and Cryogenics Advanced Development (PCAD) Project s primary objective is to develop propulsion system technologies for nontoxic or "green" propellants. The PCAD project focuses on the development of nontoxic propulsion technologies needed to provide necessary data and relevant experience to support informed decisions on implementation of nontoxic propellants for space missions. Implementation of nontoxic propellants in high performance propulsion systems offers NASA an opportunity to consider other options than current hypergolic propellants. The PCAD Project is emphasizing technology efforts in reaction control system (RCS) thruster designs, ascent main engines (AME), and descent main engines (DME). PCAD has a series of tasks and contracts to conduct risk reduction and/or retirement activities to demonstrate that nontoxic cryogenic propellants can be a feasible option for space missions. Work has focused on 1) reducing the risk of liquid oxygen/liquid methane ignition, demonstrating the key enabling technologies, and validating performance levels for reaction control engines for use on descent and ascent stages; 2) demonstrating the key enabling technologies and validating performance levels for liquid oxygen/liquid methane ascent engines; and 3) demonstrating the key enabling technologies and validating performance levels for deep throttling liquid oxygen/liquid hydrogen descent engines. The progress of these risk reduction and/or retirement activities will be presented.

Measurement of Functional Cognition and Complex Everyday Activities in Older Adults with Mild Cognitive Impairment and Mild Dementia: Validity of the Large Allen's Cognitive Level Screen.

PubMed

Wesson, Jacqueline; Clemson, Lindy; Crawford, John D; Kochan, Nicole A; Brodaty, Henry; Reppermund, Simone

2017-05-01

To explore the validity of the Large Allen's Cognitive Level Screen-5 (LACLS-5) as a performance-based measure of functional cognition, representing an ability to perform complex everyday activities in older adults with mild cognitive impairment (MCI) and mild dementia living in the community. Using cross-sectional data from the Sydney Memory and Ageing Study, 160 community-dwelling older adults with normal cognition (CN; N = 87), MCI (N = 43), or dementia (N = 30) were studied. Functional cognition (LACLS-5), complex everyday activities (Disability Assessment for Dementia [DAD]), Assessment of Motor and Process Skills [AMPS]), and neuropsychological measures were used. Participants with dementia performed worse than CN on all clinical measures, and MCI participants were intermediate. Correlational analyses showed that LACLS-5 was most strongly related to AMPS Process scores, DAD instrumental activities of daily living subscale, Mini-Mental State Exam, Block Design, Logical Memory, and Trail Making Test B. Multiple regression analysis indicated that both cognitive (Block Design) and functional measures (AMPS Process score) and sex predicted LACLS-5 performance. Finally, LACLS-5 was able to adequately discriminate between CN and dementia and between MCI and dementia but was unable to reliably distinguish between CN and MCI. Construct validity, including convergent and discriminative validity, was supported. LACLS-5 is a valid performance-based measure for evaluating functional cognition. Discriminativevalidity is acceptable for identifying mild dementia but requires further refinement for detecting MCI. Copyright © 2017 American Association for Geriatric Psychiatry. Published by Elsevier Inc. All rights reserved.

A guideline for the validation of likelihood ratio methods used for forensic evidence evaluation.

PubMed

Meuwly, Didier; Ramos, Daniel; Haraksim, Rudolf

2017-07-01

This Guideline proposes a protocol for the validation of forensic evaluation methods at the source level, using the Likelihood Ratio framework as defined within the Bayes' inference model. In the context of the inference of identity of source, the Likelihood Ratio is used to evaluate the strength of the evidence for a trace specimen, e.g. a fingermark, and a reference specimen, e.g. a fingerprint, to originate from common or different sources. Some theoretical aspects of probabilities necessary for this Guideline were discussed prior to its elaboration, which started after a workshop of forensic researchers and practitioners involved in this topic. In the workshop, the following questions were addressed: "which aspects of a forensic evaluation scenario need to be validated?", "what is the role of the LR as part of a decision process?" and "how to deal with uncertainty in the LR calculation?". The questions: "what to validate?" focuses on the validation methods and criteria and "how to validate?" deals with the implementation of the validation protocol. Answers to these questions were deemed necessary with several objectives. First, concepts typical for validation standards [1], such as performance characteristics, performance metrics and validation criteria, will be adapted or applied by analogy to the LR framework. Second, a validation strategy will be defined. Third, validation methods will be described. Finally, a validation protocol and an example of validation report will be proposed, which can be applied to the forensic fields developing and validating LR methods for the evaluation of the strength of evidence at source level under the following propositions. Copyright © 2016. Published by Elsevier B.V.

Schooling Effects on Degree Performance: A Comparison of the Predictive Validity of Aptitude Testing and Secondary School Grades at Oxford University

ERIC Educational Resources Information Center

Ogg, Tom; Zimdars, Anna; Heath, Anthony

2009-01-01

This article examines the cause of school type effects upon gaining a first class degree at Oxford University, whereby for a given level of secondary school performance, private school students perform less well at degree level. We compare the predictive power of an aptitude test and secondary school grades (GCSEs) for final examination…

A comprehensive evaluation of two MODIS evapotranspiration products over the conterminous United States: using point and gridded FLUXNET and water balance ET

USGS Publications Warehouse

Velpuri, Naga M.; Senay, Gabriel B.; Singh, Ramesh K.; Bohms, Stefanie; Verdin, James P.

2013-01-01

Remote sensing datasets are increasingly being used to provide spatially explicit large scale evapotranspiration (ET) estimates. Extensive evaluation of such large scale estimates is necessary before they can be used in various applications. In this study, two monthly MODIS 1 km ET products, MODIS global ET (MOD16) and Operational Simplified Surface Energy Balance (SSEBop) ET, are validated over the conterminous United States at both point and basin scales. Point scale validation was performed using eddy covariance FLUXNET ET (FLET) data (2001–2007) aggregated by year, land cover, elevation and climate zone. Basin scale validation was performed using annual gridded FLUXNET ET (GFET) and annual basin water balance ET (WBET) data aggregated by various hydrologic unit code (HUC) levels. Point scale validation using monthly data aggregated by years revealed that the MOD16 ET and SSEBop ET products showed overall comparable annual accuracies. For most land cover types, both ET products showed comparable results. However, SSEBop showed higher performance for Grassland and Forest classes; MOD16 showed improved performance in the Woody Savanna class. Accuracy of both the ET products was also found to be comparable over different climate zones. However, SSEBop data showed higher skill score across the climate zones covering the western United States. Validation results at different HUC levels over 2000–2011 using GFET as a reference indicate higher accuracies for MOD16 ET data. MOD16, SSEBop and GFET data were validated against WBET (2000–2009), and results indicate that both MOD16 and SSEBop ET matched the accuracies of the global GFET dataset at different HUC levels. Our results indicate that both MODIS ET products effectively reproduced basin scale ET response (up to 25% uncertainty) compared to CONUS-wide point-based ET response (up to 50–60% uncertainty) illustrating the reliability of MODIS ET products for basin-scale ET estimation. Results from this research would guide the additional parameter refinement required for the MOD16 and SSEBop algorithms in order to further improve their accuracy and performance for agro-hydrologic applications.

Implementation and application of an interactive user-friendly validation software for RADIANCE

NASA Astrophysics Data System (ADS)

Sundaram, Anand; Boonn, William W.; Kim, Woojin; Cook, Tessa S.

2012-02-01

RADIANCE extracts CT dose parameters from dose sheets using optical character recognition and stores the data in a relational database. To facilitate validation of RADIANCE's performance, a simple user interface was initially implemented and about 300 records were evaluated. Here, we extend this interface to achieve a wider variety of functions and perform a larger-scale validation. The validator uses some data from the RADIANCE database to prepopulate quality-testing fields, such as correspondence between calculated and reported total dose-length product. The interface also displays relevant parameters from the DICOM headers. A total of 5,098 dose sheets were used to test the performance accuracy of RADIANCE in dose data extraction. Several search criteria were implemented. All records were searchable by accession number, study date, or dose parameters beyond chosen thresholds. Validated records were searchable according to additional criteria from validation inputs. An error rate of 0.303% was demonstrated in the validation. Dose monitoring is increasingly important and RADIANCE provides an open-source solution with a high level of accuracy. The RADIANCE validator has been updated to enable users to test the integrity of their installation and verify that their dose monitoring is accurate and effective.

Reliability and Validity of the Inline Skating Skill Test.

PubMed

Radman, Ivan; Ruzic, Lana; Padovan, Viktoria; Cigrovski, Vjekoslav; Podnar, Hrvoje

2016-09-01

This study aimed to examine the reliability and validity of the inline skating skill test. Based on previous skating experience forty-two skaters (26 female and 16 male) were randomized into two groups (competitive level vs. recreational level). They performed the test four times, with a recovery time of 45 minutes between sessions. Prior to testing, the participants rated their skating skill using a scale from 1 to 10. The protocol included performance time measurement through a course, combining different skating techniques. Trivial changes in performance time between the repeated sessions were determined in both competitive females/males and recreational females/males (-1.7% [95% CI: -5.8-2.6%] - 2.2% [95% CI: 0.0-4.5%]). In all four subgroups, the skill test had a low mean within-individual variation (1.6% [95% CI: 1.2-2.4%] - 2.7% [95% CI: 2.1-4.0%]) and high mean inter-session correlation (ICC = 0.97 [95% CI: 0.92-0.99] - 0.99 [95% CI: 0.98-1.00]). The comparison of detected typical errors and smallest worthwhile changes (calculated as standard deviations × 0.2) revealed that the skill test was able to track changes in skaters' performances. Competitive-level skaters needed shorter time (24.4-26.4%, all p < 0.01) to complete the test in comparison to recreational-level skaters. Moreover, moderate correlation (ρ = 0.80-0.82; all p < 0.01) was observed between the participant's self-rating and achieved performance times. In conclusion, the proposed test is a reliable and valid method to evaluate inline skating skills in amateur competitive and recreational level skaters. Further studies are needed to evaluate the reproducibility of this skill test in different populations including elite inline skaters.

Reliability and Validity of the Inline Skating Skill Test

PubMed Central

Radman, Ivan; Ruzic, Lana; Padovan, Viktoria; Cigrovski, Vjekoslav; Podnar, Hrvoje

2016-01-01

This study aimed to examine the reliability and validity of the inline skating skill test. Based on previous skating experience forty-two skaters (26 female and 16 male) were randomized into two groups (competitive level vs. recreational level). They performed the test four times, with a recovery time of 45 minutes between sessions. Prior to testing, the participants rated their skating skill using a scale from 1 to 10. The protocol included performance time measurement through a course, combining different skating techniques. Trivial changes in performance time between the repeated sessions were determined in both competitive females/males and recreational females/males (-1.7% [95% CI: -5.8–2.6%] – 2.2% [95% CI: 0.0–4.5%]). In all four subgroups, the skill test had a low mean within-individual variation (1.6% [95% CI: 1.2–2.4%] – 2.7% [95% CI: 2.1–4.0%]) and high mean inter-session correlation (ICC = 0.97 [95% CI: 0.92–0.99] – 0.99 [95% CI: 0.98–1.00]). The comparison of detected typical errors and smallest worthwhile changes (calculated as standard deviations × 0.2) revealed that the skill test was able to track changes in skaters’ performances. Competitive-level skaters needed shorter time (24.4–26.4%, all p < 0.01) to complete the test in comparison to recreational-level skaters. Moreover, moderate correlation (ρ = 0.80–0.82; all p < 0.01) was observed between the participant’s self-rating and achieved performance times. In conclusion, the proposed test is a reliable and valid method to evaluate inline skating skills in amateur competitive and recreational level skaters. Further studies are needed to evaluate the reproducibility of this skill test in different populations including elite inline skaters. Key points Study evaluated the reliability and construct validity of a newly developed inline skating skill test. Evaluated test is a first protocol designed to assess specific inline skating skill. Two groups of amateur skaters with different skating proficiency repeated the skill test in four separate occasions. The results suggest that evaluated test is reliable and valid to evaluate inline skating skill in amateur skaters. PMID:27803616

Computer-Aided Techniques for Providing Operator Performance Measures.

ERIC Educational Resources Information Center

Connelly, Edward M.; And Others

This report documents the theory, structure, and implementation of a performance processor (written in FORTRAN IV) that can accept performance demonstration data representing various levels of operator's skill and, under user control, analyze data to provide candidate performance measures and validation test results. The processor accepts two…

Construct validity of the ovine model in endoscopic sinus surgery training.

PubMed

Awad, Zaid; Taghi, Ali; Sethukumar, Priya; Tolley, Neil S

2015-03-01

To demonstrate construct validity of the ovine model as a tool for training in endoscopic sinus surgery (ESS). Prospective, cross-sectional evaluation study. Over 18 consecutive months, trainees and experts were evaluated in their ability to perform a range of tasks (based on previous face validation and descriptive studies conducted by the same group) relating to ESS on the sheep-head model. Anonymized randomized video recordings of the above were assessed by two independent and blinded assessors. A validated assessment tool utilizing a five-point Likert scale was employed. Construct validity was calculated by comparing scores across training levels and experts using mean and interquartile range of global and task-specific scores. Subgroup analysis of the intermediate group ascertained previous experience. Nonparametric descriptive statistics were used, and analysis was carried out using SPSS version 21 (IBM, Armonk, NY). Reliability of the assessment tool was confirmed. The model discriminated well between different levels of expertise in global and task-specific scores. A positive correlation was noted between year in training and both global and task-specific scores (P < .001). Experience of the intermediate group was variable, and the number of ESS procedures performed under supervision had the highest impact on performance. This study describes an alternative model for ESS training and assessment. It is also the first to demonstrate construct validity of the sheep-head model for ESS training. © 2014 The American Laryngological, Rhinological and Otological Society, Inc.

Using a virtual reality game to assess goal-directed hand movements in children: A pilot feasibility study.

PubMed

Gabyzon, M Elboim; Engel-Yeger, B; Tresser, S; Springer, S

2016-01-01

Virtual reality gaming environments may be used as a supplement to the motor performance assessment tool box by providing clinicians with quantitative information regarding motor performance in terms of movement accuracy and speed, as well as sensory motor integration under different levels of dual tasking. To examine the feasibility of using the virtual reality game `Timocco' as an assessment tool for evaluating goal-directed hand movements among typically developing children. In this pilot study, 47 typically-developing children were divided into two age groups, 4-6 years old and 6-8 years old. Performance was measured using two different virtual environment games (Bubble Bath and Falling Fruit), each with two levels of difficulty. Discriminative validity (age effect) was examined by comparing the performance of the two groups, and by comparing the performance between levels of the games for each group (level effect). Test-retest reliability was examined by reassessing the older children 3-7 days after the first session. The older children performed significantly better in terms of response time, action time, game duration, and efficiency in both games compared to the younger children. Both age groups demonstrated poorer performance at the higher game level in the Bubble Bath game compared to the lower level. A similar level effect was found in the Falling Fruit game for both age groups in response time and efficiency, but not in action time. The performance of the older children was not significantly different between the two sessions at both game levels. The discriminative validity and test-retest reliability indicate the feasibility of using the Timocco virtual reality game as a tool for assessing goal-directed hand movements in children. Further studies should examine its feasibility for use in children with disabilities.

An Efficient Data Partitioning to Improve Classification Performance While Keeping Parameters Interpretable

PubMed Central

Korjus, Kristjan; Hebart, Martin N.; Vicente, Raul

2016-01-01

Supervised machine learning methods typically require splitting data into multiple chunks for training, validating, and finally testing classifiers. For finding the best parameters of a classifier, training and validation are usually carried out with cross-validation. This is followed by application of the classifier with optimized parameters to a separate test set for estimating the classifier’s generalization performance. With limited data, this separation of test data creates a difficult trade-off between having more statistical power in estimating generalization performance versus choosing better parameters and fitting a better model. We propose a novel approach that we term “Cross-validation and cross-testing” improving this trade-off by re-using test data without biasing classifier performance. The novel approach is validated using simulated data and electrophysiological recordings in humans and rodents. The results demonstrate that the approach has a higher probability of discovering significant results than the standard approach of cross-validation and testing, while maintaining the nominal alpha level. In contrast to nested cross-validation, which is maximally efficient in re-using data, the proposed approach additionally maintains the interpretability of individual parameters. Taken together, we suggest an addition to currently used machine learning approaches which may be particularly useful in cases where model weights do not require interpretation, but parameters do. PMID:27564393

An Efficient Data Partitioning to Improve Classification Performance While Keeping Parameters Interpretable.

PubMed

Korjus, Kristjan; Hebart, Martin N; Vicente, Raul

2016-01-01

Supervised machine learning methods typically require splitting data into multiple chunks for training, validating, and finally testing classifiers. For finding the best parameters of a classifier, training and validation are usually carried out with cross-validation. This is followed by application of the classifier with optimized parameters to a separate test set for estimating the classifier's generalization performance. With limited data, this separation of test data creates a difficult trade-off between having more statistical power in estimating generalization performance versus choosing better parameters and fitting a better model. We propose a novel approach that we term "Cross-validation and cross-testing" improving this trade-off by re-using test data without biasing classifier performance. The novel approach is validated using simulated data and electrophysiological recordings in humans and rodents. The results demonstrate that the approach has a higher probability of discovering significant results than the standard approach of cross-validation and testing, while maintaining the nominal alpha level. In contrast to nested cross-validation, which is maximally efficient in re-using data, the proposed approach additionally maintains the interpretability of individual parameters. Taken together, we suggest an addition to currently used machine learning approaches which may be particularly useful in cases where model weights do not require interpretation, but parameters do.

Measuring Image Navigation and Registration Performance at the 3-Sigma Level Using Platinum Quality Landmarks

NASA Technical Reports Server (NTRS)

Carr, James L.; Madani, Houria

2007-01-01

Geostationary Operational Environmental Satellite (GOES) Image Navigation and Registration (INR) performance is specified at the 3- level, meaning that 99.7% of a collection of individual measurements must comply with specification thresholds. Landmarks are measured by the Replacement Product Monitor (RPM), part of the operational GOES ground system, to assess INR performance and to close the INR loop. The RPM automatically discriminates between valid and invalid measurements enabling it to run without human supervision. In general, this screening is reliable, but a small population of invalid measurements will be falsely identified as valid. Even a small population of invalid measurements can create problems when assessing performance at the 3-sigma level. This paper describes an additional layer of quality control whereby landmarks of the highest quality ("platinum") are identified by their self-consistency. The platinum screening criteria are not simple statistical outlier tests against sigma values in populations of INR errors. In-orbit INR performance metrics for GOES-12 and GOES-13 are presented using the platinum landmark methodology.

The comprehensive care project: measuring physician performance in ambulatory practice.

PubMed

Holmboe, Eric S; Weng, Weifeng; Arnold, Gerald K; Kaplan, Sherrie H; Normand, Sharon-Lise; Greenfield, Sheldon; Hood, Sarah; Lipner, Rebecca S

2010-12-01

To investigate the feasibility, reliability, and validity of comprehensively assessing physician-level performance in ambulatory practice. Ambulatory-based general internists in 13 states participated in the assessment. We assessed physician-level performance, adjusted for patient factors, on 46 individual measures, an overall composite measure, and composite measures for chronic, acute, and preventive care. Between- versus within-physician variation was quantified by intraclass correlation coefficients (ICC). External validity was assessed by correlating performance on a certification exam. Medical records for 236 physicians were audited for seven chronic and four acute care conditions, and six age- and gender-appropriate preventive services. Performance on the individual and composite measures varied substantially within (range 5-86 percent compliance on 46 measures) and between physicians (ICC range 0.12-0.88). Reliabilities for the composite measures were robust: 0.88 for chronic care and 0.87 for preventive services. Higher certification exam scores were associated with better performance on the overall (r = 0.19; p<.01), chronic care (r = 0.14, p = .04), and preventive services composites (r = 0.17, p = .01). Our results suggest that reliable and valid comprehensive assessment of the quality of chronic and preventive care can be achieved by creating composite measures and by sampling feasible numbers of patients for each condition. © Health Research and Educational Trust.

Development and validation of a web-based questionnaire for surveying the health and working conditions of high-performance marine craft populations

PubMed Central

de Alwis, Manudul Pahansen; Lo Martire, Riccardo; Äng, Björn O; Garme, Karl

2016-01-01

Background High-performance marine craft crews are susceptible to various adverse health conditions caused by multiple interactive factors. However, there are limited epidemiological data available for assessment of working conditions at sea. Although questionnaire surveys are widely used for identifying exposures, outcomes and associated risks with high accuracy levels, until now, no validated epidemiological tool exists for surveying occupational health and performance in these populations. Aim To develop and validate a web-based questionnaire for epidemiological assessment of occupational and individual risk exposure pertinent to the musculoskeletal health conditions and performance in high-performance marine craft populations. Method A questionnaire for investigating the association between work-related exposure, performance and health was initially developed by a consensus panel under four subdomains, viz. demography, lifestyle, work exposure and health and systematically validated by expert raters for content relevance and simplicity in three consecutive stages, each iteratively followed by a consensus panel revision. The item content validity index (I-CVI) was determined as the proportion of experts giving a rating of 3 or 4. The scale content validity index (S-CVI/Ave) was computed by averaging the I-CVIs for the assessment of the questionnaire as a tool. Finally, the questionnaire was pilot tested. Results The S-CVI/Ave increased from 0.89 to 0.96 for relevance and from 0.76 to 0.94 for simplicity, resulting in 36 items in the final questionnaire. The pilot test confirmed the feasibility of the questionnaire. Conclusions The present study shows that the web-based questionnaire fulfils previously published validity acceptance criteria and is therefore considered valid and feasible for the empirical surveying of epidemiological aspects among high-performance marine craft crews and similar populations. PMID:27324717

Construct validity of individual and summary performance metrics associated with a computer-based laparoscopic simulator.

PubMed

Rivard, Justin D; Vergis, Ashley S; Unger, Bertram J; Hardy, Krista M; Andrew, Chris G; Gillman, Lawrence M; Park, Jason

2014-06-01

Computer-based surgical simulators capture a multitude of metrics based on different aspects of performance, such as speed, accuracy, and movement efficiency. However, without rigorous assessment, it may be unclear whether all, some, or none of these metrics actually reflect technical skill, which can compromise educational efforts on these simulators. We assessed the construct validity of individual performance metrics on the LapVR simulator (Immersion Medical, San Jose, CA, USA) and used these data to create task-specific summary metrics. Medical students with no prior laparoscopic experience (novices, N = 12), junior surgical residents with some laparoscopic experience (intermediates, N = 12), and experienced surgeons (experts, N = 11) all completed three repetitions of four LapVR simulator tasks. The tasks included three basic skills (peg transfer, cutting, clipping) and one procedural skill (adhesiolysis). We selected 36 individual metrics on the four tasks that assessed six different aspects of performance, including speed, motion path length, respect for tissue, accuracy, task-specific errors, and successful task completion. Four of seven individual metrics assessed for peg transfer, six of ten metrics for cutting, four of nine metrics for clipping, and three of ten metrics for adhesiolysis discriminated between experience levels. Time and motion path length were significant on all four tasks. We used the validated individual metrics to create summary equations for each task, which successfully distinguished between the different experience levels. Educators should maintain some skepticism when reviewing the plethora of metrics captured by computer-based simulators, as some but not all are valid. We showed the construct validity of a limited number of individual metrics and developed summary metrics for the LapVR. The summary metrics provide a succinct way of assessing skill with a single metric for each task, but require further validation.

Validation on milk and sprouts of EN ISO 16654:2001 - Microbiology of food and animal feeding stuffs - Horizontal method for the detection of Escherichia coli O157.

PubMed

Tozzoli, Rosangela; Maugliani, Antonella; Michelacci, Valeria; Minelli, Fabio; Caprioli, Alfredo; Morabito, Stefano

2018-05-08

In 2006, the European Committee for standardisation (CEN)/Technical Committee 275 - Food analysis - Horizontal methods/Working Group 6 - Microbiology of the food chain (TC275/WG6), launched the project of validating the method ISO 16654:2001 for the detection of Escherichia coli O157 in foodstuff by the evaluation of its performance, in terms of sensitivity and specificity, through collaborative studies. Previously, a validation study had been conducted to assess the performance of the Method No 164 developed by the Nordic Committee for Food Analysis (NMKL), which aims at detecting E. coli O157 in food as well, and is based on a procedure equivalent to that of the ISO 16654:2001 standard. Therefore, CEN established that the validation data obtained for the NMKL Method 164 could be exploited for the ISO 16654:2001 validation project, integrated with new data obtained through two additional interlaboratory studies on milk and sprouts, run in the framework of the CEN mandate No. M381. The ISO 16654:2001 validation project was led by the European Union Reference Laboratory for Escherichia coli including VTEC (EURL-VTEC), which organized the collaborative validation study on milk in 2012 with 15 participating laboratories and that on sprouts in 2014, with 14 participating laboratories. In both studies, a total of 24 samples were tested by each laboratory. Test materials were spiked with different concentration of E. coli O157 and the 24 samples corresponded to eight replicates of three levels of contamination: zero, low and high spiking level. The results submitted by the participating laboratories were analyzed to evaluate the sensitivity and specificity of the ISO 16654:2001 method when applied to milk and sprouts. The performance characteristics calculated on the data of the collaborative validation studies run under the CEN mandate No. M381 returned sensitivity and specificity of 100% and 94.4%, respectively for the milk study. As for sprouts matrix, the sensitivity resulted in 75.9% in the low level of contamination samples and 96.4% in samples spiked with high level of E. coli O157 and specificity was calculated as 99.1%. Copyright © 2018 Elsevier B.V. All rights reserved.

FUNCTIONAL PERFORMANCE TESTING OF THE HIP IN ATHLETES: A SYSTEMATIC REVIEW FOR RELIABILITY AND VALIDITY

PubMed Central

Martin, RobRoy L.

2012-01-01

Purpose/Background: The purpose of this study was to systematically review the literature for functional performance tests with evidence of reliability and validity that could be used for a young, athletic population with hip dysfunction. Methods: A search of PubMed and SPORTDiscus databases were performed to identify movement, balance, hop/jump, or agility functional performance tests from the current peer-reviewed literature used to assess function of the hip in young, athletic subjects. Results: The single-leg stance, deep squat, single-leg squat, and star excursion balance tests (SEBT) demonstrated evidence of validity and normative data for score interpretation. The single-leg stance test and SEBT have evidence of validity with association to hip abductor function. The deep squat test demonstrated evidence as a functional performance test for evaluating femoroacetabular impingement. Hop/Jump tests and agility tests have no reported evidence of reliability or validity in a population of subjects with hip pathology. Conclusions: Use of functional performance tests in the assessment of hip dysfunction has not been well established in the current literature. Diminished squat depth and provocation of pain during the single-leg balance test have been associated with patients diagnosed with FAI and gluteal tendinopathy, respectively. The SEBT and single-leg squat tests provided evidence of convergent validity through an analysis of kinematics and muscle function in normal subjects. Reliability of functional performance tests have not been established on patients with hip dysfunction. Further study is needed to establish reliability and validity of functional performance tests that can be used in a young, athletic population with hip dysfunction. Level of Evidence: 2b (Systematic Review of Literature) PMID:22893860

Measuring Assurance of Learning at the Degree Program and Academic Major Levels

ERIC Educational Resources Information Center

Marshall, Leisa Lynn

2007-01-01

In this article, the author examines the validity of performing assurance of learning (AOL) activities at the degree program level (e.g., bachelor's level) and the major level (e.g., accounting, finance). She examines 3 learning goals: management-specific knowledge, problem solving, and communication. The results strongly suggest that the AOL…

Design and validation of the INICIARE instrument, for the assessment of dependency level in acutely ill hospitalised patients.

PubMed

Morales-Asencio, José Miguel; Porcel-Gálvez, Ana María; Oliveros-Valenzuela, Rosa; Rodríguez-Gómez, Susana; Sánchez-Extremera, Lucrecia; Serrano-López, Francisco Andrés; Aranda-Gallardo, Marta; Canca-Sánchez, José Carlos; Barrientos-Trigo, Sergio

2015-03-01

The aim of this study was to establish the validity and reliability of an instrument (Inventario del NIvel de Cuidados mediante IndicAdores de clasificación de Resultados de Enfermería) used to assess the dependency level in acutely hospitalised patients. This instrument is novel, and it is based on the Nursing Outcomes Classification. Multiple existing instruments for needs assessment have been poorly validated and based predominately on interventions. Standardised Nursing Languages offer an ideal framework to develop nursing sensitive instruments. A cross-sectional validation study in two acute care hospitals in Spain. This study was implemented in two phases. First, the research team developed the instrument to be validated. In the second phase, the validation process was performed by experts, and the data analysis was conducted to establish the psychometric properties of the instrument. Seven hundred and sixty-one patient ratings performed by nurses were collected during the course of the research study. Data analysis yielded a Cronbach's alpha of 0·91. An exploratory factorial analysis identified three factors (Physiological, Instrumental and Cognitive-behavioural), which explained 74% of the variance. Inventario del NIvel de Cuidados mediante IndicAdores de clasificación de Resultados de Enfermería was demonstrated to be a valid and reliable instrument based on its use in acutely hospitalised patients to assess the level of dependency. Inventario del NIvel de Cuidados mediante IndicAdores de clasificación de Resultados de Enfermería can be used as an assessment tool in hospitalised patients during the nursing process throughout the entire hospitalisation period. It contributes information to support decisions on nursing diagnoses, interventions and outcomes. It also enables data codification in large databases. © 2014 John Wiley & Sons Ltd.

Crowd-sourced assessment of technical skills: an adjunct to urology resident surgical simulation training.

PubMed

Holst, Daniel; Kowalewski, Timothy M; White, Lee W; Brand, Timothy C; Harper, Jonathan D; Sorenson, Mathew D; Kirsch, Sarah; Lendvay, Thomas S

2015-05-01

Crowdsourcing is the practice of obtaining services from a large group of people, typically an online community. Validated methods of evaluating surgical video are time-intensive, expensive, and involve participation of multiple expert surgeons. We sought to obtain valid performance scores of urologic trainees and faculty on a dry-laboratory robotic surgery task module by using crowdsourcing through a web-based grading tool called Crowd Sourced Assessment of Technical Skill (CSATS). IRB approval was granted to test the technical skills grading accuracy of Amazon.com Mechanical Turk™ crowd-workers compared to three expert faculty surgeon graders. The two groups assessed dry-laboratory robotic surgical suturing performances of three urology residents (PGY-2, -4, -5) and two faculty using three performance domains from the validated Global Evaluative Assessment of Robotic Skills assessment tool. After an average of 2 hours 50 minutes, each of the five videos received 50 crowd-worker assessments. The inter-rater reliability (IRR) between the surgeons and crowd was 0.91 using Cronbach's alpha statistic (confidence intervals=0.20-0.92), indicating an agreement level between the two groups of "excellent." The crowds were able to discriminate the surgical level, and both the crowds and the expert faculty surgeon graders scored one senior trainee's performance above a faculty's performance. Surgery-naive crowd-workers can rapidly assess varying levels of surgical skill accurately relative to a panel of faculty raters. The crowds provided rapid feedback and were inexpensive. CSATS may be a valuable adjunct to surgical simulation training as requirements for more granular and iterative performance tracking of trainees become mandated and commonplace.

Virtual reality simulator training for laparoscopic colectomy: what metrics have construct validity?

PubMed

Shanmugan, Skandan; Leblanc, Fabien; Senagore, Anthony J; Ellis, C Neal; Stein, Sharon L; Khan, Sadaf; Delaney, Conor P; Champagne, Bradley J

2014-02-01

Virtual reality simulation for laparoscopic colectomy has been used for training of surgical residents and has been considered as a model for technical skills assessment of board-eligible colorectal surgeons. However, construct validity (the ability to distinguish between skill levels) must be confirmed before widespread implementation. This study was designed to specifically determine which metrics for laparoscopic sigmoid colectomy have evidence of construct validity. General surgeons that had performed fewer than 30 laparoscopic colon resections and laparoscopic colorectal experts (>200 laparoscopic colon resections) performed laparoscopic sigmoid colectomy on the LAP Mentor model. All participants received a 15-minute instructional warm-up and had never used the simulator before the study. Performance was then compared between each group for 21 metrics (procedural, 14; intraoperative errors, 7) to determine specifically which measurements demonstrate construct validity. Performance was compared with the Mann-Whitney U-test (p < 0.05 was significant). Fifty-three surgeons; 29 general surgeons, and 24 colorectal surgeons enrolled in the study. The virtual reality simulators for laparoscopic sigmoid colectomy demonstrated construct validity for 8 of 14 procedural metrics by distinguishing levels of surgical experience (p < 0.05). The most discriminatory procedural metrics (p < 0.01) favoring experts were reduced instrument path length, accuracy of the peritoneal/medial mobilization, and dissection of the inferior mesenteric artery. Intraoperative errors were not discriminatory for most metrics and favored general surgeons for colonic wall injury (general surgeons, 0.7; colorectal surgeons, 3.5; p = 0.045). Individual variability within the general surgeon and colorectal surgeon groups was not accounted for. The virtual reality simulators for laparoscopic sigmoid colectomy demonstrated construct validity for 8 procedure-specific metrics. However, using virtual reality simulator metrics to detect intraoperative errors did not discriminate between groups. If the virtual reality simulator continues to be used for the technical assessment of trainees and board-eligible surgeons, the evaluation of performance should be limited to procedural metrics.

Evaluating the Validity of Accommodations for English Learners through Evidence Based on Response Processes

ERIC Educational Resources Information Center

Crotts, Katrina M.

2013-01-01

English learners (ELs) represent one of the fastest growing student populations in the United States. Given that language can serve as a barrier in EL performance, test accommodations are provided to help level the playing field and allow ELs to better demonstrate their true performance level. Test accommodations on the computer offer the ability…

Applied Chaos Level Test for Validation of Signal Conditions Underlying Optimal Performance of Voice Classification Methods

ERIC Educational Resources Information Center

Liu, Boquan; Polce, Evan; Sprott, Julien C.; Jiang, Jack J.

2018-01-01

Purpose: The purpose of this study is to introduce a chaos level test to evaluate linear and nonlinear voice type classification method performances under varying signal chaos conditions without subjective impression. Study Design: Voice signals were constructed with differing degrees of noise to model signal chaos. Within each noise power, 100…

Development and Validation of a Mathematics Anxiety Scale for Students

ERIC Educational Resources Information Center

Ko, Ho Kyoung; Yi, Hyun Sook

2011-01-01

This study developed and validated a Mathematics Anxiety Scale for Students (MASS) that can be used to measure the level of mathematics anxiety that students experience in school settings and help them overcome anxiety and perform better in mathematics achievement. We conducted a series of preliminary analyses and panel reviews to evaluate quality…

Safety of High Speed Ground Transportation Systems : Analytical Methodology for Safety Validation of Computer Controlled Subsystems : Volume 2. Development of a Safety Validation Methodology

DOT National Transportation Integrated Search

1995-01-01

This report describes the development of a methodology designed to assure that a sufficiently high level of safety is achieved and maintained in computer-based systems which perform safety cortical functions in high-speed rail or magnetic levitation ...

Analytical methodology for safety validation of computer controlled subsystems. Volume 1 : state-of-the-art and assessment of safety verification/validation methodologies

DOT National Transportation Integrated Search

1995-09-01

This report describes the development of a methodology designed to assure that a sufficiently high level of safety is achieved and maintained in computer-based systems which perform safety critical functions in high-speed rail or magnetic levitation ...

Office Education. North Dakota Validated Task Listing. Competency-Based Vocational Education.

ERIC Educational Resources Information Center

North Dakota State Board for Vocational Education, Bismarck.

Intended to provide a base for vocational office education instructional programs at secondary and postsecondary levels in North Dakota, this task listing describes the skills needed to be performed by program completers, from the viewpoint of workers in office occupations. A listing of task validators (name, occupation, employer, business city,…

Shape Optimization by Bayesian-Validated Computer-Simulation Surrogates

NASA Technical Reports Server (NTRS)

Patera, Anthony T.

1997-01-01

A nonparametric-validated, surrogate approach to optimization has been applied to the computational optimization of eddy-promoter heat exchangers and to the experimental optimization of a multielement airfoil. In addition to the baseline surrogate framework, a surrogate-Pareto framework has been applied to the two-criteria, eddy-promoter design problem. The Pareto analysis improves the predictability of the surrogate results, preserves generality, and provides a means to rapidly determine design trade-offs. Significant contributions have been made in the geometric description used for the eddy-promoter inclusions as well as to the surrogate framework itself. A level-set based, geometric description has been developed to define the shape of the eddy-promoter inclusions. The level-set technique allows for topology changes (from single-body,eddy-promoter configurations to two-body configurations) without requiring any additional logic. The continuity of the output responses for input variations that cross the boundary between topologies has been demonstrated. Input-output continuity is required for the straightforward application of surrogate techniques in which simplified, interpolative models are fitted through a construction set of data. The surrogate framework developed previously has been extended in a number of ways. First, the formulation for a general, two-output, two-performance metric problem is presented. Surrogates are constructed and validated for the outputs. The performance metrics can be functions of both outputs, as well as explicitly of the inputs, and serve to characterize the design preferences. By segregating the outputs and the performance metrics, an additional level of flexibility is provided to the designer. The validated outputs can be used in future design studies and the error estimates provided by the output validation step still apply, and require no additional appeals to the expensive analysis. Second, a candidate-based a posteriori error analysis capability has been developed which provides probabilistic error estimates on the true performance for a design randomly selected near the surrogate-predicted optimal design.

The Arthroscopic Surgical Skill Evaluation Tool (ASSET).

PubMed

Koehler, Ryan J; Amsdell, Simon; Arendt, Elizabeth A; Bisson, Leslie J; Braman, Jonathan P; Bramen, Jonathan P; Butler, Aaron; Cosgarea, Andrew J; Harner, Christopher D; Garrett, William E; Olson, Tyson; Warme, Winston J; Nicandri, Gregg T

2013-06-01

Surgeries employing arthroscopic techniques are among the most commonly performed in orthopaedic clinical practice; however, valid and reliable methods of assessing the arthroscopic skill of orthopaedic surgeons are lacking. The Arthroscopic Surgery Skill Evaluation Tool (ASSET) will demonstrate content validity, concurrent criterion-oriented validity, and reliability when used to assess the technical ability of surgeons performing diagnostic knee arthroscopic surgery on cadaveric specimens. Cross-sectional study; Level of evidence, 3. Content validity was determined by a group of 7 experts using the Delphi method. Intra-articular performance of a right and left diagnostic knee arthroscopic procedure was recorded for 28 residents and 2 sports medicine fellowship-trained attending surgeons. Surgeon performance was assessed by 2 blinded raters using the ASSET. Concurrent criterion-oriented validity, interrater reliability, and test-retest reliability were evaluated. Content validity: The content development group identified 8 arthroscopic skill domains to evaluate using the ASSET. Concurrent criterion-oriented validity: Significant differences in the total ASSET score (P < .05) between novice, intermediate, and advanced experience groups were identified. Interrater reliability: The ASSET scores assigned by each rater were strongly correlated (r = 0.91, P < .01), and the intraclass correlation coefficient between raters for the total ASSET score was 0.90. Test-retest reliability: There was a significant correlation between ASSET scores for both procedures attempted by each surgeon (r = 0.79, P < .01). The ASSET appears to be a useful, valid, and reliable method for assessing surgeon performance of diagnostic knee arthroscopic surgery in cadaveric specimens. Studies are ongoing to determine its generalizability to other procedures as well as to the live operating room and other simulated environments.

Multi-Evaporator Miniature Loop Heat Pipe for Small Spacecraft Thermal Control. Part 1; New Technologies and Validation Approach

NASA Technical Reports Server (NTRS)

Ku, Jentung; Ottenstein, Laura; Douglas, Donya; Hoang, Triem

2010-01-01

Under NASA s New Millennium Program Space Technology 8 (ST 8) Project, four experiments Thermal Loop, Dependable Microprocessor, SAILMAST, and UltraFlex - were conducted to advance the maturity of individual technologies from proof of concept to prototype demonstration in a relevant environment , i.e. from a technology readiness level (TRL) of 3 to a level of 6. This paper presents the new technologies and validation approach of the Thermal Loop experiment. The Thermal Loop is an advanced thermal control system consisting of a miniature loop heat pipe (MLHP) with multiple evaporators and multiple condensers designed for future small system applications requiring low mass, low power, and compactness. The MLHP retains all features of state-of-the-art loop heat pipes (LHPs) and offers additional advantages to enhance the functionality, performance, versatility, and reliability of the system. Details of the thermal loop concept, technical advances, benefits, objectives, level 1 requirements, and performance characteristics are described. Also included in the paper are descriptions of the test articles and mathematical modeling used for the technology validation. An MLHP breadboard was built and tested in the laboratory and thermal vacuum environments for TRL 4 and TRL 5 validations, and an MLHP proto-flight unit was built and tested in a thermal vacuum chamber for the TRL 6 validation. In addition, an analytical model was developed to simulate the steady state and transient behaviors of the MLHP during various validation tests. Capabilities and limitations of the analytical model are also addressed.

Motivational Systems Theory and the Academic Performance of College Students

ERIC Educational Resources Information Center

Campbell, Michael M.

2007-01-01

This study explored the validity of the Motivational Systems Theory (MST) as a measure of performance of college students pursuing business degrees and the level of academic performance attained across gender and race lines. This goal is achieved by investigating the relationships between motivational strategies, biological factors, responsive…

A Student Assessment Tool for Standardized Patient Simulations (SAT-SPS): Psychometric analysis.

PubMed

Castro-Yuste, Cristina; García-Cabanillas, María José; Rodríguez-Cornejo, María Jesús; Carnicer-Fuentes, Concepción; Paloma-Castro, Olga; Moreno-Corral, Luis Javier

2018-05-01

The evaluation of the level of clinical competence acquired by the student is a complex process that must meet various requirements to ensure its quality. The psychometric analysis of the data collected by the assessment tools used is a fundamental aspect to guarantee the student's competence level. To conduct a psychometric analysis of an instrument which assesses clinical competence in nursing students at simulation stations with standardized patients in OSCE-format tests. The construct of clinical competence was operationalized as a set of observable and measurable behaviors, measured by the newly-created Student Assessment Tool for Standardized Patient Simulations (SAT-SPS), which was comprised of 27 items. The categories assigned to the items were 'incorrect or not performed' (0), 'acceptable' (1), and 'correct' (2). 499 nursing students. Data were collected by two independent observers during the assessment of the students' performance at a four-station OSCE with standardized patients. Descriptive statistics were used to summarize the variables. The difficulty levels and floor and ceiling effects were determined for each item. Reliability was analyzed using internal consistency and inter-observer reliability. The validity analysis was performed considering face validity, content and construct validity (through exploratory factor analysis), and criterion validity. Internal reliability and inter-observer reliability were higher than 0.80. The construct validity analysis suggested a three-factor model accounting for 37.1% of the variance. These three factors were named 'Nursing process', 'Communication skills', and 'Safe practice'. A significant correlation was found between the scores obtained and the students' grades in general, as well as with the grades obtained in subjects with clinical content. The assessment tool has proven to be sufficiently reliable and valid for the assessment of the clinical competence of nursing students using standardized patients. This tool has three main components: the nursing process, communication skills, and safety management. Copyright © 2018 Elsevier Ltd. All rights reserved.

Pilot In-Trail Procedure Validation Simulation Study

NASA Technical Reports Server (NTRS)

Bussink, Frank J. L.; Murdoch, Jennifer L.; Chamberlain, James P.; Chartrand, Ryan; Jones, Kenneth M.

2008-01-01

A Human-In-The-Loop experiment was conducted at the National Aeronautics and Space Administration (NASA) Langley Research Center (LaRC) to investigate the viability of the In-Trail Procedure (ITP) concept from a flight crew perspective, by placing participating airline pilots in a simulated oceanic flight environment. The test subject pilots used new onboard avionics equipment that provided improved information about nearby traffic and enabled them, when specific criteria were met, to request an ITP flight level change referencing one or two nearby aircraft that might otherwise block the flight level change. The subject pilots subjective assessments of ITP validity and acceptability were measured via questionnaires and discussions, and their objective performance in appropriately selecting, requesting, and performing ITP flight level changes was evaluated for each simulated flight scenario. Objective performance and subjective workload assessment data from the experiment s test conditions were analyzed for statistical and operational significance and are reported in the paper. Based on these results, suggestions are made to further improve the ITP.

An integrated radar model solution for mission level performance and cost trades

NASA Astrophysics Data System (ADS)

Hodge, John; Duncan, Kerron; Zimmerman, Madeline; Drupp, Rob; Manno, Mike; Barrett, Donald; Smith, Amelia

2017-05-01

A fully integrated Mission-Level Radar model is in development as part of a multi-year effort under the Northrop Grumman Mission Systems (NGMS) sector's Model Based Engineering (MBE) initiative to digitally interconnect and unify previously separate performance and cost models. In 2016, an NGMS internal research and development (IR and D) funded multidisciplinary team integrated radio frequency (RF), power, control, size, weight, thermal, and cost models together using a commercial-off-the-shelf software, ModelCenter, for an Active Electronically Scanned Array (AESA) radar system. Each represented model was digitally connected with standard interfaces and unified to allow end-to-end mission system optimization and trade studies. The radar model was then linked to the Air Force's own mission modeling framework (AFSIM). The team first had to identify the necessary models, and with the aid of subject matter experts (SMEs) understand and document the inputs, outputs, and behaviors of the component models. This agile development process and collaboration enabled rapid integration of disparate models and the validation of their combined system performance. This MBE framework will allow NGMS to design systems more efficiently and affordably, optimize architectures, and provide increased value to the customer. The model integrates detailed component models that validate cost and performance at the physics level with high-level models that provide visualization of a platform mission. This connectivity of component to mission models allows hardware and software design solutions to be better optimized to meet mission needs, creating cost-optimal solutions for the customer, while reducing design cycle time through risk mitigation and early validation of design decisions.

A framework to assess management performance in district health systems: a qualitative and quantitative case study in Iran.

PubMed

Tabrizi, Jafar Sadegh; Gholipour, Kamal; Iezadi, Shabnam; Farahbakhsh, Mostafa; Ghiasi, Akbar

2018-01-01

The aim was to design a district health management performance framework for Iran's healthcare system. The mixed-method study was conducted between September 2015 and May 2016 in Tabriz, Iran. In this study, the indicators of district health management performance were obtained by analyzing the 45 semi-structured surveys of experts in the public health system. Content validity of performance indicators which were generated in qualitative part were reviewed and confirmed based on content validity index (CVI). Also content validity ratio (CVR) was calculated using data acquired from a survey of 21 experts in quantitative part. The result of this study indicated that, initially, 81 indicators were considered in framework of district health management performance and, at the end, 53 indicators were validated and confirmed. These indicators were classified in 11 categories which include: human resources and organizational creativity, management and leadership, rules and ethics, planning and evaluation, district managing, health resources management and economics, community participation, quality improvement, research in health system, health information management, epidemiology and situation analysis. The designed framework model can be used to assess the district health management and facilitates performance improvement at the district level.

A cross-validation package driving Netica with python

USGS Publications Warehouse

Fienen, Michael N.; Plant, Nathaniel G.

2014-01-01

Bayesian networks (BNs) are powerful tools for probabilistically simulating natural systems and emulating process models. Cross validation is a technique to avoid overfitting resulting from overly complex BNs. Overfitting reduces predictive skill. Cross-validation for BNs is known but rarely implemented due partly to a lack of software tools designed to work with available BN packages. CVNetica is open-source, written in Python, and extends the Netica software package to perform cross-validation and read, rebuild, and learn BNs from data. Insights gained from cross-validation and implications on prediction versus description are illustrated with: a data-driven oceanographic application; and a model-emulation application. These examples show that overfitting occurs when BNs become more complex than allowed by supporting data and overfitting incurs computational costs as well as causing a reduction in prediction skill. CVNetica evaluates overfitting using several complexity metrics (we used level of discretization) and its impact on performance metrics (we used skill).

[Validation and verfication of microbiology methods].

PubMed

Camaró-Sala, María Luisa; Martínez-García, Rosana; Olmos-Martínez, Piedad; Catalá-Cuenca, Vicente; Ocete-Mochón, María Dolores; Gimeno-Cardona, Concepción

2015-01-01

Clinical microbiologists should ensure, to the maximum level allowed by the scientific and technical development, the reliability of the results. This implies that, in addition to meeting the technical criteria to ensure their validity, they must be performed with a number of conditions that allows comparable results to be obtained, regardless of the laboratory that performs the test. In this sense, the use of recognized and accepted reference methodsis the most effective tool for these guarantees. The activities related to verification and validation of analytical methods has become very important, as there is continuous development, as well as updating techniques and increasingly complex analytical equipment, and an interest of professionals to ensure quality processes and results. The definitions of validation and verification are described, along with the different types of validation/verification, and the types of methods, and the level of validation necessary depending on the degree of standardization. The situations in which validation/verification is mandatory and/or recommended is discussed, including those particularly related to validation in Microbiology. It stresses the importance of promoting the use of reference strains as controls in Microbiology and the use of standard controls, as well as the importance of participation in External Quality Assessment programs to demonstrate technical competence. The emphasis is on how to calculate some of the parameters required for validation/verification, such as the accuracy and precision. The development of these concepts can be found in the microbiological process SEIMC number 48: «Validation and verification of microbiological methods» www.seimc.org/protocols/microbiology. Copyright © 2013 Elsevier España, S.L.U. y Sociedad Española de Enfermedades Infecciosas y Microbiología Clínica. All rights reserved.

Validity of linear encoder measurement of sit-to-stand performance power in older people.

PubMed

Lindemann, U; Farahmand, P; Klenk, J; Blatzonis, K; Becker, C

2015-09-01

To investigate construct validity of linear encoder measurement of sit-to-stand performance power in older people by showing associations with relevant functional performance and physiological parameters. Cross-sectional study. Movement laboratory of a geriatric rehabilitation clinic. Eighty-eight community-dwelling, cognitively unimpaired older women (mean age 78 years). Sit-to-stand performance power and leg power were assessed using a linear encoder and the Nottingham Power Rig, respectively. Gait speed was measured on an instrumented walkway. Maximum quadriceps and hand grip strength were assessed using dynamometers. Mid-thigh muscle cross-sectional area of both legs was measured using magnetic resonance imaging. Associations of sit-to-stand performance power with power assessed by the Nottingham Power Rig, maximum gait speed and muscle cross-sectional area were r=0.646, r=0.536 and r=0.514, respectively. A linear regression model explained 50% of the variance in sit-to-stand performance power including muscle cross-sectional area (p=0.001), maximum gait speed (p=0.002), and power assessed by the Nottingham Power Rig (p=0.006). Construct validity of linear encoder measurement of sit-to-stand power was shown at functional level and morphological level for older women. This measure could be used in routine clinical practice as well as in large-scale studies. DRKS00003622. Copyright © 2015 Chartered Society of Physiotherapy. Published by Elsevier Ltd. All rights reserved.

The Comprehensive Care Project: Measuring Physician Performance in Ambulatory Practice

PubMed Central

Holmboe, Eric S; Weng, Weifeng; Arnold, Gerald K; Kaplan, Sherrie H; Normand, Sharon-Lise; Greenfield, Sheldon; Hood, Sarah; Lipner, Rebecca S

2010-01-01

Objective To investigate the feasibility, reliability, and validity of comprehensively assessing physician-level performance in ambulatory practice. Data Sources/Study Setting Ambulatory-based general internists in 13 states participated in the assessment. Study Design We assessed physician-level performance, adjusted for patient factors, on 46 individual measures, an overall composite measure, and composite measures for chronic, acute, and preventive care. Between- versus within-physician variation was quantified by intraclass correlation coefficients (ICC). External validity was assessed by correlating performance on a certification exam. Data Collection/Extraction Methods Medical records for 236 physicians were audited for seven chronic and four acute care conditions, and six age- and gender-appropriate preventive services. Principal Findings Performance on the individual and composite measures varied substantially within (range 5–86 percent compliance on 46 measures) and between physicians (ICC range 0.12–0.88). Reliabilities for the composite measures were robust: 0.88 for chronic care and 0.87 for preventive services. Higher certification exam scores were associated with better performance on the overall (r = 0.19; p <.01), chronic care (r = 0.14, p = .04), and preventive services composites (r = 0.17, p = .01). Conclusions Our results suggest that reliable and valid comprehensive assessment of the quality of chronic and preventive care can be achieved by creating composite measures and by sampling feasible numbers of patients for each condition. PMID:20819110

Estimating learning outcomes from pre- and posttest student self-assessments: a longitudinal study.

PubMed

Schiekirka, Sarah; Reinhardt, Deborah; Beißbarth, Tim; Anders, Sven; Pukrop, Tobias; Raupach, Tobias

2013-03-01

Learning outcome is an important measure for overall teaching quality and should be addressed by comprehensive evaluation tools. The authors evaluated the validity of a novel evaluation tool based on student self-assessments, which may help identify specific strengths and weaknesses of a particular course. In 2011, the authors asked 145 fourth-year students at Göttingen Medical School to self-assess their knowledge on 33 specific learning objectives in a pretest and posttest as part of a cardiorespiratory module. The authors compared performance gain calculated from self-assessments with performance gain derived from formative examinations that were closely matched to these 33 learning objectives. Eighty-three students (57.2%) completed the assessment. There was good agreement between performance gain derived from subjective data and performance gain derived from objective examinations (Pearson r=0.78; P<.0001) on the group level. The association between the two measures was much weaker when data were analyzed on the individual level. Further analysis determined a quality cutoff for performance gain derived from aggregated student self-assessments. When using this cutoff, the evaluation tool was highly sensitive in identifying specific learning objectives with favorable or suboptimal objective performance gains. The tool is easy to implement, takes initial performance levels into account, and does not require extensive pre-post testing. By providing valid estimates of actual performance gain obtained during a teaching module, it may assist medical teachers in identifying strengths and weaknesses of a particular course on the level of specific learning objectives.

Risk-based Methodology for Validation of Pharmaceutical Batch Processes.

PubMed

Wiles, Frederick

2013-01-01

In January 2011, the U.S. Food and Drug Administration published new process validation guidance for pharmaceutical processes. The new guidance debunks the long-held industry notion that three consecutive validation batches or runs are all that are required to demonstrate that a process is operating in a validated state. Instead, the new guidance now emphasizes that the level of monitoring and testing performed during process performance qualification (PPQ) studies must be sufficient to demonstrate statistical confidence both within and between batches. In some cases, three qualification runs may not be enough. Nearly two years after the guidance was first published, little has been written defining a statistical methodology for determining the number of samples and qualification runs required to satisfy Stage 2 requirements of the new guidance. This article proposes using a combination of risk assessment, control charting, and capability statistics to define the monitoring and testing scheme required to show that a pharmaceutical batch process is operating in a validated state. In this methodology, an assessment of process risk is performed through application of a process failure mode, effects, and criticality analysis (PFMECA). The output of PFMECA is used to select appropriate levels of statistical confidence and coverage which, in turn, are used in capability calculations to determine when significant Stage 2 (PPQ) milestones have been met. The achievement of Stage 2 milestones signals the release of batches for commercial distribution and the reduction of monitoring and testing to commercial production levels. Individuals, moving range, and range/sigma charts are used in conjunction with capability statistics to demonstrate that the commercial process is operating in a state of statistical control. The new process validation guidance published by the U.S. Food and Drug Administration in January of 2011 indicates that the number of process validation batches or runs required to demonstrate that a pharmaceutical process is operating in a validated state should be based on sound statistical principles. The old rule of "three consecutive batches and you're done" is no longer sufficient. The guidance, however, does not provide any specific methodology for determining the number of runs required, and little has been published to augment this shortcoming. The paper titled "Risk-based Methodology for Validation of Pharmaceutical Batch Processes" describes a statistically sound methodology for determining when a statistically valid number of validation runs has been acquired based on risk assessment and calculation of process capability.

Model-based verification and validation of the SMAP uplink processes

NASA Astrophysics Data System (ADS)

Khan, M. O.; Dubos, G. F.; Tirona, J.; Standley, S.

Model-Based Systems Engineering (MBSE) is being used increasingly within the spacecraft design community because of its benefits when compared to document-based approaches. As the complexity of projects expands dramatically with continually increasing computational power and technology infusion, the time and effort needed for verification and validation (V& V) increases geometrically. Using simulation to perform design validation with system-level models earlier in the life cycle stands to bridge the gap between design of the system (based on system-level requirements) and verifying those requirements/validating the system as a whole. This case study stands as an example of how a project can validate a system-level design earlier in the project life cycle than traditional V& V processes by using simulation on a system model. Specifically, this paper describes how simulation was added to a system model of the Soil Moisture Active-Passive (SMAP) mission's uplink process. Also discussed are the advantages and disadvantages of the methods employed and the lessons learned; which are intended to benefit future model-based and simulation-based development efforts.

Demonstration of automated proximity and docking technologies

NASA Astrophysics Data System (ADS)

Anderson, Robert L.; Tsugawa, Roy K.; Bryan, Thomas C.

An autodock was demonstrated using straightforward techniques and real sensor hardware. A simulation testbed was established and validated. The sensor design was refined with improved optical performance and image processing noise mitigation techniques, and the sensor is ready for production from off-the-shelf components. The autonomous spacecraft architecture is defined. The areas of sensors, docking hardware, propulsion, and avionics are included in the design. The Guidance Navigation and Control architecture and requirements are developed. Modular structures suitable for automated control are used. The spacecraft system manager functions including configuration, resource, and redundancy management are defined. The requirements for autonomous spacecraft executive are defined. High level decisionmaking, mission planning, and mission contingency recovery are a part of this. The next step is to do flight demonstrations. After the presentation the following question was asked. How do you define validation? There are two components to validation definition: software simulation with formal and vigorous validation, and hardware and facility performance validated with respect to software already validated against analytical profile.

Procedure-specific assessment tool for flexible pharyngo-laryngoscopy: gathering validity evidence and setting pass-fail standards.

PubMed

Melchiors, Jacob; Petersen, K; Todsen, T; Bohr, A; Konge, Lars; von Buchwald, Christian

2018-06-01

The attainment of specific identifiable competencies is the primary measure of progress in the modern medical education system. The system, therefore, requires a method for accurately assessing competence to be feasible. Evidence of validity needs to be gathered before an assessment tool can be implemented in the training and assessment of physicians. This evidence of validity must according to the contemporary theory on validity be gathered from specific sources in a structured and rigorous manner. The flexible pharyngo-laryngoscopy (FPL) is central to the otorhinolaryngologist. We aim to evaluate the flexible pharyngo-laryngoscopy assessment tool (FLEXPAT) created in a previous study and to establish a pass-fail level for proficiency. Eighteen physicians with different levels of experience (novices, intermediates, and experienced) were recruited to the study. Each performed an FPL on two patients. These procedures were video recorded, blinded, and assessed by two specialists. The score was expressed as the percentage of a possible max score. Cronbach's α was used to analyze internal consistency of the data, and a generalizability analysis was performed. The scores of the three different groups were explored, and a pass-fail level was determined using the contrasting groups' standard setting method. Internal consistency was strong with a Cronbach's α of 0.86. We found a generalizability coefficient of 0.72 sufficient for moderate stakes assessment. We found a significant difference between the novice and experienced groups (p < 0.001) and strong correlation between experience and score (Pearson's r = 0.75). The pass/fail level was established at 72% of the maximum score. Applying this pass-fail level in the test population resulted in half of the intermediary group receiving a failing score. We gathered validity evidence for the FLEXPAT according to the contemporary framework as described by Messick. Our results support a claim of validity and are comparable to other studies exploring clinical assessment tools. The high rate of physicians underperforming in the intermediary group demonstrates the need for continued educational intervention. Based on our work, we recommend the use of the FLEXPAT in clinical assessment of FPL and the application of a pass-fail level of 72% for proficiency.

Alphabus Mechanical Validation Plan and Test Campaign

NASA Astrophysics Data System (ADS)

Calvisi, G.; Bonnet, D.; Belliol, P.; Lodereau, P.; Redoundo, R.

2012-07-01

A joint team of the two leading European satellite companies (Astrium and Thales Alenia Space) worked with the support of ESA and CNES to define a product line able to efficiently address the upper segment of communications satellites : Alphabus Starting in 2009 and up to 2011 the mechanical validation of the Alphabus platform has been obtained thanks to static tests performed on dedicated static model and to environmental test performed on the first satellite based on Alphabus: Alphasat I-XL. The mechanical validation of the Alphabus platform presented an excellent opportunity to improve the validation and qualification process, with respect to static, sine vibrations, acoustic and L/V shock environment, minimizing recurrent cost of manufacturing, integration and testing. A main driver on mechanical testing is that mechanical acceptance testing at satellite level will be performed with empty tanks due to technical constraints (limitation of existing vibration devices) and programmatic advantages (test risk reduction, test schedule minimization). In this paper the impacts that such testing logic have on validation plan are briefly recalled and its actual application for Alphasat PFM mechanical test campaign is detailed.

System-Level Experimental Validations for Supersonic Commercial Transport Aircraft Entering Service in the 2018-2020 Time Period

NASA Technical Reports Server (NTRS)

Magee, Todd E.; Wilcox, Peter A.; Fugal, Spencer R.; Acheson, Kurt E.; Adamson, Eric E.; Bidwell, Alicia L.; Shaw, Stephen G.

2013-01-01

This report describes the work conducted by The Boeing Company under American Recovery and Reinvestment Act (ARRA) and NASA funding to experimentally validate the conceptual design of a supersonic airliner feasible for entry into service in the 2018 to 2020 timeframe (NASA N+2 generation). The report discusses the design, analysis and development of a low-boom concept that meets aggressive sonic boom and performance goals for a cruise Mach number of 1.8. The design is achieved through integrated multidisciplinary optimization tools. The report also describes the detailed design and fabrication of both sonic boom and performance wind tunnel models of the low-boom concept. Additionally, a description of the detailed validation wind tunnel testing that was performed with the wind tunnel models is provided along with validation comparisons with pretest Computational Fluid Dynamics (CFD). Finally, the report describes the evaluation of existing NASA sonic boom pressure rail measurement instrumentation and a detailed description of new sonic boom measurement instrumentation that was constructed for the validation wind tunnel testing.

The Reliability, Validity, and Evaluation of the Objective Structured Clinical Examination in Podiatry (Chiropody).

ERIC Educational Resources Information Center

Woodburn, Jim; Sutcliffe, Nick

1996-01-01

The Objective Structured Clinical Examination (OSCE), initially developed for undergraduate medical education, has been adapted for assessment of clinical skills in podiatry students. A 12-month pilot study found the test had relatively low levels of reliability, high construct and criterion validity, and good stability of performance over time.…

Examining the Validity of Self-Reports on Scales Measuring Students' Strategic Processing

ERIC Educational Resources Information Center

Samuelstuen, Marit S.; Braten, Ivar

2007-01-01

Background: Self-report inventories trying to measure strategic processing at a global level have been much used in both basic and applied research. However, the validity of global strategy scores is open to question because such inventories assess strategy perceptions outside the context of specific task performance. Aims: The primary aim was to…

Computer Attitude and eLearning Self-Efficacy of Undergraduate Students: Validating Potential Acceptance and Use of Online Learning Systems in Ghana

ERIC Educational Resources Information Center

Larbi-Apau, Josephine; Oti-Boadi, Mabel; Tetteh, Albert

2018-01-01

Both computer attitude and eLearning self-efficacy are critical complementary factors in determining confidence levels and behavioral belief systems, and can directly affect students' actions, performances and achievements. This study applied a multidimensional construct in validating computer attitude and eLearning self-efficacy of Psychology…

Enhanced Oceanic Operations Human-In-The-Loop In-Trail Procedure Validation Simulation Study

NASA Technical Reports Server (NTRS)

Murdoch, Jennifer L.; Bussink, Frank J. L.; Chamberlain, James P.; Chartrand, Ryan C.; Palmer, Michael T.; Palmer, Susan O.

2008-01-01

The Enhanced Oceanic Operations Human-In-The-Loop In-Trail Procedure (ITP) Validation Simulation Study investigated the viability of an ITP designed to enable oceanic flight level changes that would not otherwise be possible. Twelve commercial airline pilots with current oceanic experience flew a series of simulated scenarios involving either standard or ITP flight level change maneuvers and provided subjective workload ratings, assessments of ITP validity and acceptability, and objective performance measures associated with the appropriate selection, request, and execution of ITP flight level change maneuvers. In the majority of scenarios, subject pilots correctly assessed the traffic situation, selected an appropriate response (i.e., either a standard flight level change request, an ITP request, or no request), and executed their selected flight level change procedure, if any, without error. Workload ratings for ITP maneuvers were acceptable and not substantially higher than for standard flight level change maneuvers, and, for the majority of scenarios and subject pilots, subjective acceptability ratings and comments for ITP were generally high and positive. Qualitatively, the ITP was found to be valid and acceptable. However, the error rates for ITP maneuvers were higher than for standard flight level changes, and these errors may have design implications for both the ITP and the study's prototype traffic display. These errors and their implications are discussed.

Advanced Concept Studies for Supersonic Commercial Transports Entering Service in the 2018 to 2020 Period

NASA Technical Reports Server (NTRS)

Morgenstern, John; Norstrud, Nicole; Sokhey, Jack; Martens, Steve; Alonso, Juan J.

2013-01-01

Lockheed Martin Aeronautics Company (LM), working in conjunction with General Electric Global Research (GE GR), Rolls-Royce Liberty Works (RRLW), and Stanford University, herein presents results from the "N+2 Supersonic Validations" contract s initial 22 month phase, addressing the NASA solicitation "Advanced Concept Studies for Supersonic Commercial Transports Entering Service in the 2018 to 2020 Period." This report version adds documentation of an additional three month low boom test task. The key technical objective of this effort was to validate integrated airframe and propulsion technologies and design methodologies. These capabilities aspired to produce a viable supersonic vehicle design with environmental and performance characteristics. Supersonic testing of both airframe and propulsion technologies (including LM3: 97-023 low boom testing and April-June nozzle acoustic testing) verified LM s supersonic low-boom design methodologies and both GE and RRLW's nozzle technologies for future implementation. The N+2 program is aligned with NASA s Supersonic Project and is focused on providing system-level solutions capable of overcoming the environmental and performance/efficiency barriers to practical supersonic flight. NASA proposed "Initial Environmental Targets and Performance Goals for Future Supersonic Civil Aircraft". The LM N+2 studies are built upon LM s prior N+3 100 passenger design studies. The LM N+2 program addresses low boom design and methodology validations with wind tunnel testing, performance and efficiency goals with system level analysis, and low noise validations with two nozzle (GE and RRLW) acoustic tests.

The relationship between organizational trust and nurse administrators’ productivity in hospitals

PubMed Central

Bahrami, Susan; Hasanpour, Marzieh; Rajaeepour, Saeed; Aghahosseni, Taghi; Hodhodineghad, Nilofar

2012-01-01

Context: Management of health care organizations based on employee’s mutual trust will increase the improvement in functions and tasks. Aims: The present study was performed to investigate the relationship between organizational trust and the nurse administrators’ productivity in educational health centers of in Health-Education Centers of Isfahan University of Medical Sciences. Settings and Design: This research was a descriptive and correlational study. Materials and Methods: The population included all nurse administrators. In this research, 165 nurses were selected through random sampling method. Data collection instruments were organizational trust questionnaire based on Robbins’s model and productivity questionnaire based on Hersy and Blanchard’s model. Validity of these questionnaires was determined through content validity and their reliability was calculated through Cranach’s alpha. Statistical analysis was used: The data analysis was done using the SPSS (18) statistical software. Results: The indicators of organizational trust such as loyalty, competence, honesty, and stability were more than average level but explicitness indicator was at average level. The components of productivity such as ability, job knowledge, environmental compatibility, performance feedback, and validity were more than average level but motivation factor was at average level and organizational support was less than average level. There were a significant multiple correlations between organizational trust and productivity. Beta coefficients among organizational trust and productivity were significant and no autocorrelation existed and regression model was significant. Conclusions: Committed employees, timely performing the tasks and developing the sense of responsibility among employees can enhance production and productivity in the health care organizations. PMID:23922588

Assessment of medical residents' satisfaction.

PubMed

González-Martínez, José Francisco; García-García, José Antonio; Del Rosario Arnaud-Viñas, María; Arámbula-Morales, Enna Gabriela; Uriega-González Plata, Silvia; Mendoza-Guerrero, José Antonio

2011-01-01

Modern medical education is focused on students, and it is necessary to assess its level of satisfaction. A questionnaire was validated and we then conducted a study about the educational satisfaction level of medical residents of the Hospital General of Mexico. An observational, descriptive, cross-sectional and prospective study was conducted. A questionnaire of 21 items was validated and then applied to a representative sample of medical residents. Each item was evaluated with a scale from 0 to 10 and then gathered in groups: 0-5 = poor, 6-7 = average, 8 = good, 9 = very good, and 10 = excellent. Descriptive and inferential statistics were carried out using SPSS v.17.0. The questionnaire had internal validity with Cronbach's alpha >0.91 by item. Included in the study were 355 medical residents representing 37 different specialties. The performance perception of the ìheadî professors showed a wide heterogeneity: excellent (23.7%), very good (20.6%), good (16.9%), average (23.1%), poor (15.8%). Fourth-year residents and upward valued the educational performance higher (p = 0.001) as well as medical/surgical residents (p = 0.02). Intermediate-level residents valued the professor higher (p = 0.001), similar to students who were married or living with a partner (p <0.001). Upon contrasting the evaluation of the teacher's performance with the overall course performance, a linear, direct and significant correlation was obtained with Spearman's correlation coefficient = 0.78 and regression coefficient (p <0.001). We found a wide range of heterogeneity of results. Performance of the professors was the basic component to judge the quality of the residents' courses.

Uncertainty estimates of purity measurements based on current information: toward a "live validation" of purity methods.

PubMed

Apostol, Izydor; Kelner, Drew; Jiang, Xinzhao Grace; Huang, Gang; Wypych, Jette; Zhang, Xin; Gastwirt, Jessica; Chen, Kenneth; Fodor, Szilan; Hapuarachchi, Suminda; Meriage, Dave; Ye, Frank; Poppe, Leszek; Szpankowski, Wojciech

2012-12-01

To predict precision and other performance characteristics of chromatographic purity methods, which represent the most widely used form of analysis in the biopharmaceutical industry. We have conducted a comprehensive survey of purity methods, and show that all performance characteristics fall within narrow measurement ranges. This observation was used to develop a model called Uncertainty Based on Current Information (UBCI), which expresses these performance characteristics as a function of the signal and noise levels, hardware specifications, and software settings. We applied the UCBI model to assess the uncertainty of purity measurements, and compared the results to those from conventional qualification. We demonstrated that the UBCI model is suitable to dynamically assess method performance characteristics, based on information extracted from individual chromatograms. The model provides an opportunity for streamlining qualification and validation studies by implementing a "live validation" of test results utilizing UBCI as a concurrent assessment of measurement uncertainty. Therefore, UBCI can potentially mitigate the challenges associated with laborious conventional method validation and facilitates the introduction of more advanced analytical technologies during the method lifecycle.

Single-laboratory validation of a saponification method for the determination of four polycyclic aromatic hydrocarbons in edible oils by HPLC-fluorescence detection.

PubMed

Akdoğan, Abdullah; Buttinger, Gerhard; Wenzl, Thomas

2016-01-01

An analytical method is reported for the determination of four polycyclic aromatic hydrocarbons (benzo[a]pyrene (BaP), benz[a]anthracene (BaA), benzo[b]fluoranthene (BbF) and chrysene (CHR)) in edible oils (sesame, maize, sunflower and olive oil) by high-performance liquid chromatography. Sample preparation is based on three steps including saponification, liquid-liquid partitioning and, finally, clean-up by solid phase extraction on 2 g of silica. Guidance on single-laboratory validation of the proposed analysis method was taken from the second edition of the Eurachem guide on method validation. The lower level of the working range of the method was determined by the limits of quantification of the individual analytes, and the upper level was equal to 5.0 µg kg(-1). The limits of detection and quantification of the four PAHs ranged from 0.06 to 0.12 µg kg(-1) and from 0.13 to 0.24 µg kg(-1). Recoveries of more than 84.8% were achieved for all four PAHs at two concentration levels (2.5 and 5.0 µg kg(-1)), and expanded relative measurement uncertainties were below 20%. The performance of the validated method was in all aspects compliant with provisions set in European Union legislation for the performance of analytical methods employed in the official control of food. The applicability of the method to routine samples was evaluated based on a limited number of commercial edible oil samples.

ENSURF: multi-model sea level forecast - implementation and validation results for the IBIROOS and Western Mediterranean regions

NASA Astrophysics Data System (ADS)

Pérez, B.; Brower, R.; Beckers, J.; Paradis, D.; Balseiro, C.; Lyons, K.; Cure, M.; Sotillo, M. G.; Hacket, B.; Verlaan, M.; Alvarez Fanjul, E.

2011-04-01

ENSURF (Ensemble SURge Forecast) is a multi-model application for sea level forecast that makes use of existing storm surge or circulation models today operational in Europe, as well as near-real time tide gauge data in the region, with the following main goals: - providing an easy access to existing forecasts, as well as to its performance and model validation, by means of an adequate visualization tool - generation of better forecasts of sea level, including confidence intervals, by means of the Bayesian Model Average Technique (BMA) The system was developed and implemented within ECOOP (C.No. 036355) European Project for the NOOS and the IBIROOS regions, based on MATROOS visualization tool developed by Deltares. Both systems are today operational at Deltares and Puertos del Estado respectively. The Bayesian Modelling Average technique generates an overall forecast probability density function (PDF) by making a weighted average of the individual forecasts PDF's; the weights represent the probability that a model will give the correct forecast PDF and are determined and updated operationally based on the performance of the models during a recent training period. This implies the technique needs the availability of sea level data from tide gauges in near-real time. Results of validation of the different models and BMA implementation for the main harbours will be presented for the IBIROOS and Western Mediterranean regions, where this kind of activity is performed for the first time. The work has proved to be useful to detect problems in some of the circulation models not previously well calibrated with sea level data, to identify the differences on baroclinic and barotropic models for sea level applications and to confirm the general improvement of the BMA forecasts.

Development of the TeamOBS-PPH - targeting clinical performance in postpartum hemorrhage.

PubMed

Brogaard, Lise; Hvidman, Lone; Hinshaw, Kim; Kierkegaard, Ole; Manser, Tanja; Musaeus, Peter; Arafeh, Julie; Daniels, Kay I; Judy, Amy E; Uldbjerg, Niels

2018-06-01

This study aimed to develop a valid and reliable TeamOBS-PPH tool for assessing clinical performance in the management of postpartum hemorrhage (PPH). The tool was evaluated using video-recordings of teams managing PPH in both real-life and simulated settings. A Delphi panel consisting of 12 obstetricians from the UK, Norway, Sweden, Iceland, and Denmark achieved consensus on (i) the elements to include in the assessment tool, (ii) the weighting of each element, and (iii) the final tool. The validity and reliability were evaluated according to Cook and Beckman. (Level 1) Four raters scored four video-recordings of in situ simulations of PPH. (Level 2) Two raters scored 85 video-recordings of real-life teams managing patients with PPH ≥1000 mL in two Danish hospitals. (Level 3) Two raters scored 15 video-recordings of in situ simulations of PPH from a US hospital. The tool was designed with scores from 0 to 100. (Level 1) Teams of novices had a median score of 54 (95% CI 48-60), whereas experienced teams had a median score of 75 (95% CI 71-79; p < 0.001). (Level 2) The intra-rater [intra-class correlation (ICC) = 0.96] and inter-rater (ICC = 0.83) agreements for real-life PPH were strong. The tool was applicable in all cases: atony, retained placenta, and lacerations. (Level 3) The tool was easily adapted to in situ simulation settings in the USA (ICC = 0.86). The TeamOBS-PPH tool appears to be valid and reliable for assessing clinical performance in real-life and simulated settings. The tool will be shared as the free TeamOBS App. © 2018 Nordic Federation of Societies of Obstetrics and Gynecology.

Development and validation of a web-based questionnaire for surveying the health and working conditions of high-performance marine craft populations.

PubMed

de Alwis, Manudul Pahansen; Lo Martire, Riccardo; Äng, Björn O; Garme, Karl

2016-06-20

High-performance marine craft crews are susceptible to various adverse health conditions caused by multiple interactive factors. However, there are limited epidemiological data available for assessment of working conditions at sea. Although questionnaire surveys are widely used for identifying exposures, outcomes and associated risks with high accuracy levels, until now, no validated epidemiological tool exists for surveying occupational health and performance in these populations. To develop and validate a web-based questionnaire for epidemiological assessment of occupational and individual risk exposure pertinent to the musculoskeletal health conditions and performance in high-performance marine craft populations. A questionnaire for investigating the association between work-related exposure, performance and health was initially developed by a consensus panel under four subdomains, viz. demography, lifestyle, work exposure and health and systematically validated by expert raters for content relevance and simplicity in three consecutive stages, each iteratively followed by a consensus panel revision. The item content validity index (I-CVI) was determined as the proportion of experts giving a rating of 3 or 4. The scale content validity index (S-CVI/Ave) was computed by averaging the I-CVIs for the assessment of the questionnaire as a tool. Finally, the questionnaire was pilot tested. The S-CVI/Ave increased from 0.89 to 0.96 for relevance and from 0.76 to 0.94 for simplicity, resulting in 36 items in the final questionnaire. The pilot test confirmed the feasibility of the questionnaire. The present study shows that the web-based questionnaire fulfils previously published validity acceptance criteria and is therefore considered valid and feasible for the empirical surveying of epidemiological aspects among high-performance marine craft crews and similar populations. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

The Arthroscopic Surgical Skill Evaluation Tool (ASSET)

PubMed Central

Koehler, Ryan J.; Amsdell, Simon; Arendt, Elizabeth A; Bisson, Leslie J; Braman, Jonathan P; Butler, Aaron; Cosgarea, Andrew J; Harner, Christopher D; Garrett, William E; Olson, Tyson; Warme, Winston J.; Nicandri, Gregg T.

2014-01-01

Background Surgeries employing arthroscopic techniques are among the most commonly performed in orthopaedic clinical practice however, valid and reliable methods of assessing the arthroscopic skill of orthopaedic surgeons are lacking. Hypothesis The Arthroscopic Surgery Skill Evaluation Tool (ASSET) will demonstrate content validity, concurrent criterion-oriented validity, and reliability, when used to assess the technical ability of surgeons performing diagnostic knee arthroscopy on cadaveric specimens. Study Design Cross-sectional study; Level of evidence, 3 Methods Content validity was determined by a group of seven experts using a Delphi process. Intra-articular performance of a right and left diagnostic knee arthroscopy was recorded for twenty-eight residents and two sports medicine fellowship trained attending surgeons. Subject performance was assessed by two blinded raters using the ASSET. Concurrent criterion-oriented validity, inter-rater reliability, and test-retest reliability were evaluated. Results Content validity: The content development group identified 8 arthroscopic skill domains to evaluate using the ASSET. Concurrent criterion-oriented validity: Significant differences in total ASSET score (p<0.05) between novice, intermediate, and advanced experience groups were identified. Inter-rater reliability: The ASSET scores assigned by each rater were strongly correlated (r=0.91, p <0.01) and the intra-class correlation coefficient between raters for the total ASSET score was 0.90. Test-retest reliability: there was a significant correlation between ASSET scores for both procedures attempted by each individual (r = 0.79, p<0.01). Conclusion The ASSET appears to be a useful, valid, and reliable method for assessing surgeon performance of diagnostic knee arthroscopy in cadaveric specimens. Studies are ongoing to determine its generalizability to other procedures as well as to the live OR and other simulated environments. PMID:23548808

Assessing Arthroscopic Skills Using Wireless Elbow-Worn Motion Sensors.

PubMed

Kirby, Georgina S J; Guyver, Paul; Strickland, Louise; Alvand, Abtin; Yang, Guang-Zhong; Hargrove, Caroline; Lo, Benny P L; Rees, Jonathan L

2015-07-01

Assessment of surgical skill is a critical component of surgical training. Approaches to assessment remain predominantly subjective, although more objective measures such as Global Rating Scales are in use. This study aimed to validate the use of elbow-worn, wireless, miniaturized motion sensors to assess the technical skill of trainees performing arthroscopic procedures in a simulated environment. Thirty participants were divided into three groups on the basis of their surgical experience: novices (n = 15), intermediates (n = 10), and experts (n = 5). All participants performed three standardized tasks on an arthroscopic virtual reality simulator while wearing wireless wrist and elbow motion sensors. Video output was recorded and a validated Global Rating Scale was used to assess performance; dexterity metrics were recorded from the simulator. Finally, live motion data were recorded via Bluetooth from the wireless wrist and elbow motion sensors and custom algorithms produced an arthroscopic performance score. Construct validity was demonstrated for all tasks, with Global Rating Scale scores and virtual reality output metrics showing significant differences between novices, intermediates, and experts (p < 0.001). The correlation of the virtual reality path length to the number of hand movements calculated from the wireless sensors was very high (p < 0.001). A comparison of the arthroscopic performance score levels with virtual reality output metrics also showed highly significant differences (p < 0.01). Comparisons of the arthroscopic performance score levels with the Global Rating Scale scores showed strong and highly significant correlations (p < 0.001) for both sensor locations, but those of the elbow-worn sensors were stronger and more significant (p < 0.001) than those of the wrist-worn sensors. A new wireless assessment of surgical performance system for objective assessment of surgical skills has proven valid for assessing arthroscopic skills. The elbow-worn sensors were shown to achieve an accurate assessment of surgical dexterity and performance. The validation of an entirely objective assessment of arthroscopic skill with wireless elbow-worn motion sensors introduces, for the first time, a feasible assessment system for the live operating theater with the added potential to be applied to other surgical and interventional specialties. Copyright © 2015 by The Journal of Bone and Joint Surgery, Incorporated.

Evolving the Principles and Practice of Validation for New Alternative Approaches to Toxicity Testing.

PubMed

Whelan, Maurice; Eskes, Chantra

Validation is essential for the translation of newly developed alternative approaches to animal testing into tools and solutions suitable for regulatory applications. Formal approaches to validation have emerged over the past 20 years or so and although they have helped greatly to progress the field, it is essential that the principles and practice underpinning validation continue to evolve to keep pace with scientific progress. The modular approach to validation should be exploited to encourage more innovation and flexibility in study design and to increase efficiency in filling data gaps. With the focus now on integrated approaches to testing and assessment that are based on toxicological knowledge captured as adverse outcome pathways, and which incorporate the latest in vitro and computational methods, validation needs to adapt to ensure it adds value rather than hinders progress. Validation needs to be pursued both at the method level, to characterise the performance of in vitro methods in relation their ability to detect any association of a chemical with a particular pathway or key toxicological event, and at the methodological level, to assess how integrated approaches can predict toxicological endpoints relevant for regulatory decision making. To facilitate this, more emphasis needs to be given to the development of performance standards that can be applied to classes of methods and integrated approaches that provide similar information. Moreover, the challenge of selecting the right reference chemicals to support validation needs to be addressed more systematically, consistently and in a manner that better reflects the state of the science. Above all however, validation requires true partnership between the development and user communities of alternative methods and the appropriate investment of resources.

Preliminary results of the Geoid Slope Validation Survey 2014 in Iowa

NASA Astrophysics Data System (ADS)

Wang, Y. M.; Becker, C.; Breidenbach, S.; Geoghegan, C.; Martin, D.; Winester, D.; Hanson, T.; Mader, G. L.; Eckl, M. C.

2014-12-01

The National Geodetic Survey conducted a second Geoid Slope Validation Survey in the summer of 2014 (GSVS14). The survey took place in Iowa along U.S Route 30. The survey line is approximately 200 miles long (325 km), extending from Denison, IA to Cedar Rapids, IA. There are over 200 official survey bench marks. A leveling survey was performed, conforming to 1st order, class II specifications. A GPS survey was performed using 24 to 48 hour occupations. Absolute gravity, relative gravity, and gravity gradient measurements were also collected during the survey. In addition, deflections of the vertical were acquired at 200 eccentric survey benchmarks using the Compact Digital Astrometric Camera (CODIAC) camera. This paper presents the preliminary results of the survey, including the accuracy analysis of the leveling data, GPS ellipsoidal heights, and the deflections of the vertical which serves as an independent data set in addition to the GPS/leveling implied geoid heights.

Development and validation of a prognostic nomogram for terminally ill cancer patients.

PubMed

Feliu, Jaime; Jiménez-Gordo, Ana María; Madero, Rosario; Rodríguez-Aizcorbe, José Ramón; Espinosa, Enrique; Castro, Javier; Acedo, Jesús Domingo; Martínez, Beatriz; Alonso-Babarro, Alberto; Molina, Raquel; Cámara, Juan Carlos; García-Paredes, María Luisa; González-Barón, Manuel

2011-11-02

Determining life expectancy in terminally ill cancer patients is a difficult task. We aimed to develop and validate a nomogram to predict the length of survival in patients with terminal disease. From February 1, 2003, to December 31, 2005, 406 consecutive terminally ill patients were entered into the study. We analyzed 38 features prognostic of life expectancy among terminally ill patients by multivariable Cox regression and identified the most accurate and parsimonious model by backward variable elimination according to the Akaike information criterion. Five clinical and laboratory variables were built into a nomogram to estimate the probability of patient survival at 15, 30, and 60 days. We validated and calibrated the nomogram with an external validation cohort of 474 patients who were treated from June 1, 2006, through December 31, 2007. The median overall survival was 29.1 days for the training set and 18.3 days for the validation set. Eastern Cooperative Oncology Group performance status, lactate dehydrogenase levels, lymphocyte levels, albumin levels, and time from initial diagnosis to diagnosis of terminal disease were retained in the multivariable Cox proportional hazards model as independent prognostic factors of survival and formed the basis of the nomogram. The nomogram had high predictive performance, with a bootstrapped corrected concordance index of 0.70, and it showed good calibration. External independent validation revealed 68% predictive accuracy. We developed a highly accurate tool that uses basic clinical and analytical information to predict the probability of survival at 15, 30, and 60 days in terminally ill cancer patients. This tool can help physicians making decisions on clinical care at the end of life.

Reference Proteome Extracts for Mass Spec Instrument Performance Validation and Method Development

PubMed Central

Rosenblatt, Mike; Urh, Marjeta; Saveliev, Sergei

2014-01-01

Biological samples of high complexity are required to test protein mass spec sample preparation procedures and validate mass spec instrument performance. Total cell protein extracts provide the needed sample complexity. However, to be compatible with mass spec applications, such extracts should meet a number of design requirements: compatibility with LC/MS (free of detergents, etc.)high protein integrity (minimal level of protein degradation and non-biological PTMs)compatibility with common sample preparation methods such as proteolysis, PTM enrichment and mass-tag labelingLot-to-lot reproducibility Here we describe total protein extracts from yeast and human cells that meet the above criteria. Two extract formats have been developed: Intact protein extracts with primary use for sample preparation method development and optimizationPre-digested extracts (peptides) with primary use for instrument validation and performance monitoring

Validation of the GreenLight™ Simulator and development of a training curriculum for photoselective vaporisation of the prostate.

PubMed

Aydin, Abdullatif; Muir, Gordon H; Graziano, Manuela E; Khan, Muhammad Shamim; Dasgupta, Prokar; Ahmed, Kamran

2015-06-01

To assess face, content and construct validity, and feasibility and acceptability of the GreenLight™ Simulator as a training tool for photoselective vaporisation of the prostate (PVP), and to establish learning curves and develop an evidence-based training curriculum. This prospective, observational and comparative study, recruited novice (25 participants), intermediate (14) and expert-level urologists (seven) from the UK and Europe at the 28th European Association of Urological Surgeons Annual Meeting 2013. A group of novices (12 participants) performed 10 sessions of subtask training modules followed by a long operative case, whereas a second group (13) performed five sessions of a given case module. Intermediate and expert groups performed all training modules once, followed by one operative case. The outcome measures for learning curves and construct validity were time to task, coagulation time, vaporisation time, average sweep speed, average laser distance, blood loss, operative errors, and instrument cost. Face and content validity, feasibility and acceptability were addressed through a quantitative survey. Construct validity was demonstrated in two of five training modules (P = 0.038; P = 0.018) and in a considerable number of case metrics (P = 0.034). Learning curves were seen in all five training modules (P < 0.001) and significant reduction in case operative time (P < 0.001) and error (P = 0.017) were seen. An evidence-based training curriculum, to help trainees acquire transferable skills, was produced using the results. This study has shown the GreenLight Simulator to be a valid and useful training tool for PVP. It is hoped that by using the training curriculum for the GreenLight Simulator, novice trainees can acquire skills and knowledge to a predetermined level of proficiency. © 2014 The Authors. BJU International © 2014 BJU International.

Electrotactile Feedback Improves Performance and Facilitates Learning in the Routine Grasping Task.

PubMed

Isaković, Milica; Belić, Minja; Štrbac, Matija; Popović, Igor; Došen, Strahinja; Farina, Dario; Keller, Thierry

2016-06-13

Aim of this study was to investigate the feasibility of electrotactile feedback in closed loop training of force control during the routine grasping task. The feedback was provided using an array electrode and a simple six-level spatial coding, and the experiment was conducted in three amputee subjects. The psychometric tests confirmed that the subjects could perceive and interpret the electrotactile feedback with a high success rate. The subjects performed the routine grasping task comprising 4 blocks of 60 grasping trials. In each trial, the subjects employed feedforward control to close the hand and produce the desired grasping force (four levels). First (baseline) and the last (validation) session were performed in open loop, while the second and the third session (training) included electrotactile feedback. The obtained results confirmed that using the feedback improved the accuracy and precision of the force control. In addition, the subjects performed significantly better in the validation vs. baseline session, therefore suggesting that electrotactile feedback can be used for learning and training of myoelectric control.

Full immersion simulation: validation of a distributed simulation environment for technical and non-technical skills training in Urology.

PubMed

Brewin, James; Tang, Jessica; Dasgupta, Prokar; Khan, Muhammad S; Ahmed, Kamran; Bello, Fernando; Kneebone, Roger; Jaye, Peter

2015-07-01

To evaluate the face, content and construct validity of the distributed simulation (DS) environment for technical and non-technical skills training in endourology. To evaluate the educational impact of DS for urology training. DS offers a portable, low-cost simulated operating room environment that can be set up in any open space. A prospective mixed methods design using established validation methodology was conducted in this simulated environment with 10 experienced and 10 trainee urologists. All participants performed a simulated prostate resection in the DS environment. Outcome measures included surveys to evaluate the DS, as well as comparative analyses of experienced and trainee urologist's performance using real-time and 'blinded' video analysis and validated performance metrics. Non-parametric statistical methods were used to compare differences between groups. The DS environment demonstrated face, content and construct validity for both non-technical and technical skills. Kirkpatrick level 1 evidence for the educational impact of the DS environment was shown. Further studies are needed to evaluate the effect of simulated operating room training on real operating room performance. This study has shown the validity of the DS environment for non-technical, as well as technical skills training. DS-based simulation appears to be a valuable addition to traditional classroom-based simulation training. © 2014 The Authors BJU International © 2014 BJU International Published by John Wiley & Sons Ltd.

Development of Level 1b Calibration and Validation Readiness, Implementation and Management Plans for GOES-R

NASA Technical Reports Server (NTRS)

Kunkee, David B.; Farley, Robert W.; Kwan, Betty P.; Hecht, James H.; Walterscheid, Richard L.; Claudepierre, Seth G.; Bishop, Rebecca L.; Gelinas, Lynette J.; Deluccia, Frank J.

2017-01-01

A complement of Readiness, Implementation and Management Plans (RIMPs) to facilitate management of post-launch product test activities for the official Geostationary Operational Environmental Satellite (GOES-R) Level 1b (L1b) products have been developed and documented. Separate plans have been created for each of the GOES-R sensors including: the Advanced Baseline Imager (ABI), the Extreme ultraviolet and X-ray Irradiance Sensors (EXIS), Geostationary Lightning Mapper (GLM), GOES-R Magnetometer (MAG), the Space Environment In-Situ Suite (SEISS), and the Solar Ultraviolet Imager (SUVI). The GOES-R program has implemented these RIMPs in order to address the full scope of CalVal activities required for a successful demonstration of GOES-R L1b data product quality throughout the three validation stages: Beta, Provisional and Full Validation. For each product maturity level, the RIMPs include specific performance criteria and required artifacts that provide evidence a given validation stage has been reached, the timing when each stage will be complete, a description of every applicable Post-Launch Product Test (PLPT), roles and responsibilities of personnel, upstream dependencies, and analysis methods and tools to be employed during validation. Instrument level Post-Launch Tests (PLTs) are also referenced and apply primarily to functional check-out of the instruments.

Using remote sensing for validation of a large scale hydrologic and hydrodynamic model in the Amazon

NASA Astrophysics Data System (ADS)

Paiva, R. C.; Bonnet, M.; Buarque, D. C.; Collischonn, W.; Frappart, F.; Mendes, C. B.

2011-12-01

We present the validation of the large-scale, catchment-based hydrological MGB-IPH model in the Amazon River basin. In this model, physically-based equations are used to simulate the hydrological processes, such as the Penman Monteith method to estimate evapotranspiration, or the Moore and Clarke infiltration model. A new feature recently introduced in the model is a 1D hydrodynamic module for river routing. It uses the full Saint-Venant equations and a simple floodplain storage model. River and floodplain geometry parameters are extracted from SRTM DEM using specially developed GIS algorithms that provide catchment discretization, estimation of river cross-sections geometry and water storage volume variations in the floodplains. The model was forced using satellite-derived daily rainfall TRMM 3B42, calibrated against discharge data and first validated using daily discharges and water levels from 111 and 69 stream gauges, respectively. Then, we performed a validation against remote sensing derived hydrological products, including (i) monthly Terrestrial Water Storage (TWS) anomalies derived from GRACE, (ii) river water levels derived from ENVISAT satellite altimetry data (212 virtual stations from Santos da Silva et al., 2010) and (iii) a multi-satellite monthly global inundation extent dataset at ~25 x 25 km spatial resolution (Papa et al., 2010). Validation against river discharges shows good performance of the MGB-IPH model. For 70% of the stream gauges, the Nash and Suttcliffe efficiency index (ENS) is higher than 0.6 and at Óbidos, close to Amazon river outlet, ENS equals 0.9 and the model bias equals,-4.6%. Largest errors are located in drainage areas outside Brazil and we speculate that it is due to the poor quality of rainfall datasets in these areas poorly monitored and/or mountainous. Validation against water levels shows that model is performing well in the major tributaries. For 60% of virtual stations, ENS is higher than 0.6. But, similarly, largest errors are also located in drainage areas outside Brazil, mostly Japurá River, and in the lower Amazon River. In the latter, correlation with observations is high but the model underestimates the amplitude of water levels. We also found a large bias between model and ENVISAT water levels, ranging from -3 to -15 m. The model provided TWS in good accordance with GRACE estimates. ENS values for TWS over the whole Amazon equals 0.93. We also analyzed results in 21 sub-regions of 4 x 4°. ENS is smaller than 0.8 only in 5 areas, and these are found mostly in the northwest part of the Amazon, possibly due to same errors reported in discharge results. Flood extent validation is under development, but a previous analysis in Brazilian part of Solimões River basin suggests a good model performance. The authors are grateful for the financial and operational support from the brazilian agencies FINEP, CNPq and ANA and from the french observatories HYBAM and SOERE RBV.

Addressing EO-1 Spacecraft Pulsed Plasma Thruster EMI Concerns

NASA Technical Reports Server (NTRS)

Zakrzwski, C. M.; Davis, Mitch; Sarmiento, Charles; Bauer, Frank H. (Technical Monitor)

2001-01-01

The Pulsed Plasma Thruster (PPT) Experiment on the Earth Observing One (EO-1) spacecraft has been designed to demonstrate the capability of a new generation PPT to perform spacecraft attitude control. Results from PPT unit level radiated electromagnetic interference (EMI) tests led to concerns about potential interference problems with other spacecraft subsystems. Initial plans to address these concerns included firing the PPT at the spacecraft level both in atmosphere, with special ground support equipment. and in vacuum. During the spacecraft level tests, additional concerns where raised about potential harm to the Advanced Land Imager (ALI). The inadequacy of standard radiated emission test protocol to address pulsed electromagnetic discharges and the lack of resources required to perform compatibility tests between the PPT and an ALI test unit led to changes in the spacecraft level validation plan. An EMI shield box for the PPT was constructed and validated for spacecraft level ambient testing. Spacecraft level vacuum tests of the PPT were deleted. Implementation of the shield box allowed for successful spacecraft level testing of the PPT while eliminating any risk to the ALI. The ALI demonstration will precede the PPT demonstration to eliminate any possible risk of damage of ALI from PPT operation.

A systematic approach to the Planck LFI end-to-end test and its application to the DPC Level 1 pipeline

NASA Astrophysics Data System (ADS)

Frailis, M.; Maris, M.; Zacchei, A.; Morisset, N.; Rohlfs, R.; Meharga, M.; Binko, P.; Türler, M.; Galeotta, S.; Gasparo, F.; Franceschi, E.; Butler, R. C.; D'Arcangelo, O.; Fogliani, S.; Gregorio, A.; Lowe, S. R.; Maggio, G.; Malaspina, M.; Mandolesi, N.; Manzato, P.; Pasian, F.; Perrotta, F.; Sandri, M.; Terenzi, L.; Tomasi, M.; Zonca, A.

2009-12-01

The Level 1 of the Planck LFI Data Processing Centre (DPC) is devoted to the handling of the scientific and housekeeping telemetry. It is a critical component of the Planck ground segment which has to strictly commit to the project schedule to be ready for the launch and flight operations. In order to guarantee the quality necessary to achieve the objectives of the Planck mission, the design and development of the Level 1 software has followed the ESA Software Engineering Standards. A fundamental step in the software life cycle is the Verification and Validation of the software. The purpose of this work is to show an example of procedures, test development and analysis successfully applied to a key software project of an ESA mission. We present the end-to-end validation tests performed on the Level 1 of the LFI-DPC, by detailing the methods used and the results obtained. Different approaches have been used to test the scientific and housekeeping data processing. Scientific data processing has been tested by injecting signals with known properties directly into the acquisition electronics, in order to generate a test dataset of real telemetry data and reproduce as much as possible nominal conditions. For the HK telemetry processing, validation software have been developed to inject known parameter values into a set of real housekeeping packets and perform a comparison with the corresponding timelines generated by the Level 1. With the proposed validation and verification procedure, where the on-board and ground processing are viewed as a single pipeline, we demonstrated that the scientific and housekeeping processing of the Planck-LFI raw data is correct and meets the project requirements.

The evaluation of lumbar multifidus muscle function via palpation: reliability and validity of a new clinical test.

PubMed

Hebert, Jeffrey J; Koppenhaver, Shane L; Teyhen, Deydre S; Walker, Bruce F; Fritz, Julie M

2015-06-01

The lumbar multifidus muscle provides an important contribution to lumbar spine stability, and the restoration of lumbar multifidus function is a frequent goal of rehabilitation. Currently, there are no reliable and valid physical examination procedures available to assess lumbar multifidus function among patients with low back pain. To examine the inter-rater reliability and concurrent validity of the multifidus lift test (MLT) to identify lumbar multifidus dysfunction among patients with low back pain. A cross-sectional analysis of reliability and concurrent validity performed in a university outpatient research facility. Thirty-two persons aged 18 to 60 years with current low back pain and a minimum modified Oswestry disability score of 20%. Study participants were excluded if they reported a history of lumbar spine surgery, lumbar radiculopathy, medical red flags, osteoporosis, or had recently been treated with spinal manipulation or trunk stabilization exercises. Concurrent measures of lumbar multifidus muscle function at the L4-L5 and L5-S1 levels were obtained with the MLT (index test) and real-time ultrasound imaging (reference standard). The inter-rater reliability of the MLT was examined by measuring the level of agreement between two blinded examiners. Concurrent validity of the MLT was investigated by comparing clinicians' judgments with real-time ultrasound imaging measures of lumbar multifidus function. Inter-rater reliability of the MLT was substantial to excellent (κ=0.75 to 0.81, p≤.01) and free from errors of bias and prevalence. When performed at L4-L5 or L5-S1, the MLT demonstrated evidence of concurrent validity through its relationship with the reference standard results at L4-L5 (rbis=0.59-0.73, p≤.01). The MLT generally failed to demonstrate a relationship with the reference standard results from the L5-S1 level. Our results provide preliminary evidence supporting the reliability and validity of the MLT to assess lumbar multifidus function at the L4-L5 spinal level. Additional research examining the measurement properties and utility of this test should be undertaken before confident implementation with patients. Copyright © 2015 Elsevier Inc. All rights reserved.

Young and restless: validation of the Mind-Wandering Questionnaire (MWQ) reveals disruptive impact of mind-wandering for youth

PubMed Central

Mrazek, Michael D.; Phillips, Dawa T.; Franklin, Michael S.; Broadway, James M.; Schooler, Jonathan W.

2013-01-01

Mind-wandering is the focus of extensive investigation, yet until recently there has been no validated scale to directly measure trait levels of task-unrelated thought. Scales commonly used to assess mind-wandering lack face validity, measuring related constructs such as daydreaming or behavioral errors. Here we report four studies validating a Mind-Wandering Questionnaire (MWQ) across college, high school, and middle school samples. The 5-item scale showed high internal consistency, as well as convergent validity with existing measures of mind-wandering and related constructs. Trait levels of mind-wandering, as measured by the MWQ, were correlated with task-unrelated thought measured by thought sampling during a test of reading comprehension. In both middle school and high school samples, mind-wandering during testing was associated with worse reading comprehension. By contrast, elevated trait levels of mind-wandering predicted worse mood, less life-satisfaction, greater stress, and lower self-esteem. By extending the use of thought sampling to measure mind-wandering among adolescents, our findings also validate the use of this methodology with younger populations. Both the MWQ and thought sampling indicate that mind-wandering is a pervasive—and problematic—influence on the performance and well-being of adolescents. PMID:23986739

Validation of minicams for measuring concentrations of chemical agent in environmental air

DOE Office of Scientific and Technical Information (OSTI.GOV)

Menton, R.G.; Hayes, T.L.; Chou, Y.L.

1993-05-13

Environmental monitoring for chemical agents is necessary to ensure that notification and appropriate action will be taken in the, event that there is a release exceeding control limits of such agents into the workplace outside of engineering controls. Prior to implementing new analytical procedures for environmental monitoring, precision and accuracy (PA) tests are conducted to ensure that an agent monitoring system performs according to specified accuracy, precision, and sensitivity requirements. This testing not only establishes the accuracy and precision of the method, but also determines what factors can affect the method's performance. Performance measures that are particularly important in agentmore » monitoring include the Detection Limit (DL), Decision Limit (DC), Found Action Level (FAL), and the Target Action Level (TAL). PA experiments were performed at Battelle's Medical Research and Evaluation Facility (MREF) to validate the use of the miniature chemical agent monitoring system (MINICAMs) for measuring environmental air concentrations of sulfur mustard (HD). This presentation discusses the experimental and statistical approaches for characterizing the performance of MINICAMS for measuring HD in air.« less

Assessment of construct validity of a virtual reality laparoscopy simulator.

PubMed

Rosenthal, Rachel; Gantert, Walter A; Hamel, Christian; Hahnloser, Dieter; Metzger, Juerg; Kocher, Thomas; Vogelbach, Peter; Scheidegger, Daniel; Oertli, Daniel; Clavien, Pierre-Alain

2007-08-01

The aim of this study was to assess whether virtual reality (VR) can discriminate between the skills of novices and intermediate-level laparoscopic surgical trainees (construct validity), and whether the simulator assessment correlates with an expert's evaluation of performance. Three hundred and seven (307) participants of the 19th-22nd Davos International Gastrointestinal Surgery Workshops performed the clip-and-cut task on the Xitact LS 500 VR simulator (Xitact S.A., Morges, Switzerland). According to their previous experience in laparoscopic surgery, participants were assigned to the basic course (BC) or the intermediate course (IC). Objective performance parameters recorded by the simulator were compared to the standardized assessment by the course instructors during laparoscopic pelvitrainer and conventional surgery exercises. IC participants performed significantly better on the VR simulator than BC participants for the task completion time as well as the economy of movement of the right instrument, not the left instrument. Participants with maximum scores in the pelvitrainer cholecystectomy task performed the VR trial significantly faster, compared to those who scored less. In the conventional surgery task, a significant difference between those who scored the maximum and those who scored less was found not only for task completion time, but also for economy of movement of the right instrument. VR simulation provides a valid assessment of psychomotor skills and some basic aspects of spatial skills in laparoscopic surgery. Furthermore, VR allows discrimination between trainees with different levels of experience in laparoscopic surgery establishing construct validity for the Xitact LS 500 clip-and-cut task. Virtual reality may become the gold standard to assess and monitor surgical skills in laparoscopic surgery.

A semi-automatic method for left ventricle volume estimate: an in vivo validation study

NASA Technical Reports Server (NTRS)

Corsi, C.; Lamberti, C.; Sarti, A.; Saracino, G.; Shiota, T.; Thomas, J. D.

2001-01-01

This study aims to the validation of the left ventricular (LV) volume estimates obtained by processing volumetric data utilizing a segmentation model based on level set technique. The validation has been performed by comparing real-time volumetric echo data (RT3DE) and magnetic resonance (MRI) data. A validation protocol has been defined. The validation protocol was applied to twenty-four estimates (range 61-467 ml) obtained from normal and pathologic subjects, which underwent both RT3DE and MRI. A statistical analysis was performed on each estimate and on clinical parameters as stroke volume (SV) and ejection fraction (EF). Assuming MRI estimates (x) as a reference, an excellent correlation was found with volume measured by utilizing the segmentation procedure (y) (y=0.89x + 13.78, r=0.98). The mean error on SV was 8 ml and the mean error on EF was 2%. This study demonstrated that the segmentation technique is reliably applicable on human hearts in clinical practice.

System-Level Radiation Hardening

NASA Technical Reports Server (NTRS)

Ladbury, Ray

2014-01-01

Although system-level radiation hardening can enable the use of high-performance components and enhance the capabilities of a spacecraft, hardening techniques can be costly and can compromise the very performance designers sought from the high-performance components. Moreover, such techniques often result in a complicated design, especially if several complex commercial microcircuits are used, each posing its own hardening challenges. The latter risk is particularly acute for Commercial-Off-The-Shelf components since high-performance parts (e.g. double-data-rate synchronous dynamic random access memories - DDR SDRAMs) may require other high-performance commercial parts (e.g. processors) to support their operation. For these reasons, it is essential that system-level radiation hardening be a coordinated effort, from setting requirements through testing up to and including validation.

Virtual reality simulation training in Otolaryngology.

PubMed

Arora, Asit; Lau, Loretta Y M; Awad, Zaid; Darzi, Ara; Singh, Arvind; Tolley, Neil

2014-01-01

To conduct a systematic review of the validity data for the virtual reality surgical simulator platforms available in Otolaryngology. Ovid and Embase databases searched July 13, 2013. Four hundred and nine abstracts were independently reviewed by 2 authors. Thirty-six articles which fulfilled the search criteria were retrieved and viewed in full text. These articles were assessed for quantitative data on at least one aspect of face, content, construct or predictive validity. Papers were stratified by simulator, sub-specialty and further classified by the validation method used. There were 21 articles reporting applications for temporal bone surgery (n = 12), endoscopic sinus surgery (n = 6) and myringotomy (n = 3). Four different simulator platforms were validated for temporal bone surgery and two for each of the other surgical applications. Face/content validation represented the most frequent study type (9/21). Construct validation studies performed on temporal bone and endoscopic sinus surgery simulators showed that performance measures reliably discriminated between different experience levels. Simulation training improved cadaver temporal bone dissection skills and operating room performance in sinus surgery. Several simulator platforms particularly in temporal bone surgery and endoscopic sinus surgery are worthy of incorporation into training programmes. Standardised metrics are necessary to guide curriculum development in Otolaryngology. Copyright © 2013 Surgical Associates Ltd. Published by Elsevier Ltd. All rights reserved.

Individual Passive Chemical Sampler Testing Continued Chemical Agent and TIC Performance Validation

DTIC Science & Technology

2002-04-01

period of high temperature, although the atmosphere was wet. 4.3 Post-Deployment Activities The deployment of the samplers did not go as...4.4 Day 0 Adsorption and Recovery Comparison Between Gore Low-Level and Gore High -Level Samplers at Varying Temperatures...43 Figure 4.5 Day 0 Adsorption and Recovery Comparison Between SKC High Level and Gore High -Level Samplers

Isokinetic knee strength qualities as predictors of jumping performance in high-level volleyball athletes: multiple regression approach.

PubMed

Sattler, Tine; Sekulic, Damir; Spasic, Miodrag; Osmankac, Nedzad; Vicente João, Paulo; Dervisevic, Edvin; Hadzic, Vedran

2016-01-01

Previous investigations noted potential importance of isokinetic strength in rapid muscular performances, such as jumping. This study aimed to identify the influence of isokinetic-knee-strength on specific jumping performance in volleyball. The secondary aim of the study was to evaluate reliability and validity of the two volleyball-specific jumping tests. The sample comprised 67 female (21.96±3.79 years; 68.26±8.52 kg; 174.43±6.85 cm) and 99 male (23.62±5.27 years; 84.83±10.37 kg; 189.01±7.21 cm) high- volleyball players who competed in 1st and 2nd National Division. Subjects were randomly divided into validation (N.=55 and 33 for males and females, respectively) and cross-validation subsamples (N.=54 and 34 for males and females, respectively). Set of predictors included isokinetic tests, to evaluate the eccentric and concentric strength capacities of the knee extensors, and flexors for dominant and non-dominant leg. The main outcome measure for the isokinetic testing was peak torque (PT) which was later normalized for body mass and expressed as PT/Kg. Block-jump and spike-jump performances were measured over three trials, and observed as criteria. Forward stepwise multiple regressions were calculated for validation subsamples and then cross-validated. Cross validation included correlations between and t-test differences between observed and predicted scores; and Bland Altman graphics. Jumping tests were found to be reliable (spike jump: ICC of 0.79 and 0.86; block-jump: ICC of 0.86 and 0.90; for males and females, respectively), and their validity was confirmed by significant t-test differences between 1st vs. 2nd division players. Isokinetic variables were found to be significant predictors of jumping performance in females, but not among males. In females, the isokinetic-knee measures were shown to be stronger and more valid predictors of the block-jump (42% and 64% of the explained variance for validation and cross-validation subsample, respectively) than that of the spike-jump (39% and 34% of the explained variance for validation and cross-validation subsample, respectively). Differences between prediction models calculated for males and females are mostly explained by gender-specific biomechanics of jumping. Study defined importance of knee-isokinetic-strength in volleyball jumping performance in female athletes. Further studies should evaluate association between ankle-isokinetic-strength and volleyball-specific jumping performances. Results reinforce the need for the cross-validation of the prediction-models in sport and exercise sciences.

ENSURF: multi-model sea level forecast - implementation and validation results for the IBIROOS and Western Mediterranean regions

NASA Astrophysics Data System (ADS)

Pérez, B.; Brouwer, R.; Beckers, J.; Paradis, D.; Balseiro, C.; Lyons, K.; Cure, M.; Sotillo, M. G.; Hackett, B.; Verlaan, M.; Fanjul, E. A.

2012-03-01

ENSURF (Ensemble SURge Forecast) is a multi-model application for sea level forecast that makes use of several storm surge or circulation models and near-real time tide gauge data in the region, with the following main goals: 1. providing easy access to existing forecasts, as well as to its performance and model validation, by means of an adequate visualization tool; 2. generation of better forecasts of sea level, including confidence intervals, by means of the Bayesian Model Average technique (BMA). The Bayesian Model Average technique generates an overall forecast probability density function (PDF) by making a weighted average of the individual forecasts PDF's; the weights represent the Bayesian likelihood that a model will give the correct forecast and are continuously updated based on the performance of the models during a recent training period. This implies the technique needs the availability of sea level data from tide gauges in near-real time. The system was implemented for the European Atlantic facade (IBIROOS region) and Western Mediterranean coast based on the MATROOS visualization tool developed by Deltares. Results of validation of the different models and BMA implementation for the main harbours are presented for these regions where this kind of activity is performed for the first time. The system is currently operational at Puertos del Estado and has proved to be useful in the detection of calibration problems in some of the circulation models, in the identification of the systematic differences between baroclinic and barotropic models for sea level forecasts and to demonstrate the feasibility of providing an overall probabilistic forecast, based on the BMA method.

Domestic violence on children: development and validation of an instrument to evaluate knowledge of health professionals 1

PubMed Central

Oliveira, Lanuza Borges; Soares, Fernanda Amaral; Silveira, Marise Fagundes; de Pinho, Lucinéia; Caldeira, Antônio Prates; Leite, Maísa Tavares de Souza

2016-01-01

ABSTRACT Objective: to develop and validate an instrument to evaluate the knowledge of health professionals about domestic violence on children. Method: this was a study conducted with 194 physicians, nurses and dentists. A literature review was performed for preparation of the items and identification of the dimensions. Apparent and content validation was performed using analysis of three experts and 27 professors of the pediatric health discipline. For construct validation, Cronbach's alpha was used, and the Kappa test was applied to verify reproducibility. The criterion validation was conducted using the Student's t-test. Results: the final instrument included 56 items; the Cronbach alpha was 0.734, the Kappa test showed a correlation greater than 0.6 for most items, and the Student t-test showed a statistically significant value to the level of 5% for the two selected variables: years of education and using the Family Health Strategy. Conclusion: the instrument is valid and can be used as a promising tool to develop or direct actions in public health and evaluate knowledge about domestic violence on children. PMID:27556878

The Model Human Processor and the Older Adult: Parameter Estimation and Validation Within a Mobile Phone Task

PubMed Central

Jastrzembski, Tiffany S.; Charness, Neil

2009-01-01

The authors estimate weighted mean values for nine information processing parameters for older adults using the Card, Moran, and Newell (1983) Model Human Processor model. The authors validate a subset of these parameters by modeling two mobile phone tasks using two different phones and comparing model predictions to a sample of younger (N = 20; Mage = 20) and older (N = 20; Mage = 69) adults. Older adult models fit keystroke-level performance at the aggregate grain of analysis extremely well (R = 0.99) and produced equivalent fits to previously validated younger adult models. Critical path analyses highlighted points of poor design as a function of cognitive workload, hardware/software design, and user characteristics. The findings demonstrate that estimated older adult information processing parameters are valid for modeling purposes, can help designers understand age-related performance using existing interfaces, and may support the development of age-sensitive technologies. PMID:18194048

The Model Human Processor and the older adult: parameter estimation and validation within a mobile phone task.

PubMed

Jastrzembski, Tiffany S; Charness, Neil

2007-12-01

The authors estimate weighted mean values for nine information processing parameters for older adults using the Card, Moran, and Newell (1983) Model Human Processor model. The authors validate a subset of these parameters by modeling two mobile phone tasks using two different phones and comparing model predictions to a sample of younger (N = 20; M-sub(age) = 20) and older (N = 20; M-sub(age) = 69) adults. Older adult models fit keystroke-level performance at the aggregate grain of analysis extremely well (R = 0.99) and produced equivalent fits to previously validated younger adult models. Critical path analyses highlighted points of poor design as a function of cognitive workload, hardware/software design, and user characteristics. The findings demonstrate that estimated older adult information processing parameters are valid for modeling purposes, can help designers understand age-related performance using existing interfaces, and may support the development of age-sensitive technologies.

A Framework for Performing Verification and Validation in Reuse Based Software Engineering

NASA Technical Reports Server (NTRS)

Addy, Edward A.

1997-01-01

Verification and Validation (V&V) is currently performed during application development for many systems, especially safety-critical and mission- critical systems. The V&V process is intended to discover errors, especially errors related to critical processing, as early as possible during the development process. The system application provides the context under which the software artifacts are validated. This paper describes a framework that extends V&V from an individual application system to a product line of systems that are developed within an architecture-based software engineering environment. This framework includes the activities of traditional application-level V&V, and extends these activities into domain engineering and into the transition between domain engineering and application engineering. The framework includes descriptions of the types of activities to be performed during each of the life-cycle phases, and provides motivation for the activities.

Development and Validity of a Silicone Renal Tumor Model for Robotic Partial Nephrectomy Training.

PubMed

Monda, Steven M; Weese, Jonathan R; Anderson, Barrett G; Vetter, Joel M; Venkatesh, Ramakrishna; Du, Kefu; Andriole, Gerald L; Figenshau, Robert S

2018-04-01

To provide a training tool to address the technical challenges of robot-assisted laparoscopic partial nephrectomy, we created silicone renal tumor models using 3-dimensional printed molds of a patient's kidney with a mass. In this study, we assessed the face, content, and construct validity of these models. Surgeons of different training levels completed 4 simulations on silicone renal tumor models. Participants were surveyed on the usefulness and realism of the model as a training tool. Performance was measured using operation-specific metrics, self-reported operative demands (NASA Task Load Index [NASA TLX]), and blinded expert assessment (Global Evaluative Assessment of Robotic Surgeons [GEARS]). Twenty-four participants included attending urologists, endourology fellows, urology residents, and medical students. Post-training surveys of expert participants yielded mean results of 79.2 on the realism of the model's overall feel and 90.2 on the model's overall usefulness for training. Renal artery clamp times and GEARS scores were significantly better in surgeons further in training (P ≤.005 and P ≤.025). Renal artery clamp times, preserved renal parenchyma, positive margins, NASA TLX, and GEARS scores were all found to improve across trials (P <.001, P = .025, P = .024, P ≤.020, and P ≤.006, respectively). Face, content, and construct validity were demonstrated in the use of a silicone renal tumor model in a cohort of surgeons of different training levels. Expert participants deemed the model useful and realistic. Surgeons of higher training levels performed better than less experienced surgeons in various study metrics, and improvements within individuals were observed over sequential trials. Future studies should aim to assess model predictive validity, namely, the association between model performance improvements and improvements in live surgery. Copyright © 2018 Elsevier Inc. All rights reserved.

Head-to-Head Comparison and Evaluation of 92 Plasma Protein Biomarkers for Early Detection of Colorectal Cancer in a True Screening Setting.

PubMed

Chen, Hongda; Zucknick, Manuela; Werner, Simone; Knebel, Phillip; Brenner, Hermann

2015-07-15

Novel noninvasive blood-based screening tests are strongly desirable for early detection of colorectal cancer. We aimed to conduct a head-to-head comparison of the diagnostic performance of 92 plasma-based tumor-associated protein biomarkers for early detection of colorectal cancer in a true screening setting. Among all available 35 carriers of colorectal cancer and a representative sample of 54 men and women free of colorectal neoplasms recruited in a cohort of screening colonoscopy participants in 2005-2012 (N = 5,516), the plasma levels of 92 protein biomarkers were measured. ROC analyses were conducted to evaluate the diagnostic performance. A multimarker algorithm was developed through the Lasso logistic regression model and validated in an independent validation set. The .632+ bootstrap method was used to adjust for the potential overestimation of diagnostic performance. Seventeen protein markers were identified to show statistically significant differences in plasma levels between colorectal cancer cases and controls. The adjusted area under the ROC curves (AUC) of these 17 individual markers ranged from 0.55 to 0.70. An eight-marker classifier was constructed that increased the adjusted AUC to 0.77 [95% confidence interval (CI), 0.59-0.91]. When validating this algorithm in an independent validation set, the AUC was 0.76 (95% CI, 0.65-0.85), and sensitivities at cutoff levels yielding 80% and 90% specificities were 65% (95% CI, 41-80%) and 44% (95% CI, 24-72%), respectively. The identified profile of protein biomarkers could contribute to the development of a powerful multimarker blood-based test for early detection of colorectal cancer. ©2015 American Association for Cancer Research.

Validation of a virtual reality-based robotic surgical skills curriculum.

PubMed

Connolly, Michael; Seligman, Johnathan; Kastenmeier, Andrew; Goldblatt, Matthew; Gould, Jon C

2014-05-01

The clinical application of robotic-assisted surgery (RAS) is rapidly increasing. The da Vinci Surgical System™ is currently the only commercially available RAS system. The skills necessary to perform robotic surgery are unique from those required for open and laparoscopic surgery. A validated laparoscopic surgical skills curriculum (fundamentals of laparoscopic surgery or FLS™) has transformed the way surgeons acquire laparoscopic skills. There is a need for a similar skills training and assessment tool specific for robotic surgery. Based on previously published data and expert opinion, we developed a robotic skills curriculum. We sought to evaluate this curriculum for evidence of construct validity (ability to discriminate between users of different skill levels). Four experienced surgeons (>20 RAS) and 20 novice surgeons (first-year medical students with no surgical or RAS experience) were evaluated. The curriculum comprised five tasks utilizing the da Vinci™ Skills Simulator (Pick and Place, Camera Targeting 2, Peg Board 2, Matchboard 2, and Suture Sponge 3). After an orientation to the robot and a period of acclimation in the simulator, all subjects completed three consecutive repetitions of each task. Computer-derived performance metrics included time, economy of motion, master work space, instrument collisions, excessive force, distance of instruments out of view, drops, missed targets, and overall scores (a composite of all metrics). Experienced surgeons significantly outperformed novice surgeons in most metrics. Statistically significant differences were detected for each task in regards to mean overall scores and mean time (seconds) to completion. The curriculum we propose is a valid method of assessing and distinguishing robotic surgical skill levels on the da Vinci Si™ Surgical System. Further study is needed to establish proficiency levels and to demonstrate that training on the simulator with the proposed curriculum leads to improved robotic surgical performance in the operating room.

Implementing the Science Assessment Standards: Developing and validating a set of laboratory assessment tasks in high school biology

NASA Astrophysics Data System (ADS)

Saha, Gouranga Chandra

Very often a number of factors, especially time, space and money, deter many science educators from using inquiry-based, hands-on, laboratory practical tasks as alternative assessment instruments in science. A shortage of valid inquiry-based laboratory tasks for high school biology has been cited. Driven by this need, this study addressed the following three research questions: (1) How can laboratory-based performance tasks be designed and developed that are doable by students for whom they are designed/written? (2) Do student responses to the laboratory-based performance tasks validly represent at least some of the intended process skills that new biology learning goals want students to acquire? (3) Are the laboratory-based performance tasks psychometrically consistent as individual tasks and as a set? To answer these questions, three tasks were used from the six biology tasks initially designed and developed by an iterative process of trial testing. Analyses of data from 224 students showed that performance-based laboratory tasks that are doable by all students require careful and iterative process of development. Although the students demonstrated more skill in performing than planning and reasoning, their performances at the item level were very poor for some items. Possible reasons for the poor performances have been discussed and suggestions on how to remediate the deficiencies have been made. Empirical evidences for validity and reliability of the instrument have been presented both from the classical and the modern validity criteria point of view. Limitations of the study have been identified. Finally implications of the study and directions for further research have been discussed.

Performance Tested Method multiple laboratory validation study of ELISA-based assays for the detection of peanuts in food.

PubMed

Park, Douglas L; Coates, Scott; Brewer, Vickery A; Garber, Eric A E; Abouzied, Mohamed; Johnson, Kurt; Ritter, Bruce; McKenzie, Deborah

2005-01-01

Performance Tested Method multiple laboratory validations for the detection of peanut protein in 4 different food matrixes were conducted under the auspices of the AOAC Research Institute. In this blind study, 3 commercially available ELISA test kits were validated: Neogen Veratox for Peanut, R-Biopharm RIDASCREEN FAST Peanut, and Tepnel BioKits for Peanut Assay. The food matrixes used were breakfast cereal, cookies, ice cream, and milk chocolate spiked at 0 and 5 ppm peanut. Analyses of the samples were conducted by laboratories representing industry and international and U.S governmental agencies. All 3 commercial test kits successfully identified spiked and peanut-free samples. The validation study required 60 analyses on test samples at the target level 5 microg peanut/g food and 60 analyses at a peanut-free level, which was designed to ensure that the lower 95% confidence limit for the sensitivity and specificity would not be <90%. The probability that a test sample contains an allergen given a prevalence rate of 5% and a positive test result using a single test kit analysis with 95% sensitivity and 95% specificity, which was demonstrated for these test kits, would be 50%. When 2 test kits are run simultaneously on all samples, the probability becomes 95%. It is therefore recommended that all field samples be analyzed with at least 2 of the validated kits.

Evaluation of the performance of a micromethod for measuring urinary iodine by using six sigma quality metrics.

PubMed

Hussain, Husniza; Khalid, Norhayati Mustafa; Selamat, Rusidah; Wan Nazaimoon, Wan Mohamud

2013-09-01

The urinary iodine micromethod (UIMM) is a modification of the conventional method and its performance needs evaluation. UIMM performance was evaluated using the method validation and 2008 Iodine Deficiency Disorders survey data obtained from four urinary iodine (UI) laboratories. Method acceptability tests and Sigma quality metrics were determined using total allowable errors (TEas) set by two external quality assurance (EQA) providers. UIMM obeyed various method acceptability test criteria with some discrepancies at low concentrations. Method validation data calculated against the UI Quality Program (TUIQP) TEas showed that the Sigma metrics were at 2.75, 1.80, and 3.80 for 51±15.50 µg/L, 108±32.40 µg/L, and 149±38.60 µg/L UI, respectively. External quality control (EQC) data showed that the performance of the laboratories was within Sigma metrics of 0.85-1.12, 1.57-4.36, and 1.46-4.98 at 46.91±7.05 µg/L, 135.14±13.53 µg/L, and 238.58±17.90 µg/L, respectively. No laboratory showed a calculated total error (TEcalc)

Development of a proficiency-based virtual reality simulation training curriculum for laparoscopic appendicectomy.

PubMed

Sirimanna, Pramudith; Gladman, Marc A

2017-10-01

Proficiency-based virtual reality (VR) training curricula improve intraoperative performance, but have not been developed for laparoscopic appendicectomy (LA). This study aimed to develop an evidence-based training curriculum for LA. A total of 10 experienced (>50 LAs), eight intermediate (10-30 LAs) and 20 inexperienced (<10 LAs) operators performed guided and unguided LA tasks on a high-fidelity VR simulator using internationally relevant techniques. The ability to differentiate levels of experience (construct validity) was measured using simulator-derived metrics. Learning curves were analysed. Proficiency benchmarks were defined by the performance of the experienced group. Intermediate and experienced participants completed a questionnaire to evaluate the realism (face validity) and relevance (content validity). Of 18 surgeons, 16 (89%) considered the VR model to be visually realistic and 17 (95%) believed that it was representative of actual practice. All 'guided' modules demonstrated construct validity (P < 0.05), with learning curves that plateaued between sessions 6 and 9 (P < 0.01). When comparing inexperienced to intermediates to experienced, the 'unguided' LA module demonstrated construct validity for economy of motion (5.00 versus 7.17 versus 7.84, respectively; P < 0.01) and task time (864.5 s versus 477.2 s versus 352.1 s, respectively, P < 0.01). Construct validity was also confirmed for number of movements, path length and idle time. Validated modules were used for curriculum construction, with proficiency benchmarks used as performance goals. A VR LA model was realistic and representative of actual practice and was validated as a training and assessment tool. Consequently, the first evidence-based internationally applicable training curriculum for LA was constructed, which facilitates skill acquisition to proficiency. © 2017 Royal Australasian College of Surgeons.

Validation of a novel basic virtual reality simulator, the LAP-X, for training basic laparoscopic skills.

PubMed

Kawaguchi, Koji; Egi, Hiroyuki; Hattori, Minoru; Sawada, Hiroyuki; Suzuki, Takahisa; Ohdan, Hideki

2014-10-01

Virtual reality surgical simulators are becoming popular as a means of providing trainees with an opportunity to practice laparoscopic skills. The Lap-X (Epona Medical, Rotterdam, the Netherlands) is a novel VR simulator for training basic skills in laparoscopic surgery. The objective of this study was to validate the LAP-X laparoscopic virtual reality simulator by assessing the face and construct validity in order to determine whether the simulator is adequate for basic skills training. The face and content validity were evaluated using a structured questionnaire. To assess the construct validity, the participants, nine expert surgeons (median age: 40 (32-45)) (>100 laparoscopic procedures) and 11 novices performed three basic laparoscopic tasks using the Lap-X. The participants reported a high level of content validity. No significant differences were found between the expert surgeons and the novices (Ps > 0.246). The performance of the expert surgeons on the three tasks was significantly better than that of the novices in all parameters (Ps < 0.05). This study demonstrated the face, content and construct validity of the Lap-X. The Lap-X holds real potential as a home and hospital training device.

Validity of deterministic record linkage using multiple indirect personal identifiers: linking a large registry to claims data.

PubMed

Setoguchi, Soko; Zhu, Ying; Jalbert, Jessica J; Williams, Lauren A; Chen, Chih-Ying

2014-05-01

Linking patient registries with administrative databases can enhance the utility of the databases for epidemiological and comparative effectiveness research. However, registries often lack direct personal identifiers, and the validity of record linkage using multiple indirect personal identifiers is not well understood. Using a large contemporary national cardiovascular device registry and 100% Medicare inpatient data, we linked hospitalization-level records. The main outcomes were the validity measures of several deterministic linkage rules using multiple indirect personal identifiers compared with rules using both direct and indirect personal identifiers. Linkage rules using 2 or 3 indirect, patient-level identifiers (ie, date of birth, sex, admission date) and hospital ID produced linkages with sensitivity of 95% and specificity of 98% compared with a gold standard linkage rule using a combination of both direct and indirect identifiers. Ours is the first large-scale study to validate the performance of deterministic linkage rules without direct personal identifiers. When linking hospitalization-level records in the absence of direct personal identifiers, provider information is necessary for successful linkage. © 2014 American Heart Association, Inc.

Factor- and Item-Level Analyses of the 38-Item Activities Scale for Kids-Performance

ERIC Educational Resources Information Center

Bagley, Anita M.; Gorton, George E.; Bjornson, Kristie; Bevans, Katherine; Stout, Jean L.; Narayanan, Unni; Tucker, Carole A.

2011-01-01

Aim: Children and adolescents highly value their ability to participate in relevant daily life and recreational activities. The Activities Scale for Kids-performance (ASKp) instrument measures the frequency of performance of 30 common childhood activities, and has been shown to be valid and reliable. A revised and expanded 38-item ASKp (ASKp38)…

The objective structured clinical examination revisited for postgraduate trainees in general practice.

PubMed

Schoenmakers, Birgitte; Wens, Johan

2014-03-04

To investigate if the psychometric qualities of an OSCE consisting of more complex simulated patient encounters remain valid and reliable in the assessment of postgraduate trainees in general practice. In this intervention study without control group, the traditional OSCE was formally replaced by the new, complex version. The study population was composed by all postgraduate trainees (second and third phase) in general practice during the ongoing academic year. Data were handled and collected as part of the formal assessment program. Univariate analyses, the variance of scores and multivariate analyses were performed to assess the test qualities. A total of 340 students participated. Average final scores were slightly higher for third-phase students (t-test, p =0.05). Overall test scores were equally distributed on station level, circuit level and phase level. A multiple regression analysis revealed that test scores were dependent on the stations and circuits, but not on the master phase. In a changing learning environment, assessment and evaluation strategies require reorientation. The reliability and validity of the OSCE remain subject to discussion. In particular, when it comes to content and design, the traditional OSCE might underestimate the performance level of postgraduate trainees in general practice. A reshaping of this OSCE to a more sophisticated design with more complex patient encounters appears to restore the validity of the test results.

vvtools v. 1.0

DOE Office of Scientific and Technical Information (OSTI.GOV)

Drake, Richard R.

Vvtools is a suite of testing tools, with a focus on reproducible verification and validation. They are written in pure Python, and contain a test harness and an automated process management tool. Users of vvtools can develop suites of verification and validation tests and run them on small to large high performance computing resources in an automated and reproducible way. The test harness enables complex processes to be performed in each test and even supports a one-level parent/child dependency between tests. It includes a built in capability to manage workloads requiring multiple processors and platforms that use batch queueing systems.

[Reliability and validity of the Chinese version on Alcohol Use Disorders Identification Test].

PubMed

Zhang, C; Yang, G P; Li, Z; Li, X N; Li, Y; Hu, J; Zhang, F Y; Zhang, X J

2017-08-10

Objective: To assess the reliability and validity of the Chinese version on Alcohol Use Disorders Identification Test (AUDIT) among medical students in China and to provide correct way of application on the recommended scales. Methods: An E-questionnaire was developed and sent to medical students in five different colleges. Students were all active volunteers to accept the testings. Cronbach's α and split-half reliability were calculated to evaluate the reliability of AUDIT while content, contract, discriminant and convergent validity were performed to measure the validity of the scales. Results: The overall Cronbach's α of AUDIT was 0.782 and the split-half reliability was 0.711. Data showed that the domain Cronbach's α and split-half reliability were 0.796 and 0.794 for hazardous alcohol use, 0.561 and 0.623 for dependence symptoms, and 0.647 and 0.640 for harmful alcohol use. Results also showed that the content validity index on the levels of items I-CVI) were from 0.83 to 1.00, the content validity index of scale level (S-CVI/UA) was 0.90, content validity index of average scale level (S-CVI/Ave) was 0.99 and the content validity ratios (CVR) were from 0.80 to 1.00. The simplified version of AUDIT supported a presupposed three-factor structure which could explain 61.175% of the total variance revealed through exploratory factor analysis. AUDIT semed to have good convergent and discriminant validity, with the success rate of calibration experiment as 100%. Conclusion: AUDIT showed good reliability and validity among medical students in China thus worth for promotion on its use.

Assessing the validity and reliability of the Pool Activity Level (PAL) Checklist for use with older people with dementia.

PubMed

Wenborn, Jennifer; Challis, David; Pool, Jackie; Burgess, Jane; Elliott, Nicola; Orrell, Martin

2008-03-01

Activity is key to maintaining physical and mental health and well-being. However, as dementia affects the ability to engage in activity, care-givers can find it difficult to provide appropriate activities. The Pool Activity Level (PAL) Checklist guides the selection of appropriate, personally meaningful activities. The aim of this study was to assess the reliability and validity of the PAL Checklist when used with older people with dementia. A postal questionnaire sent to activity providers assessed content validity. Validity and reliability were measured in a sample of 60 older people with dementia. The questionnaire response rate was 83% (102/122). Most respondents felt no important items were missing. Seven of the nine activities were ranked as 'very important' or 'essential' by at least 77% of the sample, indicating very good content validity. Correlation with measures of cognition, severity of dementia and activity performance demonstrated strong concurrent validity. Inter-item correlation indicated strong construct validity. Cronbach's alpha coefficient measured internal consistency as excellent (0.95). All items achieved acceptable test-retest reliability, and the majority demonstrated acceptable inter-rater reliability. We conclude that the PAL Checklist demonstrates adequate validity and reliability when used with older people with dementia and appears a useful tool for a variety of care settings.

Multilevel Safety Climate and Safety Performance in the Construction Industry: Development and Validation of a Top-Down Mechanism

PubMed Central

Gao, Ran; Chan, Albert P.C.; Utama, Wahyudi P.; Zahoor, Hafiz

2016-01-01

The character of construction projects exposes front-line workers to dangers and accidents. Safety climate has been confirmed to be a predictor of safety performance in the construction industry. This study aims to explore the underlying mechanisms of the relationship between multilevel safety climate and safety performance. An integrated model was developed to study how particular safety climate factors of one level affect those of other levels, and then affect safety performance from the top down. A questionnaire survey was administered on six construction sites in Vietnam. A total of 1030 valid questionnaires were collected from this survey. Approximately half of the data were used to conduct exploratory factor analysis (EFA) and the remaining data were submitted to structural equation modeling (SEM). Top management commitment (TMC) and supervisors’ expectation (SE) were identified as factors to represent organizational safety climate (OSC) and supervisor safety climate (SSC), respectively, and coworkers’ caring and communication (CCC) and coworkers’ role models (CRM) were identified as factors to denote coworker safety climate (CSC). SEM results show that OSC factor is positively related to SSC factor and CSC factors significantly. SSC factor could partially mediate the relationship between OSC factor and CSC factors, as well as the relationship between OSC factor and safety performance. CSC factors partially mediate the relationship between OSC factor and safety performance, and the relationship between SSC factor and safety performance. The findings imply that a positive safety culture should be established both at the organizational level and the group level. Efforts from all top management, supervisors, and coworkers should be provided to improve safety performance in the construction industry. PMID:27834823

Multilevel Safety Climate and Safety Performance in the Construction Industry: Development and Validation of a Top-Down Mechanism.

PubMed

Gao, Ran; Chan, Albert P C; Utama, Wahyudi P; Zahoor, Hafiz

2016-11-08

The character of construction projects exposes front-line workers to dangers and accidents. Safety climate has been confirmed to be a predictor of safety performance in the construction industry. This study aims to explore the underlying mechanisms of the relationship between multilevel safety climate and safety performance. An integrated model was developed to study how particular safety climate factors of one level affect those of other levels, and then affect safety performance from the top down. A questionnaire survey was administered on six construction sites in Vietnam. A total of 1030 valid questionnaires were collected from this survey. Approximately half of the data were used to conduct exploratory factor analysis (EFA) and the remaining data were submitted to structural equation modeling (SEM). Top management commitment (TMC) and supervisors' expectation (SE) were identified as factors to represent organizational safety climate (OSC) and supervisor safety climate (SSC), respectively, and coworkers' caring and communication (CCC) and coworkers' role models (CRM) were identified as factors to denote coworker safety climate (CSC). SEM results show that OSC factor is positively related to SSC factor and CSC factors significantly. SSC factor could partially mediate the relationship between OSC factor and CSC factors, as well as the relationship between OSC factor and safety performance. CSC factors partially mediate the relationship between OSC factor and safety performance, and the relationship between SSC factor and safety performance. The findings imply that a positive safety culture should be established both at the organizational level and the group level. Efforts from all top management, supervisors, and coworkers should be provided to improve safety performance in the construction industry.

Development, optimization and validation of gas chromatographic fingerprinting of Brazilian commercial diesel fuel for quality control.

PubMed

dos Santos, Bruno César Diniz Brito; Flumignan, Danilo Luiz; de Oliveira, José Eduardo

2012-10-01

A three-step development, optimization and validation strategy is described for gas chromatography (GC) fingerprints of Brazilian commercial diesel fuel. A suitable GC-flame ionization detection (FID) system was selected to assay a complex matrix such as diesel. The next step was to improve acceptable chromatographic resolution with reduced analysis time, which is recommended for routine applications. Full three-level factorial designs were performed to improve flow rate, oven ramps, injection volume and split ratio in the GC system. Finally, several validation parameters were performed. The GC fingerprinting can be coupled with pattern recognition and multivariate regressions analyses to determine fuel quality and fuel physicochemical parameters. This strategy can also be applied to develop fingerprints for quality control of other fuel types.

The UKCAT-12 study: educational attainment, aptitude test performance, demographic and socio-economic contextual factors as predictors of first year outcome in a cross-sectional collaborative study of 12 UK medical schools.

PubMed

McManus, I C; Dewberry, Chris; Nicholson, Sandra; Dowell, Jonathan S

2013-11-14

Most UK medical schools use aptitude tests during student selection, but large-scale studies of predictive validity are rare. This study assesses the United Kingdom Clinical Aptitude Test (UKCAT), and its four sub-scales, along with measures of educational attainment, individual and contextual socio-economic background factors, as predictors of performance in the first year of medical school training. A prospective study of 4,811 students in 12 UK medical schools taking the UKCAT from 2006 to 2008 as a part of the medical school application, for whom first year medical school examination results were available in 2008 to 2010. UKCAT scores and educational attainment measures (General Certificate of Education (GCE): A-levels, and so on; or Scottish Qualifications Authority (SQA): Scottish Highers, and so on) were significant predictors of outcome. UKCAT predicted outcome better in female students than male students, and better in mature than non-mature students. Incremental validity of UKCAT taking educational attainment into account was significant, but small. Medical school performance was also affected by sex (male students performing less well), ethnicity (non-White students performing less well), and a contextual measure of secondary schooling, students from secondary schools with greater average attainment at A-level (irrespective of public or private sector) performing less well. Multilevel modeling showed no differences between medical schools in predictive ability of the various measures. UKCAT sub-scales predicted similarly, except that Verbal Reasoning correlated positively with performance on Theory examinations, but negatively with Skills assessments. This collaborative study in 12 medical schools shows the power of large-scale studies of medical education for answering previously unanswerable but important questions about medical student selection, education and training. UKCAT has predictive validity as a predictor of medical school outcome, particularly in mature applicants to medical school. UKCAT offers small but significant incremental validity which is operationally valuable where medical schools are making selection decisions based on incomplete measures of educational attainment. The study confirms the validity of using all the existing measures of educational attainment in full at the time of selection decision-making. Contextual measures provide little additional predictive value, except that students from high attaining secondary schools perform less well, an effect previously shown for UK universities in general.

The UKCAT-12 study: educational attainment, aptitude test performance, demographic and socio-economic contextual factors as predictors of first year outcome in a cross-sectional collaborative study of 12 UK medical schools

PubMed Central

2013-01-01

Background Most UK medical schools use aptitude tests during student selection, but large-scale studies of predictive validity are rare. This study assesses the United Kingdom Clinical Aptitude Test (UKCAT), and its four sub-scales, along with measures of educational attainment, individual and contextual socio-economic background factors, as predictors of performance in the first year of medical school training. Methods A prospective study of 4,811 students in 12 UK medical schools taking the UKCAT from 2006 to 2008 as a part of the medical school application, for whom first year medical school examination results were available in 2008 to 2010. Results UKCAT scores and educational attainment measures (General Certificate of Education (GCE): A-levels, and so on; or Scottish Qualifications Authority (SQA): Scottish Highers, and so on) were significant predictors of outcome. UKCAT predicted outcome better in female students than male students, and better in mature than non-mature students. Incremental validity of UKCAT taking educational attainment into account was significant, but small. Medical school performance was also affected by sex (male students performing less well), ethnicity (non-White students performing less well), and a contextual measure of secondary schooling, students from secondary schools with greater average attainment at A-level (irrespective of public or private sector) performing less well. Multilevel modeling showed no differences between medical schools in predictive ability of the various measures. UKCAT sub-scales predicted similarly, except that Verbal Reasoning correlated positively with performance on Theory examinations, but negatively with Skills assessments. Conclusions This collaborative study in 12 medical schools shows the power of large-scale studies of medical education for answering previously unanswerable but important questions about medical student selection, education and training. UKCAT has predictive validity as a predictor of medical school outcome, particularly in mature applicants to medical school. UKCAT offers small but significant incremental validity which is operationally valuable where medical schools are making selection decisions based on incomplete measures of educational attainment. The study confirms the validity of using all the existing measures of educational attainment in full at the time of selection decision-making. Contextual measures provide little additional predictive value, except that students from high attaining secondary schools perform less well, an effect previously shown for UK universities in general. PMID:24229380

Validity and reproducibility of a food frequency questionnaire for assessment of fruit and vegetable intake in Iranian adults*

PubMed Central

Mohammadifard, Noushin; Omidvar, Nasrin; Houshiarrad, Anahita; Neyestani, Tirang; Naderi, Gholam-Ali; Soleymani, Bahram

2011-01-01

BACKGROUND: This study's aim was to design and validate a semi-quantitative food frequency questionnaire (FFQ) for assessment of fruits and vegetables (FV) consumption in adults of Isfahan by comparing the FFQ with dietary reference method and blood plasma levels of beta-carotene, vitamin C, and retinol. METHODS: This validation study was performed on 123 healthy adults of Isfahan. FV intake was assessed using a 110-item FFQ. Data collection was performed during two different time periods to control for seasonal effects, fall/winter (cold season) and spring/summer (warm season). In each phase a FFQ and 1 day recall, and 2 days of food records as the dietary reference method were completed and plasma vitamin C, beta-carotene and retinol were measured. Data was analyzed by Pearson or Spearman and intraclass correlations. RESULTS: Serum Lipids, sex, age, body mass index (BMI) and educational level adjusted Pearson correlation coefficient of FV with plasma vitamin C, beta-carotene and retinol were 0.55, 0.47 and 0.28 in the cold season (p < 0.05) and 0.52, 0.45 and 0.35 in the warm season (p < 0.001), respectively. Energy and fat intake, sex, age, BMI and educational level adjusted Pearson correlation coefficient for FV with dietary reference method in the cold and warm seasons were 0.62 and 0.60, respectively (p < 0.001). Intraclass correlation for reproducibility of FFQ in FV was 0.65 (p<0.001). CONCLUSIONS: The designed FFQ had a good criterion validity and reproducibility for assessment of FV intake. Thus, it can serve as a valid tool in epidemiological studies to assess fruit and vegetable intake. PMID:22973322

Validating a work group climate assessment tool for improving the performance of public health organizations

PubMed Central

Perry, Cary; LeMay, Nancy; Rodway, Greg; Tracy, Allison; Galer, Joan

2005-01-01

Background This article describes the validation of an instrument to measure work group climate in public health organizations in developing countries. The instrument, the Work Group Climate Assessment Tool (WCA), was applied in Brazil, Mozambique, and Guinea to assess the intermediate outcomes of a program to develop leadership for performance improvement. Data were collected from 305 individuals in 42 work groups, who completed a self-administered questionnaire. Methods The WCA was initially validated using Cronbach's alpha reliability coefficient and exploratory factor analysis. This article presents the results of a second validation study to refine the initial analyses to account for nested data, to provide item-level psychometrics, and to establish construct validity. Analyses included eigenvalue decomposition analysis, confirmatory factor analysis, and validity and reliability analyses. Results This study confirmed the validity and reliability of the WCA across work groups with different demographic characteristics (gender, education, management level, and geographical location). The study showed that there is agreement between the theoretical construct of work climate and the items in the WCA tool across different populations. The WCA captures a single perception of climate rather than individual sub-scales of clarity, support, and challenge. Conclusion The WCA is useful for comparing the climates of different work groups, tracking the changes in climate in a single work group over time, or examining differences among individuals' perceptions of their work group climate. Application of the WCA before and after a leadership development process can help work groups hold a discussion about current climate and select a target for improvement. The WCA provides work groups with a tool to take ownership of their own group climate through a process that is simple and objective and that protects individual confidentiality. PMID:16223447

Development of an Itemwise Efficiency Scoring Method: Concurrent, Convergent, Discriminant, and Neuroimaging-Based Predictive Validity Assessed in a Large Community Sample

PubMed Central

Moore, Tyler M.; Reise, Steven P.; Roalf, David R.; Satterthwaite, Theodore D.; Davatzikos, Christos; Bilker, Warren B.; Port, Allison M.; Jackson, Chad T.; Ruparel, Kosha; Savitt, Adam P.; Baron, Robert B.; Gur, Raquel E.; Gur, Ruben C.

2016-01-01

Traditional “paper-and-pencil” testing is imprecise in measuring speed and hence limited in assessing performance efficiency, but computerized testing permits precision in measuring itemwise response time. We present a method of scoring performance efficiency (combining information from accuracy and speed) at the item level. Using a community sample of 9,498 youths age 8-21, we calculated item-level efficiency scores on four neurocognitive tests, and compared the concurrent, convergent, discriminant, and predictive validity of these scores to simple averaging of standardized speed and accuracy-summed scores. Concurrent validity was measured by the scores' abilities to distinguish men from women and their correlations with age; convergent and discriminant validity were measured by correlations with other scores inside and outside of their neurocognitive domains; predictive validity was measured by correlations with brain volume in regions associated with the specific neurocognitive abilities. Results provide support for the ability of itemwise efficiency scoring to detect signals as strong as those detected by standard efficiency scoring methods. We find no evidence of superior validity of the itemwise scores over traditional scores, but point out several advantages of the former. The itemwise efficiency scoring method shows promise as an alternative to standard efficiency scoring methods, with overall moderate support from tests of four different types of validity. This method allows the use of existing item analysis methods and provides the convenient ability to adjust the overall emphasis of accuracy versus speed in the efficiency score, thus adjusting the scoring to the real-world demands the test is aiming to fulfill. PMID:26866796

The Role of Structural Models in the Solar Sail Flight Validation Process

NASA Technical Reports Server (NTRS)

Johnston, John D.

2004-01-01

NASA is currently soliciting proposals via the New Millennium Program ST-9 opportunity for a potential Solar Sail Flight Validation (SSFV) experiment to develop and operate in space a deployable solar sail that can be steered and provides measurable acceleration. The approach planned for this experiment is to test and validate models and processes for solar sail design, fabrication, deployment, and flight. These models and processes would then be used to design, fabricate, and operate scaleable solar sails for future space science missions. There are six validation objectives planned for the ST9 SSFV experiment: 1) Validate solar sail design tools and fabrication methods; 2) Validate controlled deployment; 3) Validate in space structural characteristics (focus of poster); 4) Validate solar sail attitude control; 5) Validate solar sail thrust performance; 6) Characterize the sail's electromagnetic interaction with the space environment. This poster presents a top-level assessment of the role of structural models in the validation process for in-space structural characteristics.

Assessing teamwork performance in obstetrics: A systematic search and review of validated tools.

PubMed

Fransen, Annemarie F; de Boer, Liza; Kienhorst, Dieneke; Truijens, Sophie E; van Runnard Heimel, Pieter J; Oei, S Guid

2017-09-01

Teamwork performance is an essential component for the clinical efficiency of multi-professional teams in obstetric care. As patient safety is related to teamwork performance, it has become an important learning goal in simulation-based education. In order to improve teamwork performance, reliable assessment tools are required. These can be used to provide feedback during training courses, or to compare learning effects between different types of training courses. The aim of the current study is to (1) identify the available assessment tools to evaluate obstetric teamwork performance in a simulated environment, and (2) evaluate their psychometric properties in order to identify the most valuable tool(s) to use. We performed a systematic search in PubMed, MEDLINE, and EMBASE to identify articles describing assessment tools for the evaluation of obstetric teamwork performance in a simulated environment. In order to evaluate the quality of the identified assessment tools the standards and grading rules have been applied as recommended by the Accreditation Council for Graduate Medical Education (ACGME) Committee on Educational Outcomes. The included studies were also assessed according to the Oxford Centre for Evidence Based Medicine (OCEBM) levels of evidence. This search resulted in the inclusion of five articles describing the following six tools: Clinical Teamwork Scale, Human Factors Rating Scale, Global Rating Scale, Assessment of Obstetric Team Performance, Global Assessment of Obstetric Team Performance, and the Teamwork Measurement Tool. Based on the ACGME guidelines we assigned a Class 3, level C of evidence, to all tools. Regarding the OCEBM levels of evidence, a level 3b was assigned to two studies and a level 4 to four studies. The Clinical Teamwork Scale demonstrated the most comprehensive validation, and the Teamwork Measurement Tool demonstrated promising results, however it is recommended to further investigate its reliability. Copyright © 2017. Published by Elsevier B.V.

Gardener and Landscape Worker. Student Material. Competency Based Education Curriculum.

ERIC Educational Resources Information Center

Long, Diana

This secondary-level, competency-based curriculum contains modules for Gardener and Landscape Worker. A companion teacher's guide is available separately--see note. Each module contains a number of West Virginia-validated Gardener and Landscape Worker tasks/competencies with a performance guide listing the steps needed to perform each task,…

Nursery and Greenhouse Worker. Student Material. Competency Based Education Curriculum.

ERIC Educational Resources Information Center

Long, Diana

This secondary-level, competency-based curriculum contains 11 modules for Nursery and Greenhouse Worker. A companion teacher's guide is available separately--see note. Each module contains a number of West Virginia-validated Nursery and Greenhouse Worker tasks/competencies with a performance guide listing the steps needed to perform each task,…

Rater Expertise in a Second Language Speaking Assessment: The Influence of Training and Experience

ERIC Educational Resources Information Center

Davis, Lawrence Edward

2012-01-01

Speaking performance tests typically employ raters to produce scores; accordingly, variability in raters' scoring decisions has important consequences for test reliability and validity. One such source of variability is the rater's level of expertise in scoring. Therefore, it is important to understand how raters' performance is influenced by…

Family Living and Parenthood. Performance Objectives and Criterion-Referenced Test Items.

ERIC Educational Resources Information Center

Missouri Univ., Columbia. Instructional Materials Lab.

This guide was developed to assist home economics teachers in implementing the Missouri Vocational Instructional Management System into the home economics curriculum at the local level through a family living and parenthood semester course. The course contains a minimum of two performance objectives for each competency developed and validated by…

Normative data for the Rappel libre/Rappel indicé à 16 items (16-item Free and Cued Recall) in the elderly Quebec-French population.

PubMed

Dion, Mélissa; Potvin, Olivier; Belleville, Sylvie; Ferland, Guylaine; Renaud, Mélanie; Bherer, Louis; Joubert, Sven; Vallet, Guillaume T; Simard, Martine; Rouleau, Isabelle; Lecomte, Sarah; Macoir, Joël; Hudon, Carol

2015-01-01

Performance on verbal memory tests is generally associated with socio-demographic variables such as age, sex, and education level. Performance also varies between different cultural groups. The present study aimed to establish normative data for the Rappel libre/Rappel indicé à 16 items (16-item Free and Cued Recall; RL/RI-16), a French adaptation of the Free and Cued Selective Reminding Test (Buschke, 1984; Grober, Buschke, Crystal, Bang, & Dresner, 1988). The sample consisted of 566 healthy French-speaking older adults (50-88 years old) from the province of Quebec, Canada. Normative data for the RL/RI-16 were derived from 80% of the total sample (normative sample) and cross-validated using the remaining participants (20%; validation sample). The effects of participants' age, sex, and education level were assessed on different indices of memory performance. Results indicated that these variables were independently associated with performance. Normative data are presented as regression equations with standard deviations (symmetric distributions) and percentiles (asymmetric distributions).

Evidencing the association between swimming capacities and performance indicators in water polo: a multiple regression study.

PubMed

Kontic, Dean; Zenic, Natasa; Uljevic, Ognjen; Sekulic, Damir; Lesnik, Blaz

2017-06-01

Swimming capacities are hypothesized to be important determinants of water polo performance but there is an evident lack of studies examining different swimming capacities in relation to specific offensive and defensive performance variables in this sport. The aim of this study was to determine the relationship between five swimming capacities and six performance determinants in water polo. The sample comprised 79 high-level youth water polo players (all males, 17-18 years of age). The variables included six performance-related variables (agility in offence and defense, efficacy in offence and defense, polyvalence in offence and defense), and five swimming-capacity tests (water polo sprint test [15 m], swimming sprint test [25 m], short-distance [100 m], aerobic endurance [400 m] and an anaerobic lactate endurance test [4× 50 m]). First, multiple regressions were calculated for one-half of the sample of subjects which were then validated with the remaining half of the sample. The 25-m swim was not included in the regression analyses due to the multicollinearity with other predictors. The originally calculated regression models were validated for defensive agility (R=0.67 and R=0.55 for the original regression calculation and validation subsample, respectively) offensive agility (R=0.59 and R=0.61), and offensive efficacy (R=0.64 and R=0.58). Anaerobic lactate endurance is a significant predictor of offensive and defensive agility, while 15 m sprint significantly contributes to offensive efficacy. Swimming capacities are not found to be related to the polyvalence of the players. The most superior offensive performance can be expected from those players with a high level of anaerobic lactate endurance and advanced sprinting capacity, while anaerobic lactate endurance is recognized as most important quality in defensive duties. Future studies should observe players' polyvalence in relation to (theoretical) knowledge of technical and tactical tasks. Results reinforce the need for the cross-validation of the prediction-models in sport and exercise sciences.

Determination of Ochratoxin A in Rye and Rye-Based Products by Fluorescence Polarization Immunoassay

PubMed Central

Lippolis, Vincenzo; Porricelli, Anna C. R.; Cortese, Marina; Zanardi, Sandro; Pascale, Michelangelo

2017-01-01

A rapid fluorescence polarization immunoassay (FPIA) was optimized and validated for the determination of ochratoxin A (OTA) in rye and rye crispbread. Samples were extracted with a mixture of acetonitrile/water (60:40, v/v) and purified by SPE-aminopropyl column clean-up before performing the FPIA. Overall mean recoveries were 86 and 95% for spiked rye and rye crispbread with relative standard deviations lower than 6%. Limits of detection (LOD) of the optimized FPIA was 0.6 μg/kg for rye and rye crispbread, respectively. Good correlations (r > 0.977) were observed between OTA contents in contaminated samples obtained by FPIA and high-performance liquid chromatography (HPLC) with immunoaffinity cleanup used as reference method. Furthermore, single laboratory validation and small-scale collaborative trials were carried out for the determination of OTA in rye according to Regulation 519/2014/EU laying down procedures for the validation of screening methods. The precision profile of the method, cut-off level and rate of false suspect results confirm the satisfactory analytical performances of assay as a screening method. These findings show that the optimized FPIA is suitable for high-throughput screening, and permits reliable quantitative determination of OTA in rye and rye crispbread at levels that fall below the EU regulatory limits. PMID:28954398

Measuring Exposure to Direct-to-Consumer Advertising—A Validation Study in the Context of Cancer-Related Treatment Advertising

PubMed Central

Tan, Andy SL; Hornik, Robert C

2014-01-01

This research examines two recurrent conceptual issues of measuring media exposure in survey research in the context of cancer-related direct-to-consumer advertising (CR-DTCA)—the level of content specificity of survey items and the benefits of providing exemplars to aid recall. We evaluated three candidate measures of cancer patients’ self-reported exposure to CR-DTCA; these measures varied in content specificity and provision of ad exemplars. Using data from two distinct population-based surveys, we assessed the performance of each measure based on several reliability and validity criteria. Results across both surveys indicate that all three measures performed equally well in terms of internal consistency, convergent, nomological, and discriminant validity with a few minor differences between these measures. Increased content specificity or inclusion of ad exemplars did not result in better performance of the exposure measures. Participants were able to extrapolate from ad exemplars to report their exposure to broad categories of CR-DTCA. The briefest of the three measures posed the lowest level of survey costs among the three measures and was deployed successfully for both mailed and internet-based survey administration. We discussed future directions for application of these findings in DTCA research for other illness and for media exposure research more generally. PMID:24693332

Use of immersive virtual reality to assess episodic memory: A validation study in older adults.

PubMed

Corriveau Lecavalier, Nick; Ouellet, Émilie; Boller, Benjamin; Belleville, Sylvie

2018-05-29

Virtual reality (VR) allows for the creation of ecological environments that could be used for cognitive assessment and intervention. This study comprises two parts that describe and assess an immersive VR task, the Virtual Shop, which can be used to measure episodic memory. Part 1 addresses its applicability in healthy older adults by measuring presence, motivation, and cybersickness symptoms. Part 2 addresses its construct validity by investigating correlations between performance in the VR task and on a traditional experimental memory task, and by measuring whether the VR task is sensitive to age-related memory differences. Fifty-seven older and 20 younger adults were assessed in the Virtual Shop, in which they memorised and fetched 12 familiar items. Part 1 showed high levels of presence, higher levels of motivation for the VR than for the traditional task, and negligible cybersickness symptoms. Part 2 indicates that memory performance in the VR task is positively correlated with performance on a traditional memory task for both age groups, and age-related differences were found on the VR and traditional memory tasks. Thus, the use of VR is feasible in older adults and the Virtual Shop is a valid task to assess and train episodic memory in this population.

Development and validation of criterion-referenced clinically relevant fitness standards for maintaining physical independence in later years.

PubMed

Rikli, Roberta E; Jones, C Jessie

2013-04-01

To develop and validate criterion-referenced fitness standards for older adults that predict the level of capacity needed for maintaining physical independence into later life. The proposed standards were developed for use with a previously validated test battery for older adults-the Senior Fitness Test (Rikli, R. E., & Jones, C. J. (2001). Development and validation of a functional fitness test for community--residing older adults. Journal of Aging and Physical Activity, 6, 127-159; Rikli, R. E., & Jones, C. J. (1999a). Senior fitness test manual. Champaign, IL: Human Kinetics.). A criterion measure to assess physical independence was identified. Next, scores from a subset of 2,140 "moderate-functioning" older adults from a larger cross-sectional database, together with findings from longitudinal research on physical capacity and aging, were used as the basis for proposing fitness standards (performance cut points) associated with having the ability to function independently. Validity and reliability analyses were conducted to test the standards for their accuracy and consistency as predictors of physical independence. Performance standards are presented for men and women ages 60-94 indicating the level of fitness associated with remaining physically independent until late in life. Reliability and validity indicators for the standards ranged between .79 and .97. The proposed standards provide easy-to-use, previously unavailable methods for evaluating physical capacity in older adults relative to that associated with physical independence. Most importantly, the standards can be used in planning interventions that target specific areas of weakness, thus reducing risk for premature loss of mobility and independence.

Validating YouTube Factors Affecting Learning Performance

NASA Astrophysics Data System (ADS)

Pratama, Yoga; Hartanto, Rudy; Suning Kusumawardani, Sri

2018-03-01

YouTube is often used as a companion medium or a learning supplement. One of the educational places that often uses is Jogja Audio School (JAS) which focuses on music production education. Music production is a difficult material to learn, especially at the audio mastering. With tutorial contents from YouTube, students find it easier to learn and understand audio mastering and improved their learning performance. This study aims to validate the role of YouTube as a medium of learning in improving student’s learning performance by looking at the factors that affect student learning performance. The sample involves 100 respondents from JAS at audio mastering level. The results showed that student learning performance increases seen from factors that have a significant influence of motivation, instructional content, and YouTube usefulness. Overall findings suggest that YouTube has a important role to student learning performance in music production education and as an innovative and efficient learning medium.

Development of an objective assessment tool for total laparoscopic hysterectomy: A Delphi method among experts and evaluation on a virtual reality simulator.

PubMed

Knight, Sophie; Aggarwal, Rajesh; Agostini, Aubert; Loundou, Anderson; Berdah, Stéphane; Crochet, Patrice

2018-01-01

Total Laparoscopic hysterectomy (LH) requires an advanced level of operative skills and training. The aim of this study was to develop an objective scale specific for the assessment of technical skills for LH (H-OSATS) and to demonstrate feasibility of use and validity in a virtual reality setting. The scale was developed using a hierarchical task analysis and a panel of international experts. A Delphi method obtained consensus among experts on relevant steps that should be included into the H-OSATS scale for assessment of operative performances. Feasibility of use and validity of the scale were evaluated by reviewing video recordings of LH performed on a virtual reality laparoscopic simulator. Three groups of operators of different levels of experience were assessed in a Marseille teaching hospital (10 novices, 8 intermediates and 8 experienced surgeons). Correlations with scores obtained using a recognised generic global rating tool (OSATS) were calculated. A total of 76 discrete steps were identified by the hierarchical task analysis. 14 experts completed the two rounds of the Delphi questionnaire. 64 steps reached consensus and were integrated in the scale. During the validation process, median time to rate each video recording was 25 minutes. There was a significant difference between the novice, intermediate and experienced group for total H-OSATS scores (133, 155.9 and 178.25 respectively; p = 0.002). H-OSATS scale demonstrated high inter-rater reliability (intraclass correlation coefficient [ICC] = 0.930; p<0.001) and test retest reliability (ICC = 0.877; p<0.001). High correlations were found between total H-OSATS scores and OSATS scores (rho = 0.928; p<0.001). The H-OSATS scale displayed evidence of validity for assessment of technical performances for LH performed on a virtual reality simulator. The implementation of this scale is expected to facilitate deliberate practice. Next steps should focus on evaluating the validity of the scale in the operating room.

Mitigating Task Saturation in Critical Care Air Transport Team Red Flag Checklist

DTIC Science & Technology

2015-04-14

Cincinnati. Team and individual performances were scored using a validated assessment tool for NOTECHS. Salivary cortisol levels were measured at baseline...deployment experience (pɘ.04) continued to be significant. Salivary cortisol levels increased by 0.124μg/dL over baseline as the result of the...Figure Page 1 Salivary cortisol levels for all 48 participants in the simulations ........................................3

Validation environment for AIPS/ALS: Implementation and results

NASA Technical Reports Server (NTRS)

Segall, Zary; Siewiorek, Daniel; Caplan, Eddie; Chung, Alan; Czeck, Edward; Vrsalovic, Dalibor

1990-01-01

The work is presented which was performed in porting the Fault Injection-based Automated Testing (FIAT) and Programming and Instrumentation Environments (PIE) validation tools, to the Advanced Information Processing System (AIPS) in the context of the Ada Language System (ALS) application, as well as an initial fault free validation of the available AIPS system. The PIE components implemented on AIPS provide the monitoring mechanisms required for validation. These mechanisms represent a substantial portion of the FIAT system. Moreover, these are required for the implementation of the FIAT environment on AIPS. Using these components, an initial fault free validation of the AIPS system was performed. The implementation is described of the FIAT/PIE system, configured for fault free validation of the AIPS fault tolerant computer system. The PIE components were modified to support the Ada language. A special purpose AIPS/Ada runtime monitoring and data collection was implemented. A number of initial Ada programs running on the PIE/AIPS system were implemented. The instrumentation of the Ada programs was accomplished automatically inside the PIE programming environment. PIE's on-line graphical views show vividly and accurately the performance characteristics of Ada programs, AIPS kernel and the application's interaction with the AIPS kernel. The data collection mechanisms were written in a high level language, Ada, and provide a high degree of flexibility for implementation under various system conditions.

How Many Batches Are Needed for Process Validation under the New FDA Guidance?

PubMed

Yang, Harry

2013-01-01

The newly updated FDA Guidance for Industry on Process Validation: General Principles and Practices ushers in a life cycle approach to process validation. While the guidance no longer considers the use of traditional three-batch validation appropriate, it does not prescribe the number of validation batches for a prospective validation protocol, nor does it provide specific methods to determine it. This potentially could leave manufacturers in a quandary. In this paper, I develop a Bayesian method to address the issue. By combining process knowledge gained from Stage 1 Process Design (PD) with expected outcomes of Stage 2 Process Performance Qualification (PPQ), the number of validation batches for PPQ is determined to provide a high level of assurance that the process will consistently produce future batches meeting quality standards. Several examples based on simulated data are presented to illustrate the use of the Bayesian method in helping manufacturers make risk-based decisions for Stage 2 PPQ, and they highlight the advantages of the method over traditional Frequentist approaches. The discussions in the paper lend support for a life cycle and risk-based approach to process validation recommended in the new FDA guidance. The newly updated FDA Guidance for Industry on Process Validation: General Principles and Practices ushers in a life cycle approach to process validation. While the guidance no longer considers the use of traditional three-batch validation appropriate, it does not prescribe the number of validation batches for a prospective validation protocol, nor does it provide specific methods to determine it. This potentially could leave manufacturers in a quandary. In this paper, I develop a Bayesian method to address the issue. By combining process knowledge gained from Stage 1 Process Design (PD) with expected outcomes of Stage 2 Process Performance Qualification (PPQ), the number of validation batches for PPQ is determined to provide a high level of assurance that the process will consistently produce future batches meeting quality standards. Several examples based on simulated data are presented to illustrate the use of the Bayesian method in helping manufacturers make risk-based decisions for Stage 2 PPQ, and THEY highlight the advantages of the method over traditional Frequentist approaches. The discussions in the paper lend support for a life cycle and risk-based approach to process validation recommended in the new FDA guidance.

Development and validation of a pediatric sports activity rating scale: the Hospital for Special Surgery Pediatric Functional Activity Brief Scale (HSS Pedi-FABS).

PubMed

Fabricant, Peter D; Robles, Alex; Downey-Zayas, Timothy; Do, Huong T; Marx, Robert G; Widmann, Roger F; Green, Daniel W

2013-10-01

Having simple and reliable validated outcome measures is vital to conducting high-quality outcomes research in the field of orthopaedic surgery. Activity level is a key prognostic variable for patients with sports injuries. There is a paucity of such activity scales for children and adolescents who are otherwise healthy and athletically active. In addition to frequency and intensity of athletic activity, level of play and coach/trainer supervision are important variables unique to children and adolescents that are not captured in available adult scoring systems. To create and validate a concise and comprehensive activity rating scale for athletically active children and adolescents 10 to 18 years of age. Cohort study (diagnosis); Level of evidence, 2. Item generation was performed with a panel of orthopaedic surgeons and adolescent athletes. Item reduction, pilot testing and scale refinement resulted in a final 8-item instrument, the Hospital for Special Surgery Pediatric Functional Activity Brief Scale (HSS Pedi-FABS). Existing methods were used to determine reliability and validation. The Flesch-Kincaid score was calculated at a 6.6th-grade reading level (approximately 13 years old); therefore, although all subjects provided their own answers, parents were allowed to assist children younger than 13 years with reading the questionnaire. Scale reliability was excellent (test-retest reliability, intraclass correlation coefficient = 0.91; internal consistency, Cronbach alpha = .914), and there were no floor or ceiling effects. There was also robust construct validity: Convergent validity testing revealed positive correlations between the HSS Pedi-FABS and level of competition in athletic activity, number of reported hours of athletic activity per week, and existing comparable adult and pediatric scales. Discriminant validity was shown with age, body mass index, and type of sport as measured by the Daniel scale. The 8-item HSS Pedi-FABS can be used to reliably and accurately evaluate activity level as a prognostic variable for clinical research studies. It is a simple, reliable, and valid metric to assess activity in children and adolescents 10 to 18 years of age. This instrument will lead to better evaluation of posttreatment outcomes and patient-reported activity for child and adolescent athletes.

Simulation-based assessment to identify critical gaps in safe anesthesia resident performance.

PubMed

Blum, Richard H; Boulet, John R; Cooper, Jeffrey B; Muret-Wagstaff, Sharon L

2014-01-01

Valid methods are needed to identify anesthesia resident performance gaps early in training. However, many assessment tools in medicine have not been properly validated. The authors designed and tested use of a behaviorally anchored scale, as part of a multiscenario simulation-based assessment system, to identify high- and low-performing residents with regard to domains of greatest concern to expert anesthesiology faculty. An expert faculty panel derived five key behavioral domains of interest by using a Delphi process (1) Synthesizes information to formulate a clear anesthetic plan; (2) Implements a plan based on changing conditions; (3) Demonstrates effective interpersonal and communication skills with patients and staff; (4) Identifies ways to improve performance; and (5) Recognizes own limits. Seven simulation scenarios spanning pre-to-postoperative encounters were used to assess performances of 22 first-year residents and 8 fellows from two institutions. Two of 10 trained faculty raters blinded to trainee program and training level scored each performance independently by using a behaviorally anchored rating scale. Residents, fellows, facilitators, and raters completed surveys. Evidence supporting the reliability and validity of the assessment scores was procured, including a high generalizability coefficient (ρ = 0.81) and expected performance differences between first-year resident and fellow participants. A majority of trainees, facilitators, and raters judged the assessment to be useful, realistic, and representative of critical skills required for safe practice. The study provides initial evidence to support the validity of a simulation-based performance assessment system for identifying critical gaps in safe anesthesia resident performance early in training.

A whole blood gene expression-based signature for smoking status

PubMed Central

2012-01-01

Background Smoking is the leading cause of preventable death worldwide and has been shown to increase the risk of multiple diseases including coronary artery disease (CAD). We sought to identify genes whose levels of expression in whole blood correlate with self-reported smoking status. Methods Microarrays were used to identify gene expression changes in whole blood which correlated with self-reported smoking status; a set of significant genes from the microarray analysis were validated by qRT-PCR in an independent set of subjects. Stepwise forward logistic regression was performed using the qRT-PCR data to create a predictive model whose performance was validated in an independent set of subjects and compared to cotinine, a nicotine metabolite. Results Microarray analysis of whole blood RNA from 209 PREDICT subjects (41 current smokers, 4 quit ≤ 2 months, 64 quit > 2 months, 100 never smoked; NCT00500617) identified 4214 genes significantly correlated with self-reported smoking status. qRT-PCR was performed on 1,071 PREDICT subjects across 256 microarray genes significantly correlated with smoking or CAD. A five gene (CLDND1, LRRN3, MUC1, GOPC, LEF1) predictive model, derived from the qRT-PCR data using stepwise forward logistic regression, had a cross-validated mean AUC of 0.93 (sensitivity=0.78; specificity=0.95), and was validated using 180 independent PREDICT subjects (AUC=0.82, CI 0.69-0.94; sensitivity=0.63; specificity=0.94). Plasma from the 180 validation subjects was used to assess levels of cotinine; a model using a threshold of 10 ng/ml cotinine resulted in an AUC of 0.89 (CI 0.81-0.97; sensitivity=0.81; specificity=0.97; kappa with expression model = 0.53). Conclusion We have constructed and validated a whole blood gene expression score for the evaluation of smoking status, demonstrating that clinical and environmental factors contributing to cardiovascular disease risk can be assessed by gene expression. PMID:23210427

Grid workflow validation using ontology-based tacit knowledge: A case study for quantitative remote sensing applications

NASA Astrophysics Data System (ADS)

Liu, Jia; Liu, Longli; Xue, Yong; Dong, Jing; Hu, Yingcui; Hill, Richard; Guang, Jie; Li, Chi

2017-01-01

Workflow for remote sensing quantitative retrieval is the ;bridge; between Grid services and Grid-enabled application of remote sensing quantitative retrieval. Workflow averts low-level implementation details of the Grid and hence enables users to focus on higher levels of application. The workflow for remote sensing quantitative retrieval plays an important role in remote sensing Grid and Cloud computing services, which can support the modelling, construction and implementation of large-scale complicated applications of remote sensing science. The validation of workflow is important in order to support the large-scale sophisticated scientific computation processes with enhanced performance and to minimize potential waste of time and resources. To research the semantic correctness of user-defined workflows, in this paper, we propose a workflow validation method based on tacit knowledge research in the remote sensing domain. We first discuss the remote sensing model and metadata. Through detailed analysis, we then discuss the method of extracting the domain tacit knowledge and expressing the knowledge with ontology. Additionally, we construct the domain ontology with Protégé. Through our experimental study, we verify the validity of this method in two ways, namely data source consistency error validation and parameters matching error validation.

Development and Validation of a Rapid (13)C6-Glucose Isotope Dilution UPLC-MRM Mass Spectrometry Method for Use in Determining System Accuracy and Performance of Blood Glucose Monitoring Devices.

PubMed

Matsunami, Risë K; Angelides, Kimon; Engler, David A

2015-05-18

There is currently considerable discussion about the accuracy of blood glucose concentrations determined by personal blood glucose monitoring systems (BGMS). To date, the FDA has allowed new BGMS to demonstrate accuracy in reference to other glucose measurement systems that use the same or similar enzymatic-based methods to determine glucose concentration. These types of reference measurement procedures are only comparative in nature and are subject to the same potential sources of error in measurement and system perturbations as the device under evaluation. It would be ideal to have a completely orthogonal primary method that could serve as a true standard reference measurement procedure for establishing the accuracy of new BGMS. An isotope-dilution liquid chromatography/mass spectrometry (ID-UPLC-MRM) assay was developed using (13)C6-glucose as a stable isotope analogue to specifically measure glucose concentration in human plasma, and validated for use against NIST standard reference materials, and against fresh isolates of whole blood and plasma into which exogenous glucose had been spiked. Assay performance was quantified to NIST-traceable dry weight measures for both glucose and (13)C6-glucose. The newly developed assay method was shown to be rapid, highly specific, sensitive, accurate, and precise for measuring plasma glucose levels. The assay displayed sufficient dynamic range and linearity to measure across the range of both normal and diabetic blood glucose levels. Assay performance was measured to within the same uncertainty levels (<1%) as the NIST definitive method for glucose measurement in human serum. The newly developed ID UPLC-MRM assay can serve as a validated reference measurement procedure to which new BGMS can be assessed for glucose measurement performance. © 2015 Diabetes Technology Society.

Development and Validation of a Rapid 13C6-Glucose Isotope Dilution UPLC-MRM Mass Spectrometry Method for Use in Determining System Accuracy and Performance of Blood Glucose Monitoring Devices

PubMed Central

Matsunami, Risë K.; Angelides, Kimon; Engler, David A.

2015-01-01

Background: There is currently considerable discussion about the accuracy of blood glucose concentrations determined by personal blood glucose monitoring systems (BGMS). To date, the FDA has allowed new BGMS to demonstrate accuracy in reference to other glucose measurement systems that use the same or similar enzymatic-based methods to determine glucose concentration. These types of reference measurement procedures are only comparative in nature and are subject to the same potential sources of error in measurement and system perturbations as the device under evaluation. It would be ideal to have a completely orthogonal primary method that could serve as a true standard reference measurement procedure for establishing the accuracy of new BGMS. Methods: An isotope-dilution liquid chromatography/mass spectrometry (ID-UPLC-MRM) assay was developed using 13C6-glucose as a stable isotope analogue to specifically measure glucose concentration in human plasma, and validated for use against NIST standard reference materials, and against fresh isolates of whole blood and plasma into which exogenous glucose had been spiked. Assay performance was quantified to NIST-traceable dry weight measures for both glucose and 13C6-glucose. Results: The newly developed assay method was shown to be rapid, highly specific, sensitive, accurate, and precise for measuring plasma glucose levels. The assay displayed sufficient dynamic range and linearity to measure across the range of both normal and diabetic blood glucose levels. Assay performance was measured to within the same uncertainty levels (<1%) as the NIST definitive method for glucose measurement in human serum. Conclusions: The newly developed ID UPLC-MRM assay can serve as a validated reference measurement procedure to which new BGMS can be assessed for glucose measurement performance. PMID:25986627

Understanding of visual attention by adult humans (Homo sapiens): a partial replication of Povinelli, Bierschwale, and Cech (1999).

PubMed

Thomas, Emily; Murphy, Mary; Pitt, Rebecca; Rivers, Angela; Leavens, David A

2008-11-01

Povinelli, Bierschwale, and Cech (1999) reported that when tested on a visual attention task, the behavior of juvenile chimpanzees did not support a high-level understanding of visual attention. This study replicates their research using adult humans and aims to investigate the validity of their experimental design. Participants were trained to respond to pointing cues given by an experimenter, and then tested on their ability to locate hidden objects from visual cues. Povinelli et al.'s assertion that the generalization of pointing to gaze is indicative of a high-level framework was not supported by our findings: Training improved performance only on initial probe trials when the experimenter's gaze was not directed at the baited cup. Furthermore, participants performed above chance on such trials, the same result exhibited by chimpanzees and used as evidence by Povinelli et al. to support a low-level framework. These findings, together with the high performance of participants in an incongruent condition, in which the experimenter pointed to or gazed at an unbaited container, challenge the validity of their experimental design. (PsycINFO Database Record (c) 2008 APA, all rights reserved).

Effectively Coping With Task Stress: A Study of the Validity of the Trait Emotional Intelligence Questionnaire-Short Form (TEIQue-SF).

PubMed

O'Connor, Peter; Nguyen, Jessica; Anglim, Jeromy

2017-01-01

In this study, we investigated the validity of the Trait Emotional Intelligence Questionnaire-Short Form (TEIQue-SF; Petrides, 2009) in the context of task-induced stress. We used a total sample of 225 volunteers to investigate (a) the incremental validity of the TEIQue-SF over other predictors of coping with task-induced stress, and (b) the construct validity of the TEIQue-SF by examining the mechanisms via which scores from the TEIQue-SF predict coping outcomes. Results demonstrated that the TEIQue-SF possessed incremental validity over the Big Five personality traits in the prediction of emotion-focused coping. Results also provided support for the construct validity of the TEIQue-SF by demonstrating that this measure predicted adaptive coping via emotion-focused channels. Specifically, results showed that, following a task stressor, the TEIQue-SF predicted low negative affect and high task performance via high levels of emotion-focused coping. Consistent with the purported theoretical nature of the trait emotional intelligence (EI) construct, trait EI as assessed by the TEIQue-SF primarily enhances affect and performance in stressful situations by regulating negative emotions.

Ultrasensitive NIR-SERRS Probes with Multiplexed Ratiometric Quantification for In Vivo Antibody Leads Validation.

PubMed

Kang, Homan; Jeong, Sinyoung; Jo, Ahla; Chang, Hyejin; Yang, Jin-Kyoung; Jeong, Cheolhwan; Kyeong, San; Lee, Youn Woo; Samanta, Animesh; Maiti, Kaustabh Kumar; Cha, Myeong Geun; Kim, Taek-Keun; Lee, Sukmook; Jun, Bong-Hyun; Chang, Young-Tae; Chung, Junho; Lee, Ho-Young; Jeong, Dae Hong; Lee, Yoon-Sik

2018-02-01

Immunotargeting ability of antibodies may show significant difference between in vitro and in vivo. To select antibody leads with high affinity and specificity, it is necessary to perform in vivo validation of antibody candidates following in vitro antibody screening. Herein, a robust in vivo validation of anti-tetraspanin-8 antibody candidates against human colon cancer using ratiometric quantification method is reported. The validation is performed on a single mouse and analyzed by multiplexed surface-enhanced Raman scattering using ultrasensitive and near infrared (NIR)-active surface-enhanced resonance Raman scattering nanoprobes (NIR-SERRS dots). The NIR-SERRS dots are composed of NIR-active labels and Au/Ag hollow-shell assembled silica nanospheres. A 93% of NIR-SERRS dots is detectable at a single-particle level and signal intensity is 100-fold stronger than that from nonresonant molecule-labeled spherical Au NPs (80 nm). The result of SERRS-based antibody validation is comparable to that of the conventional method using single-photon-emission computed tomography. The NIR-SERRS-based strategy is an alternate validation method which provides cost-effective and accurate multiplexing measurements for antibody-based drug development. © 2017 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim.

Multi-Evaporator Miniature Loop Heat Pipe for Small Spacecraft Thermal Control. Part 2; Validation Results

NASA Technical Reports Server (NTRS)

Ku, Jentung; Ottenstein, Laura; Douglas, Donya; Hoang, Triem

2010-01-01

Under NASA s New Millennium Program Space Technology 8 (ST 8) Project, Goddard Space Fight Center has conducted a Thermal Loop experiment to advance the maturity of the Thermal Loop technology from proof of concept to prototype demonstration in a relevant environment , i.e. from a technology readiness level (TRL) of 3 to a level of 6. The thermal Loop is an advanced thermal control system consisting of a miniature loop heat pipe (MLHP) with multiple evaporators and multiple condensers designed for future small system applications requiring low mass, low power, and compactness. The MLHP retains all features of state-of-the-art loop heat pipes (LHPs) and offers additional advantages to enhance the functionality, performance, versatility, and reliability of the system. An MLHP breadboard was built and tested in the laboratory and thermal vacuum environments for the TRL 4 and TRL 5 validations, respectively, and an MLHP proto-flight unit was built and tested in a thermal vacuum chamber for the TRL 6 validation. In addition, an analytical model was developed to simulate the steady state and transient behaviors of the MLHP during various validation tests. The MLHP demonstrated excellent performance during experimental tests and the analytical model predictions agreed very well with experimental data. All success criteria at various TRLs were met. Hence, the Thermal Loop technology has reached a TRL of 6. This paper presents the validation results, both experimental and analytical, of such a technology development effort.

Validation of antibiotic residue tests for dairy goats.

PubMed

Zeng, S S; Hart, S; Escobar, E N; Tesfai, K

1998-03-01

The SNAP test, LacTek test (B-L and CEF), Charm Bacillus sterothermophilus var. calidolactis disk assay (BsDA), and Charm II Tablet Beta-lactam sequential test were validated using antibiotic-fortified and -incurred goat milk following the protocol for test kit validations of the U.S. Food and Drug Administration Center for Veterinary Medicine. SNAP, Charm BsDA, and Charm II Tablet Sequential tests were sensitive and reliable in detecting antibiotic residues in goat milk. All three assays showed greater than 90% sensitivity and specificity at tolerance and detection levels. However, caution should be taken in interpreting test results at detection levels. Because of the high sensitivity of these three tests, false-violative results could be obtained in goat milk containing antibiotic residues below the tolerance level. Goat milk testing positive by these tests must be confirmed using a more sophisticated methodology, such as high-performance liquid chromatography, before the milk is condemned. LacTek B-L test did not detect several antibiotics, including penicillin G, in goat milk at tolerance levels. However, LacTek CEF was excellent in detecting ceftiofur residue in goat milk.

Finite element analysis of dental implants with validation: to what extent can we expect the model to predict biological phenomena? A literature review and proposal for classification of a validation process.

PubMed

Chang, Yuanhan; Tambe, Abhijit Anil; Maeda, Yoshinobu; Wada, Masahiro; Gonda, Tomoya

2018-03-08

A literature review of finite element analysis (FEA) studies of dental implants with their model validation process was performed to establish the criteria for evaluating validation methods with respect to their similarity to biological behavior. An electronic literature search of PubMed was conducted up to January 2017 using the Medical Subject Headings "dental implants" and "finite element analysis." After accessing the full texts, the context of each article was searched using the words "valid" and "validation" and articles in which these words appeared were read to determine whether they met the inclusion criteria for the review. Of 601 articles published from 1997 to 2016, 48 that met the eligibility criteria were selected. The articles were categorized according to their validation method as follows: in vivo experiments in humans (n = 1) and other animals (n = 3), model experiments (n = 32), others' clinical data and past literature (n = 9), and other software (n = 2). Validation techniques with a high level of sufficiency and efficiency are still rare in FEA studies of dental implants. High-level validation, especially using in vivo experiments tied to an accurate finite element method, needs to become an established part of FEA studies. The recognition of a validation process should be considered when judging the practicality of an FEA study.

Developing and validating a conceptual survey to assess introductory physics students’ understanding of magnetism

NASA Astrophysics Data System (ADS)

Li, Jing; Singh, Chandralekha

2017-03-01

Development of validated physics surveys on various topics is important for investigating the extent to which students master those concepts after traditional instruction and for assessing innovative curricula and pedagogies that can improve student understanding significantly. Here, we discuss the development and validation of a conceptual multiple-choice survey related to magnetism suitable for introductory physics courses. The survey was developed taking into account common students’ difficulties with magnetism concepts covered in introductory physics courses found in our investigation and the incorrect choices to the multiple-choice questions were designed based upon those common student difficulties. After the development and validation of the survey, it was administered to introductory physics students in various classes in paper-pencil format before and after traditional lecture-based instruction in relevant concepts. We compared the performance of students on the survey in the algebra-based and calculus-based introductory physics courses before and after traditional lecture-based instruction in relevant magnetism concepts. We discuss the common difficulties of introductory physics students with magnetism concepts we found via the survey. We also administered the survey to upper-level undergraduates majoring in physics and PhD students to benchmark the survey and compared their performance with those of traditionally taught introductory physics students for whom the survey is intended. A comparison with the base line data on the validated magnetism survey from traditionally taught introductory physics courses and upper-level undergraduate and PhD students discussed in this paper can help instructors assess the effectiveness of curricula and pedagogies which is especially designed to help students integrate conceptual and quantitative understanding and develop a good grasp of the concepts. In particular, if introductory physics students’ average performance in a class is significantly better than those of students in traditionally taught courses described here (and particularly when it is comparable to that of physics PhD students’ average performance discussed here), the curriculum or pedagogy used in that introductory class can be deemed effective. Moreover, we discuss the use of the survey to investigate gender differences in student performance.

Development of diagnostic test instruments to reveal level student conception in kinematic and dynamics

NASA Astrophysics Data System (ADS)

Handhika, J.; Cari, C.; Suparmi, A.; Sunarno, W.; Purwandari, P.

2018-03-01

The purpose of this research was to develop a diagnostic test instrument to reveal students' conceptions in kinematics and dynamics. The diagnostic test was developed based on the content indicator the concept of (1) displacement and distance, (2) instantaneous and average velocity, (3) zero and constant acceleration, (4) gravitational acceleration (5) Newton's first Law, (6) and Newton's third Law. The diagnostic test development model includes: Diagnostic test requirement analysis, formulating test-making objectives, developing tests, checking the validity of the content and the performance of reliability, and application of tests. The Content Validation Index (CVI) results in the category are highly relevant, with a value of 0.85. Three questions get negative Content Validation Ratio CVR) (-0.6), after revised distractors and clarify visual presentation; the CVR become 1 (highly relevant). This test was applied, obtained 16 valid test items, with Cronbach Alpha value of 0.80. It can conclude that diagnostic test can be used to reveal the level of students conception in kinematics and dynamics.

Assessing Individual Social Capital Capacity: The Development and Validation of a Network Accessibility Scale

ERIC Educational Resources Information Center

Hatala, John-Paul

2009-01-01

Any organization that is able to promote the importance of increased levels of social capital and individuals who can leverage and use the resources that exist within the network may experience higher levels of performance. This study sought to add to our knowledge about individuals' accessing social resources for the purpose of accomplishing…

Validation of Questionnaire-Assessed Physical Activity in Comparison With Objective Measures Using Accelerometers and Physical Performance Measures Among Community-Dwelling Adults Aged ≥85 Years in Tokyo, Japan.

PubMed

Oguma, Yuko; Osawa, Yusuke; Takayama, Michiyo; Abe, Yukiko; Tanaka, Shigeho; Lee, I-Min; Arai, Yasumichi

2017-04-01

To date, there is no physical activity (PA) questionnaire with convergent and construct validity for the oldest-old. The aim of the current study was to investigate the validity of questionnaire-assessed PA in comparison with objective measures determined by uniaxial and triaxial accelerometers and physical performance measures in the oldest-old. Participants were 155 elderly (mean age 90 years) who were examined at the university and agreed to wear an accelerometer for 7 days in the 3-year-follow-up survey of the Tokyo Oldest-Old Survey of Total Health. Fifty-nine participants wore a uniaxial and triaxial accelerometer simultaneously. Self-rated walking, exercise, and household PA were measured using a modified Zutphen PA Questionnaire (PAQ). Several physical performance tests were done, and the associations among PAQ, accelerometer-assessed PA, and physical performances were compared by Spearman's correlation coefficients. Significant, low to moderate correlations between PA measures were seen on questionnaire and accelerometer assessments (ρ = 0.19 to 0.34). Questionnaireassessed PA measure were correlated with a range of lower extremity performance (ρ = 0.21 to 0.29). This PAQ demonstrated convergent and construct validity. Our findings suggest that the PAQ can reasonably be used in this oldest-old population to rank their PA level.

Development and Validation of High Precision Thermal, Mechanical, and Optical Models for the Space Interferometry Mission

NASA Technical Reports Server (NTRS)

Lindensmith, Chris A.; Briggs, H. Clark; Beregovski, Yuri; Feria, V. Alfonso; Goullioud, Renaud; Gursel, Yekta; Hahn, Inseob; Kinsella, Gary; Orzewalla, Matthew; Phillips, Charles

2006-01-01

SIM Planetquest (SIM) is a large optical interferometer for making microarcsecond measurements of the positions of stars, and to detect Earth-sized planets around nearby stars. To achieve this precision, SIM requires stability of optical components to tens of picometers per hour. The combination of SIM s large size (9 meter baseline) and the high stability requirement makes it difficult and costly to measure all aspects of system performance on the ground. To reduce risks, costs and to allow for a design with fewer intermediate testing stages, the SIM project is developing an integrated thermal, mechanical and optical modeling process that will allow predictions of the system performance to be made at the required high precision. This modeling process uses commercial, off-the-shelf tools and has been validated against experimental results at the precision of the SIM performance requirements. This paper presents the description of the model development, some of the models, and their validation in the Thermo-Opto-Mechanical (TOM3) testbed which includes full scale brassboard optical components and the metrology to test them at the SIM performance requirement levels.

Validity of Level of Supervision Scales for Assessing Pediatric Fellows on the Common Pediatric Subspecialty Entrustable Professional Activities.

PubMed

Mink, Richard B; Schwartz, Alan; Herman, Bruce E; Turner, David A; Curran, Megan L; Myers, Angela; Hsu, Deborah C; Kesselheim, Jennifer C; Carraccio, Carol L

2018-02-01

Entrustable professional activities (EPAs) represent the routine and essential activities that physicians perform in practice. Although some level of supervision scales have been proposed, they have not been validated. In this study, the investigators created level of supervision scales for EPAs common to the pediatric subspecialties and then examined their validity in a study conducted by the Subspecialty Pediatrics Investigator Network (SPIN). SPIN Steering Committee members used a modified Delphi process to develop unique scales for six of the seven common EPAs. The investigators sought validity evidence in a multisubspecialty study in which pediatric fellowship program directors and Clinical Competency Committees used the scales to evaluate fellows in fall 2014 and spring 2015. Separate scales for the six EPAs, each with five levels of progressive entrustment, were created. In both fall and spring, more than 300 fellows in each year of training from over 200 programs were assessed. In both periods and for each EPA, there was a progressive increase in entrustment levels, with second-year fellows rated higher than first-year fellows (P < .001) and third-year fellows rated higher than second-year fellows (P < .001). For each EPA, spring ratings were higher (P < .001) than those in the fall. Interrater reliability was high (Janson and Olsson's iota = 0.73). The supervision scales developed for these six common pediatric subspecialty EPAs demonstrated strong validity evidence for use in EPA-based assessment of pediatric fellows. They may also inform the development of scales in other specialties.

Assessment of MARMOT. A Mesoscale Fuel Performance Code

DOE Office of Scientific and Technical Information (OSTI.GOV)

Tonks, M. R.; Schwen, D.; Zhang, Y.

2015-04-01

MARMOT is the mesoscale fuel performance code under development as part of the US DOE Nuclear Energy Advanced Modeling and Simulation Program. In this report, we provide a high level summary of MARMOT, its capabilities, and its current state of validation. The purpose of MARMOT is to predict the coevolution of microstructure and material properties of nuclear fuel and cladding. It accomplished this using the phase field method coupled to solid mechanics and heat conduction. MARMOT is based on the Multiphysics Object-Oriented Simulation Environment (MOOSE), and much of its basic capability in the areas of the phase field method, mechanics,more » and heat conduction come directly from MOOSE modules. However, additional capability specific to fuel and cladding is available in MARMOT. While some validation of MARMOT has been completed in the areas of fission gas behavior and grain growth, much more validation needs to be conducted. However, new mesoscale data needs to be obtained in order to complete this validation.« less

Onboard FPGA-based SAR processing for future spaceborne systems

NASA Technical Reports Server (NTRS)

Le, Charles; Chan, Samuel; Cheng, Frank; Fang, Winston; Fischman, Mark; Hensley, Scott; Johnson, Robert; Jourdan, Michael; Marina, Miguel; Parham, Bruce;

Measurement of Habitual Physical Activity Performance in Adolescents with Cerebral Palsy: A Systematic Review

ERIC Educational Resources Information Center

Clanchy, Kelly M.; Tweedy, Sean M.; Boyd, Roslyn

2011-01-01

Aim: This systematic review compares the validity, reliability, and clinical use of habitual physical activity (HPA) performance measures in adolescents with cerebral palsy (CP). Method: Measures of HPA across Gross Motor Function Classification System (GMFCS) levels I-V for adolescents (10-18y) with CP were included if at least 60% of items…

The Definition and Measurement of Small Military Unit Team Functions. Final Report, July 1980-October 1981.

ERIC Educational Resources Information Center

Shiflett, Samuel; And Others

A study was undertaken to improve the measurement of small team performance within the Army. A provisional taxonomy of team-level performance functions was field-validated; criteria and measures of the functions were developed; and their reliability was examined. The provisional taxonomy, used for observing Army field training exercises, was used…

Assessment of communication, professionalism, and surgical skills in an objective structured performance-related examination (OSPRE): a psychometric study.

PubMed

Ponton-Carss, Alicia; Hutchison, Carol; Violato, Claudio

2011-10-01

The purpose of this study was to investigate the reliability and validity of a performance assessment of communication, professionalism, and surgical skills competencies for surgery residents. Fourteen residents from the general surgery program of the University of Calgary were assessed in 7 surgical simulation stations that included communication and professionalism skills. The internal consistency reliability of the checklists and global rating scales combined was adequate for communication (α = .75-.92) and surgical skills (α = .86-.96), but not for professionalism (α = 0). There was evidence of validity as surgical skills performance improved as a function of postgraduate year level but not for the professionalism checklist. Surgical skills and communication correlated in the 2 stations assessed (r = .55 and .57; P < .05). There is evidence for both reliability and validity for simultaneously assessing surgical skills and communication skills. Further instrument development is required to assess professionalism in a structured examination context. Copyright © 2011 Elsevier Inc. All rights reserved.

Validity of a smartphone protractor to measure sagittal parameters in adult spinal deformity.

PubMed

Kunkle, William Aaron; Madden, Michael; Potts, Shannon; Fogelson, Jeremy; Hershman, Stuart

2017-10-01

Smartphones have become an integral tool in the daily life of health-care professionals (Franko 2011). Their ease of use and wide availability often make smartphones the first tool surgeons use to perform measurements. This technique has been validated for certain orthopedic pathologies (Shaw 2012; Quek 2014; Milanese 2014; Milani 2014), but never to assess sagittal parameters in adult spinal deformity (ASD). This study was designed to assess the validity, reproducibility, precision, and efficiency of using a smartphone protractor application to measure sagittal parameters commonly measured in ASD assessment and surgical planning. This study aimed to (1) determine the validity of smartphone protractor applications, (2) determine the intra- and interobserver reliability of smartphone protractor applications when used to measure sagittal parameters in ASD, (3) determine the efficiency of using a smartphone protractor application to measure sagittal parameters, and (4) elucidate whether a physician's level of experience impacts the reliability or validity of using a smartphone protractor application to measure sagittal parameters in ASD. An experimental validation study was carried out. Thirty standard 36″ standing lateral radiographs were examined. Three separate measurements were performed using a marker and protractor; then at a separate time point, three separate measurements were performed using a smartphone protractor application for all 30 radiographs. The first 10 radiographs were then re-measured two more times, for a total of three measurements from both the smartphone protractor and marker and protractor. The parameters included lumbar lordosis, pelvic incidence, and pelvic tilt. Three raters performed all measurements-a junior level orthopedic resident, a senior level orthopedic resident, and a fellowship-trained spinal deformity surgeon. All data, including the time to perform the measurements, were recorded, and statistical analysis was performed to determine intra- and interobserver reliability, as well as accuracy, efficiency, and precision. Statistical analysis using the intra- and interclass correlation coefficient was calculated using R (version 3.3.2, 2016) to determine the degree of intra- and interobserver reliability. High rates of intra- and interobserver reliability were observed between the junior resident, senior resident, and attending surgeon when using the smartphone protractor application as demonstrated by high inter- and intra-class correlation coefficients greater than 0.909 and 0.874 respectively. High rates of inter- and intraobserver reliability were also seen between the junior resident, senior resident, and attending surgeon when a marker and protractor were used as demonstrated by high inter- and intra-class correlation coefficients greater than 0.909 and 0.807 respectively. The lumbar lordosis, pelvic incidence, and pelvic tilt values were accurately measured by all three raters, with excellent inter- and intra-class correlation coefficient values. When the first 10 radiographs were re-measured at different time points, a high degree of precision was noted. Measurements performed using the smartphone application were consistently faster than using a marker and protractor-this difference reached statistical significance of p<.05. Adult spinal deformity radiographic parameters can be measured accurately, precisely, reliably, and more efficiently using a smartphone protractor application than with a standard protractor and wax pencil. A high degree of intra- and interobserver reliability was seen between the residents and attending surgeon, indicating measurements made with a smartphone protractor are unaffected by an observer's level of experience. As a result, smartphone protractors may be used when planning ASD surgery. Copyright © 2017 Elsevier Inc. All rights reserved.

Validating hierarchical verbal autopsy expert algorithms in a large data set with known causes of death.

PubMed

Kalter, Henry D; Perin, Jamie; Black, Robert E

2016-06-01

Physician assessment historically has been the most common method of analyzing verbal autopsy (VA) data. Recently, the World Health Organization endorsed two automated methods, Tariff 2.0 and InterVA-4, which promise greater objectivity and lower cost. A disadvantage of the Tariff method is that it requires a training data set from a prior validation study, while InterVA relies on clinically specified conditional probabilities. We undertook to validate the hierarchical expert algorithm analysis of VA data, an automated, intuitive, deterministic method that does not require a training data set. Using Population Health Metrics Research Consortium study hospital source data, we compared the primary causes of 1629 neonatal and 1456 1-59 month-old child deaths from VA expert algorithms arranged in a hierarchy to their reference standard causes. The expert algorithms were held constant, while five prior and one new "compromise" neonatal hierarchy, and three former child hierarchies were tested. For each comparison, the reference standard data were resampled 1000 times within the range of cause-specific mortality fractions (CSMF) for one of three approximated community scenarios in the 2013 WHO global causes of death, plus one random mortality cause proportions scenario. We utilized CSMF accuracy to assess overall population-level validity, and the absolute difference between VA and reference standard CSMFs to examine particular causes. Chance-corrected concordance (CCC) and Cohen's kappa were used to evaluate individual-level cause assignment. Overall CSMF accuracy for the best-performing expert algorithm hierarchy was 0.80 (range 0.57-0.96) for neonatal deaths and 0.76 (0.50-0.97) for child deaths. Performance for particular causes of death varied, with fairly flat estimated CSMF over a range of reference values for several causes. Performance at the individual diagnosis level was also less favorable than that for overall CSMF (neonatal: best CCC = 0.23, range 0.16-0.33; best kappa = 0.29, 0.23-0.35; child: best CCC = 0.40, 0.19-0.45; best kappa = 0.29, 0.07-0.35). Expert algorithms in a hierarchy offer an accessible, automated method for assigning VA causes of death. Overall population-level accuracy is similar to that of more complex machine learning methods, but without need for a training data set from a prior validation study.

Validity and reliability of the robotic objective structured assessment of technical skills

PubMed Central

Siddiqui, Nazema Y.; Galloway, Michael L.; Geller, Elizabeth J.; Green, Isabel C.; Hur, Hye-Chun; Langston, Kyle; Pitter, Michael C.; Tarr, Megan E.; Martino, Martin A.

2015-01-01

Objective Objective structured assessments of technical skills (OSATS) have been developed to measure the skill of surgical trainees. Our aim was to develop an OSATS specifically for trainees learning robotic surgery. Study Design This is a multi-institutional study in eight academic training programs. We created an assessment form to evaluate robotic surgical skill through five inanimate exercises. Obstetrics/gynecology, general surgery, and urology residents, fellows, and faculty completed five robotic exercises on a standard training model. Study sessions were recorded and randomly assigned to three blinded judges who scored performance using the assessment form. Construct validity was evaluated by comparing scores between participants with different levels of surgical experience; inter- and intra-rater reliability were also assessed. Results We evaluated 83 residents, 9 fellows, and 13 faculty, totaling 105 participants; 88 (84%) were from obstetrics/gynecology. Our assessment form demonstrated construct validity, with faculty and fellows performing significantly better than residents (mean scores: 89 ± 8 faculty; 74 ± 17 fellows; 59 ± 22 residents, p<0.01). In addition, participants with more robotic console experience scored significantly higher than those with fewer prior console surgeries (p<0.01). R-OSATS demonstrated good inter-rater reliability across all five drills (mean Cronbach's α: 0.79 ± 0.02). Intra-rater reliability was also high (mean Spearman's correlation: 0.91 ± 0.11). Conclusions We developed an assessment form for robotic surgical skill that demonstrates construct validity, inter- and intra-rater reliability. When paired with standardized robotic skill drills this form may be useful to distinguish between levels of trainee performance. PMID:24807319

Predictive performance models and multiple task performance

NASA Technical Reports Server (NTRS)

Wickens, Christopher D.; Larish, Inge; Contorer, Aaron

1989-01-01

Five models that predict how performance of multiple tasks will interact in complex task scenarios are discussed. The models are shown in terms of the assumptions they make about human operator divided attention. The different assumptions about attention are then empirically validated in a multitask helicopter flight simulation. It is concluded from this simulation that the most important assumption relates to the coding of demand level of different component tasks.

Technology Readiness of the NEXT Ion Propulsion System

NASA Technical Reports Server (NTRS)

Benson, Scott W.; Patterson, Michael J.

2008-01-01

The NASA's Evolutionary Xenon Thruster (NEXT) ion propulsion system has been in advanced technology development under the NASA In-Space Propulsion Technology project. The highest fidelity hardware planned has now been completed by the government/industry team, including: a flight prototype model (PM) thruster, an engineering model (EM) power processing unit, EM propellant management assemblies, a breadboard gimbal, and control unit simulators. Subsystem and system level technology validation testing is in progress. To achieve the objective Technology Readiness Level 6, environmental testing is being conducted to qualification levels in ground facilities simulating the space environment. Additional tests have been conducted to characterize the performance range and life capability of the NEXT thruster. This paper presents the status and results of technology validation testing accomplished to date, the validated subsystem and system capabilities, and the plans for completion of this phase of NEXT development. The next round of competed planetary science mission announcements of opportunity, and directed mission decisions, are anticipated to occur in 2008 and 2009. Progress to date, and the success of on-going technology validation, indicate that the NEXT ion propulsion system will be a primary candidate for mission consideration in these upcoming opportunities.

Assessments on GOCE-based Gravity Field Model Comparisons with Terrestrial Data Using Wavelet Decomposition and Spectral Enhancement Approaches

NASA Astrophysics Data System (ADS)

Erol, Serdar; Serkan Isık, Mustafa; Erol, Bihter

2016-04-01

The recent Earth gravity field satellite missions data lead significant improvement in Global Geopotential Models in terms of both accuracy and resolution. However the improvement in accuracy is not the same everywhere in the Earth and therefore quantifying the level of improvement locally is necessary using the independent data. The validations of the level-3 products from the gravity field satellite missions, independently from the estimation procedures of these products, are possible using various arbitrary data sets, as such the terrestrial gravity observations, astrogeodetic vertical deflections, GPS/leveling data, the stationary sea surface topography. Quantifying the quality of the gravity field functionals via recent products has significant importance for determination of the regional geoid modeling, base on the satellite and terrestrial data fusion with an optimal algorithm, beside the statistical reporting the improvement rates depending on spatial location. In the validations, the errors and the systematic differences between the data and varying spectral content of the compared signals should be considered in order to have comparable results. In this manner this study compares the performance of Wavelet decomposition and spectral enhancement techniques in validation of the GOCE/GRACE based Earth gravity field models using GPS/leveling and terrestrial gravity data in Turkey. The terrestrial validation data are filtered using Wavelet decomposition technique and the numerical results from varying levels of decomposition are compared with the results which are derived using the spectral enhancement approach with contribution of an ultra-high resolution Earth gravity field model. The tests include the GO-DIR-R5, GO-TIM-R5, GOCO05S, EIGEN-6C4 and EGM2008 global models. The conclusion discuss the superiority and drawbacks of both concepts as well as reporting the performance of tested gravity field models with an estimate of their contribution to modeling the geoid in Turkish territory.

From dV-Trainer to Real Robotic Console: The Limitations of Robotic Skill Training.

PubMed

Yang, Kun; Zhen, Hang; Hubert, Nicolas; Perez, Manuela; Wang, Xing Huan; Hubert, Jacques

To investigate operators' performance quality, mental stress, and ergonomic habits through a training curriculum on robotic simulators. Forty volunteers without robotic surgery experience were recruited to practice 2 exercises on a dV-Trainer (dVT) for 14 hours. The simulator software (M-score a ) provided an automatic evaluation of the overall score for the surgeons' performance. Each participant provided a subjective difficulty score (validity to be proven) for each exercise. Their ergonomic habits were evaluated based on the workspace range and armrest load-validated criteria for evaluating the proficiency of using the armrest. They then repeated the same tasks on a da Vinci Surgical Skill Simulator for a final-level test. Their final scores were compared with their initial scores and the scores of 5 experts on the da Vinci Surgical Skill Simulator. A total of 14 hours of training on the dVT significantly improved the surgeons' performance scores to the expert level with a significantly reduced workload, but their ergonomic score was still far from the expert level. Sufficient training on the dVT improves novices' performance, reduces psychological stress, and inculcates better ergonomic habits. Among the evaluated criteria, novices had the most difficulty in achieving expert levels of ergonomic skills. The training benefits of robotic surgery simulators should be determined with quantified variables. The detection of the limitations during robotic training curricula could guide the targeted training and improve the training effect. Copyright © 2017. Published by Elsevier Inc.

A Deep Machine Learning Method for Classifying Cyclic Time Series of Biological Signals Using Time-Growing Neural Network.

PubMed

Gharehbaghi, Arash; Linden, Maria

2017-10-12

This paper presents a novel method for learning the cyclic contents of stochastic time series: the deep time-growing neural network (DTGNN). The DTGNN combines supervised and unsupervised methods in different levels of learning for an enhanced performance. It is employed by a multiscale learning structure to classify cyclic time series (CTS), in which the dynamic contents of the time series are preserved in an efficient manner. This paper suggests a systematic procedure for finding the design parameter of the classification method for a one-versus-multiple class application. A novel validation method is also suggested for evaluating the structural risk, both in a quantitative and a qualitative manner. The effect of the DTGNN on the performance of the classifier is statistically validated through the repeated random subsampling using different sets of CTS, from different medical applications. The validation involves four medical databases, comprised of 108 recordings of the electroencephalogram signal, 90 recordings of the electromyogram signal, 130 recordings of the heart sound signal, and 50 recordings of the respiratory sound signal. Results of the statistical validations show that the DTGNN significantly improves the performance of the classification and also exhibits an optimal structural risk.

Reinforced Carbon-Carbon Subcomponent Flat Plate Impact Testing for Space Shuttle Orbiter Return to Flight

NASA Technical Reports Server (NTRS)

Melis, Matthew E.; Brand, Jeremy H.; Pereira, J. Michael; Revilock, Duane M.

2007-01-01

Following the tragedy of the Space Shuttle Columbia on February 1, 2003, a major effort commenced to develop a better understanding of debris impacts and their effect on the Space Shuttle subsystems. An initiative to develop and validate physics-based computer models to predict damage from such impacts was a fundamental component of this effort. To develop the models it was necessary to physically characterize Reinforced Carbon-Carbon (RCC) and various debris materials which could potentially shed on ascent and impact the Orbiter RCC leading edges. The validated models enabled the launch system community to use the impact analysis software LS DYNA to predict damage by potential and actual impact events on the Orbiter leading edge and nose cap thermal protection systems. Validation of the material models was done through a three-level approach: fundamental tests to obtain independent static and dynamic material model properties of materials of interest, sub-component impact tests to provide highly controlled impact test data for the correlation and validation of the models, and full-scale impact tests to establish the final level of confidence for the analysis methodology. This paper discusses the second level subcomponent test program in detail and its application to the LS DYNA model validation process. The level two testing consisted of over one hundred impact tests in the NASA Glenn Research Center Ballistic Impact Lab on 6 by 6 in. and 6 by 12 in. flat plates of RCC and evaluated three types of debris projectiles: BX 265 External Tank foam, ice, and PDL 1034 External Tank foam. These impact tests helped determine the level of damage generated in the RCC flat plates by each projectile. The information obtained from this testing validated the LS DYNA damage prediction models and provided a certain level of confidence to begin performing analysis for full-size RCC test articles for returning NASA to flight with STS 114 and beyond.

First validation of the PASSPORT training environment for arthroscopic skills.

PubMed

Tuijthof, Gabriëlle J M; van Sterkenburg, Maayke N; Sierevelt, Inger N; van Oldenrijk, Jakob; Van Dijk, C Niek; Kerkhoffs, Gino M M J

2010-02-01

The demand for high quality care is in contrast to reduced training time for residents to develop arthroscopic skills. Thereto, simulators are introduced to train skills away from the operating room. In our clinic, a physical simulation environment to Practice Arthroscopic Surgical Skills for Perfect Operative Real-life Treatment (PASSPORT) is being developed. The PASSPORT concept consists of maintaining the normal arthroscopic equipment, replacing the human knee joint by a phantom, and integrating registration devices to provide performance feedback. The first prototype of the knee phantom allows inspection, treatment of menisci, irrigation, and limb stressing. PASSPORT was evaluated for face and construct validity. Construct validity was assessed by measuring the performance of two groups with different levels of arthroscopic experience (20 surgeons and 8 residents). Participants performed a navigation task five times on PASSPORT. Task times were recorded. Face validity was assessed by completion of a short questionnaire on the participants' impressions and comments for improvements. Construct validity was demonstrated as the surgeons (median task time 19.7 s [8.0-37.6]) were more efficient than the residents (55.2 s [27.9-96.6]) in task completion for each repetition (Mann-Whitney U test, P < 0.05). The prototype of the knee phantom sufficiently imitated limb outer appearance (79%), portal resistance (82%), and arthroscopic view (81%). Improvements are required for the stressing device and the material of cruciate ligaments. Our physical simulation environment (PASSPORT) demonstrates its potential to evolve as a training modality. In future, automated performance feedback is aimed for.

Validity and Reliability of Baseline Testing in a Standardized Environment.

PubMed

Higgins, Kathryn L; Caze, Todd; Maerlender, Arthur

2017-08-11

The Immediate Postconcussion Assessment and Cognitive Testing (ImPACT) is a computerized neuropsychological test battery commonly used to determine cognitive recovery from concussion based on comparing post-injury scores to baseline scores. This model is based on the premise that ImPACT baseline test scores are a valid and reliable measure of optimal cognitive function at baseline. Growing evidence suggests that this premise may not be accurate and a large contributor to invalid and unreliable baseline test scores may be the protocol and environment in which baseline tests are administered. This study examined the effects of a standardized environment and administration protocol on the reliability and performance validity of athletes' baseline test scores on ImPACT by comparing scores obtained in two different group-testing settings. Three hundred-sixty one Division 1 cohort-matched collegiate athletes' baseline data were assessed using a variety of indicators of potential performance invalidity; internal reliability was also examined. Thirty-one to thirty-nine percent of the baseline cases had at least one indicator of low performance validity, but there were no significant differences in validity indicators based on environment in which the testing was conducted. Internal consistency reliability scores were in the acceptable to good range, with no significant differences between administration conditions. These results suggest that athletes may be reliably performing at levels lower than their best effort would produce. © The Author 2017. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

Yo-Yo IR2 testing of elite and sub-elite soccer players: performance, heart rate response and correlations to other interval tests.

PubMed

Ingebrigtsen, Jørgen; Bendiksen, Mads; Randers, Morten Bredsgaard; Castagna, Carlo; Krustrup, Peter; Holtermann, Andreas

2012-01-01

We examined performance, heart rate response and construct validity of the Yo-Yo IR2 test by testing 111 elite and 92 sub-elite soccer players from Norway and Denmark. VO₂max, Yo-Yo IR1 and repeated sprint tests (RSA) (n = 51) and match-analyses (n = 39) were also performed. Yo-Yo IR2 and Yo-Yo IR1 performance was 41 and 25% better (P < 0.01) for elite than sub-elite players, respectively, and heart rate after 2 and 4 min of the Yo-Yo IR2 test was 20 and 15 bpm (9 and 6% HRmax), respectively, lower (P < 0.01) for elite players. RSA performance and VO₂max was not different between competitive levels (P > 0.05). For top-teams, Yo-Yo IR2 performance (28%) and sprinting distance (25%) during match were greater (P < 0.05) than for bottom-teams. For elite and sub-elite players, Yo-Yo IR2 performance was correlated (P < 0.05) with Yo-Yo IR1 performance (r = 0.74 and 0.76) and mean RSA time (r = -0.74 and -0.34). We conclude that the Yo-Yo IR2 test has a high discriminant and concurrent validity, as it discriminates between players of different within- and between-league competitive levels and is correlated to other frequently used intermittent elite soccer tests.

Further evaluation of the EORTC QLQ-C30 psychometric properties in a large Brazilian cancer patient cohort as a function of their educational status.

PubMed

Paiva, Carlos Eduardo; Carneseca, Estela Cristina; Barroso, Eliane Marçon; de Camargos, Mayara Goulart; Alfano, Ana Camila Callado; Rugno, Fernanda Capella; Paiva, Bianca Sakamoto Ribeiro

2014-08-01

The European Organization for Research and Treatment of Cancer Core Quality of Life Questionnaire (EORTC QLQ-C30) is considered a valid instrument for use in Brazil. However, the previous Brazilian validation study included only 30 lung cancer patients and only measured test-retest reliability. The aim of this study was to evaluate the psychometric properties of the EORTC QLQ-C30 in a sample of cancer patients at different educational levels who completed the instrument administered by an interviewer. Data from six prospective studies conducted by the same group of researchers were combined in this study (N = 986). Reliability was assessed using Cronbach's alpha coefficient, all values of which were >0.7, with the exception of cognitive functioning, social functioning, and nausea and vomiting (α = 0.57, α = 0.69, and α = 0.68, respectively). In multi-trait scaling analysis, convergent and divergent validity were considered adequate (validity indices were 91.6 and 97.4%). In general, moderate to strong correlations were found between the subscales of the EORTC QLQ-C30 and its respective dimensions from the WHOQOL-bref, the hospital anxiety and depression scale, and the Edmonton Symptom Assessment System (ESAS) instruments. In addition, the EORTC QLQ-C30 was able to differentiate groups of patients with distinct performance statuses and types of treatment (known-group validation). Statistical analyses were also performed on educational status, yielding similar results. Detailed psychometric property data using the EORTC QLQ-C30 in Brazil are added by this study. In addition, we demonstrated that this instrument is in general reliable and valid regardless of the patient educational level.

Construct Validity of the Societal Outreach Scale (SOS).

PubMed

Fike, David S; Denton, Jason; Walk, Matt; Kish, Jennifer; Gorman, Ira

2018-04-01

The American Physical Therapy Association (APTA) has been working toward a vision of increasing professional focus on societal-level health. However, performance of social responsibility and related behaviors by physical therapists remain relatively poorly integrated into practice. Promoting a focus on societal outreach is necessary for all health care professionals to impact the health of their communities. The objective was to document the validity of the 14-item Societal Outreach Scale (SOS) for use with practicing physical therapists. This study used a cross-sectional survey. The SOS was transmitted via email to all therapists who were licensed and practicing in 10 states in the United States that were purposefully selected to assure a broad representation. A sample of 2612 usable responses was received. Factor analysis was applied to assess construct validity of the instrument. Of alternate models, a 3-factor model best demonstrated goodness of fit with the sample data according to conventional indices (standardized root mean squared residual = .03, comparative fit index .96, root mean square error of approximation = .06). The 3 factors measured by the SOS were labeled Societal-Level Health Advocacy, Community Engagement/Social Integration, and Political Engagement. Internal consistency reliability was 0.7 for all factors. The 3-factor SOS demonstrated acceptable validity and reliability. Though the sample included a broad representation of physical therapists, this was a single cross-sectional study. Additional confirmatory factor analysis, reliability testing, and word refinement of the tool are warranted. Given the construct validity and reliability of the 3-factor SOS, it is recommended for use as a validated instrument to measure physical therapists' performance of social responsibility and related behaviors.

Excellent Patient Care Processes in Poor Hospitals? Why Hospital-Level and Patient-Level Care Quality-Outcome Relationships Can Differ.

PubMed

Finney, John W; Humphreys, Keith; Kivlahan, Daniel R; Harris, Alex H S

2016-04-01

Studies finding weak or nonexistent relationships between hospital performance on providing recommended care and hospital-level clinical outcomes raise questions about the value and validity of process of care performance measures. Such findings may cause clinicians to question the effectiveness of the care process presumably captured by the performance measure. However, one cannot infer from hospital-level results whether patients who received the specified care had comparable, worse or superior outcomes relative to patients not receiving that care. To make such an inference has been labeled the "ecological fallacy," an error that is well known among epidemiologists and sociologists, but less so among health care researchers and policy makers. We discuss such inappropriate inferences in the health care performance measurement field and illustrate how and why process measure-outcome relationships can differ at the patient and hospital levels. We also offer recommendations for appropriate multilevel analyses to evaluate process measure-outcome relationships at the patient and hospital levels and for a more effective role for performance measure bodies and research funding organizations in encouraging such multilevel analyses.

Progress & Frontiers in PV Performance

DOE Office of Scientific and Technical Information (OSTI.GOV)

Deline, Chris; DiOrio, Nick; Jordan, Dirk

2016-09-12

PowerPoint slides for a presentation given at Solar Power International 2016. Presentation includes System Advisor Model (SAM) introduction and battery modeling, bifacial PV modules and modeling, shade modeling and module level power electronics (MLPE), degradation rates, and PVWatts updates and validation.

A multicenter prospective cohort study on camera navigation training for key user groups in minimally invasive surgery.

PubMed

Graafland, Maurits; Bok, Kiki; Schreuder, Henk W R; Schijven, Marlies P

2014-06-01

Untrained laparoscopic camera assistants in minimally invasive surgery (MIS) may cause suboptimal view of the operating field, thereby increasing risk for errors. Camera navigation is often performed by the least experienced member of the operating team, such as inexperienced surgical residents, operating room nurses, and medical students. The operating room nurses and medical students are currently not included as key user groups in structured laparoscopic training programs. A new virtual reality laparoscopic camera navigation (LCN) module was specifically developed for these key user groups. This multicenter prospective cohort study assesses face validity and construct validity of the LCN module on the Simendo virtual reality simulator. Face validity was assessed through a questionnaire on resemblance to reality and perceived usability of the instrument among experts and trainees. Construct validity was assessed by comparing scores of groups with different levels of experience on outcome parameters of speed and movement proficiency. The results obtained show uniform and positive evaluation of the LCN module among expert users and trainees, signifying face validity. Experts and intermediate experience groups performed significantly better in task time and camera stability during three repetitions, compared to the less experienced user groups (P < .007). Comparison of learning curves showed significant improvement of proficiency in time and camera stability for all groups during three repetitions (P < .007). The results of this study show face validity and construct validity of the LCN module. The module is suitable for use in training curricula for operating room nurses and novice surgical trainees, aimed at improving team performance in minimally invasive surgery. © The Author(s) 2013.

Examining the validity of self-reports on scales measuring students' strategic processing.

PubMed

Samuelstuen, Marit S; Bråten, Ivar

2007-06-01

Self-report inventories trying to measure strategic processing at a global level have been much used in both basic and applied research. However, the validity of global strategy scores is open to question because such inventories assess strategy perceptions outside the context of specific task performance. The primary aim was to examine the criterion-related and construct validity of the global strategy data obtained with the Cross-Curricular Competencies (CCC) scale. Additionally, we wanted to compare the validity of these data with the validity of data obtained with a task-specific self-report inventory focusing on the same types of strategies. The sample included 269 10th-grade students from 12 different junior high schools. Global strategy use as assessed with the CCC was compared with task-specific strategy use reported in three different reading situations. Moreover, relationships between scores on the CCC and scores on measures of text comprehension were examined and compared with relationships between scores on the task-specific strategy measure and the same comprehension measures. The comparison between the CCC strategy scores and the task-specific strategy scores suggested only modest criterion-related validity for the data obtained with the global strategy inventory. The CCC strategy scores were also not related to the text comprehension measures, indicating poor construct validity. In contrast, the task-specific strategy scores were positively related to the comprehension measures, indicating good construct validity. Attempts to measure strategic processing at a global level seem to have limited validity and utility.

Construction of a web-based questionnaire for longitudinal investigation of work exposure, musculoskeletal pain and performance impairments in high-performance marine craft populations

PubMed Central

de Alwis, Manudul Pahansen; Äng, Björn Olov; Garme, Karl

2017-01-01

Objective High-performance marine craft personnel (HPMCP) are regularly exposed to vibration and repeated shock (VRS) levels exceeding maximum limitations stated by international legislation. Whereas such exposure reportedly is detrimental to health and performance, the epidemiological data necessary to link these adverse effects causally to VRS are not available in the scientific literature, and no suitable tools for acquiring such data exist. This study therefore constructed a questionnaire for longitudinal investigations in HPMCP. Methods A consensus panel defined content domains, identified relevant items and outlined a questionnaire. The relevance and simplicity of the questionnaire’s content were then systematically assessed by expert raters in three consecutive stages, each followed by revisions. An item-level content validity index (I-CVI) was computed as the proportion of experts rating an item as relevant and simple, and a scale-level content validity index (S-CVI/Ave) as the average I-CVI across items. The thresholds for acceptable content validity were 0.78 and 0.90, respectively. Finally, a dynamic web version of the questionnaire was constructed and pilot tested over a 1-month period during a marine exercise in a study population sample of eight subjects, while accelerometers simultaneously quantified VRS exposure. Results Content domains were defined as work exposure, musculoskeletal pain and human performance, and items were selected to reflect these constructs. Ratings from nine experts yielded S-CVI/Ave of 0.97 and 1.00 for relevance and simplicity, respectively, and the pilot test suggested that responses were sensitive to change in acceleration and that the questionnaire, following some adjustments, was feasible for its intended purpose. Conclusions A dynamic web-based questionnaire for longitudinal survey of key variables in HPMCP was constructed. Expert ratings supported that the questionnaire content is relevant, simple and sufficiently comprehensive, and the pilot test suggested that the questionnaire is feasible for longitudinal measurements in the study population. PMID:28729320

Technical skills assessment toolbox: a review using the unitary framework of validity.

PubMed

Ghaderi, Iman; Manji, Farouq; Park, Yoon Soo; Juul, Dorthea; Ott, Michael; Harris, Ilene; Farrell, Timothy M

2015-02-01

The purpose of this study was to create a technical skills assessment toolbox for 35 basic and advanced skills/procedures that comprise the American College of Surgeons (ACS)/Association of Program Directors in Surgery (APDS) surgical skills curriculum and to provide a critical appraisal of the included tools, using contemporary framework of validity. Competency-based training has become the predominant model in surgical education and assessment of performance is an essential component. Assessment methods must produce valid results to accurately determine the level of competency. A search was performed, using PubMed and Google Scholar, to identify tools that have been developed for assessment of the targeted technical skills. A total of 23 assessment tools for the 35 ACS/APDS skills modules were identified. Some tools, such as Operative Performance Rating System (OSATS) and Objective Structured Assessment of Technical Skill (OPRS), have been tested for more than 1 procedure. Therefore, 30 modules had at least 1 assessment tool, with some common surgical procedures being addressed by several tools. Five modules had none. Only 3 studies used Messick's framework to design their validity studies. The remaining studies used an outdated framework on the basis of "types of validity." When analyzed using the contemporary framework, few of these studies demonstrated validity for content, internal structure, and relationship to other variables. This study provides an assessment toolbox for common surgical skills/procedures. Our review shows that few authors have used the contemporary unitary concept of validity for development of their assessment tools. As we progress toward competency-based training, future studies should provide evidence for various sources of validity using the contemporary framework.

V & V Within Reuse-Based Software Engineering

NASA Technical Reports Server (NTRS)

Addy, Edward A.

1996-01-01

Verification and validation (V&V) is used to increase the level of assurance of critical software, particularly that of safety-critical and mission critical software. This paper describes the working group's success in identifying V&V tasks that could be performed in the domain engineering and transition levels of reuse-based software engineering. The primary motivation for V&V at the domain level is to provide assurance that the domain requirements are correct and that the domain artifacts correctly implement the domain requirements. A secondary motivation is the possible elimination of redundant V&V activities at the application level. The group also considered the criteria and motivation for performing V&V in domain engineering.

Calibration of Clinical Audio Recording and Analysis Systems for Sound Intensity Measurement.

PubMed

Maryn, Youri; Zarowski, Andrzej

2015-11-01

Sound intensity is an important acoustic feature of voice/speech signals. Yet recordings are performed with different microphone, amplifier, and computer configurations, and it is therefore crucial to calibrate sound intensity measures of clinical audio recording and analysis systems on the basis of output of a sound-level meter. This study was designed to evaluate feasibility, validity, and accuracy of calibration methods, including audiometric speech noise signals and human voice signals under typical speech conditions. Calibration consisted of 3 comparisons between data from 29 measurement microphone-and-computer systems and data from the sound-level meter: signal-specific comparison with audiometric speech noise at 5 levels, signal-specific comparison with natural voice at 3 levels, and cross-signal comparison with natural voice at 3 levels. Intensity measures from recording systems were then linearly converted into calibrated data on the basis of these comparisons, and validity and accuracy of calibrated sound intensity were investigated. Very strong correlations and quasisimilarity were found between calibrated data and sound-level meter data across calibration methods and recording systems. Calibration of clinical sound intensity measures according to this method is feasible, valid, accurate, and representative for a heterogeneous set of microphones and data acquisition systems in real-life circumstances with distinct noise contexts.

Validity of the Nike+ device during walking and running.

PubMed

Kane, N A; Simmons, M C; John, D; Thompson, D L; Bassett, D R; Basset, D R

2010-02-01

We determined the validity of the Nike+ device for estimating speed, distance, and energy expenditure (EE) during walking and running. Twenty trained individuals performed a maximal oxygen uptake test and underwent anthropometric and body composition testing. Each participant was outfitted with a Nike+ sensor inserted into the shoe and an Apple iPod nano. They performed eight 6-min stages on the treadmill, including level walking at 55, 82, and 107 m x min(-1), inclined walking (82 m x min(-1)) at 5 and 10% grades, and level running at 134, 161, and 188 m x min(-1). Speed was measured using a tachometer and EE was measured by indirect calorimetry. Results showed that the Nike+ device overestimated the speed of level walking at 55 m x min(-1) by 20%, underestimated the speed of level walking at 107 m x min(-1) by 12%, but closely estimated the speed of level walking at 82 m x min(-1), and level running at all speeds (p<0.05). Similar results were found for distance. The Nike+ device overestimated the EE of level walking by 18-37%, but closely estimated the EE of level running (p<0.05). In conclusion the Nike+ in-shoe device provided reasonable estimates of speed and distance during level running at the three speeds tested in this study. However, it overestimated EE during level walking and it did not detect the increased cost of inclined locomotion.

Field validation of protocols developed to evaluate in-line mastitis detection systems.

PubMed

Kamphuis, C; Dela Rue, B T; Eastwood, C R

2016-02-01

This paper reports on a field validation of previously developed protocols for evaluating the performance of in-line mastitis-detection systems. The protocols outlined 2 requirements of these systems: (1) to detect cows with clinical mastitis (CM) promptly and accurately to enable timely and appropriate treatment and (2) to identify cows with high somatic cell count (SCC) to manage bulk milk SCC levels. Gold standard measures, evaluation tests, performance measures, and performance targets were proposed. The current study validated the protocols on commercial dairy farms with automated in-line mastitis-detection systems using both electrical conductivity (EC) and SCC sensor systems that both monitor at whole-udder level. The protocol for requirement 1 was applied on 3 commercial farms. For requirement 2, the protocol was applied on 6 farms; 3 of them had low bulk milk SCC (128×10(3) cells/mL) and were the same farms as used for field evaluation of requirement 1. Three farms with high bulk milk SCC (270×10(3) cells/mL) were additionally enrolled. The field evaluation methodology and results were presented at a workshop including representation from 7 international suppliers of in-line mastitis-detection systems. Feedback was sought on the acceptance of standardized performance evaluation protocols and recommended refinements to the protocols. Although the methodology for requirement 1 was relatively labor intensive and required organizational skills over an extended period, no major issues were encountered during the field validation of both protocols. The validation, thus, proved the protocols to be practical. Also, no changes to the data collection process were recommended by the technology supplier representatives. However, 4 recommendations were made to refine the protocols: inclusion of an additional analysis that ignores small (low-density) clot observations in the definition of CM, extension of the time window from 4 to 5 milkings for timely alerts for CM, setting a maximum number of 10 milkings for the time window to detect a CM episode, and presentation of sensitivity for a larger range of false alerts per 1,000 milkings replacing minimum performance targets. The recommended refinements are discussed with suggested changes to the original protocols. The information presented is intended to inform further debate toward achieving international agreement on standard protocols to evaluate performance of in-line mastitis-detection systems. Copyright © 2016 American Dairy Science Association. Published by Elsevier Inc. All rights reserved.

GPM Ground Validation: Pre to Post-Launch Era

NASA Astrophysics Data System (ADS)

Petersen, Walt; Skofronick-Jackson, Gail; Huffman, George

2015-04-01

NASA GPM Ground Validation (GV) activities have transitioned from the pre to post-launch era. Prior to launch direct validation networks and associated partner institutions were identified world-wide, covering a plethora of precipitation regimes. In the U.S. direct GV efforts focused on use of new operational products such as the NOAA Multi-Radar Multi-Sensor suite (MRMS) for TRMM validation and GPM radiometer algorithm database development. In the post-launch, MRMS products including precipitation rate, accumulation, types and data quality are being routinely generated to facilitate statistical GV of instantaneous (e.g., Level II orbit) and merged (e.g., IMERG) GPM products. Toward assessing precipitation column impacts on product uncertainties, range-gate to pixel-level validation of both Dual-Frequency Precipitation Radar (DPR) and GPM microwave imager data are performed using GPM Validation Network (VN) ground radar and satellite data processing software. VN software ingests quality-controlled volumetric radar datasets and geo-matches those data to coincident DPR and radiometer level-II data. When combined MRMS and VN datasets enable more comprehensive interpretation of both ground and satellite-based estimation uncertainties. To support physical validation efforts eight (one) field campaigns have been conducted in the pre (post) launch era. The campaigns span regimes from northern latitude cold-season snow to warm tropical rain. Most recently the Integrated Precipitation and Hydrology Experiment (IPHEx) took place in the mountains of North Carolina and involved combined airborne and ground-based measurements of orographic precipitation and hydrologic processes underneath the GPM Core satellite. One more U.S. GV field campaign (OLYMPEX) is planned for late 2015 and will address cold-season precipitation estimation, process and hydrology in the orographic and oceanic domains of western Washington State. Finally, continuous direct and physical validation measurements are also being conducted at the NASA Wallops Flight Facility multi-radar, gauge and disdrometer facility located in coastal Virginia. This presentation will summarize the evolution of the NASA GPM GV program from pre to post-launch eras and place focus on evaluation of year-1 post-launch GPM satellite datasets including Level II GPROF, DPR and Combined algorithms, and Level III IMERG products.

Enhanced high-performance liquid chromatography method for the determination of retinoic acid in plasma. Development, optimization and validation.

PubMed

Teglia, Carla M; Gil García, María D; Galera, María Martínez; Goicoechea, Héctor C

2014-08-01

When determining endogenous compounds in biological samples, the lack of blank or analyte-free matrix samples involves the use of alternative strategies for calibration and quantitation. This article deals with the development, optimization and validation of a high performance liquid chromatography method for the determination of retinoic acid in plasma, obtaining at the same time information about its isomers, taking into account the basal concentration of these endobiotica. An experimental design was used for the optimization of three variables: mobile phase composition, flow rate and column temperature through a central composite design. Four responses were selected for optimization purposes (area under the peaks, quantity of peaks, analysis time and resolution between the first principal peak and the following one). The optimum conditions resulted in a mobile phase consisting of methanol 83.4% (v/v), acetonitrile 0.6% (v/v) and acid aqueous solution 16.0% (v/v); flow rate of 0.68 mL min(-1) and an column temperature of 37.10 °C. Detection was performed at 350 nm by a diode array detector. The method was validated following a holistic approach that included not only the classical parameters related to method performance but also the robustness and the expected proportion of acceptable results lying inside predefined acceptability intervals, i.e., the uncertainty of measurements. The method validation results indicated a high selectivity and good precision characteristics that were studied at four concentration levels, with RSD less than 5.0% for retinoic acid (less than 7.5% for the LOQ concentration level), in intra and inter-assay precision studies. Linearity was proved for a range from 0.00489 to 15.109 ng mL(-1) of retinoic acid and the recovery, which was studied at four different fortification levels in phuman plasma samples, varied from 99.5% to 106.5% for retinoic acid. The applicability of the method was demonstrated by determining retinoic acid and obtaining information about its isomers in human and frog plasma samples from different origins. Copyright © 2014 Elsevier B.V. All rights reserved.

Formulating Spatially Varying Performance in the Statistical Fusion Framework

PubMed Central

Landman, Bennett A.

2012-01-01

To date, label fusion methods have primarily relied either on global (e.g. STAPLE, globally weighted vote) or voxelwise (e.g. locally weighted vote) performance models. Optimality of the statistical fusion framework hinges upon the validity of the stochastic model of how a rater errs (i.e., the labeling process model). Hitherto, approaches have tended to focus on the extremes of potential models. Herein, we propose an extension to the STAPLE approach to seamlessly account for spatially varying performance by extending the performance level parameters to account for a smooth, voxelwise performance level field that is unique to each rater. This approach, Spatial STAPLE, provides significant improvements over state-of-the-art label fusion algorithms in both simulated and empirical data sets. PMID:22438513

British isles lupus assessment group 2004 index is valid for assessment of disease activity in systemic lupus erythematosus

PubMed Central

Yee, Chee-Seng; Farewell, Vernon; Isenberg, David A; Rahman, Anisur; Teh, Lee-Suan; Griffiths, Bridget; Bruce, Ian N; Ahmad, Yasmeen; Prabu, Athiveeraramapandian; Akil, Mohammed; McHugh, Neil; D'Cruz, David; Khamashta, Munther A; Maddison, Peter; Gordon, Caroline

2007-01-01

Objective To determine the construct and criterion validity of the British Isles Lupus Assessment Group 2004 (BILAG-2004) index for assessing disease activity in systemic lupus erythematosus (SLE). Methods Patients with SLE were recruited into a multicenter cross-sectional study. Data on SLE disease activity (scores on the BILAG-2004 index, Classic BILAG index, and Systemic Lupus Erythematosus Disease Activity Index 2000 [SLEDAI-2K]), investigations, and therapy were collected. Overall BILAG-2004 and overall Classic BILAG scores were determined by the highest score achieved in any of the individual systems in the respective index. Erythrocyte sedimentation rates (ESRs), C3 levels, C4 levels, anti–double-stranded DNA (anti-dsDNA) levels, and SLEDAI-2K scores were used in the analysis of construct validity, and increase in therapy was used as the criterion for active disease in the analysis of criterion validity. Statistical analyses were performed using ordinal logistic regression for construct validity and logistic regression for criterion validity. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated. Results Of the 369 patients with SLE, 92.7% were women, 59.9% were white, 18.4% were Afro-Caribbean and 18.4% were South Asian. Their mean ± SD age was 41.6 ± 13.2 years and mean disease duration was 8.8 ± 7.7 years. More than 1 assessment was obtained on 88.6% of the patients, and a total of 1,510 assessments were obtained. Increasing overall scores on the BILAG-2004 index were associated with increasing ESRs, decreasing C3 levels, decreasing C4 levels, elevated anti-dsDNA levels, and increasing SLEDAI-2K scores (all P < 0.01). Increase in therapy was observed more frequently in patients with overall BILAG-2004 scores reflecting higher disease activity. Scores indicating active disease (overall BILAG-2004 scores of A and B) were significantly associated with increase in therapy (odds ratio [OR] 19.3, P < 0.01). The BILAG-2004 and Classic BILAG indices had comparable sensitivity, specificity, PPV, and NPV. Conclusion These findings show that the BILAG-2004 index has construct and criterion validity. PMID:18050213

How Non-Linearity and Grade-Level Differences Complicate the Validation of Observation Protocols

ERIC Educational Resources Information Center

Lazarev, Valeriy; Newman, Denis

2013-01-01

Teacher evaluation is currently a major policy issue at all levels of the K-12 system driven in large part by current US Department of Education requirements. The main objective of this study is to explore the patterns of relationship between observational scores and value-added measures of teacher performance in math classrooms and the variation…

The Relationship between Lexical Frequency Profiling Measures and Rater Judgements of Spoken and Written General English Language Proficiency on the CELPIP-General Test

ERIC Educational Resources Information Center

Douglas, Scott Roy

2015-01-01

Independent confirmation that vocabulary in use unfolds across levels of performance as expected can contribute to a more complete understanding of validity in standardized English language tests. This study examined the relationship between Lexical Frequency Profiling (LFP) measures and rater judgements of test-takers' overall levels of…

Validity of a self-administered food frequency questionnaire in the estimation of heterocyclic aromatic amines.

PubMed

Iwasaki, Motoki; Mukai, Tomomi; Takachi, Ribeka; Ishihara, Junko; Totsuka, Yukari; Tsugane, Shoichiro

2014-08-01

Clarification of the putative etiologic role of heterocyclic aromatic amines (HAAs) in the development of cancer requires a validated assessment tool for dietary HAAs. This study primarily aimed to evaluate the validity of a food frequency questionnaire (FFQ) in estimating HAA intake, using 2-amino-1-methyl-6-phenylimidazo[4,5-b]pyridine (PhIP) level in human hair as the reference method. We first updated analytical methods of PhIP using liquid chromatography-electrospray ionization/tandem mass spectrometry (LC-ESI/MS/MS) and measured 44 fur samples from nine rats from a feeding study as part-verification of the quantitative performance of LC-ESI/MS/MS. We next measured PhIP level in human hair samples from a validation study of the FFQ (n = 65). HAA intake from the FFQ was estimated using information on intake from six fish items and seven meat items and data on HAA content in each food item. Correlation coefficients between PhIP level in human hair and HAA intake from the FFQ were calculated. The animal feeding study of PhIP found a significant dose-response relationship between dosage and PhIP in rat fur. Mean level was 53.8 pg/g hair among subjects with values over the limit of detection (LOD) (n = 57). We found significant positive correlation coefficients between PhIP in human hair and HAA intake from the FFQ, with Spearman rank correlation coefficients of 0.35 for all subjects, 0.21 for subjects with over LOD values, and 0.34 for subjects with over limit of quantification. Findings from the validation study suggest that the FFQ is reasonably valid for the assessment of HAA intake.

Estimating groundwater levels using system identification models in Nzhelele and Luvuvhu areas, Limpopo Province, South Africa

NASA Astrophysics Data System (ADS)

Makungo, Rachel; Odiyo, John O.

2017-08-01

This study was focused on testing the ability of a coupled linear and non-linear system identification model in estimating groundwater levels. System identification provides an alternative approach for estimating groundwater levels in areas that lack data required by physically-based models. It also overcomes the limitations of physically-based models due to approximations, assumptions and simplifications. Daily groundwater levels for 4 boreholes, rainfall and evaporation data covering the period 2005-2014 were used in the study. Seventy and thirty percent of the data were used to calibrate and validate the model, respectively. Correlation coefficient (R), coefficient of determination (R2), root mean square error (RMSE), percent bias (PBIAS), Nash Sutcliffe coefficient of efficiency (NSE) and graphical fits were used to evaluate the model performance. Values for R, R2, RMSE, PBIAS and NSE ranged from 0.8 to 0.99, 0.63 to 0.99, 0.01-2.06 m, -7.18 to 1.16 and 0.68 to 0.99, respectively. Comparisons of observed and simulated groundwater levels for calibration and validation runs showed close agreements. The model performance mostly varied from satisfactory, good, very good and excellent. Thus, the model is able to estimate groundwater levels. The calibrated models can reasonably capture description between input and output variables and can, thus be used to estimate long term groundwater levels.

Making Decisions about Adult Learners Based on Performances on Functional Competency Measures.

ERIC Educational Resources Information Center

Bunch, Michael B.

The validity and dependability of functional competency tests for adults are examined as they relate to the information needs of instructional decision makers. Test data from the Adult Performance Level (APL) Program (funded by the U.S. Office of Education at the University of Texas at Austin) is used to illustrate key points. In the discussion of…

The Florida Community College Accountability Plan: An Analysis of Institutional Characteristics and Success at Meeting State Defined Performance Measures.

ERIC Educational Resources Information Center

Windham, Patricia W.; Hackett, E. Raymond

In response to the increasing use of state-based performance indicators for postsecondary education, a study was undertaken to review the reliability and validity of state-level indicators in the Florida Community College System (FCCS). Data were collected from literature reviews and the 1996 FCCS Accountability Report, detailing outcomes for 17…

Testing Math or Testing Language? The Construct Validity of the KeyMath-Revised for Children with Intellectual Disability and Language Difficulties

ERIC Educational Resources Information Center

Rhodes, Katherine T.; Branum-Martin, Lee; Morris, Robin D.; Romski, MaryAnn; Sevcik, Rose A.

2015-01-01

Although it is often assumed that mathematics ability alone predicts mathematics test performance, linguistic demands may also predict achievement. This study examined the role of language in mathematics assessment performance for children with intellectual disability (ID) at less severe levels, on the KeyMath-Revised Inventory (KM-R) with a…

Method for the determination of catechin and epicatechin enantiomers in cocoa-based ingredients and products by high-performance liquid chromatography: single-laboratory validation.

PubMed

Machonis, Philip R; Jones, Matthew A; Schaneberg, Brian T; Kwik-Uribe, Catherine L

2012-01-01

A single-laboratory validation study was performed for an HPLC method to identify and quantify the flavanol enantiomers (+)- and (-)-epicatechin and (+)- and (-)-catechin in cocoa-based ingredients and products. These compounds were eluted isocratically with an ammonium acetate-methanol mobile phase applied to a modified beta-cyclodextrin chiral stationary phase and detected using fluorescence. Spike recovery experiments using appropriate matrix blanks, along with cocoa extract, cocoa powder, and dark chocolate, were used to evaluate accuracy, repeatability, specificity, LOD, LOQ, and linearity of the method as performed by a single analyst on multiple days. In all samples analyzed, (-)-epicatechin was the predominant flavanol and represented 68-91% of the total monomeric flavanols detected. For the cocoa-based products, within-day (intraday) precision for (-)-epicatechin was between 1.46-3.22%, for (+)-catechin between 3.66-6.90%, and for (-)-catechin between 1.69-6.89%; (+)-epicatechin was not detected in these samples. Recoveries for the three sample types investigated ranged from 82.2 to 102.1% at the 50% spiking level, 83.7 to 102.0% at the 100% spiking level, and 80.4 to 101.1% at the 200% spiking level. Based on performance results, this method may be suitable for routine laboratory use in analysis of cocoa-based ingredients and products.

Physical performance tests after stroke: reliability and validity.

PubMed

Maeda, A; Yuasa, T; Nakamura, K; Higuchi, S; Motohashi, Y

2000-01-01

To evaluate the reliability and validity of the modified physical performance tests for stroke survivors who live in a community. The subjects included 40 stroke survivors and 40 apparently healthy independent elderly persons. The physical performance tests for the stroke survivors comprised two physical capacity evaluation tasks that represented physical abilities necessary to perform the main activities of daily living, e.g., standing-up ability (time needed to stand up from bed rest) and walking ability (time needed to walk 10 m). Regarding the reliability of tests, significant correlations were confirmed between test and retest of physical performance tests with both short and long intervals in individuals after stroke. Regarding the validity of tests, the authors studied the significant correlations between the maximum isometric strength of the quardriceps muscle and the time needed to walk 10 m, centimeters reached while sitting and reaching, and the time needed to stand up from bed rest. The authors confirmed that there were significant correlations between the instrumental activity of daily living and the time needed to stand up from bed rest, along with the time needed to walk 10 m for the stroke survivors. These physical performance tests are useful guides for evaluating a level of activity of daily living and physical frailty of stroke survivors living in a community.

Tests for the Assessment of Sport-Specific Performance in Olympic Combat Sports: A Systematic Review With Practical Recommendations.

PubMed

Chaabene, Helmi; Negra, Yassine; Bouguezzi, Raja; Capranica, Laura; Franchini, Emerson; Prieske, Olaf; Hbacha, Hamdi; Granacher, Urs

2018-01-01

The regular monitoring of physical fitness and sport-specific performance is important in elite sports to increase the likelihood of success in competition. This study aimed to systematically review and to critically appraise the methodological quality, validation data, and feasibility of the sport-specific performance assessment in Olympic combat sports like amateur boxing, fencing, judo, karate, taekwondo, and wrestling. A systematic search was conducted in the electronic databases PubMed, Google-Scholar, and Science-Direct up to October 2017. Studies in combat sports were included that reported validation data (e.g., reliability, validity, sensitivity) of sport-specific tests. Overall, 39 studies were eligible for inclusion in this review. The majority of studies (74%) contained sample sizes <30 subjects. Nearly, 1/3 of the reviewed studies lacked a sufficient description (e.g., anthropometrics, age, expertise level) of the included participants. Seventy-two percent of studies did not sufficiently report inclusion/exclusion criteria of their participants. In 62% of the included studies, the description and/or inclusion of a familiarization session (s) was either incomplete or not existent. Sixty-percent of studies did not report any details about the stability of testing conditions. Approximately half of the studies examined reliability measures of the included sport-specific tests (intraclass correlation coefficient [ICC] = 0.43-1.00). Content validity was addressed in all included studies, criterion validity (only the concurrent aspect of it) in approximately half of the studies with correlation coefficients ranging from r = -0.41 to 0.90. Construct validity was reported in 31% of the included studies and predictive validity in only one. Test sensitivity was addressed in 13% of the included studies. The majority of studies (64%) ignored and/or provided incomplete information on test feasibility and methodological limitations of the sport-specific test. In 28% of the included studies, insufficient information or a complete lack of information was provided in the respective field of the test application. Several methodological gaps exist in studies that used sport-specific performance tests in Olympic combat sports. Additional research should adopt more rigorous validation procedures in the application and description of sport-specific performance tests in Olympic combat sports.

Initial interlaboratory validation of an analytical method for the determination of lead in canned tuna to be used for monitoring and regulatory purposes.

PubMed

Santiago, E C; Bello, F B B

2003-06-01

The Association of Official Analytical Chemists (AOAC) Standard Method 972.23 (dry ashing and flame atomic absorption spectrophotometry (FAAS)), applied to the analysis of lead in tuna, was validated in three selected local laboratories to determine the acceptability of the method to both the Codex Alimentarius Commission (Codex) and the European Union (EU) Commission for monitoring lead in canned tuna. Initial validation showed that the standard AOAC method as performed in the three participating laboratories cannot satisfy the Codex/EU proposed criteria for the method detection limit for monitoring lead in fish at the present regulation level of 0.5 mg x kg(-1). Modification of the standard method by chelation/concentration of the digest solution before FAAS analysis showed that the modified method has the potential to meet Codex/EU criteria on sensitivity, accuracy and precision at the specified regulation level.

A need for an augmented review when reviewing rehabilitation research.

PubMed

Gerber, Lynn H; Nava, Andrew; Garfinkel, Steven; Goel, Divya; Weinstein, Ali A; Cai, Cindy

2016-10-01

There is a need for additional strategies for performing systematic reviews (SRs) to improve translation of findings into practice and to influence health policy. SRs critically appraise research methodology and determine level of evidence of research findings. The standard type of SR identifies randomized controlled trials (RCTs) as providing the most valid data and highest level of evidence. RCTs are not among the most frequently used research design in disability and health research. RCTs usually measure impairments for the primary research outcome rather than improved function, participation or societal integration. It forces a choice between "validity" and "utility/relevance." Other approaches have effectively been used to assess the validity of alternative research designs, whose outcomes focus on function and patient-reported outcomes. We propose that utilizing existing evaluation tools that measure knowledge, dissemination and utility of findings, may help improve the translation of findings into practice and health policy. Copyright © 2016 Elsevier Inc. All rights reserved.

The construct and criterion validity of the multi-source feedback process to assess physician performance: a meta-analysis

PubMed Central

Al Ansari, Ahmed; Donnon, Tyrone; Al Khalifa, Khalid; Darwish, Abdulla; Violato, Claudio

2014-01-01

Background The purpose of this study was to conduct a meta-analysis on the construct and criterion validity of multi-source feedback (MSF) to assess physicians and surgeons in practice. Methods In this study, we followed the guidelines for the reporting of observational studies included in a meta-analysis. In addition to PubMed and MEDLINE databases, the CINAHL, EMBASE, and PsycINFO databases were searched from January 1975 to November 2012. All articles listed in the references of the MSF studies were reviewed to ensure that all relevant publications were identified. All 35 articles were independently coded by two authors (AA, TD), and any discrepancies (eg, effect size calculations) were reviewed by the other authors (KA, AD, CV). Results Physician/surgeon performance measures from 35 studies were identified. A random-effects model of weighted mean effect size differences (d) resulted in: construct validity coefficients for the MSF system on physician/surgeon performance across different levels in practice ranged from d=0.14 (95% confidence interval [CI] 0.40–0.69) to d=1.78 (95% CI 1.20–2.30); construct validity coefficients for the MSF on physician/surgeon performance on two different occasions ranged from d=0.23 (95% CI 0.13–0.33) to d=0.90 (95% CI 0.74–1.10); concurrent validity coefficients for the MSF based on differences in assessor group ratings ranged from d=0.50 (95% CI 0.47–0.52) to d=0.57 (95% CI 0.55–0.60); and predictive validity coefficients for the MSF on physician/surgeon performance across different standardized measures ranged from d=1.28 (95% CI 1.16–1.41) to d=1.43 (95% CI 0.87–2.00). Conclusion The construct and criterion validity of the MSF system is supported by small to large effect size differences based on the MSF process and physician/surgeon performance across different clinical and nonclinical domain measures. PMID:24600300

Stressors, academic performance, and learned resourcefulness in baccalaureate nursing students.

PubMed

Goff, Anne-Marie

2011-01-01

High stress levels in nursing students may affect memory, concentration, and problem-solving ability, and may lead to decreased learning, coping, academic performance, and retention. College students with higher levels of learned resourcefulness develop greater self-confidence, motivation, and academic persistence, and are less likely to become anxious, depressed, and frustrated, but no studies specifically involve nursing students. This explanatory correlational study used Gadzella's Student-life Stress Inventory (SSI) and Rosenbaum's Self Control Scale (SCS) to explore learned resourcefulness, stressors, and academic performance in 53 baccalaureate nursing students. High levels of personal and academic stressors were evident, but not significant predictors of academic performance (p = .90). Age was a significant predictor of academic performance (p = < .01) and males and African-American/Black participants had higher learned resourcefulness scores than females and Caucasians. Studies in larger, more diverse samples are necessary to validate these findings.

Diagnostic Tools for Performance Evaluation of Innovative In-Situ Remediation Technologies at Chlorinated Solvent-Contaminated Sites

DTIC Science & Technology

2011-07-01

to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. PLEASE DO NOT...these innovative methods with conventional diagnostic tools that are currently used for assessing bioremediation performance. 132 Rula Deeb (510) 596...conventional diagnostic tools that are currently used for assessing bioremediation performance. DEMONSTRATION RESULTS 3-D multi-level systems

Validation studies and proficiency testing.

PubMed

Ankilam, Elke; Heinze, Petra; Kay, Simon; Van den Eede, Guy; Popping, Bert

2002-01-01

Genetically modified organisms (GMOs) entered the European food market in 1996. Current legislation demands the labeling of food products if they contain <1% GMO, as assessed for each ingredient of the product. To create confidence in the testing methods and to complement enforcement requirements, there is an urgent need for internationally validated methods, which could serve as reference methods. To date, several methods have been submitted to validation trials at an international level; approaches now exist that can be used in different circumstances and for different food matrixes. Moreover, the requirement for the formal validation of methods is clearly accepted; several national and international bodies are active in organizing studies. Further validation studies, especially on the quantitative polymerase chain reaction methods, need to be performed to cover the rising demand for new extraction methods and other background matrixes, as well as for novel GMO constructs.

SeaWiFS Technical Report Series. Volume 38; SeaWiFS Calibration and Validation Quality Control Procedures

NASA Technical Reports Server (NTRS)

Hooker, Stanford B. (Editor); Firestone, Elaine R. (Editor); McClain, Charles R.; Darzi, Michael; Barnes, Robert A.; Eplee, Robert E.; Firestone, James K.; Patt, Frederick S.; Robinson, Wayne D.; Schieber, Brian D.;

Identifying Outliers of Non-Gaussian Groundwater State Data Based on Ensemble Estimation for Long-Term Trends

NASA Astrophysics Data System (ADS)

Park, E.; Jeong, J.; Choi, J.; Han, W. S.; Yun, S. T.

2016-12-01

Three modified outlier identification methods: the three sigma rule (3s), inter quantile range (IQR) and median absolute deviation (MAD), which take advantage of the ensemble regression method are proposed. For validation purposes, the performance of the methods is compared using simulated and actual groundwater data with a few hypothetical conditions. In the validations using simulated data, all of the proposed methods reasonably identify outliers at a 5% outlier level; whereas, only the IQR method performs well for identifying outliers at a 30% outlier level. When applying the methods to real groundwater data, the outlier identification performance of the IQR method is found to be superior to the other two methods. However, the IQR method is found to have a limitation in the false identification of excessive outliers, which may be supplemented by joint applications with the other methods (i.e., the 3s rule and MAD methods). The proposed methods can be also applied as a potential tool for future anomaly detection by model training based on currently available data.

Testing Math or Testing Language? The Construct Validity of the KeyMath-Revised for Children With Intellectual Disability and Language Difficulties.

PubMed

Rhodes, Katherine T; Branum-Martin, Lee; Morris, Robin D; Romski, MaryAnn; Sevcik, Rose A

2015-11-01

Although it is often assumed that mathematics ability alone predicts mathematics test performance, linguistic demands may also predict achievement. This study examined the role of language in mathematics assessment performance for children with intellectual disability (ID) at less severe levels, on the KeyMath-Revised Inventory (KM-R) with a sample of 264 children, in grades 2-5. Using confirmatory factor analysis, the hypothesis that the KM-R would demonstrate discriminant validity with measures of language abilities in a two-factor model was compared to two plausible alternative models. Results indicated that KM-R did not have discriminant validity with measures of children's language abilities and was a multidimensional test of both mathematics and language abilities for this population of test users. Implications are considered for test development, interpretation, and intervention.

Mental State Assessment and Validation Using Personalized Physiological Biometrics

PubMed Central

Patel, Aashish N.; Howard, Michael D.; Roach, Shane M.; Jones, Aaron P.; Bryant, Natalie B.; Robinson, Charles S. H.; Clark, Vincent P.; Pilly, Praveen K.

2018-01-01

Mental state monitoring is a critical component of current and future human-machine interfaces, including semi-autonomous driving and flying, air traffic control, decision aids, training systems, and will soon be integrated into ubiquitous products like cell phones and laptops. Current mental state assessment approaches supply quantitative measures, but their only frame of reference is generic population-level ranges. What is needed are physiological biometrics that are validated in the context of task performance of individuals. Using curated intake experiments, we are able to generate personalized models of three key biometrics as useful indicators of mental state; namely, mental fatigue, stress, and attention. We demonstrate improvements to existing approaches through the introduction of new features. Furthermore, addressing the current limitations in assessing the efficacy of biometrics for individual subjects, we propose and employ a multi-level validation scheme for the biometric models by means of k-fold cross-validation for discrete classification and regression testing for continuous prediction. The paper not only provides a unified pipeline for extracting a comprehensive mental state evaluation from a parsimonious set of sensors (only EEG and ECG), but also demonstrates the use of validation techniques in the absence of empirical data. Furthermore, as an example of the application of these models to novel situations, we evaluate the significance of correlations of personalized biometrics to the dynamic fluctuations of accuracy and reaction time on an unrelated threat detection task using a permutation test. Our results provide a path toward integrating biometrics into augmented human-machine interfaces in a judicious way that can help to maximize task performance.

Mental State Assessment and Validation Using Personalized Physiological Biometrics.

PubMed

Patel, Aashish N; Howard, Michael D; Roach, Shane M; Jones, Aaron P; Bryant, Natalie B; Robinson, Charles S H; Clark, Vincent P; Pilly, Praveen K

2018-01-01

Mental state monitoring is a critical component of current and future human-machine interfaces, including semi-autonomous driving and flying, air traffic control, decision aids, training systems, and will soon be integrated into ubiquitous products like cell phones and laptops. Current mental state assessment approaches supply quantitative measures, but their only frame of reference is generic population-level ranges. What is needed are physiological biometrics that are validated in the context of task performance of individuals. Using curated intake experiments, we are able to generate personalized models of three key biometrics as useful indicators of mental state; namely, mental fatigue, stress, and attention. We demonstrate improvements to existing approaches through the introduction of new features. Furthermore, addressing the current limitations in assessing the efficacy of biometrics for individual subjects, we propose and employ a multi-level validation scheme for the biometric models by means of k -fold cross-validation for discrete classification and regression testing for continuous prediction. The paper not only provides a unified pipeline for extracting a comprehensive mental state evaluation from a parsimonious set of sensors (only EEG and ECG), but also demonstrates the use of validation techniques in the absence of empirical data. Furthermore, as an example of the application of these models to novel situations, we evaluate the significance of correlations of personalized biometrics to the dynamic fluctuations of accuracy and reaction time on an unrelated threat detection task using a permutation test. Our results provide a path toward integrating biometrics into augmented human-machine interfaces in a judicious way that can help to maximize task performance.

The Persian Version of the "Life Satisfaction Scale": Construct Validity and Test-Re-Test Reliability among Iranian Older Adults.

PubMed

Moghadam, Manije; Salavati, Mahyar; Sahaf, Robab; Rassouli, Maryam; Moghadam, Mojgan; Kamrani, Ahmad Ali Akbari

2018-03-01

After forward-backward translation, the LSS was administered to 334 Persian speaking, cognitively healthy elderly aged 60 years and over recruited through convenience sampling. To analyze the validity of the model's constructs and the relationships between the constructs, a confirmatory factor analysis followed by PLS analysis was performed. The Construct validity was further investigated by calculating the correlations between the LSS and the "Short Form Health Survey" (SF-36) subscales measuring similar and dissimilar constructs. The LSS was re-administered to 50 participants a month later to assess the reliability. For the eight-factor model of the life satisfaction construct, adequate goodness of fit between the hypothesized model and the model derived from the sample data was attained (positive and statistically significant beta coefficients, good R-squares and acceptable GoF). Construct validity was supported by convergent and discriminant validity, and correlations between the LSS and SF-36 subscales. Minimum Intraclass Correlation Coefficient level of 0.60 was exceeded by all subscales. Minimum level of reliability indices (Cronbach's α, composite reliability and indicator reliability) was exceeded by all subscales. The Persian-version of the Life Satisfaction Scale is a reliable and valid instrument, with psychometric properties which are consistent with the original version.

Measuring comparative hospital performance.

PubMed

Griffith, John R; Alexander, Jeffrey A; Jelinek, Richard C

2002-01-01

Leading healthcare provider organizations now use a "balanced scorecard" of performance measures, expanding information reviewed at the governance level to include financial, customer, and internal performance information, as well as providing an opportunity to learn and grow to provide better strategic guidance. The approach, successfully used by other industries, uses competitor data and benchmarks to identify opportunities for improved mission achievement. This article evaluates one set of nine multidimensional hospital performance measures derived from Medicare reports (cash flow, asset turnover, mortality, complications, length of inpatient stay, cost per case, occupancy, change in occupancy, and percent of revenue from outpatient care). The study examines the content validity, reliability and sensitivity, validity of comparison, and independence and concludes that seven of the nine measures (all but the two occupancy measures) represent a potentially useful set for evaluating most U.S. hospitals. This set reflects correctable differences in performance between hospitals serving similar populations, that is, the measures reflect relative performance and identify opportunities to make the organization more successful.

Validation of the tool assessment of clinical education (AssCE): A study using Delphi method and clinical experts.

PubMed

Löfmark, Anna; Mårtensson, Gunilla

2017-03-01

The aim of the present study was to establish the validity of the tool Assessment of Clinical Education (AssCE). The tool is widely used in Sweden and some Nordic countries for assessing nursing students' performance in clinical education. It is important that the tools in use be subjected to regular audit and critical reviews. The validation process, performed in two stages, was concluded with a high level of congruence. In the first stage, Delphi technique was used to elaborate the AssCE tool using a group of 35 clinical nurse lecturers. After three rounds, we reached consensus. In the second stage, a group of 46 clinical nurse lecturers representing 12 universities in Sweden and Norway audited the revised version of the AssCE in relation to learning outcomes from the last clinical course at their respective institutions. Validation of the revised AssCE was established with high congruence between the factors in the AssCE and examined learning outcomes. The revised AssCE tool seems to meet its objective to be a validated assessment tool for use in clinical nursing education. Copyright © 2016 Elsevier Ltd. All rights reserved.

Laser metrology and optic active control system for GAIA

NASA Astrophysics Data System (ADS)

D'Angelo, F.; Bonino, L.; Cesare, S.; Castorina, G.; Mottini, S.; Bertinetto, F.; Bisi, M.; Canuto, E.; Musso, F.

2017-11-01

The Laser Metrology and Optic Active Control (LM&OAC) program has been carried out under ESA contract with the purpose to design and validate a laser metrology system and an actuation mechanism to monitor and control at microarcsec level the stability of the Basic Angle (angle between the lines of sight of the two telescopes) of GAIA satellite. As part of the program, a breadboard (including some EQM elements) of the laser metrology and control system has been built and submitted to functional, performance and environmental tests. In the followings we describe the mission requirements, the system architecture, the breadboard design, and finally the performed validation tests. Conclusion and appraisals from this experience are also reported.

Characterization of the faulted behavior of digital computers and fault tolerant systems

NASA Technical Reports Server (NTRS)

Bavuso, Salvatore J.; Miner, Paul S.

1989-01-01

A development status evaluation is presented for efforts conducted at NASA-Langley since 1977, toward the characterization of the latent fault in digital fault-tolerant systems. Attention is given to the practical, high speed, generalized gate-level logic system simulator developed, as well as to the validation methodology used for the simulator, on the basis of faultable software and hardware simulations employing a prototype MIL-STD-1750A processor. After validation, latency tests will be performed.

Development, validation and operating room-transfer of a six-step laparoscopic training program for the vesicourethral anastomosis.

PubMed

Klein, Jan; Teber, Dogu; Frede, Tom; Stock, Christian; Hruza, Marcel; Gözen, Ali; Seemann, Othmar; Schulze, Michael; Rassweiler, Jens

2013-03-01

Development and full validation of a laparoscopic training program for stepwise learning of a reproducible application of a standardized laparoscopic anastomosis technique and integration into the clinical course. The training of vesicourethral anastomosis (VUA) was divided into six simple standardized steps. To fix the objective criteria, four experienced surgeons performed the stepwise training protocol. Thirty-eight participants with no previous laparoscopic experience were investigated in their training performance. The times needed to manage each training step and the total training time were recorded. The integration into the clinical course was investigated. The training results and the corresponding steps during laparoscopic radical prostatectomy (LRP) were analyzed. Data analysis of corresponding operating room (OR) sections of 793 LRP was performed. Based on the validity, criteria were determined. In the laboratory section, a significant reduction of OR time for every step was seen in all participants. Coordination: 62%; longitudinal incision: 52%; inverted U-shape incision: 43%; plexus: 47%. Anastomosis catheter model: 38%. VUA: 38%. The laboratory section required a total time of 29 hours (minimum: 16 hours; maximum: 42 hours). All participants had shorter execution times in the laboratory than under real conditions. The best match was found within the VUA model. To perform an anastomosis under real conditions, 25% more time was needed. By using the training protocol, the performance of the VUA is comparable to that of an surgeon with experience of about 50 laparoscopic VUA. Data analysis proved content, construct, and prognostic validity. The use of stepwise training approaches enables a surgeon to learn and reproduce complex reconstructive surgical tasks: eg, the VUA in a safe environment. The validity of the designed system is given at all levels and should be used as a standard in the clinical surgical training in laparoscopic reconstructive urology.

Performance indicators for public mental healthcare: a systematic international inventory

PubMed Central

2012-01-01

Background The development and use of performance indicators (PI) in the field of public mental health care (PMHC) has increased rapidly in the last decade. To gain insight in the current state of PI for PMHC in nations and regions around the world, we conducted a structured review of publications in scientific peer-reviewed journals supplemented by a systematic inventory of PI published in policy documents by (non-) governmental organizations. Methods Publications on PI for PMHC were identified through database- and internet searches. Final selection was based on review of the full content of the publications. Publications were ordered by nation or region and chronologically. Individual PI were classified by development method, assessment level, care domain, performance dimension, diagnostic focus, and data source. Finally, the evidence on feasibility, data reliability, and content-, criterion-, and construct validity of the PI was evaluated. Results A total of 106 publications were included in the sample. The majority of the publications (n = 65) were peer-reviewed journal articles and 66 publications specifically dealt with performance of PMHC in the United States. The objectives of performance measurement vary widely from internal quality improvement to increasing transparency and accountability. The characteristics of 1480 unique PI were assessed. The majority of PI is based on stakeholder opinion, assesses care processes, is not specific to any diagnostic group, and utilizes administrative data sources. The targeted quality dimensions varied widely across and within nations depending on local professional or political definitions and interests. For all PI some evidence for the content validity and feasibility has been established. Data reliability, criterion- and construct validity have rarely been assessed. Only 18 publications on criterion validity were included. These show significant associations in the expected direction on the majority of PI, but mixed results on a noteworthy number of others. Conclusions PI have been developed for a broad range of care levels, domains, and quality dimensions of PMHC. To ensure their usefulness for the measurement of PMHC performance and advancement of transparency, accountability and quality improvement in PMHC, future research should focus on assessment of the psychometric properties of PI. PMID:22433251

Development and validation of a casemix classification to predict costs of specialist palliative care provision across inpatient hospice, hospital and community settings in the UK: a study protocol

PubMed Central

Guo, Ping; Dzingina, Mendwas; Firth, Alice M; Davies, Joanna M; Douiri, Abdel; O’Brien, Suzanne M; Pinto, Cathryn; Pask, Sophie; Higginson, Irene J; Eagar, Kathy; Murtagh, Fliss E M

2018-01-01

Introduction Provision of palliative care is inequitable with wide variations across conditions and settings in the UK. Lack of a standard way to classify by case complexity is one of the principle obstacles to addressing this. We aim to develop and validate a casemix classification to support the prediction of costs of specialist palliative care provision. Methods and analysis Phase I: A cohort study to determine the variables and potential classes to be included in a casemix classification. Data are collected from clinicians in palliative care services across inpatient hospice, hospital and community settings on: patient demographics, potential complexity/casemix criteria and patient-level resource use. Cost predictors are derived using multivariate regression and then incorporated into a classification using classification and regression trees. Internal validation will be conducted by bootstrapping to quantify any optimism in the predictive performance (calibration and discrimination) of the developed classification. Phase II: A mixed-methods cohort study across settings for external validation of the classification developed in phase I. Patient and family caregiver data will be collected longitudinally on demographics, potential complexity/casemix criteria and patient-level resource use. This will be triangulated with data collected from clinicians on potential complexity/casemix criteria and patient-level resource use, and with qualitative interviews with patients and caregivers about care provision across difference settings. The classification will be refined on the basis of its performance in the validation data set. Ethics and dissemination The study has been approved by the National Health Service Health Research Authority Research Ethics Committee. The results are expected to be disseminated in 2018 through papers for publication in major palliative care journals; policy briefs for clinicians, commissioning leads and policy makers; and lay summaries for patients and public. Trial registration number ISRCTN90752212. PMID:29550781

Development and validation of a casemix classification to predict costs of specialist palliative care provision across inpatient hospice, hospital and community settings in the UK: a study protocol.

PubMed

Guo, Ping; Dzingina, Mendwas; Firth, Alice M; Davies, Joanna M; Douiri, Abdel; O'Brien, Suzanne M; Pinto, Cathryn; Pask, Sophie; Higginson, Irene J; Eagar, Kathy; Murtagh, Fliss E M

2018-03-17

Provision of palliative care is inequitable with wide variations across conditions and settings in the UK. Lack of a standard way to classify by case complexity is one of the principle obstacles to addressing this. We aim to develop and validate a casemix classification to support the prediction of costs of specialist palliative care provision. Phase I: A cohort study to determine the variables and potential classes to be included in a casemix classification. Data are collected from clinicians in palliative care services across inpatient hospice, hospital and community settings on: patient demographics, potential complexity/casemix criteria and patient-level resource use. Cost predictors are derived using multivariate regression and then incorporated into a classification using classification and regression trees. Internal validation will be conducted by bootstrapping to quantify any optimism in the predictive performance (calibration and discrimination) of the developed classification. Phase II: A mixed-methods cohort study across settings for external validation of the classification developed in phase I. Patient and family caregiver data will be collected longitudinally on demographics, potential complexity/casemix criteria and patient-level resource use. This will be triangulated with data collected from clinicians on potential complexity/casemix criteria and patient-level resource use, and with qualitative interviews with patients and caregivers about care provision across difference settings. The classification will be refined on the basis of its performance in the validation data set. The study has been approved by the National Health Service Health Research Authority Research Ethics Committee. The results are expected to be disseminated in 2018 through papers for publication in major palliative care journals; policy briefs for clinicians, commissioning leads and policy makers; and lay summaries for patients and public. ISRCTN90752212. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

Development and Relative Validity of a Food Frequency Questionnaire to Assess Intakes of Total and Free Sugars in Australian Toddlers.

PubMed

Devenish, Gemma; Mukhtar, Aqif; Begley, Andrea; Do, Loc; Scott, Jane

2017-11-08

Background : Dental research into early childhood caries is hindered by a lack of suitable dietary assessment tools that have been developed and validated for the population and outcomes of interest. The aim of this study was to develop and investigate the relative validity and reproducibility of the Study of Mothers' and Infants' Life Events Food Frequency Questionnaire (SMILE-FFQ), to assess the total and free sugars intakes of Australian toddlers. Methods : The SMILE-FFQ was designed to capture the leading dietary contributors to dental caries risk in toddlers aged 18-30 months via a proxy report. Ninety-five parents of Australian toddlers completed the questionnaire online before and after providing three 24-h recalls (24HR), collected on non-consecutive days using the multipass method. Total and free sugars were compared between the two SMILE-FFQ administrations and between each SMILE-FFQ and the 24HR using multiple statistical tests and standardised validity criteria. Correlation (Pearson), mean difference (Wilcoxon rank test) and Bland Altman analyses were conducted to compare absolute values, with cross-classification (Chi-Square and Weighted Kappa) used to compare agreement across tertiles. Results : All reproducibility tests showed good agreement except weighted kappa, which showed acceptable agreement. Relative validity tests revealed a mix of good and acceptable agreement, with total sugars performing better at the individual level than free sugars. Compared to the 24HR, the SMILE-FFQ tended to underestimate absolute values at lower levels and overestimate them at higher levels. Conclusions : The combined findings of the various tests indicate that the SMILE-FFQ performs comparably to the 24HR for assessing both total and free sugars among individuals, is most effective for ranking participants rather than determining absolute intakes, and is therefore suitable for use in observational studies of Australian toddlers.

Robotic surgery training: construct validity of Global Evaluative Assessment of Robotic Skills (GEARS).

PubMed

Sánchez, Renata; Rodríguez, Omaira; Rosciano, José; Vegas, Liumariel; Bond, Verónica; Rojas, Aram; Sanchez-Ismayel, Alexis

2016-09-01

The objective of this study is to determine the ability of the GEARS scale (Global Evaluative Assessment of Robotic Skills) to differentiate individuals with different levels of experience in robotic surgery, as a fundamental validation. This is a cross-sectional study that included three groups of individuals with different levels of experience in robotic surgery (expert, intermediate, novice) their performance were assessed by GEARS applied by two reviewers. The difference between groups was determined by Mann-Whitney test and the consistency between the reviewers was studied by Kendall W coefficient. The agreement between the reviewers of the scale GEARS was 0.96. The score was 29.8 ± 0.4 to experts, 24 ± 2.8 to intermediates and 16 ± 3 to novices, with a statistically significant difference between all of them (p < 0.05). All parameters from the scale allow discriminating between different levels of experience, with exception of the depth perception item. We conclude that the scale GEARS was able to differentiate between individuals with different levels of experience in robotic surgery and, therefore, is a validated and useful tool to evaluate surgeons in training.

Estimation of Particulate Mass and Manganese Exposure Levels among Welders

PubMed Central

Hobson, Angela; Seixas, Noah; Sterling, David; Racette, Brad A.

2011-01-01

Background: Welders are frequently exposed to Manganese (Mn), which may increase the risk of neurological impairment. Historical exposure estimates for welding-exposed workers are needed for epidemiological studies evaluating the relationship between welding and neurological or other health outcomes. The objective of this study was to develop and validate a multivariate model to estimate quantitative levels of welding fume exposures based on welding particulate mass and Mn concentrations reported in the published literature. Methods: Articles that described welding particulate and Mn exposures during field welding activities were identified through a comprehensive literature search. Summary measures of exposure and related determinants such as year of sampling, welding process performed, type of ventilation used, degree of enclosure, base metal, and location of sampling filter were extracted from each article. The natural log of the reported arithmetic mean exposure level was used as the dependent variable in model building, while the independent variables included the exposure determinants. Cross-validation was performed to aid in model selection and to evaluate the generalizability of the models. Results: A total of 33 particulate and 27 Mn means were included in the regression analysis. The final model explained 76% of the variability in the mean exposures and included welding process and degree of enclosure as predictors. There was very little change in the explained variability and root mean squared error between the final model and its cross-validation model indicating the final model is robust given the available data. Conclusions: This model may be improved with more detailed exposure determinants; however, the relatively large amount of variance explained by the final model along with the positive generalizability results of the cross-validation increases the confidence that the estimates derived from this model can be used for estimating welder exposures in absence of individual measurement data. PMID:20870928

Development of an Integrated Nozzle for a Symmetric, RBCC Launch Vehicle Configuration

NASA Technical Reports Server (NTRS)

Smith, Timothy D.; Canabal, Francisco, III; Rice, Tharen; Blaha, Bernard

2000-01-01

The development of rocket based combined cycle (RBCC) engines is highly dependent upon integrating several different modes of operation into a single system. One of the key components to develop acceptable performance levels through each mode of operation is the nozzle. It must be highly integrated to serve the expansion processes of both rocket and air-breathing modes without undue weight, drag, or complexity. The NASA GTX configuration requires a fixed geometry, altitude-compensating nozzle configuration. The initial configuration, used mainly to estimate weight and cooling requirements was a 1 So half-angle cone, which cuts a concave surface from a point within the flowpath to the vehicle trailing edge. Results of 3-D CFD calculations on this geometry are presented. To address the critical issues associated with integrated, fixed geometry, multimode nozzle development, the GTX team has initiated a series of tasks to evolve the nozzle design, and validate performance levels. An overview of these tasks is given. The first element is a design activity to develop tools for integration of efficient expansion surfaces With the existing flowpath and vehicle aft-body, and to develop a second-generation nozzle design. A preliminary result using a "streamline-tracing" technique is presented. As the nozzle design evolves, a combination of 3-D CFD analysis and experimental evaluation will be used to validate the design procedure and determine the installed performance for propulsion cycle modeling. The initial experimental effort will consist of cold-flow experiments designed to validate the general trends of the streamline-tracing methodology and anchor the CFD analysis. Experiments will also be conducted to simulate nozzle performance during each mode of operation. As the design matures, hot-fire tests will be conducted to refine performance estimates and anchor more sophisticated reacting-flow analysis.

Memory Alteration Test to Detect Amnestic Mild Cognitive Impairment and Early Alzheimer's Dementia in Population with Low Educational Level.

PubMed

Custodio, Nilton; Lira, David; Herrera-Perez, Eder; Montesinos, Rosa; Castro-Suarez, Sheila; Cuenca-Alfaro, José; Valeriano-Lorenzo, Lucía

2017-01-01

Background/Aims : Short tests to early detection of the cognitive impairment are necessary in primary care setting, particularly in populations with low educational level. The aim of this study was to assess the performance of Memory Alteration Test (M@T) to discriminate controls, patients with amnestic Mild Cognitive Impairment (aMCI) and patients with early Alzheimer's Dementia (AD) in a sample of individuals with low level of education. Methods : Cross-sectional study to assess the performance of the M@T (study test), compared to the neuropsychological evaluation (gold standard test) scores in 247 elderly subjects with low education level from Lima-Peru. The cognitive evaluation included three sequential stages: (1) screening (to detect cases with cognitive impairment); (2) nosological diagnosis (to determinate specific disease); and (3) classification (to differentiate disease subtypes). The subjects with negative results for all stages were considered as cognitively normal (controls). The test performance was assessed by means of area under the receiver operating characteristic (ROC) curve. We calculated validity measures (sensitivity, specificity and correctly classified percentage), the internal consistency (Cronbach's alpha coefficient), and concurrent validity (Pearson's ratio coefficient between the M@T and Clinical Dementia Rating (CDR) scores). Results : The Cronbach's alpha coefficient was 0.79 and Pearson's ratio coefficient was 0.79 ( p < 0.01). The AUC of M@T to discriminate between early AD and aMCI was 99.60% (sensitivity = 100.00%, specificity = 97.53% and correctly classified = 98.41%) and to discriminate between aMCI and controls was 99.56% (sensitivity = 99.17%, specificity = 91.11%, and correctly classified = 96.99%). Conclusions : The M@T is a short test with a good performance to discriminate controls, aMCI and early AD in individuals with low level of education from urban settings.

Development of an objective assessment tool for total laparoscopic hysterectomy: A Delphi method among experts and evaluation on a virtual reality simulator

PubMed Central

Knight, Sophie; Aggarwal, Rajesh; Agostini, Aubert; Loundou, Anderson; Berdah, Stéphane

2018-01-01

Introduction Total Laparoscopic hysterectomy (LH) requires an advanced level of operative skills and training. The aim of this study was to develop an objective scale specific for the assessment of technical skills for LH (H-OSATS) and to demonstrate feasibility of use and validity in a virtual reality setting. Material and methods The scale was developed using a hierarchical task analysis and a panel of international experts. A Delphi method obtained consensus among experts on relevant steps that should be included into the H-OSATS scale for assessment of operative performances. Feasibility of use and validity of the scale were evaluated by reviewing video recordings of LH performed on a virtual reality laparoscopic simulator. Three groups of operators of different levels of experience were assessed in a Marseille teaching hospital (10 novices, 8 intermediates and 8 experienced surgeons). Correlations with scores obtained using a recognised generic global rating tool (OSATS) were calculated. Results A total of 76 discrete steps were identified by the hierarchical task analysis. 14 experts completed the two rounds of the Delphi questionnaire. 64 steps reached consensus and were integrated in the scale. During the validation process, median time to rate each video recording was 25 minutes. There was a significant difference between the novice, intermediate and experienced group for total H-OSATS scores (133, 155.9 and 178.25 respectively; p = 0.002). H-OSATS scale demonstrated high inter-rater reliability (intraclass correlation coefficient [ICC] = 0.930; p<0.001) and test retest reliability (ICC = 0.877; p<0.001). High correlations were found between total H-OSATS scores and OSATS scores (rho = 0.928; p<0.001). Conclusion The H-OSATS scale displayed evidence of validity for assessment of technical performances for LH performed on a virtual reality simulator. The implementation of this scale is expected to facilitate deliberate practice. Next steps should focus on evaluating the validity of the scale in the operating room. PMID:29293635

A Novel Model for Predicting Incident Moderate to Severe Anemia and Iron Deficiency in Patients with Newly Diagnosed Ulcerative Colitis.

PubMed

Khan, Nabeel; Patel, Dhruvan; Shah, Yash; Yang, Yu-Xiao

2017-05-01

Anemia and iron deficiency are common complications of ulcerative colitis (UC). We aimed to develop and internally validate a prediction model for the incidence of moderate to severe anemia and iron deficiency anemia (IDA) in newly diagnosed patients with UC. Multivariable logistic regression was performed among a nationwide cohort of patients who were newly diagnosed with UC in the VA health-care system. Model development was performed in a random two-third of the total cohort and then validated in the remaining one-third of the cohort. As candidate predictors, we examined routinely available data at the time of UC diagnosis including demographics, medications, laboratory results, and endoscopy findings. A total of 789 patients met the inclusion criteria. For the outcome of moderate to severe anemia, age, albumin level and mild anemia at UC diagnosis were predictors selected for the model. The AUC for this model was 0.69 (95% CI 0.64-0.74). For the outcome of moderate to severe anemia with evidence of iron deficiency, the predictors included African-American ethnicity, mild anemia, age, and albumin level at UC diagnosis. The AUC was 0.76, (95% CI 0.69-0.82). Calibration was consistently good in all models (Hosmer-Lemeshow goodness of fit p > 0.05). The models performed similarly in the internal validation cohort. We developed and internally validated a prognostic model for predicting the risk of moderate to severe anemia and IDA among newly diagnosed patients with UC. This will help identify patients at high risk of these complications, who could benefit from surveillance and preventive measures.

Teamwork methods for accountable care: relational coordination and TeamSTEPPS®.

PubMed

Gittell, Jody Hoffer; Beswick, Joanne; Goldmann, Don; Wallack, Stanley S

2015-01-01

To deliver greater value in the accountable care context, the Institute of Medicine argues for a culture of teamwork at multiple levels--across professional and organizational siloes and with patients and their families and communities. The logic of performance improvement is that data are needed to target interventions and to assess their impact. We argue that efforts to build teamwork will benefit from teamwork measures that provide diagnostic information regarding the current state and teamwork interventions that can respond to the opportunities identified in the current state. We identify teamwork measures and teamwork interventions that are validated and that can work across multiple levels of teamwork. We propose specific ways to combine them for optimal effectiveness. We review measures of teamwork documented by Valentine, Nembhard, and Edmondson and select those that they identified as satisfying the four criteria for psychometric validation and as being unbounded and therefore able to measure teamwork across multiple levels. We then consider teamwork interventions that are widely used in the U.S. health care context, are well validated based on their association with outcomes, and are capable of working at multiple levels of teamwork. We select the top candidate in each category and propose ways to combine them for optimal effectiveness. We find relational coordination is a validated multilevel teamwork measure and TeamSTEPPS® is a validated multilevel teamwork intervention and propose specific ways for the relational coordination measure to enhance the TeamSTEPPS intervention. Health care systems and change agents seeking to respond to the challenges of accountable care can use TeamSTEPPS as a validated multilevel teamwork intervention methodology, enhanced by relational coordination as a validated multilevel teamwork measure with diagnostic capacity to pinpoint opportunities for improving teamwork along specific dimensions (e.g., shared knowledge, timely communication) and in specific role relationships (e.g., nurse/medical assistant, emergency unit/medical unit, primary care/specialty care).

Reliability, sensitivity and validity of the assistant referee intermittent endurance test (ARIET) - a modified Yo-Yo IE2 test for elite soccer assistant referees.

PubMed

Castagna, Carlo; Bendiksen, Mads; Impellizzeri, Franco M; Krustrup, Peter

2012-01-01

We examined the reliability and validity of the assistant referee intermittent endurance test (ARIET), a modified Yo-Yo IE2 test including shuttles of sideways running. The ARIET was carried out on 198 Italian (Serie A-B, Lega-Pro and National Level) and 47 Danish elite soccer assistant referees. Reproducibility was tested for 41 assistant referees on four occasions each separated by one week. The ARIET intraclass correlation coefficients and typical error of measurement ranged from 0.96 to 0.99 and 3.1 to 5.7%, respectively. ARIET performance for Serie A and B was 23 and 25% greater than in Lega-Pro (P < 0.001). The lowest cut-off value derived from receiving operator characteristic discriminating Serie A-B from Lega-Pro was 1300 m. The ARIET performance was significantly correlated with VO(2max) (r = 0.78, P < 0.001), %HR(max) after 4 min of ARIET (r = - 0.81, P < 0.001) and Yo-Yo IR1 performance (r = 0.95, P < 0.001), but not sprint performance (r = -0.15; P = 0.58). The results showed that ARIET is a reproducible and valid test that is able to discriminate between assistant referees of different competitive levels. The lack of correlation with sprinting ability and close correlations with aerobic power, intermittent shuttle running and sub-maximal ARIET heart rate loading provide evidence that ARIET is a relevant test for assessment of intermittent endurance capacity of soccer assistant referees.

A method for monitoring intensity during aquatic resistance exercises.

PubMed

Colado, Juan C; Tella, Victor; Triplett, N Travis

2008-11-01

The aims of this study were (i) to check whether monitoring of both the rhythm of execution and the perceived effort is a valid tool for reproducing the same intensity of effort in different sets of the same aquatic resistance exercise (ARE) and (ii) to assess whether this method allows the ARE to be put at the same intensity level as its equivalent carried out on dry land. Four healthy trained young men performed horizontal shoulder abduction and adduction (HSAb/Ad) movements in water and on dry land. Muscle activation was recorded using surface electromyography of 1 stabilizer and several agonist muscles. Before the final tests, the ARE movement cadence was established individually following a rhythmic digitalized sequence of beats to define the alternate HSAb/Ad movements. This cadence allowed the subject to perform 15 repetitions at a perceived exertion of 9-10 using Hydro-Tone Bells. After that, each subject performed 2 nonconsecutive ARE sets. The dry land exercises (1 set of HSAb and 1 set of HSAd) were performed using a dual adjustable pulley cable motion machine, with the previous selection of weights that allowed the same movement cadence to be maintained and the completion of the same repetitions in each of the sets as with the ARE. The average normalized data were compared for the exercises in order to determine possible differences in muscle activity. The results show the validity of this method for reproducing the intensity of effort in different sets of the same ARE, but is not valid for matching the same intensity level as kinematically similar land-based exercises.

Testing for the validity of purchasing power parity theory both in the long-run and the short-run for ASEAN-5

NASA Astrophysics Data System (ADS)

Choji, Niri Martha; Sek, Siok Kun

2017-11-01

The purchasing power parity theory says that the trade rates among two nations ought to be equivalent to the proportion of the total price levels between the two nations. For more than a decade, there has been substantial interest in testing for the validity of the Purchasing Power Parity (PPP) empirically. This paper performs a series of tests to see if PPP is valid for ASEAN-5 nations for the period of 2000-2016 using monthly data. For this purpose, we conducted four different tests of stationarity, two cointegration tests (Pedroni and Westerlund), and also the VAR model. The stationarity (unit root) tests reveal that the variables are not stationary at levels however stationary at first difference. Cointegration test results did not reject the H0 of no cointegration implying the absence long-run association among the variables and results of the VAR model did not reveal a strong short-run relationship. Based on the data, we, therefore, conclude that PPP is not valid in long-and short-run for ASEAN-5 during 2000-2016.

Performance of Portable Ventilators at Altitude

DTIC Science & Technology

2015-03-30

collection of information if it does not display a currently valid OMB control number. PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. 1. REPORT...Deploying ventilators that can maintain a consistent tidal volume (VT) delivery at various altitudes is imperative for lung protection when...performance of mechanical ventilators calibrated for operation at sea level. Deploying ventilators that can maintain a consistent tidal volume (VT) delivery

Estimation of water table level and nitrate pollution based on geostatistical and multiple mass transport models

NASA Astrophysics Data System (ADS)

Matiatos, Ioannis; Varouhakis, Emmanouil A.; Papadopoulou, Maria P.

2015-04-01

As the sustainable use of groundwater resources is a great challenge for many countries in the world, groundwater modeling has become a very useful and well established tool for studying groundwater management problems. Based on various methods used to numerically solve algebraic equations representing groundwater flow and contaminant mass transport, numerical models are mainly divided into Finite Difference-based and Finite Element-based models. The present study aims at evaluating the performance of a finite difference-based (MODFLOW-MT3DMS), a finite element-based (FEFLOW) and a hybrid finite element and finite difference (Princeton Transport Code-PTC) groundwater numerical models simulating groundwater flow and nitrate mass transport in the alluvial aquifer of Trizina region in NE Peloponnese, Greece. The calibration of groundwater flow in all models was performed using groundwater hydraulic head data from seven stress periods and the validation was based on a series of hydraulic head data for two stress periods in sufficient numbers of observation locations. The same periods were used for the calibration of nitrate mass transport. The calibration and validation of the three models revealed that the simulated values of hydraulic heads and nitrate mass concentrations coincide well with the observed ones. The models' performance was assessed by performing a statistical analysis of these different types of numerical algorithms. A number of metrics, such as Mean Absolute Error (MAE), Root Mean Square Error (RMSE), Bias, Nash Sutcliffe Model Efficiency (NSE) and Reliability Index (RI) were used allowing the direct comparison of models' performance. Spatiotemporal Kriging (STRK) was also applied using separable and non-separable spatiotemporal variograms to predict water table level and nitrate concentration at each sampling station for two selected hydrological stress periods. The predictions were validated using the respective measured values. Maps of water table level and nitrate concentrations were produced and compared with those obtained from groundwater and mass transport numerical models. Preliminary results showed similar efficiency of the spatiotemporal geostatistical method with the numerical models. However data requirements of the former model were significantly less. Advantages and disadvantages of the methods performance were analysed and discussed indicating the characteristics of the different approaches.

Dynamic testing in schizophrenia: does training change the construct validity of a test?

PubMed

Wiedl, Karl H; Schöttke, Henning; Green, Michael F; Nuechterlein, Keith H

2004-01-01

Dynamic testing typically involves specific interventions for a test to assess the extent to which test performance can be modified, beyond level of baseline (static) performance. This study used a dynamic version of the Wisconsin Card Sorting Test (WCST) that is based on cognitive remediation techniques within a test-training-test procedure. From results of previous studies with schizophrenia patients, we concluded that the dynamic and static versions of the WCST should have different construct validity. This hypothesis was tested by examining the patterns of correlations with measures of executive functioning, secondary verbal memory, and verbal intelligence. Results demonstrated a specific construct validity of WCST dynamic (i.e., posttest) scores as an index of problem solving (Tower of Hanoi) and secondary verbal memory and learning (Auditory Verbal Learning Test), whereas the impact of general verbal capacity and selective attention (Verbal IQ, Stroop Test) was reduced. It is concluded that the construct validity of the test changes with dynamic administration and that this difference helps to explain why the dynamic version of the WCST predicts functional outcome better than the static version.

GOES-R L1b Readiness Implementation and Management Plan

NASA Technical Reports Server (NTRS)

Kunkee, David; Farley, Robert; Kwan, Betty; Walterscheid, Richard; Hecht, James; Claudepierre, Seth.; De Luccia, Frank

2017-01-01

A complement of Readiness, Implementation and Management Plans (RIMPs) to facilitate management of post-launch product test activities for the official Geostationary Operational Environmental Satellite (GOES-R) Level 1b (L1b) products have been developed and documented. Separate plans have been created for each of the GOES-R sensors including: the Advanced Baseline Imager (ABI), the Extreme ultraviolet and X-ray Irradiance Sensors (EXIS), Geostationary Lightning Mapper (GLM), GOES-R Magnetometer (MAG), the Space Environment In-Situ Suite (SEISS), and the Solar Ultraviolet Imager (SUVI). The GOES-R program has implemented these RIMPs in order to address the full scope of CalVal activities required for a successful demonstration of GOES-R L1b data product quality throughout the three validation stages: Beta, Provisional and Full Validation. For each product maturity level, the RIMPs include specific performance criteria and required artifacts that provide evidence a given validation stage has been reached, the timing when each stage will be complete, a description of every applicable Post-Launch Product Test (PLPT), roles and responsibilities of personnel, upstream dependencies, and analysis methods and tools to be employed during validation. Instrument level Post-Launch Tests (PLTs) are also referenced and apply primarily to functional check-out of the instruments.

High-throughput method for the determination of residues of β-lactam antibiotics in bovine milk by LC-MS/MS.

PubMed

Jank, Louise; Martins, Magda Targa; Arsand, Juliana Bazzan; Hoff, Rodrigo Barcellos; Barreto, Fabiano; Pizzolato, Tânia Mara

2015-01-01

This study describes the development and validation procedures for scope extension of a method for the determination of β-lactam antibiotic residues (ampicillin, amoxicillin, penicillin G, penicillin V, oxacillin, cloxacillin, dicloxacillin, nafcillin, ceftiofur, cefquinome, cefoperazone, cephapirine, cefalexin and cephalonium) in bovine milk. Sample preparation was performed by liquid-liquid extraction (LLE) followed by two clean-up steps, including low temperature purification (LTP) and a solid phase dispersion clean-up. Extracts were analysed using a liquid chromatography-electrospray-tandem mass spectrometry system (LC-ESI-MS/MS). Chromatographic separation was performed in a C18 column, using methanol and water (both with 0.1% of formic acid) as mobile phase. Method validation was performed according to the criteria of Commission Decision 2002/657/EC. Main validation parameters such as linearity, limit of detection, decision limit (CCα), detection capability (CCβ), accuracy, and repeatability were determined and were shown to be adequate. The method was applied to real samples (more than 250) and two milk samples had levels above maximum residues limits (MRLs) for cloxacillin - CLX and cefapirin - CFAP.

Multi-Evaporator Miniature Loop Heat Pipe for Small Spacecraft Thermal Control

NASA Technical Reports Server (NTRS)

Ku, Jentung; Ottenstein, Laura; Douglas, Donya

2008-01-01

This paper presents the development of the Thermal Loop experiment under NASA's New Millennium Program Space Technology 8 (ST8) Project. The Thermal Loop experiment was originally planned for validating in space an advanced heat transport system consisting of a miniature loop heat pipe (MLHP) with multiple evaporators and multiple condensers. Details of the thermal loop concept, technical advances and benefits, Level 1 requirements and the technology validation approach are described. An MLHP breadboard has been built and tested in the laboratory and thermal vacuum environments, and has demonstrated excellent performance that met or exceeded the design requirements. The MLHP retains all features of state-of-the-art loop heat pipes and offers additional advantages to enhance the functionality, performance, versatility, and reliability of the system. In addition, an analytical model has been developed to simulate the steady state and transient operation of the MHLP, and the model predictions agreed very well with experimental results. A protoflight MLHP has been built and is being tested in a thermal vacuum chamber to validate its performance and technical readiness for a flight experiment.

Development of an ultra high performance liquid chromatography method for determining triamcinolone acetonide in hydrogels using the design of experiments/design space strategy in combination with process capability index.

PubMed

Oliva, Alexis; Monzón, Cecilia; Santoveña, Ana; Fariña, José B; Llabrés, Matías

2016-07-01

An ultra high performance liquid chromatography method was developed and validated for the quantitation of triamcinolone acetonide in an injectable ophthalmic hydrogel to determine the contribution of analytical method error in the content uniformity measurement. During the development phase, the design of experiments/design space strategy was used. For this, the free R-program was used as a commercial software alternative, a fast efficient tool for data analysis. The process capability index was used to find the permitted level of variation for each factor and to define the design space. All these aspects were analyzed and discussed under different experimental conditions by the Monte Carlo simulation method. Second, a pre-study validation procedure was performed in accordance with the International Conference on Harmonization guidelines. The validated method was applied for the determination of uniformity of dosage units and the reasons for variability (inhomogeneity and the analytical method error) were analyzed based on the overall uncertainty. © 2016 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim.

Use of Latent Class Analysis to define groups based on validity, cognition, and emotional functioning.

PubMed

Morin, Ruth T; Axelrod, Bradley N

Latent Class Analysis (LCA) was used to classify a heterogeneous sample of neuropsychology data. In particular, we used measures of performance validity, symptom validity, cognition, and emotional functioning to assess and describe latent groups of functioning in these areas. A data-set of 680 neuropsychological evaluation protocols was analyzed using a LCA. Data were collected from evaluations performed for clinical purposes at an urban medical center. A four-class model emerged as the best fitting model of latent classes. The resulting classes were distinct based on measures of performance validity and symptom validity. Class A performed poorly on both performance and symptom validity measures. Class B had intact performance validity and heightened symptom reporting. The remaining two Classes performed adequately on both performance and symptom validity measures, differing only in cognitive and emotional functioning. In general, performance invalidity was associated with worse cognitive performance, while symptom invalidity was associated with elevated emotional distress. LCA appears useful in identifying groups within a heterogeneous sample with distinct performance patterns. Further, the orthogonal nature of performance and symptom validities is supported.

Fundamental arthroscopic skill differentiation with virtual reality simulation.

PubMed

Rose, Kelsey; Pedowitz, Robert

2015-02-01

The purpose of this study was to investigate the use and validity of virtual reality modules as part of the educational approach to mastering arthroscopy in a safe environment by assessing the ability to distinguish between experience levels. Additionally, the study aimed to evaluate whether experts have greater ambidexterity than do novices. Three virtual reality modules (Swemac/Augmented Reality Systems, Linkoping, Sweden) were created to test fundamental arthroscopic skills. Thirty participants-10 experts consisting of faculty, 10 intermediate participants consisting of orthopaedic residents, and 10 novices consisting of medical students-performed each exercise. Steady and Telescope was designed to train centering and image stability. Steady and Probe was designed to train basic triangulation. Track and Moving Target was designed to train coordinated motions of arthroscope and probe. Metrics reflecting speed, accuracy, and efficiency of motion were used to measure construct validity. Steady and Probe and Track a Moving Target both exhibited construct validity, with better performance by experts and intermediate participants than by novices (P < .05), whereas Steady and Telescope did not show validity. There was an overall trend toward better ambidexterity as a function of greater surgical experience, with experts consistently more proficient than novices throughout all 3 modules. This study represents a new way to assess basic arthroscopy skills using virtual reality modules developed through task deconstruction. Participants with the most arthroscopic experience performed better and were more consistent than novices on all 3 virtual reality modules. Greater arthroscopic experience correlates with more symmetry of ambidextrous performance. However, further adjustment of the modules may better simulate fundamental arthroscopic skills and discriminate between experience levels. Arthroscopy training is a critical element of orthopaedic surgery resident training. Developing techniques to safely and effectively train these skills is critical for patient safety and resident education. Copyright © 2015 Arthroscopy Association of North America. Published by Elsevier Inc. All rights reserved.

Psychometric properties of the motor diagnostics in the German football talent identification and development programme.

PubMed

HÖner, Oliver; Votteler, Andreas; Schmid, Markus; Schultz, Florian; Roth, Klaus

2015-01-01

The utilisation of motor performance tests for talent identification in youth sports is discussed intensively in talent research. This article examines the reliability, differential stability and validity of the motor diagnostics conducted nationwide by the German football talent identification and development programme and provides reference values for a standardised interpretation of the diagnostics results. Highly selected players (the top 4% of their age groups, U12-U15) took part in the diagnostics at 17 measurement points between spring 2004 and spring 2012 (N = 68,158). The heterogeneous test battery measured speed abilities and football-specific technical skills (sprint, agility, dribbling, ball control, shooting, juggling). For all measurement points, the overall score and the speed tests showed high internal consistency, high test-retest reliability and satisfying differential stability. The diagnostics demonstrated satisfying factorial-related validity with plausible and stable loadings on the two empirical factors "speed" and "technical skills". The score, and the technical skills dribbling and juggling, differentiated the most among players of different performance levels and thus showed the highest criterion-related validity. Satisfactory psychometric properties for the diagnostics are an important prerequisite for a scientifically sound rating of players' actual motor performance and for the future examination of the prognostic validity for success in adulthood.

The Eysenckian personality factors and their correlations with academic performance.

PubMed

Poropat, Arthur E

2011-03-01

BACKGROUND. The relationship between personality and academic performance has long been explored, and a recent meta-analysis established that measures of the five-factor model (FFM) dimension of Conscientiousness have similar validity to intelligence measures. Although currently dominant, the FFM is only one of the currently accepted models of personality, and has limited theoretical support. In contrast, the Eysenckian personality model was developed to assess a specific theoretical model and is still commonly used in educational settings and research. AIMS. This meta-analysis assessed the validity of the Eysenckian personality measures for predicting academic performance. SAMPLE. Statistics were obtained for correlations with Psychoticism, Extraversion, and Neuroticism (20-23 samples; N from 8,013 to 9,191), with smaller aggregates for the Lie scale (7 samples; N= 3,910). METHODS. The Hunter-Schmidt random effects method was used to estimate population correlations between the Eysenckian personality measures and academic performance. Moderating effects were tested using weighted least squares regression. RESULTS. Significant but modest validities were reported for each scale. Neuroticism and Extraversion had relationships with academic performance that were consistent with previous findings, while Psychoticism appears to be linked to academic performance because of its association with FFM Conscientiousness. Age and educational level moderated correlations with Neuroticism and Extraversion, and gender had no moderating effect. Correlations varied significantly based on the measurement instrument used. CONCLUSIONS. The Eysenckian scales do not add to the prediction of academic performance beyond that provided by FFM scales. Several measurement problems afflict the Eysenckian scales, including low to poor internal reliability and complex factor structures. In particular, the measurement and validity problems of Psychoticism mean its continued use in academic settings is unjustified. © 2010 The Author. British Journal of Educational Psychology. © 2010 The British Psychological Society.

A Self-Validation Method for High-Temperature Thermocouples Under Oxidizing Atmospheres

NASA Astrophysics Data System (ADS)

Mokdad, S.; Failleau, G.; Deuzé, T.; Briaudeau, S.; Kozlova, O.; Sadli, M.

2015-08-01

Thermocouples are prone to significant drift in use particularly when they are exposed to high temperatures. Indeed, high-temperature exposure can affect the response of a thermocouple progressively by changing the structure of the thermoelements and inducing inhomogeneities. Moreover, an oxidizing atmosphere contributes to thermocouple drift by changing the chemical nature of the metallic wires by the effect of oxidation. In general, severe uncontrolled drift of thermocouples results from these combined influences. A periodic recalibration of the thermocouple can be performed, but sometimes it is not possible to remove the sensor out of the process. Self-validation methods for thermocouples provide a solution to avoid this drawback, but there are currently no high-temperature contact thermometers with self-validation capability at temperatures up to . LNE-Cnam has developed fixed-point devices integrated to the thermocouples consisting of machined alumina-based devices for operation under oxidizing atmospheres. These devices require small amounts of pure metals (typically less than 2 g). They are suitable for self-validation of high-temperature thermocouples up to . In this paper the construction and the characterization of these integrated fixed-point devices are described. The phase-transition plateaus of gold, nickel, and palladium, which enable coverage of the temperature range between and , are assessed with this self-validation technique. Results of measurements performed at LNE-Cnam with the integrated self-validation module at several levels of temperature will be presented. The performance of the devices are assessed and discussed, in terms of robustness and metrological characteristics. Uncertainty budgets are also proposed and detailed.

Validation of the Movie for the Assessment of Social Cognition in Adolescents with ASD: Fixation Duration and Pupil Dilation as Predictors of Performance.

PubMed

Müller, Nico; Baumeister, Sarah; Dziobek, Isabel; Banaschewski, Tobias; Poustka, Luise

2016-09-01

Impaired social cognition is one of the core characteristics of autism spectrum disorders (ASD). Appropriate measures of social cognition for high-functioning adolescents with ASD are, however, lacking. The Movie for the Assessment of Social Cognition (MASC) uses dynamic social stimuli, ensuring ecological validity, and has proven to be a sensitive measure in adulthood. In the current study, 33 adolescents with ASD and 23 controls were administered the MASC, while concurrent eye tracking was used to relate gaze behavior to performance levels. The ASD group exhibited reduced MASC scores, with social cognition performance being explained by shorter fixation duration on eyes and decreased pupil dilation. These potential diagnostic markers are discussed as indicators of different processing of social information in ASD.

[Development and validation of the Family Vulnerability Index to Disability and Dependence (FVI-DD)].

PubMed

Amendola, Fernanda; Alvarenga, Márcia Regina Martins; Latorre, Maria do Rosário Dias de Oliveira; Oliveira, Maria Amélia de Campos

2014-02-01

This exploratory, descriptive, cross-sectional, and quantitative study aimed to develop and validate an index of family vulnerability to disability and dependence (FVI-DD). This study was adapted from the Family Development Index, with the addition of social and health indicators of disability and dependence. The instrument was applied to 248 families in the city of Sao Paulo, followed by exploratory factor analysis. Factor validation was performed using the concurrent and discriminant validity of the Lawton scale and Katz Index. The descriptive level adopted for the study was p < 0.05. The final vulnerability index comprised 50 questions classified into seven factors contemplating social and health dimensions, and this index exhibited good internal consistency (Cronbach's alpha = 0.82). FVI-DD was validated using both the Lawton scale and Katz Index. We conclude that FVI-DD can accurately and reliably assess family vulnerability to disability and dependence.

Overview of a Proposed Flight Validation of Aerocapture System Technology for Planetary Missions

NASA Technical Reports Server (NTRS)

Keys, Andrew S.; Hall, Jeffery L.; Oh, David; Munk, Michelle M.

2006-01-01

Aerocapture System Technology for Planetary Missions is being proposed to NASA's New Millennium Program for flight aboard the Space Technology 9 (ST9) flight opportunity. The proposed ST9 aerocapture mission is a system-level flight validation of the aerocapture maneuver as performed by an instrumented, high-fidelity flight vehicle within a true in-space and atmospheric environment. Successful validation of the aerocapture maneuver will be enabled through the flight validation of an advanced guidance, navigation, and control system as developed by Ball Aerospace and two advanced Thermal Protection System (TPS) materials, Silicon Refined Ablative Material-20 (SRAM-20) and SRAM-14, as developed by Applied Research Associates (ARA) Ablatives Laboratory. The ST9 aerocapture flight validation will be sufficient for immediate infusion of these technologies into NASA science missions being proposed for flight to a variety of Solar System destinations possessing a significant planetary atmosphere.

Psychometric viability of measures of functional performance commonly used for people with dementia: a systematic review of measurement properties.

PubMed

Fox, Benjamin; Henwood, Timothy; Keogh, Justin; Neville, Christine

2016-08-01

Confidence in findings can only be drawn from measurement tools that have sound psychometric properties for the population with which they are used. Within a dementia specific population, measures of physical function have been poorly justified in exercise intervention studies, with justification of measures based on validity or reliability studies from dissimilar clinical populations, such as people with bronchitis or healthy older adults without dementia. To review the reliability and validity of quantitative measures of pre-identified physical function, as commonly used within exercise intervention literature for adults with dementia. Participants were adults, aged 65 years and older, with a confirmed medical diagnosis of dementia. n/a Desired studies were observational and cross-sectional and that assessed measures from a pre-identified list of measures of physical function. Studies that assessed the psychometric constructs of reliability and validity were targeted. COSMIN taxology was used to define reliability and validity. This included, but were not limited to, Intra-Class Correlations, Kappa, Cronbach's Alpha, Chi Squared, Standard Error of Measurement, Minimal Detectable Change and Limits of Agreement. Published material was sourced from the following four databases: MEDLINE, EMBASE, CINAHL and ISI Web of Science. Grey literature was searched for using ALOIS, Google Scholar and ProQuest. The COSMIN checklist was used to assess methodological quality of included studies. Assessment was completed by two reviewers independently. Reliability and validity data was extracted from included studies using standardized Joanna Briggs Institute data collection forms. Extraction was completed by two reviewers. A narrative synthesis of measurement properties of the tools used to measure physical function was performed. Quantitative meta-analysis was conducted for Intra-Class Correlation Coefficients only. With respect to relative reliability, studies reporting assessed measures had intraclass correlation coefficients greater than 0.71, indicating their suitability for use at a group level. However, a consistent finding among studies that included assessment of absolute reliability was that intra individual variation was too large for meaningful measurement of individuals. This was indicated by large Minimal Detectable Change (MDC) scores. Walk Speed has the smallest reported Mimimal Detectable Change score at 0.11m/s. This represented a change of 35% before statistical variation could be eliminated as the cause for this change. All measures had large MDC values. Walk Speed had the smallest MDC values at 0.11m/s, which represented a necessary change of 35%. Only a limited number of studies assessed the validity of measures. This supports the use of these measures in a very narrow selection of circumstances (see Summary of Findings). In summary, measures have shown appropriate levels of relative reliability. This supports their use at the group level. However, large levels of intra-individual variation undermine their applicability at the individual level. Limited studies of validity were available to this review, which limits a conclusion on whether measures are valid for people with dementia.

Competitive-level differences in Yo-Yo intermittent recovery and twelve minute run test performance in soccer referees.

PubMed

Castagna, Carlo; Abt, Grant; D'Ottavio, Stefano

2005-11-01

The aim of this study was to examine yo-yo intermittent recovery test (Yo-Yo test) and 12-minute run test (12MRT) performances in experienced soccer referees of different competitive levels. Three groups (n = 14 each) of experienced Italian soccer referees officiating in the first (series AB, top-level), third (series C, medium-level), and fourth (series D, low-level) division, were randomly submitted to the 12MRT and the Yo-Yo test during 2 testing sessions, 48-hours apart. 12MRT performances were 3,000 +/- 112 m; 2,894 +/- 99 m; and 2,896 +/- 171 m for top-level, medium-level and low-level referees, respectively (p > 0.05). In the Yo-Yo test, the top-level, medium-level, and low-level referees covered 1,874 +/- 431 m; 1,360 +/- 172 m; and 1,272 +/- 215 m, respectively. The test performances of top-level referees in the Yo-Yo test was significantly different from those scored by medium-level and low-level referees (p < 0.05). After the Yo-Yo test, blood lactate concentrations (BLC) were higher in the medium-level and low-level referees compared with the top-level referees (p < 0.05). The results of the present study show that the Yo-Yo test and not the 12MRT can discriminate endurance performance in experienced elite level soccer referees. With respect to its discriminative and match performance validity, the Yo-Yo test may be considered a relevant field test to assess endurance preparedness for experienced soccer referees and a useful tool in talent selection.

Phonological Models.

ERIC Educational Resources Information Center

Ballard, W.L.

1968-01-01

The article discusses models of synchronic and diachronic phonology and suggests changes in them. The basic generative model of phonology is outlined with the author's reinterpretations. The systematic phonemic level is questioned in terms of its unreality with respect to linguistic performance and its lack of validity with respect to historical…

Development and External Validation of a Prognostic Nomogram for Metastatic Uveal Melanoma

PubMed Central

Valpione, Sara; Moser, Justin C.; Parrozzani, Raffaele; Bazzi, Marco; Mansfield, Aaron S.; Mocellin, Simone; Pigozzo, Jacopo; Midena, Edoardo; Markovic, Svetomir N.; Aliberti, Camillo; Campana, Luca G.; Chiarion-Sileni, Vanna

2015-01-01

Background Approximately 50% of patients with uveal melanoma (UM) will develop metastatic disease, usually involving the liver. The outcome of metastatic UM (mUM) is generally poor and no standard therapy has been established. Additionally, clinicians lack a validated prognostic tool to evaluate these patients. The aim of this work was to develop a reliable prognostic nomogram for clinicians. Patients and Methods Two cohorts of mUM patients, from Veneto Oncology Institute (IOV) (N=152) and Mayo Clinic (MC) (N=102), were analyzed to develop and externally validate, a prognostic nomogram. Results The median survival of mUM was 17.2 months in the IOV cohort and 19.7 in the MC cohort. Percentage of liver involvement (HR 1.6), elevated levels of serum LDH (HR 1.6), and a WHO performance status=1 (HR 1.5) or 2–3 (HR 4.6) were associated with worse prognosis. Longer disease-free interval from diagnosis of UM to that of mUM conferred a survival advantage (HR 0.9). The nomogram had a concordance probability of 0.75 (SE .006) in the development dataset (IOV), and 0.80 (SE .009) in the external validation (MC). Nomogram predictions were well calibrated. Conclusions The nomogram, which includes percentage of liver involvement, LDH levels, WHO performance status and disease free-interval accurately predicts the prognosis of mUM and could be useful for decision-making and risk stratification for clinical trials. PMID:25780931

Endogenous protein "barcode" for data validation and normalization in quantitative MS analysis.

PubMed

Lee, Wooram; Lazar, Iulia M

2014-07-01

Quantitative proteomic experiments with mass spectrometry detection are typically conducted by using stable isotope labeling and label-free quantitation approaches. Proteins with housekeeping functions and stable expression level such actin, tubulin, and glyceraldehyde-3-phosphate dehydrogenase are frequently used as endogenous controls. Recent studies have shown that the expression level of such common housekeeping proteins is, in fact, dependent on various factors such as cell type, cell cycle, or disease status and can change in response to a biochemical stimulation. The interference of such phenomena can, therefore, substantially compromise their use for data validation, alter the interpretation of results, and lead to erroneous conclusions. In this work, we advance the concept of a protein "barcode" for data normalization and validation in quantitative proteomic experiments. The barcode comprises a novel set of proteins that was generated from cell cycle experiments performed with MCF7, an estrogen receptor positive breast cancer cell line, and MCF10A, a nontumorigenic immortalized breast cell line. The protein set was selected from a list of ~3700 proteins identified in different cellular subfractions and cell cycle stages of MCF7/MCF10A cells, based on the stability of spectral count data generated with an LTQ ion trap mass spectrometer. A total of 11 proteins qualified as endogenous standards for the nuclear and 62 for the cytoplasmic barcode, respectively. The validation of the protein sets was performed with a complementary SKBR3/Her2+ cell line.

Shoulder model validation and joint contact forces during wheelchair activities.

PubMed

Morrow, Melissa M B; Kaufman, Kenton R; An, Kai-Nan

2010-09-17

Chronic shoulder impingement is a common problem for manual wheelchair users. The loading associated with performing manual wheelchair activities of daily living is substantial and often at a high frequency. Musculoskeletal modeling and optimization techniques can be used to estimate the joint contact forces occurring at the shoulder to assess the soft tissue loading during an activity and to possibly identify activities and strategies that place manual wheelchair users at risk for shoulder injuries. The purpose of this study was to validate an upper extremity musculoskeletal model and apply the model to wheelchair activities for analysis of the estimated joint contact forces. Upper extremity kinematics and handrim wheelchair kinetics were measured over three conditions: level propulsion, ramp propulsion, and a weight relief lift. The experimental data were used as input to a subject-specific musculoskeletal model utilizing optimization to predict joint contact forces of the shoulder during all conditions. The model was validated using a mean absolute error calculation. Model results confirmed that ramp propulsion and weight relief lifts place the shoulder under significantly higher joint contact loading than level propulsion. In addition, they exhibit large superior contact forces that could contribute to impingement. This study highlights the potential impingement risk associated with both the ramp and weight relief lift activities. Level propulsion was shown to have a low relative risk of causing injury, but with consideration of the frequency with which propulsion is performed, this observation is not conclusive.

The validity of parental reports on motor skills performance level in preschool children: a comparison with a standardized motor test.

PubMed

Zysset, Annina E; Kakebeeke, Tanja H; Messerli-Bürgy, Nadine; Meyer, Andrea H; Stülb, Kerstin; Leeger-Aschmann, Claudia S; Schmutz, Einat A; Arhab, Amar; Ferrazzini, Valentina; Kriemler, Susi; Munsch, Simone; Puder, Jardena J; Jenni, Oskar G

2018-05-01

Motor skills are interrelated with essential domains of childhood such as cognitive and social development. Thus, the evaluation of motor skills and the identification of atypical or delayed motor development is crucial in pediatric practice (e.g., during well-child visits). Parental reports on motor skills may serve as possible indicators to decide whether further assessment of a child is necessary or not. We compared parental reports on fundamental motor skills performance level (e.g., hopping, throwing), based on questions frequently asked in pediatric practice, with a standardized motor test in 389 children (46.5% girls/53.5% boys, M age = 3.8 years, SD = 0.5, range 3.0-5.0 years) from the Swiss Preschoolers' Health Study (SPLASHY). Motor skills were examined using the Zurich Neuromotor Assessment 3-5 (ZNA3-5), and parents filled in an online questionnaire on fundamental motor skills performance level. The results showed that the answers from the parental report correlated only weakly with the objectively assessed motor skills (r = .225, p < .001). Although a parental screening instrument for motor skills would be desirable, the parent's report used in this study was not a valid indicator for children's fundamental motor skills. Thus, we may recommend to objectively examine motor skills in clinical practice and not to exclusively rely on parental report. What is Known: • Early assessment of motor skills in preschool children is important because motor skills are essential for the engagement in social activities and the development of cognitive abilities. Atypical or delayed motor development can be an indicator for different developmental needs or disorders. • Pediatricians frequently ask parents about the motor competences of their child during well-child visits. What is New: • The parental report on fundamental motor skills performance level used in this study was not a reliable indicator for describing motor development in the preschool age. • Standardized examinations of motor skills are required to validly assess motor development in preschoolers.

Numerical aerodynamic simulation facility. Preliminary study extension

NASA Technical Reports Server (NTRS)

1978-01-01

The production of an optimized design of key elements of the candidate facility was the primary objective of this report. This was accomplished by effort in the following tasks: (1) to further develop, optimize and describe the function description of the custom hardware; (2) to delineate trade off areas between performance, reliability, availability, serviceability, and programmability; (3) to develop metrics and models for validation of the candidate systems performance; (4) to conduct a functional simulation of the system design; (5) to perform a reliability analysis of the system design; and (6) to develop the software specifications to include a user level high level programming language, a correspondence between the programming language and instruction set and outline the operation system requirements.

Creation of a novel simulator for minimally invasive neurosurgery: fusion of 3D printing and special effects.

PubMed

Weinstock, Peter; Rehder, Roberta; Prabhu, Sanjay P; Forbes, Peter W; Roussin, Christopher J; Cohen, Alan R

2017-07-01

OBJECTIVE Recent advances in optics and miniaturization have enabled the development of a growing number of minimally invasive procedures, yet innovative training methods for the use of these techniques remain lacking. Conventional teaching models, including cadavers and physical trainers as well as virtual reality platforms, are often expensive and ineffective. Newly developed 3D printing technologies can recreate patient-specific anatomy, but the stiffness of the materials limits fidelity to real-life surgical situations. Hollywood special effects techniques can create ultrarealistic features, including lifelike tactile properties, to enhance accuracy and effectiveness of the surgical models. The authors created a highly realistic model of a pediatric patient with hydrocephalus via a unique combination of 3D printing and special effects techniques and validated the use of this model in training neurosurgery fellows and residents to perform endoscopic third ventriculostomy (ETV), an effective minimally invasive method increasingly used in treating hydrocephalus. METHODS A full-scale reproduction of the head of a 14-year-old adolescent patient with hydrocephalus, including external physical details and internal neuroanatomy, was developed via a unique collaboration of neurosurgeons, simulation engineers, and a group of special effects experts. The model contains "plug-and-play" replaceable components for repetitive practice. The appearance of the training model (face validity) and the reproducibility of the ETV training procedure (content validity) were assessed by neurosurgery fellows and residents of different experience levels based on a 14-item Likert-like questionnaire. The usefulness of the training model for evaluating the performance of the trainees at different levels of experience (construct validity) was measured by blinded observers using the Objective Structured Assessment of Technical Skills (OSATS) scale for the performance of ETV. RESULTS A combination of 3D printing technology and casting processes led to the creation of realistic surgical models that include high-fidelity reproductions of the anatomical features of hydrocephalus and allow for the performance of ETV for training purposes. The models reproduced the pulsations of the basilar artery, ventricles, and cerebrospinal fluid (CSF), thus simulating the experience of performing ETV on an actual patient. The results of the 14-item questionnaire showed limited variability among participants' scores, and the neurosurgery fellows and residents gave the models consistently high ratings for face and content validity. The mean score for the content validity questions (4.88) was higher than the mean score for face validity (4.69) (p = 0.03). On construct validity scores, the blinded observers rated performance of fellows significantly higher than that of residents, indicating that the model provided a means to distinguish between novice and expert surgical skills. CONCLUSIONS A plug-and-play lifelike ETV training model was developed through a combination of 3D printing and special effects techniques, providing both anatomical and haptic accuracy. Such simulators offer opportunities to accelerate the development of expertise with respect to new and novel procedures as well as iterate new surgical approaches and innovations, thus allowing novice neurosurgeons to gain valuable experience in surgical techniques without exposing patients to risk of harm.

An evidence-based virtual reality training program for novice laparoscopic surgeons.

PubMed

Aggarwal, Rajesh; Grantcharov, Teodor P; Eriksen, Jens R; Blirup, Dorthe; Kristiansen, Viggo B; Funch-Jensen, Peter; Darzi, Ara

2006-08-01

To develop an evidence-based virtual reality laparoscopic training curriculum for novice laparoscopic surgeons to achieve a proficient level of skill prior to participating in live cases. Technical skills for laparoscopic surgery must be acquired within a competency-based curriculum that begins in the surgical skills laboratory. Implementation of this program necessitates the definition of the validity, learning curves and proficiency criteria on the training tool. The study recruited 40 surgeons, classified into experienced (performed >100 laparoscopic cholecystectomies) or novice groups (<10 laparoscopic cholecystectomies). Ten novices and 10 experienced surgeons were tested on basic tasks, and 11 novices and 9 experienced surgeons on a procedural module for dissection of Calot triangle. Performance of the 2 groups was assessed using time, error, and economy of movement parameters. All basic tasks demonstrated construct validity (Mann-Whitney U test, P < 0.05), and learning curves for novices plateaued at a median of 7 repetitions (Friedman's test, P < 0.05). Expert surgeons demonstrated a learning rate at a median of 2 repetitions (P < 0.05). Performance on the dissection module demonstrated significant differences between experts and novices (P < 0.002); learning curves for novice subjects plateaued at the fourth repetition (P < 0.05). Expert benchmark criteria were defined for validated parameters on each task. A competency-based training curriculum for novice laparoscopic surgeons has been defined. This can serve to ensure that junior trainees have acquired prerequisite levels of skill prior to entering the operating room, and put them directly into practice.

Detecting coached neuropsychological dysfunction: a simulation experiment regarding mild traumatic brain injury.

PubMed

Lau, Lily; Basso, Michael R; Estevis, Eduardo; Miller, Ashley; Whiteside, Douglas M; Combs, Dennis; Arentsen, Timothy J

2017-11-01

Performance validity tests (PVTs) and symptom validity tests (SVTs) are often administered during neuropsychological evaluations. Examinees may be coached to avoid detection by measures of response validity. Relatively little research has evaluated whether graduated levels of coaching has differential effects upon PVT and SVT performance. Accordingly, the present experiment evaluated the effect of graduated levels of coaching upon the classification accuracy of commonly used PVTs and SVTs and the currently accepted criterion of failing two or more PVTs or SVTs. Participants simulated symptoms associated with mild traumatic brain injury (TBI). One group was provided superficial information concerning cognitive, emotional, and physical symptoms. Another group was provided detailed information about such symptoms. A third group was provided detailed information about symptoms and guidance how to evade detection by PVTs. These groups were compared to an honest-responding group. Extending prior experiments, stand-alone and embedded PVT measures were administered in addition to SVTs. The three simulator groups were readily identified by PVTs and SVTs, but a meaningful minority of those provided test-taking strategies eluded detection. The Word Memory Test emerged as the most sensitive indicator of simulated mild TBI symptoms. PVTs achieved more sensitive detection of simulated head injury status than SVTs. Individuals coached to modify test-taking performance were marginally more successful in eluding detection by PVTs and SVTs than those coached with respect to TBI symptoms only. When the criterion of failing two or more PVTs or SVTs was applied, only 5% eluded detection.

Determination of low-level acrylamide in drinking water by liquid chromatography/tandem mass spectrometry.

PubMed

Lucentini, Luca; Ferretti, Emanuele; Veschetti, Enrico; Achene, Laura; Turrio-Baldassarri, Luigi; Ottaviani, Massimo; Bogialli, Sara

2009-01-01

A simple and sensitive liquid chromatographic-tandem mass spectrometric (LC/MS/MS) method has been developed and validated to confirm and quantify acrylamide monomer (AA) in drinking water using [13C3] acrylamide as internal standard (IS). After a preconcentration by solid-phase extraction with spherical activated carbon, analytes were chromatographed on IonPac ICE-AS1 column (9 x 250 mm) under isocratic conditions using acetonitrile-water-0.1 M formic acid (43 + 52 + 5, v/v/v) as the mobile phase. Analysis was achieved using a triple-quadrupole mass analyzer equipped with a turbo ion spray interface. For confirmation and quantification of the analytes, MS data acquisition was performed in the multireaction monitoring mode, selecting 2 precursor ion to product ion transitions for both AA and IS. The method was validated for linearity, sensitivity, accuracy, precision, extraction efficiency, and matrix effect. Linearity in tap water was observed over the concentration range 0.1-2.0 microg/L. Limits of detection and quantification were 0.02 and 0.1 microg/L, respectively. Interday and intraday assays were performed across 3 validation levels (0.1, 0.5, and 1.5 microg/L). Accuracy (as mean recovery) ranged from 89.3 to 96.2% with relative standard deviation <7.98%. Performance characteristics of this LC/MS/MS method make it suitable for regulatory confirmatory analysis of AA in drinking water in compliance with European Union and U.S. Environmental Protection Agency standards.

Advancing the argument for validity of the Alberta Context Tool with healthcare aides in residential long-term care

PubMed Central

2011-01-01

Background Organizational context has the potential to influence the use of new knowledge. However, despite advances in understanding the theoretical base of organizational context, its measurement has not been adequately addressed, limiting our ability to quantify and assess context in healthcare settings and thus, advance development of contextual interventions to improve patient care. We developed the Alberta Context Tool (the ACT) to address this concern. It consists of 58 items representing 10 modifiable contextual concepts. We reported the initial validation of the ACT in 2009. This paper presents the second stage of the psychometric validation of the ACT. Methods We used the Standards for Educational and Psychological Testing to frame our validity assessment. Data from 645 English speaking healthcare aides from 25 urban residential long-term care facilities (nursing homes) in the three Canadian Prairie Provinces were used for this stage of validation. In this stage we focused on: (1) advanced aspects of internal structure (e.g., confirmatory factor analysis) and (2) relations with other variables validity evidence. To assess reliability and validity of scores obtained using the ACT we conducted: Cronbach's alpha, confirmatory factor analysis, analysis of variance, and tests of association. We also assessed the performance of the ACT when individual responses were aggregated to the care unit level, because the instrument was developed to obtain unit-level scores of context. Results Item-total correlations exceeded acceptable standards (> 0.3) for the majority of items (51 of 58). We ran three confirmatory factor models. Model 1 (all ACT items) displayed unacceptable fit overall and for five specific items (1 item on adequate space for resident care in the Organizational Slack-Space ACT concept and 4 items on use of electronic resources in the Structural and Electronic Resources ACT concept). This prompted specification of two additional models. Model 2 used the 7 scaled ACT concepts while Model 3 used the 3 count-based ACT concepts. Both models displayed substantially improved fit in comparison to Model 1. Cronbach's alpha for the 10 ACT concepts ranged from 0.37 to 0.92 with 2 concepts performing below the commonly accepted standard of 0.70. Bivariate associations between the ACT concepts and instrumental research utilization levels (which the ACT should predict) were statistically significant at the 5% level for 8 of the 10 ACT concepts. The majority (8/10) of the ACT concepts also showed a statistically significant trend of increasing mean scores when arrayed across the lowest to the highest levels of instrumental research use. Conclusions The validation process in this study demonstrated additional empirical support for construct validity of the ACT, when completed by healthcare aides in nursing homes. The overall pattern of the data was consistent with the structure hypothesized in the development of the ACT and supports the ACT as an appropriate measure for assessing organizational context in nursing homes. Caution should be applied in using the one space and four electronic resource items that displayed misfit in this study with healthcare aides until further assessments are made. PMID:21767378

SCIAMACHY validation by aircraft remote measurements: design, execution, and first results of the SCIA-VALUE mission

NASA Astrophysics Data System (ADS)

Fix, A.; Ehret, G.; Flentje, H.; Poberaj, G.; Gottwald, M.; Finkenzeller, H.; Bremer, H.; Bruns, M.; Burrows, J. P.; Kleinböhl, A.; Küllmann, H.; Kuttippurath, J.; Richter, A.; Wang, P.; Heue, K.-P.; Platt, U.; Wagner, T.

2004-12-01

For the first time three different remote sensing instruments - a sub-millimeter radiometer, a differential optical absorption spectrometer in the UV-visible spectral range, and a lidar - were deployed aboard DLR's meteorological research aircraft Falcon 20 to validate a large number of SCIAMACHY level 2 and off-line data products such as O3, NO2, N2O, BrO, OClO, H2O, aerosols, and clouds. Within two main validation campaigns of the SCIA-VALUE mission (SCIAMACHY VALidation and Utilization Experiment) extended latitudinal cross-sections stretching from polar regions to the tropics as well as longitudinal cross sections at polar latitudes at about 70° N and the equator have been generated. This contribution gives an overview over the campaigns performed and reports on the observation strategy for achieving the validation goals. We also emphasize the synergetic use of the novel set of aircraft instrumentation and the usefulness of this innovative suite of remote sensing instruments for satellite validation.

SCIAMACHY validation by aircraft remote sensing: design, execution, and first measurement results of the SCIA-VALUE mission

NASA Astrophysics Data System (ADS)

Fix, A.; Ehret, G.; Flentje, H.; Poberaj, G.; Gottwald, M.; Finkenzeller, H.; Bremer, H.; Bruns, M.; Burrows, J. P.; Kleinböhl, A.; Küllmann, H.; Kuttippurath, J.; Richter, A.; Wang, P.; Heue, K.-P.; Platt, U.; Pundt, I.; Wagner, T.

2005-05-01

For the first time three different remote sensing instruments - a sub-millimeter radiometer, a differential optical absorption spectrometer in the UV-visible spectral range, and a lidar - were deployed aboard DLR's meteorological research aircraft Falcon 20 to validate a large number of SCIAMACHY level 2 and off-line data products such as O3, NO2, N2O, BrO, OClO, H2O, aerosols, and clouds. Within two validation campaigns of the SCIA-VALUE mission (SCIAMACHY VALidation and Utilization Experiment) extended latitudinal cross-sections stretching from polar regions to the tropics as well as longitudinal cross sections at polar latitudes at about 70° N and the equator were generated. This contribution gives an overview over the campaigns performed and reports on the observation strategy for achieving the validation goals. We also emphasize the synergetic use of the novel set of aircraft instrumentation and the usefulness of this innovative suite of remote sensing instruments for satellite validation.

Development of System-level Performance Measures for Evaluation of Models of Care for Inflammatory Arthritis in Canada.

PubMed

Barber, Claire E H; Marshall, Deborah A; Mosher, Dianne P; Akhavan, Pooneh; Tucker, Lori; Houghton, Kristin; Batthish, Michelle; Levy, Deborah M; Schmeling, Heinrike; Ellsworth, Janet; Tibollo, Heidi; Grant, Sean; Khodyakov, Dmitry; Lacaille, Diane

2016-03-01

To develop system-level performance measures for evaluating the care of patients with inflammatory arthritis (IA), including rheumatoid arthritis (RA), psoriatic arthritis, ankylosing spondylitis, and juvenile idiopathic arthritis. This study involved several methodological phases. Over multiple rounds, various participants were asked to help define a set of candidate measurement themes. A systematic search was conducted of existing guidelines and measures. A set of 6 performance measures was defined and presented to 50 people, including patients with IA, rheumatologists, allied health professionals, and researchers using a 3-round, online, modified Delphi process. Participants rated the validity, feasibility, relevance, and likelihood of use of the measures. Measures with median ratings ≥ 7 for validity and relevance were included in the final set. Six performance measures were developed evaluating the following aspects of care, with each measure being applied separately for each type of IA except where specified: waiting times for rheumatology consultation for patients with new onset IA, percentage of patients with IA seen by a rheumatologist, percentage of patients with IA seen in yearly followup by a rheumatologist, percentage of patients with RA treated with a disease-modifying antirheumatic drug (DMARD), time to DMARD therapy in RA, and number of rheumatologists per capita. The first set of system-level performance measures for IA care in Canada has been developed with broad input. The measures focus on timely access to care and initiation of appropriate treatment for patients with IA, and are likely to be of interest to other arthritis care systems internationally.

Cerebral NIRS performance testing with molded and 3D-printed phantoms (Conference Presentation)

NASA Astrophysics Data System (ADS)

Wang, Jianting; Huang, Stanley; Chen, Yu; Welle, Cristin G.; Pfefer, T. Joshua

2017-03-01

Near-infrared spectroscopy (NIRS) has emerged as a low-cost, portable approach for rapid, point-of-care detection of hematomas caused by traumatic brain injury. As a new technology, there is a need to develop standardized test methods for objective, quantitative performance evaluation of these devices. Towards this goal, we have developed and studied two types of phantom-based testing approaches. The first involves 3D-printed phantoms incorporating hemoglobin-filled inclusions. Phantom layers representing specific cerebral tissues were printed using photopolymers doped with varying levels of titanium oxide and black resin. The accuracy, precision and spectral dependence of printed phantom optical properties were validated using spectrophotometry. The phantom also includes a hematoma inclusion insert which was filled with a hemoglobin solution. Oxygen saturation levels were modified by adding sodium dithionite at calibrated concentrations. The second phantom approach involves molded silicone layers with a superficial region - simulating the scalp and skull - comprised of removable layers to vary hematoma size and depth, and a bottom layer representing brain matter. These phantoms were tested with both a commercial hematoma detector and a custom NIRS system to optimize their designs and validate their utility in performing inter-device comparisons. The effects of hematoma depth, diameter, and height, as well as tissue optical properties and biological variables including hemoglobin saturation level and scalp/skull thickness were studied. Results demonstrate the ability to quantitatively compare NIRS device performance and indicate the promise of using 3D printing to achieve phantoms with realistic variations in tissue optical properties for evaluating biophotonic device performance.

Leisure-time physical activity associates with cognitive decline

PubMed Central

Willey, Joshua Z.; Gardener, Hannah; Caunca, Michelle R.; Moon, Yeseon Park; Dong, Chuanhui; Cheung, Yuen K.; Sacco, Ralph L.; Elkind, Mitchell S.V.

2016-01-01

Objective: Because leisure-time physical activity (LTPA) is protective against incident dementia, we hypothesized that LTPA is protective against decline in domain-specific cognitive performance. Methods: As part of the Northern Manhattan Study, LTPA was ascertained at enrollment using a validated in-person questionnaire. We assessed cognition in participants in the Northern Manhattan Study MRI substudy using a standard neuropsychological examination (NPE) (n = 1,228), and a repeat examination was performed 5 years later (n = 876). LTPA was summarized as the maximum intensity of any activity performed, classified as none to light intensity (physical inactivity) (90%) vs moderate to heavy intensity (10%). The NPE was subcategorized using standardized z scores over validated domains: processing speed, semantic memory, episodic memory, and executive function. We used multivariable linear regression models to examine the association of LTPA with initial and change in cognitive performance. Analyses were adjusted for sociodemographics, cardiovascular disease risk factors, and MRI findings (white matter hyperintensity volume, silent brain infarcts, cerebral volume). Results: No/low levels of LTPA were associated with worse executive function, semantic memory, and processing speed scores on the first NPE. The associations were slightly attenuated and no longer significant after adjusting for vascular risk factors. Cognitively unimpaired participants reporting no/low LTPA vs moderate/high levels declined more over time in processing speed (β = −0.231 ± 0.112, p = 0.040) and episodic memory (β = −0.223 ± 0.117, p = 0.057) adjusting for sociodemographic and vascular risk factors. Conclusions: A low level of LTPA is independently associated with greater decline in cognitive performance over time across domains. PMID:27009261

Reliability and validity of the adapted Resistance Training Skills Battery for Children.

PubMed

Furzer, Bonnie J; Bebich-Philip, Marc D; Wright, Kemi E; Reid, Siobhan L; Thornton, Ashleigh L

2017-12-29

Resistance training (RT) is emerging as a training modality to improve motor function and facilitate physical activity participation in children across the motor proficiency spectrum. Although RT competency assessments have been established and validated among adolescent cohorts, the extent to which these methods are suitable for assessing children's RT skills is unknown. This project aimed to assess the psychometric properties of the adapted Resistance Training Skills Battery for Children (RTSBc), in children with varying motor proficiency. Repeated measures design with 40 participants (M age=8.2±1.7years) displaying varying levels of motor proficiency. Participants performed the adapted RTSBc on two occasions, receiving a score for their execution of each component, in addition to an overall RT skill quotient child (RTSQc). Cronbach's alpha, intra-class correlation (ICC), Bland-Altman analysis, and typical error were used to assess test-retest reliability. To examine construct validity, exploratory factor analysis was performed alongside computing correlations between participants' muscle strength, motor proficiency, age, lean muscle mass, and RTSQc. The RTSBc displayed an acceptable level of internal consistency (alpha=0.86) and test-retest reliability (ICC range=0.86-0.99). Exploratory factor analysis supported internal test structure, with all six RT skills loading strongly on a single factor (range 0.56-0.89). Analyses of structural validity revealed positive correlations for RTSQc in relation to motor proficiency (r=0.52, p<0.001) and strength scores (r=0.61, p<0.001). Analyses revealed support for the construct validity and test-retest reliability of the RTSBc, providing preliminary evidence that the RTSBc is appropriate for use in the assessment of children's RT competency. Copyright © 2018 Sports Medicine Australia. Published by Elsevier Ltd. All rights reserved.

Development and Validation of a Statistical Shape Modeling-Based Finite Element Model of the Cervical Spine Under Low-Level Multiple Direction Loading Conditions

PubMed Central

Bredbenner, Todd L.; Eliason, Travis D.; Francis, W. Loren; McFarland, John M.; Merkle, Andrew C.; Nicolella, Daniel P.

2014-01-01

Cervical spinal injuries are a significant concern in all trauma injuries. Recent military conflicts have demonstrated the substantial risk of spinal injury for the modern warfighter. Finite element models used to investigate injury mechanisms often fail to examine the effects of variation in geometry or material properties on mechanical behavior. The goals of this study were to model geometric variation for a set of cervical spines, to extend this model to a parametric finite element model, and, as a first step, to validate the parametric model against experimental data for low-loading conditions. Individual finite element models were created using cervical spine (C3–T1) computed tomography data for five male cadavers. Statistical shape modeling (SSM) was used to generate a parametric finite element model incorporating variability of spine geometry, and soft-tissue material property variation was also included. The probabilistic loading response of the parametric model was determined under flexion-extension, axial rotation, and lateral bending and validated by comparison to experimental data. Based on qualitative and quantitative comparison of the experimental loading response and model simulations, we suggest that the model performs adequately under relatively low-level loading conditions in multiple loading directions. In conclusion, SSM methods coupled with finite element analyses within a probabilistic framework, along with the ability to statistically validate the overall model performance, provide innovative and important steps toward describing the differences in vertebral morphology, spinal curvature, and variation in material properties. We suggest that these methods, with additional investigation and validation under injurious loading conditions, will lead to understanding and mitigating the risks of injury in the spine and other musculoskeletal structures. PMID:25506051

Pulse design for multilevel systems by utilizing Lie transforms

NASA Astrophysics Data System (ADS)

Kang, Yi-Hao; Chen, Ye-Hong; Shi, Zhi-Cheng; Huang, Bi-Hua; Song, Jie; Xia, Yan

2018-03-01

We put forward a scheme to design pulses to manipulate multilevel systems with Lie transforms. A formula to reverse construct a control Hamiltonian is given and is applied in pulse design in the three- and four-level systems as examples. To demonstrate the validity of the scheme, we perform numerical simulations, which show the population transfers for cascaded three-level and N -type four-level Rydberg atoms can be completed successfully with high fidelities. Therefore, the scheme may benefit quantum information tasks based on multilevel systems.

[SCREENING OF NUTRITIONAL STATUS AMONG ELDERLY PEOPLE AT FAMILY MEDICINE].

PubMed

Račić, M; Ivković, N; Kusmuk, S

2015-11-01

The prevalence of malnutrition in elderly is high. Malnutrition or risk of malnutrition can be detected by use of nutritional screening or assessment tools. This systematic review aimed to identify tools that would be reliable, valid, sensitive and specific for nutritional status screening in patients older than 65 at family medicine. The review was performed following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. Studies were retrieved using MEDLINE (via Ovid), PubMed and Cochrane Library electronic databases and by manual searching of relevant articles listed in reference list of key publications. The electronic databases were searched using defined key words adapted to each database and using MESH terms. Manual revision of reviews and original articles was performed using Electronic Journals Library. Included studies involved development and validation of screening tools in the community-dwelling elderly population. The tools, subjected to validity and reliability testing for use in the community-dwelling elderly population were Mini Nutritional Assessment (MNA), Mini Nutritional Assessment-Short Form (MNA-SF), Nutrition Screening Initiative (NSI), which includes DETERMINE list, Level I and II Screen, Seniors in the Community: Risk Evaluation for Eating, and Nutrition (SCREEN I and SCREEN II), Subjective Global Assessment (SGA), Nutritional Risk Index (NRI), and Malaysian and South African tool. MNA and MNA-SF appear to have highest reliability and validity for screening of community-dwelling elderly, while the reliability and validity of SCREEN II are good. The authors conclude that whilst several tools have been developed, most have not undergone extensive testing to demonstrate their ability to identify nutritional risk. MNA and MNA-SF have the highest reliability and validity for screening of nutritional status in the community-dwelling elderly, and the reliability and validity of SCREEN II are satisfactory. These instruments also contain all three nutritional status indicators and are practical for use in family medicine. However, the gold standard for screening cannot be set because testing of reliability and continuous validation in the study with a higher level of evidence need to be conducted in family medicine.

Laparoscopic Common Bile Duct Exploration Four-Task Training Model: Construct Validity

PubMed Central

Otaño, Natalia; Rodríguez, Omaira; Sánchez, Renata; Benítez, Gustavo; Schweitzer, Michael

2012-01-01

Background: Training models in laparoscopic surgery allow the surgical team to practice procedures in a safe environment. We have proposed the use of a 4-task, low-cost inert model to practice critical steps of laparoscopic common bile duct exploration. Methods: The performance of 3 groups with different levels of expertise in laparoscopic surgery, novices (A), intermediates (B), and experts (C), was evaluated using a low-cost inert model in the following tasks: (1) intraoperative cholangiography catheter insertion, (2) transcystic exploration, (3) T-tube placement, and (4) choledochoscope management. Kruskal-Wallis and Mann-Whitney tests were used to identify differences among the groups. Results: A total of 14 individuals were evaluated: 5 novices (A), 5 intermediates (B), and 4 experts (C). The results involving intraoperative cholangiography catheter insertion were similar among the 3 groups. As for the other tasks, the expert had better results than the other 2, in which no significant differences occurred. The proposed model is able to discriminate among individuals with different levels of expertise, indicating that the abilities that the model evaluates are relevant in the surgeon's performance in CBD exploration. Conclusions: Construct validity for tasks 2 and 3 was demonstrated. However, task 1 was no capable of distinguishing between groups, and task 4 was not statistically validated. PMID:22906323

The script concordance test in radiation oncology: validation study of a new tool to assess clinical reasoning

PubMed Central

Lambert, Carole; Gagnon, Robert; Nguyen, David; Charlin, Bernard

2009-01-01

Background The Script Concordance test (SCT) is a reliable and valid tool to evaluate clinical reasoning in complex situations where experts' opinions may be divided. Scores reflect the degree of concordance between the performance of examinees and that of a reference panel of experienced physicians. The purpose of this study is to demonstrate SCT's usefulness in radiation oncology. Methods A 90 items radiation oncology SCT was administered to 155 participants. Three levels of experience were tested: medical students (n = 70), radiation oncology residents (n = 38) and radiation oncologists (n = 47). Statistical tests were performed to assess reliability and to document validity. Results After item optimization, the test comprised 30 cases and 70 questions. Cronbach alpha was 0.90. Mean scores were 51.62 (± 8.19) for students, 71.20 (± 9.45) for residents and 76.67 (± 6.14) for radiation oncologists. The difference between the three groups was statistically significant when compared by the Kruskall-Wallis test (p < 0.001). Conclusion The SCT is reliable and useful to discriminate among participants according to their level of experience in radiation oncology. It appears as a useful tool to document the progression of reasoning during residency training. PMID:19203358

MicroRNA Expression Profiling to Identify and Validate Reference Genes for the Relative Quantification of microRNA in Rectal Cancer.

PubMed

Eriksen, Anne Haahr Mellergaard; Andersen, Rikke Fredslund; Pallisgaard, Niels; Sørensen, Flemming Brandt; Jakobsen, Anders; Hansen, Torben Frøstrup

2016-01-01

MicroRNAs (miRNAs) play important roles in regulating biological processes at the post-transcriptional level. Deregulation of miRNAs has been observed in cancer, and miRNAs are being investigated as potential biomarkers regarding diagnosis, prognosis and prediction in cancer management. Real-time quantitative polymerase chain reaction (RT-qPCR) is commonly used, when measuring miRNA expression. Appropriate normalisation of RT-qPCR data is important to ensure reliable results. The aim of the present study was to identify stably expressed miRNAs applicable as normaliser candidates in future studies of miRNA expression in rectal cancer. We performed high-throughput miRNA profiling (OpenArray®) on ten pairs of laser micro-dissected rectal cancer tissue and adjacent stroma. A global mean expression normalisation strategy was applied to identify the most stably expressed miRNAs for subsequent validation. In the first validation experiment, a panel of miRNAs were analysed on 25 pairs of micro dissected rectal cancer tissue and adjacent stroma. Subsequently, the same miRNAs were analysed in 28 pairs of rectal cancer tissue and normal rectal mucosa. From the miRNA profiling experiment, miR-645, miR-193a-5p, miR-27a and let-7g were identified as stably expressed, both in malignant and stromal tissue. In addition, NormFinder confirmed high expression stability for the four miRNAs. In the RT-qPCR based validation experiments, no significant difference between tumour and stroma/normal rectal mucosa was detected for the mean of the normaliser candidates miR-27a, miR-193a-5p and let-7g (first validation P = 0.801, second validation P = 0.321). MiR-645 was excluded from the data analysis, because it was undetected in 35 of 50 samples (first validation) and in 24 of 56 samples (second validation), respectively. Significant difference in expression level of RNU6B was observed between tumour and adjacent stromal (first validation), and between tumour and normal rectal mucosa (second validation). We recommend the mean expression of miR-27a, miR-193a-5p and let-7g as normalisation factor, when performing miRNA expression analyses by RT-qPCR on rectal cancer tissue.

Measuring verbal and non-verbal communication in aphasia: reliability, validity, and sensitivity to change of the Scenario Test.

PubMed

van der Meulen, Ineke; van de Sandt-Koenderman, W Mieke E; Duivenvoorden, Hugo J; Ribbers, Gerard M

2010-01-01

This study explores the psychometric qualities of the Scenario Test, a new test to assess daily-life communication in severe aphasia. The test is innovative in that it: (1) examines the effectiveness of verbal and non-verbal communication; and (2) assesses patients' communication in an interactive setting, with a supportive communication partner. To determine the reliability, validity, and sensitivity to change of the Scenario Test and discuss its clinical value. The Scenario Test was administered to 122 persons with aphasia after stroke and to 25 non-aphasic controls. Analyses were performed for the entire group of persons with aphasia, as well as for a subgroup of persons unable to communicate verbally (n = 43). Reliability (internal consistency, test-retest reliability, inter-judge, and intra-judge reliability) and validity (internal validity, convergent validity, known-groups validity) and sensitivity to change were examined using standard psychometric methods. The Scenario Test showed high levels of reliability. Internal consistency (Cronbach's alpha = 0.96; item-rest correlations = 0.58-0.82) and test-retest reliability (ICC = 0.98) were high. Agreement between judges in total scores was good, as indicated by the high inter- and intra-judge reliability (ICC = 0.86-1.00). Agreement in scores on the individual items was also good (square-weighted kappa values 0.61-0.92). The test demonstrated good levels of validity. A principal component analysis for categorical data identified two dimensions, interpreted as general communication and communicative creativity. Correlations with three other instruments measuring communication in aphasia, that is, Spontaneous Speech interview from the Aachen Aphasia Test (AAT), Amsterdam-Nijmegen Everyday Language Test (ANELT), and Communicative Effectiveness Index (CETI), were moderate to strong (0.50-0.85) suggesting good convergent validity. Group differences were observed between persons with aphasia and non-aphasic controls, as well as between persons with aphasia unable to use speech to convey information and those able to communicate verbally; this indicates good known-groups validity. The test was sensitive to changes in performance, measured over a period of 6 months. The data support the reliability and validity of the Scenario Test as an instrument for examining daily-life communication in aphasia. The test focuses on multimodal communication; its psychometric qualities enable future studies on the effect of Alternative and Augmentative Communication (AAC) training in aphasia.

Design and Validation of an Infrared Badal Optometer for Laser Speckle (IBOLS)

PubMed Central

Teel, Danielle F. W.; Copland, R. James; Jacobs, Robert J.; Wells, Thad; Neal, Daniel R.; Thibos, Larry N.

2009-01-01

Purpose To validate the design of an infrared wavefront aberrometer with a Badal optometer employing the principle of laser speckle generated by a spinning disk and infrared light. The instrument was designed for subjective meridional refraction in infrared light by human patients. Methods Validation employed a model eye with known refractive error determined with an objective infrared wavefront aberrometer. The model eye was used to produce a speckle pattern on an artificial retina with controlled amounts of ametropia introduced with auxiliary ophthalmic lenses. A human observer performed the psychophysical task of observing the speckle pattern (with the aid of a video camera sensitive to infrared radiation) formed on the artificial retina. Refraction was performed by adjusting the vergence of incident light with the Badal optometer to nullify the motion of laser speckle. Validation of the method was performed for different levels of spherical ametropia and for various configurations of an astigmatic model eye. Results Subjective measurements of meridional refractive error over the range −4D to + 4D agreed with astigmatic refractive errors predicted by the power of the model eye in the meridian of motion of the spinning disk. Conclusions Use of a Badal optometer to control laser speckle is a valid method for determining subjective refractive error at infrared wavelengths. Such an instrument will be useful for comparing objective measures of refractive error obtained for the human eye with autorefractors and wavefront aberrometers that employ infrared radiation. PMID:18772719

Development and validation of trauma surgical skills metrics: Preliminary assessment of performance after training.

PubMed

Shackelford, Stacy; Garofalo, Evan; Shalin, Valerie; Pugh, Kristy; Chen, Hegang; Pasley, Jason; Sarani, Babak; Henry, Sharon; Bowyer, Mark; Mackenzie, Colin F

2015-07-01

Maintaining trauma-specific surgical skills is an ongoing challenge for surgical training programs. An objective assessment of surgical skills is needed. We hypothesized that a validated surgical performance assessment tool could detect differences following a training intervention. We developed surgical performance assessment metrics based on discussion with expert trauma surgeons, video review of 10 experts and 10 novice surgeons performing three vascular exposure procedures and lower extremity fasciotomy on cadavers, and validated the metrics with interrater reliability testing by five reviewers blinded to level of expertise and a consensus conference. We tested these performance metrics in 12 surgical residents (Year 3-7) before and 2 weeks after vascular exposure skills training in the Advanced Surgical Skills for Exposure in Trauma (ASSET) course. Performance was assessed in three areas as follows: knowledge (anatomic, management), procedure steps, and technical skills. Time to completion of procedures was recorded, and these metrics were combined into a single performance score, the Trauma Readiness Index (TRI). Wilcoxon matched-pairs signed-ranks test compared pretraining/posttraining effects. Mean time to complete procedures decreased by 4.3 minutes (from 13.4 minutes to 9.1 minutes). The performance component most improved by the 1-day skills training was procedure steps, completion of which increased by 21%. Technical skill scores improved by 12%. Overall knowledge improved by 3%, with 18% improvement in anatomic knowledge. TRI increased significantly from 50% to 64% with ASSET training. Interrater reliability of the surgical performance assessment metrics was validated with single intraclass correlation coefficient of 0.7 to 0.98. A trauma-relevant surgical performance assessment detected improvements in specific procedure steps and anatomic knowledge taught during a 1-day course, quantified by the TRI. ASSET training reduced time to complete vascular control by one third. Future applications include assessing specific skills in a larger surgeon cohort, assessing military surgical readiness, and quantifying skill degradation with time since training.

Validation of NH3 satellite observations by ground-based FTIR measurements

NASA Astrophysics Data System (ADS)

Dammers, Enrico; Palm, Mathias; Van Damme, Martin; Shephard, Mark; Cady-Pereira, Karen; Capps, Shannon; Clarisse, Lieven; Coheur, Pierre; Erisman, Jan Willem

2016-04-01

Global emissions of reactive nitrogen have been increasing to an unprecedented level due to human activities and are estimated to be a factor four larger than pre-industrial levels. Concentration levels of NOx are declining, but ammonia (NH3) levels are increasing around the globe. While NH3 at its current concentrations poses significant threats to the environment and human health, relatively little is known about the total budget and global distribution. Surface observations are sparse and mainly available for north-western Europe, the United States and China and are limited by the high costs and poor temporal and spatial resolution. Since the lifetime of atmospheric NH3 is short, on the order of hours to a few days, due to efficient deposition and fast conversion to particulate matter, the existing surface measurements are not sufficient to estimate global concentrations. Advanced space-based IR-sounders such as the Tropospheric Emission Spectrometer (TES), the Infrared Atmospheric Sounding Interferometer (IASI), and the Cross-track Infrared Sounder (CrIS) enable global observations of atmospheric NH3 that help overcome some of the limitations of surface observations. However, the satellite NH3 retrievals are complex requiring extensive validation. Presently there have only been a few dedicated satellite NH3 validation campaigns performed with limited spatial, vertical or temporal coverage. Recently a retrieval methodology was developed for ground-based Fourier Transform Infrared Spectroscopy (FTIR) instruments to obtain vertical concentration profiles of NH3. Here we show the applicability of retrieved columns from nine globally distributed stations with a range of NH3 pollution levels to validate satellite NH3 products.

EmptyHeaded: A Relational Engine for Graph Processing

PubMed Central

Aberger, Christopher R.; Tu, Susan; Olukotun, Kunle; Ré, Christopher

2016-01-01

There are two types of high-performance graph processing engines: low- and high-level engines. Low-level engines (Galois, PowerGraph, Snap) provide optimized data structures and computation models but require users to write low-level imperative code, hence ensuring that efficiency is the burden of the user. In high-level engines, users write in query languages like datalog (SociaLite) or SQL (Grail). High-level engines are easier to use but are orders of magnitude slower than the low-level graph engines. We present EmptyHeaded, a high-level engine that supports a rich datalog-like query language and achieves performance comparable to that of low-level engines. At the core of EmptyHeaded’s design is a new class of join algorithms that satisfy strong theoretical guarantees but have thus far not achieved performance comparable to that of specialized graph processing engines. To achieve high performance, EmptyHeaded introduces a new join engine architecture, including a novel query optimizer and data layouts that leverage single-instruction multiple data (SIMD) parallelism. With this architecture, EmptyHeaded outperforms high-level approaches by up to three orders of magnitude on graph pattern queries, PageRank, and Single-Source Shortest Paths (SSSP) and is an order of magnitude faster than many low-level baselines. We validate that EmptyHeaded competes with the best-of-breed low-level engine (Galois), achieving comparable performance on PageRank and at most 3× worse performance on SSSP. PMID:28077912

Requirements for facilities and measurement techniques to support CFD development for hypersonic aircraft

NASA Technical Reports Server (NTRS)

Sellers, William L., III; Dwoyer, Douglas L.

1992-01-01

The design of a hypersonic aircraft poses unique challenges to the engineering community. Problems with duplicating flight conditions in ground based facilities have made performance predictions risky. Computational fluid dynamics (CFD) has been proposed as an additional means of providing design data. At the present time, CFD codes are being validated based on sparse experimental data and then used to predict performance at flight conditions with generally unknown levels of uncertainty. This paper will discuss the facility and measurement techniques that are required to support CFD development for the design of hypersonic aircraft. Illustrations are given of recent success in combining experimental and direct numerical simulation in CFD model development and validation for hypersonic perfect gas flows.

Testing the Construct Validity of a Virtual Reality Hip Arthroscopy Simulator.

PubMed

Khanduja, Vikas; Lawrence, John E; Audenaert, Emmanuel

2017-03-01

To test the construct validity of the hip diagnostics module of a virtual reality hip arthroscopy simulator. Nineteen orthopaedic surgeons performed a simulated arthroscopic examination of a healthy hip joint using a 70° arthroscope in the supine position. Surgeons were categorized as either expert (those who had performed 250 hip arthroscopies or more) or novice (those who had performed fewer than this). Twenty-one specific targets were visualized within the central and peripheral compartments; 9 via the anterior portal, 9 via the anterolateral portal, and 3 via the posterolateral portal. This was immediately followed by a task testing basic probe examination of the joint in which a series of 8 targets were probed via the anterolateral portal. During the tasks, the surgeon's performance was evaluated by the simulator using a set of predefined metrics including task duration, number of soft tissue and bone collisions, and distance travelled by instruments. No repeat attempts at the tasks were permitted. Construct validity was then evaluated by comparing novice and expert group performance metrics over the 2 tasks using the Mann-Whitney test, with a P value of less than .05 considered significant. On the visualization task, the expert group outperformed the novice group on time taken (P = .0003), number of collisions with soft tissue (P = .001), number of collisions with bone (P = .002), and distance travelled by the arthroscope (P = .02). On the probe examination, the 2 groups differed only in the time taken to complete the task (P = .025) with no significant difference in other metrics. Increased experience in hip arthroscopy was reflected by significantly better performance on the virtual reality simulator across 2 tasks, supporting its construct validity. This study validates a virtual reality hip arthroscopy simulator and supports its potential for developing basic arthroscopic skills. Level III. Copyright © 2016 Arthroscopy Association of North America. All rights reserved.

A Perspective on Computational Human Performance Models as Design Tools

NASA Technical Reports Server (NTRS)

Jones, Patricia M.

2010-01-01

The design of interactive systems, including levels of automation, displays, and controls, is usually based on design guidelines and iterative empirical prototyping. A complementary approach is to use computational human performance models to evaluate designs. An integrated strategy of model-based and empirical test and evaluation activities is particularly attractive as a methodology for verification and validation of human-rated systems for commercial space. This talk will review several computational human performance modeling approaches and their applicability to design of display and control requirements.

A Novel Health Evaluation Strategy for Multifunctional Self-Validating Sensors

PubMed Central

Shen, Zhengguang; Wang, Qi

2013-01-01

The performance evaluation of sensors is very important in actual application. In this paper, a theory based on multi-variable information fusion is studied to evaluate the health level of multifunctional sensors. A novel conception of health reliability degree (HRD) is defined to indicate a quantitative health level, which is different from traditional so-called qualitative fault diagnosis. To evaluate the health condition from both local and global perspectives, the HRD of a single sensitive component at multiple time points and the overall multifunctional sensor at a single time point are defined, respectively. The HRD methodology is emphasized by using multi-variable data fusion technology coupled with a grey comprehensive evaluation method. In this method, to acquire the distinct importance of each sensitive unit and the sensitivity of different time points, the information entropy and analytic hierarchy process method are used, respectively. In order to verify the feasibility of the proposed strategy, a health evaluating experimental system for multifunctional self-validating sensors was designed. The five different health level situations have been discussed. Successful results show that the proposed method is feasible, the HRD could be used to quantitatively indicate the health level and it does have a fast response to the performance changes of multifunctional sensors. PMID:23291576

10 CFR 26.131 - Cutoff levels for validity screening and initial validity tests.

Code of Federal Regulations, 2010 CFR

2010-01-01

... 10 Energy 1 2010-01-01 2010-01-01 false Cutoff levels for validity screening and initial validity tests. 26.131 Section 26.131 Energy NUCLEAR REGULATORY COMMISSION FITNESS FOR DUTY PROGRAMS Licensee Testing Facilities § 26.131 Cutoff levels for validity screening and initial validity tests. (a) Each...

10 CFR 26.131 - Cutoff levels for validity screening and initial validity tests.

Code of Federal Regulations, 2011 CFR

2011-01-01

... 10 Energy 1 2011-01-01 2011-01-01 false Cutoff levels for validity screening and initial validity tests. 26.131 Section 26.131 Energy NUCLEAR REGULATORY COMMISSION FITNESS FOR DUTY PROGRAMS Licensee Testing Facilities § 26.131 Cutoff levels for validity screening and initial validity tests. (a) Each...

Liquid chromatography-tandem mass spectrometric assay for the T790M mutant EGFR inhibitor osimertinib (AZD9291) in human plasma.

PubMed

Rood, Johannes J M; van Bussel, Mark T J; Schellens, Jan H M; Beijnen, Jos H; Sparidans, Rolf W

2016-09-15

A method for the quantitative analysis by ultra-performance liquid chromatography-tandem mass spectrometry of the highly selective irreversible covalent inhibitor of EGFR-TK, osimertinib in human plasma was developed and validated, using pazopanib as an internal standard. The validation was performed in a range from 1 to 1000ng/ml, with the lowest level corresponding to the lower limit of quantitation. Gradient elution was performed on a 1.8μm particle trifunctional bonded C18 column by 1% (v/v) formic acid in water, and acetonitrile as mobile phase. The analyte was detected in the selected reaction monitoring mode of a triple quadrupole mass spectrometer after positive ionization with the heated electrospray interface. Within-day precisions ranged from 3.4 to 10.3%, and between-day precisions from 3.8 to 10.4%, accuracies were 95.5-102.8%. Plasma (either lithium heparin or sodium EDTA) pretreatment was performed by salting-out assisted liquid-liquid extraction using acetonitrile and magnesium sulfate. This method was used to analyze the osimertinib blood plasma levels of five adult patients with metastatic T790M mutated non-small cellular lung carcinoma for therapeutic drug monitoring purposes. Copyright © 2016 Elsevier B.V. All rights reserved.

Expert Advisor (EA) Evaluation System Using Web-based ELECTRE Method in Foreign Exchange (Forex) Market

NASA Astrophysics Data System (ADS)

Satibi; Widodo, Catur Edi; Farikhin

2018-02-01

This research aims to optimize forex trading profit automatically using EA but its still keep considering accuracy and drawdown levels. The evaluation system will classify EA performance based on trading market sessions (Sydney, Tokyo, London and New York) to determine the right EA to be used in certain market sessions. This evaluation system is a web-based ELECTRE methods that interact in real-time with EA through web service and are able to present real-time charts performance dashboard using web socket protocol communications. Web applications are programmed using NodeJs. In the testing period, all EAs had been simulated 24 hours in all market sessions for three months, the best EA is valued by its profit, accuracy and drawdown criteria that calculated using web-based ELECTRE method. The ideas of this research are to compare the best EA on testing period with collaboration performances of each best classified EA by market sessions. This research uses three months historical data of EUR/USD as testing period and other 3 months as validation period. As a result, performance of collaboration four best EA classified by market sessions can increase profits percentage consistently in testing and validation periods and keep securing accuracy and drawdown levels.

Spanish version of the Time Management Behavior Questionnaire for university students.

PubMed

García-Ros, Rafael; Pérez-González, Francisco

2012-11-01

The main objective of the study is to analyze the psychometric properties and predictive capacity on academic performance in university contexts of a Spanish adaptation of the Time Management Behavior Questionnaire. The scale was applied to 462 students newly admitted at the Universitat de València in the 2006-2007 school year. The analyses performed made it possible to reproduce the factorial structure of the original version of the questionnaire with slight modifications in the ascription of various items. The underlying factorial structure includes four interrelated dimensions (Establishing objectives and priorities, Time management tools, Perception of time control and Preference for disorganization), which present satisfactory levels of reliability and an adequate convergent validity with the Time management subscale of the Motivated Strategies for Learning Questionnaire. The scores on the dimensions of time management show significant levels of association with academic performance in the first year of university studies, especially highlighting the predictive capacity of the subscale dealing with the Establishment of objectives and priorities. These results show the reliability and validity of this adaptation of the scale for evaluating how the students manage their academic time, and predicting their performance in the year they initiate the degree program, thus aiding in the development of intervention proposals directed towards improving these skills.

Development and validation of the Sports Athlete Foot and Ankle Score: an instrument for sports-related ankle injuries.

PubMed

Morssinkhof, M L A; Wang, O; James, L; van der Heide, H J L; Winson, I G

2013-09-01

Many existing scoring systems assess ankle function, but there is no evidence that any of them has been validated in a group of patients with a higher demand on their ankle function. Problems include ceiling effects, not being able to detect change or they do not contain a sports-subscale. The aim of this study was to create a validated self-administered scoring system for ankle injuries in the higher performing athlete. First, 26 patients were interviewed to solicit opinions needed to create the final score, which is modified from the Foot and Ankle Outcome Score (FAOS). Second, SAFAS was validated in a group of 25 athletes with and 14 athletes without ankle injury. It is a self-administered region specific sports foot and ankle score that contains four subscales assessing the levels of symptoms, pain, daily living and sports. The Spearman correlation coefficients between SAFAS and the Foot and Ankle Ability Measure (FAAM) ranged from 0.78 to 0.88. Content validity is established by key informant interviews, expert opinions and a high satisfaction rate of 75%. Cronbach's alpha indicated good internal consistency of each subscale ranging from 0.77 to 0.92. SAFAS has shown good evidence for being a valid instrudent for assessing sports-related ankle injuries in high-performing athletes. Copyright © 2013 European Foot and Ankle Society. Published by Elsevier Ltd. All rights reserved.

Validation of a Visual-Spatial Secondary Task to Assess Automaticity in Laparoscopic Skills.

PubMed

Castillo, Richard; Alvarado, Juan; Moreno, Pablo; Billeke, Pablo; Martínez, Carlos; Varas, Julián; Jarufe, Nicolás

2017-12-26

Our objective was to assess reliability and validity of a visual-spatial secondary task (VSST) as a method to measure automaticity on a basic simulated laparoscopic skill model. In motor skill acquisition, expertise is defined by automaticity. The highest level of performance with less cognitive and attentional resources characterizes this stage, allowing experts to perform multiple tasks. Conventional validated parameters as operative time, objective assessment skills scales (OSATS), and movement economy, are insufficient to distinguish if an individual has reached the more advanced learning phases, such as automaticity. There is literature about using a VSST as an attention indicator that correlates with the automaticity level. Novices with completed and approved Fundamentals of Laparoscopic Surgery course, and laparoscopy experts were enrolled for an experimental study and measured under dual tasks conditions. Each participant performed the test giving priority to the primary task while at the same time they responded to a VSST. The primary task consisted of 4 interrupted laparoscopic stitches (ILS) on a bench-model. The VSST was a screen that showed different patterns that the surgeon had to recognize and press a pedal while doing the stitches (PsychoPsy software, Python, MacOS). Novices were overtrained on ILS until they reach at least 100 repetitions and then were retested. Participants were video recorded and then assessed by 2 blinded evaluators who measured operative time and OSATS. These scores were considered indicators of quality for the primary task. The VSST performance was measured by the detectability index (DI), which is a ratio between correct and wrong detections. A reliable evaluation was defined as two measures of DI with less than 10% of difference, maintaining the cutoff scores for performance on the primary task (operative time <110 seg and OSATS >17 points). Novices (n = 11) achieved reliable measure of the test after 2 (2-5) repetitions on the preassessment and 3.75 (2-5) on the postassessment (p = 0.04); whereas laparoscopy experts (n = 4) did it after 3.5 (3-4) repetitions. Proficiency cutoff scores for the primary task were achieved on every measure for novices (prepost overtraining) and experts. Expert performance on VSST was DI 0.78 (0.69-0.87). Novice performance was significantly better on postassessment (DI-pre 0.48 [0.06-0.71] vs DI-post 0.78 [0.48-0.95], p = 0.003). Overtraining consisted in 140 (100-210) repetitions of ILS for all novices, made in 8 hours (3-15). By categorizing DI based on expert performance, novices with DI-post >0.65 achieved better OSATS score and less operative time than novices with DI-post<0.65 (p = 0.007 y, p = 0.089, respectively). Measuring automaticity is feasible using a VSST. This instrument is reliable and has a face, content and construct validity. A DI over 0.65 may be a cutoff point correlated with high standard performance on the primary task. This instrument measures performance on laparoscopic skills, and along with conventional indicators, would better define advance levels of expertise. More studies are required applying this VSST to achieve external validity by reproducing our results. Copyright © 2017 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

The Benefits of Improving Indoor Environmental Quality

ERIC Educational Resources Information Center

Lamping, Jerry

2012-01-01

As school funding levels nationwide continue to plummet amid public demands for increased student performance, an expanding body of research in the field of indoor environmental quality (IEQ) is providing greater statistical validity about the relationship between environmental conditions in school facilities and student achievement. Since the…

Validation of an Evaluation Tutoring Task Scale at the University

ERIC Educational Resources Information Center

Sáiz-Manzanares, María Consuelo; Bol-Arreba, Alfredo; Payo-Hernanz, René Jesús

2014-01-01

Introduction: Recent investigations have emphasized the need for university teachers to develop tutorial programs for students at university. Many universities are committed to broadening research on university teaching that will sharpen academic performance and levels of student satisfaction. Tutoring programs improve the development of the…

Developing Multiple Choice Tests: Tips & Techniques

ERIC Educational Resources Information Center

McCowan, Richard J.

1999-01-01

Item writing is a major responsibility of trainers. Too often, qualified staff who prepare lessons carefully and teach conscientiously use inadequate tests that do not validly reflect the true level of trainee achievement. This monograph describes techniques for constructing multiple-choice items that measure student performance accurately. It…

Evaluation and comparison of predictive individual-level general surrogates.

PubMed

Gabriel, Erin E; Sachs, Michael C; Halloran, M Elizabeth

2018-07-01

An intermediate response measure that accurately predicts efficacy in a new setting at the individual level could be used both for prediction and personalized medical decisions. In this article, we define a predictive individual-level general surrogate (PIGS), which is an individual-level intermediate response that can be used to accurately predict individual efficacy in a new setting. While methods for evaluating trial-level general surrogates, which are predictors of trial-level efficacy, have been developed previously, few, if any, methods have been developed to evaluate individual-level general surrogates, and no methods have formalized the use of cross-validation to quantify the expected prediction error. Our proposed method uses existing methods of individual-level surrogate evaluation within a given clinical trial setting in combination with cross-validation over a set of clinical trials to evaluate surrogate quality and to estimate the absolute prediction error that is expected in a new trial setting when using a PIGS. Simulations show that our method performs well across a variety of scenarios. We use our method to evaluate and to compare candidate individual-level general surrogates over a set of multi-national trials of a pentavalent rotavirus vaccine.

Observations on CFD Verification and Validation from the AIAA Drag Prediction Workshops

NASA Technical Reports Server (NTRS)

Morrison, Joseph H.; Kleb, Bil; Vassberg, John C.

2014-01-01

The authors provide observations from the AIAA Drag Prediction Workshops that have spanned over a decade and from a recent validation experiment at NASA Langley. These workshops provide an assessment of the predictive capability of forces and moments, focused on drag, for transonic transports. It is very difficult to manage the consistency of results in a workshop setting to perform verification and validation at the scientific level, but it may be sufficient to assess it at the level of practice. Observations thus far: 1) due to simplifications in the workshop test cases, wind tunnel data are not necessarily the “correct” results that CFD should match, 2) an average of core CFD data are not necessarily a better estimate of the true solution as it is merely an average of other solutions and has many coupled sources of variation, 3) outlier solutions should be investigated and understood, and 4) the DPW series does not have the systematic build up and definition on both the computational and experimental side that is required for detailed verification and validation. Several observations regarding the importance of the grid, effects of physical modeling, benefits of open forums, and guidance for validation experiments are discussed. The increased variation in results when predicting regions of flow separation and increased variation due to interaction effects, e.g., fuselage and horizontal tail, point out the need for validation data sets for these important flow phenomena. Experiences with a recent validation experiment at NASA Langley are included to provide guidance on validation experiments.

Reliability and concurrent validity of postural asymmetry measurement in adolescent idiopathic scoliosis.

PubMed

Prowse, Ashleigh; Aslaksen, Berit; Kierkegaard, Marie; Furness, James; Gerdhem, Paul; Abbott, Allan

2017-01-18

To investigate the reliability and concurrent validity of the Baseline ® Body Level/Scoliosis meter for adolescent idiopathic scoliosis postural assessment in three anatomical planes. This is an observational reliability and concurrent validity study of adolescent referrals to the Orthopaedic department for scoliosis screening at Karolinska University Hospital, Stockholm, Sweden between March-May 2012. A total of 31 adolescents with idiopathic scoliosis (13.6 ± 0.6 years old) of mild-moderate curvatures (25° ± 12°) were consecutively recruited. Measurement of cervical, thoracic and lumbar curvatures, pelvic and shoulder tilt, and axial thoracic rotation (ATR) were performed by two trained physiotherapists in one day. The intraclass correlation coefficient (ICC) was used to determine the inter-examiner reliability (ICC2,1) and the intra-rater reliability (ICC3,3) of the Baseline ® Body Level/Scoliosis meter. Spearman's correlation analyses were used to estimate concurrent validity between the Baseline ® Body Level/Scoliosis meter and Gold Standard Cobb angles from radiographs and the Orthopaedic Systems Inc. Scoliometer. There was excellent reliability between examiners for thoracic kyphosis (ICC2,1 = 0.94), ATR (ICC2,1 = 0.92) and lumbar lordosis (ICC2,1 = 0.79). There was adequate reliability between examiners for cervical lordosis (ICC2,1 = 0.51), however poor reliability for pelvic and shoulder tilt. Both devices were reproducible in the measurement of ATR when repeated by one examiner (ICC3,3 0.98-1.00). The device had a good correlation with the Scoliometer (rho = 0.78). When compared with Cobb angle from radiographs, there was a moderate correlation for ATR (rho = 0.627). The Baseline ® Body Level/Scoliosis meter provides reliable transverse and sagittal cervical, thoracic and lumbar measurements and valid transverse plan measurements of mild-moderate scoliosis deformity.

Multilevel microvibration test for performance predictions of a space optical load platform

NASA Astrophysics Data System (ADS)

Li, Shiqi; Zhang, Heng; Liu, Shiping; Wang, Yue

2018-05-01

This paper presents a framework for the multilevel microvibration analysis and test of a space optical load platform. The test framework is conducted on three levels, including instrument, subsystem, and system level. Disturbance source experimental investigations are performed to evaluate the vibration amplitude and study vibration mechanism. Transfer characteristics of space camera are validated by a subsystem test, which allows the calculation of transfer functions from various disturbance sources to optical performance outputs. In order to identify the influence of the source on the spacecraft performance, a system level microvibration measurement test has been performed on the ground. From the time domain analysis and spectrum analysis of multilevel microvibration tests, we concluded that the disturbance source has a significant effect on its installation position. After transmitted through mechanical links, the residual vibration reduces to a background noise level. In addition, the angular microvibration of the platform jitter is mainly concentrated in the rotation of y-axes. This work is applied to a real practical application involving the high resolution satellite camera system.

Pain patients' experiences of validation and invalidation from physicians before and after multimodal pain rehabilitation: Associations with pain, negative affectivity, and treatment outcome.

PubMed

Edlund, Sara M; Wurm, Matilda; Holländare, Fredrik; Linton, Steven J; Fruzzetti, Alan E; Tillfors, Maria

2017-10-01

Validating and invalidating responses play an important role in communication with pain patients, for example regarding emotion regulation and adherence to treatment. However, it is unclear how patients' perceptions of validation and invalidation relate to patient characteristics and treatment outcome. The aim of this study was to investigate the occurrence of subgroups based on pain patients' perceptions of validation and invalidation from their physicians. The stability of these perceptions and differences between subgroups regarding pain, pain interference, negative affectivity and treatment outcome were also explored. A total of 108 pain patients answered questionnaires regarding perceived validation and invalidation, pain severity, pain interference, and negative affectivity before and after pain rehabilitation treatment. Two cluster analyses using perceived validation and invalidation were performed, one on pre-scores and one on post-scores. The stability of patient perceptions from pre- to post-treatment was investigated, and clusters were compared on pain severity, pain interference, and negative affectivity. Finally, the connection between perceived validation and invalidation and treatment outcome was explored. Three clusters emerged both before and after treatment: (1) low validation and heightened invalidation, (2) moderate validation and invalidation, and (3) high validation and low invalidation. Perceptions of validation and invalidation were generally stable over time, although there were individuals whose perceptions changed. When compared to the other two clusters, the low validation/heightened invalidation cluster displayed significantly higher levels of pain interference and negative affectivity post-treatment but not pre-treatment. The whole sample significantly improved on pain interference and depression, but treatment outcome was independent of cluster. Unexpectedly, differences between clusters on pain interference and negative affectivity were only found post-treatment. This appeared to be due to the pre- and post-heightened invalidation clusters not containing the same individuals. Therefore, additional analyses were conducted to investigate the individuals who changed clusters. Results showed that patients scoring high on negative affectivity ended up in the heightened invalidation cluster post-treatment. Taken together, most patients felt understood when communicating with their rehabilitation physician. However, a smaller group of patients experienced the opposite: low levels of validation and heightened levels of invalidation. This group stood out as more problematic, reporting greater pain interference and negative affectivity when compared to the other groups after treatment. Patient perceptions were typically stable over time, but some individuals changed cluster, and these movements seemed to be related to negative affectivity and pain interference. These results do not support a connection between perceived validation and invalidation from physicians (meeting the patients pre- and post-treatment) and treatment outcome. Overall, our results suggest that there is a connection between negative affectivity and pain interference in the patients, and perceived validation and invalidation from the physicians. In clinical practice, it is important to pay attention to comorbid psychological problems and level of pain interference, since these factors may negatively influence effective communication. A focus on decreasing invalidating responses and/or increasing validating responses might be particularly important for patients with high levels of psychological problems and pain interference. Copyright © 2017. Published by Elsevier B.V.

Tests for the Assessment of Sport-Specific Performance in Olympic Combat Sports: A Systematic Review With Practical Recommendations

PubMed Central

Chaabene, Helmi; Negra, Yassine; Bouguezzi, Raja; Capranica, Laura; Franchini, Emerson; Prieske, Olaf; Hbacha, Hamdi; Granacher, Urs

2018-01-01

The regular monitoring of physical fitness and sport-specific performance is important in elite sports to increase the likelihood of success in competition. This study aimed to systematically review and to critically appraise the methodological quality, validation data, and feasibility of the sport-specific performance assessment in Olympic combat sports like amateur boxing, fencing, judo, karate, taekwondo, and wrestling. A systematic search was conducted in the electronic databases PubMed, Google-Scholar, and Science-Direct up to October 2017. Studies in combat sports were included that reported validation data (e.g., reliability, validity, sensitivity) of sport-specific tests. Overall, 39 studies were eligible for inclusion in this review. The majority of studies (74%) contained sample sizes <30 subjects. Nearly, 1/3 of the reviewed studies lacked a sufficient description (e.g., anthropometrics, age, expertise level) of the included participants. Seventy-two percent of studies did not sufficiently report inclusion/exclusion criteria of their participants. In 62% of the included studies, the description and/or inclusion of a familiarization session (s) was either incomplete or not existent. Sixty-percent of studies did not report any details about the stability of testing conditions. Approximately half of the studies examined reliability measures of the included sport-specific tests (intraclass correlation coefficient [ICC] = 0.43–1.00). Content validity was addressed in all included studies, criterion validity (only the concurrent aspect of it) in approximately half of the studies with correlation coefficients ranging from r = −0.41 to 0.90. Construct validity was reported in 31% of the included studies and predictive validity in only one. Test sensitivity was addressed in 13% of the included studies. The majority of studies (64%) ignored and/or provided incomplete information on test feasibility and methodological limitations of the sport-specific test. In 28% of the included studies, insufficient information or a complete lack of information was provided in the respective field of the test application. Several methodological gaps exist in studies that used sport-specific performance tests in Olympic combat sports. Additional research should adopt more rigorous validation procedures in the application and description of sport-specific performance tests in Olympic combat sports. PMID:29692739

Measuring competence in endoscopic sinus surgery.

PubMed

Syme-Grant, J; White, P S; McAleer, J P G

2008-02-01

Competence based education is currently being introduced into higher surgical training in the UK. Valid and reliable performance assessment tools are essential to ensure competencies are achieved. No such tools have yet been reported in the UK literature. We sought to develop and pilot test an Endoscopic Sinus Surgery Competence Assessment Tool (ESSCAT). The ESSCAT was designed for in-theatre assessment of higher surgical trainees in the UK. The ESSCAT rating matrix was developed through task analysis of ESS procedures. All otolaryngology consultants and specialist registrars in Scotland were given the opportunity to contribute to its refinement. Two cycles of in-theatre testing were used to ensure utility and gather quantitative data on validity and reliability. Videos of trainees performing surgery were used in establishing inter-rater reliability. National consultation, the consensus derived minimum standard of performance, Cronbach's alpha = 0.89 and demonstration of trainee learning (p = 0.027) during the in vivo application of the ESSCAT suggest a high level of validity. Inter-rater reliability was moderate for competence decisions (Cohen's Kappa = 0.5) and good for total scores (Intra-Class Correlation Co-efficient = 0.63). Intra-rater reliability was good for both competence decisions (Kappa = 0.67) and total scores (Kendall's Tau-b = 0.73). The ESSCAT generates a valid and reliable assessment of trainees' in-theatre performance of endoscopic sinus surgery. In conjunction with ongoing evaluation of the instrument we recommend the use of the ESSCAT in higher specialist training in otolaryngology in the UK.

The performance of seven QPrediction risk scores in an independent external sample of patients from general practice: a validation study

PubMed Central

Hippisley-Cox, Julia; Coupland, Carol; Brindle, Peter

2014-01-01

Objectives To validate the performance of a set of risk prediction algorithms developed using the QResearch database, in an independent sample from general practices contributing to the Clinical Research Data Link (CPRD). Setting Prospective open cohort study using practices contributing to the CPRD database and practices contributing to the QResearch database. Participants The CPRD validation cohort consisted of 3.3 million patients, aged 25–99 years registered at 357 general practices between 1 Jan 1998 and 31 July 2012. The validation statistics for QResearch were obtained from the original published papers which used a one-third sample of practices separate to those used to derive the score. A cohort from QResearch was used to compare incidence rates and baseline characteristics and consisted of 6.8 million patients from 753 practices registered between 1 Jan 1998 and until 31 July 2013. Outcome measures Incident events relating to seven different risk prediction scores: QRISK2 (cardiovascular disease); QStroke (ischaemic stroke); QDiabetes (type 2 diabetes); QFracture (osteoporotic fracture and hip fracture); QKidney (moderate and severe kidney failure); QThrombosis (venous thromboembolism); QBleed (intracranial bleed and upper gastrointestinal haemorrhage). Measures of discrimination and calibration were calculated. Results Overall, the baseline characteristics of the CPRD and QResearch cohorts were similar though QResearch had higher recording levels for ethnicity and family history. The validation statistics for each of the risk prediction scores were very similar in the CPRD cohort compared with the published results from QResearch validation cohorts. For example, in women, the QDiabetes algorithm explained 50% of the variation within CPRD compared with 51% on QResearch and the receiver operator curve value was 0.85 on both databases. The scores were well calibrated in CPRD. Conclusions Each of the algorithms performed practically as well in the external independent CPRD validation cohorts as they had in the original published QResearch validation cohorts. PMID:25168040

Criterion and Concurrent Validity of the activPAL™ Professional Physical Activity Monitor in Adolescent Females

PubMed Central

Dowd, Kieran P.; Harrington, Deirdre M.; Donnelly, Alan E.

2012-01-01

Background The activPAL has been identified as an accurate and reliable measure of sedentary behaviour. However, only limited information is available on the accuracy of the activPAL activity count function as a measure of physical activity, while no unit calibration of the activPAL has been completed to date. This study aimed to investigate the criterion validity of the activPAL, examine the concurrent validity of the activPAL, and perform and validate a value calibration of the activPAL in an adolescent female population. The performance of the activPAL in estimating posture was also compared with sedentary thresholds used with the ActiGraph accelerometer. Methodologies Thirty adolescent females (15 developmental; 15 cross-validation) aged 15–18 years performed 5 activities while wearing the activPAL, ActiGraph GT3X, and the Cosmed K4B2. A random coefficient statistics model examined the relationship between metabolic equivalent (MET) values and activPAL counts. Receiver operating characteristic analysis was used to determine activity thresholds and for cross-validation. The random coefficient statistics model showed a concordance correlation coefficient of 0.93 (standard error of the estimate = 1.13). An optimal moderate threshold of 2997 was determined using mixed regression, while an optimal vigorous threshold of 8229 was determined using receiver operating statistics. The activPAL count function demonstrated very high concurrent validity (r = 0.96, p<0.01) with the ActiGraph count function. Levels of agreement for sitting, standing, and stepping between direct observation and the activPAL and ActiGraph were 100%, 98.1%, 99.2% and 100%, 0%, 100%, respectively. Conclusions These findings suggest that the activPAL is a valid, objective measurement tool that can be used for both the measurement of physical activity and sedentary behaviours in an adolescent female population. PMID:23094069

Identifying outliers of non-Gaussian groundwater state data based on ensemble estimation for long-term trends

NASA Astrophysics Data System (ADS)

Jeong, Jina; Park, Eungyu; Han, Weon Shik; Kim, Kueyoung; Choung, Sungwook; Chung, Il Moon

2017-05-01

A hydrogeological dataset often includes substantial deviations that need to be inspected. In the present study, three outlier identification methods - the three sigma rule (3σ), inter quantile range (IQR), and median absolute deviation (MAD) - that take advantage of the ensemble regression method are proposed by considering non-Gaussian characteristics of groundwater data. For validation purposes, the performance of the methods is compared using simulated and actual groundwater data with a few hypothetical conditions. In the validations using simulated data, all of the proposed methods reasonably identify outliers at a 5% outlier level; whereas, only the IQR method performs well for identifying outliers at a 30% outlier level. When applying the methods to real groundwater data, the outlier identification performance of the IQR method is found to be superior to the other two methods. However, the IQR method shows limitation by identifying excessive false outliers, which may be overcome by its joint application with other methods (for example, the 3σ rule and MAD methods). The proposed methods can be also applied as potential tools for the detection of future anomalies by model training based on currently available data.

Construct and face validity of the educational computer-based environment (ECE) assessment scenarios for basic endoneurosurgery skills.

PubMed

Cagiltay, Nergiz Ercil; Ozcelik, Erol; Sengul, Gokhan; Berker, Mustafa

2017-11-01

In neurosurgery education, there is a paradigm shift from time-based training to criterion-based model for which competency and assessment becomes very critical. Even virtual reality simulators provide alternatives to improve education and assessment in neurosurgery programs and allow for several objective assessment measures, there are not many tools for assessing the overall performance of trainees. This study aims to develop and validate a tool for assessing the overall performance of participants in a simulation-based endoneurosurgery training environment. A training program was developed in two levels: endoscopy practice and beginning surgical practice based on four scenarios. Then, three experiments were conducted with three corresponding groups of participants (Experiment 1, 45 (32 beginners, 13 experienced), Experiment 2, 53 (40 beginners, 13 experienced), and Experiment 3, 26 (14 novices, 12 intermediate) participants). The results analyzed to understand the common factors among the performance measurements of these experiments. Then, a factor capable of assessing the overall skill levels of surgical residents was extracted. Afterwards, the proposed measure was tested to estimate the experience levels of the participants. Finally, the level of realism of these educational scenarios was assessed. The factor formed by time, distance, and accuracy on simulated tasks provided an overall performance indicator. The prediction correctness was very high for the beginners than the one for experienced surgeons in Experiments 1 and 2. When non-dominant hand is used in a surgical procedure-based scenario, skill levels of surgeons can be better predicted. The results indicate that the scenarios in Experiments 1 and 2 can be used as an assessment tool for the beginners, and scenario-2 in Experiment 3 can be used as an assessment tool for intermediate and novice levels. It can be concluded that forming the balance between perceived action capacities and skills is critical for better designing and developing skill assessment surgical simulation tools.

Predicting in vivo effect levels for repeat-dose systemic toxicity using chemical, biological, kinetic and study covariates.

PubMed

Truong, Lisa; Ouedraogo, Gladys; Pham, LyLy; Clouzeau, Jacques; Loisel-Joubert, Sophie; Blanchet, Delphine; Noçairi, Hicham; Setzer, Woodrow; Judson, Richard; Grulke, Chris; Mansouri, Kamel; Martin, Matthew

2018-02-01

In an effort to address a major challenge in chemical safety assessment, alternative approaches for characterizing systemic effect levels, a predictive model was developed. Systemic effect levels were curated from ToxRefDB, HESS-DB and COSMOS-DB from numerous study types totaling 4379 in vivo studies for 1247 chemicals. Observed systemic effects in mammalian models are a complex function of chemical dynamics, kinetics, and inter- and intra-individual variability. To address this complex problem, systemic effect levels were modeled at the study-level by leveraging study covariates (e.g., study type, strain, administration route) in addition to multiple descriptor sets, including chemical (ToxPrint, PaDEL, and Physchem), biological (ToxCast), and kinetic descriptors. Using random forest modeling with cross-validation and external validation procedures, study-level covariates alone accounted for approximately 15% of the variance reducing the root mean squared error (RMSE) from 0.96 log 10 to 0.85 log 10  mg/kg/day, providing a baseline performance metric (lower expectation of model performance). A consensus model developed using a combination of study-level covariates, chemical, biological, and kinetic descriptors explained a total of 43% of the variance with an RMSE of 0.69 log 10  mg/kg/day. A benchmark model (upper expectation of model performance) was also developed with an RMSE of 0.5 log 10  mg/kg/day by incorporating study-level covariates and the mean effect level per chemical. To achieve a representative chemical-level prediction, the minimum study-level predicted and observed effect level per chemical were compared reducing the RMSE from 1.0 to 0.73 log 10  mg/kg/day, equivalent to 87% of predictions falling within an order-of-magnitude of the observed value. Although biological descriptors did not improve model performance, the final model was enriched for biological descriptors that indicated xenobiotic metabolism gene expression, oxidative stress, and cytotoxicity, demonstrating the importance of accounting for kinetics and non-specific bioactivity in predicting systemic effect levels. Herein, we generated an externally predictive model of systemic effect levels for use as a safety assessment tool and have generated forward predictions for over 30,000 chemicals.

Proficiency training on a virtual reality robotic surgical skills curriculum.

PubMed

Bric, Justin; Connolly, Michael; Kastenmeier, Andrew; Goldblatt, Matthew; Gould, Jon C

2014-12-01

The clinical application of robotic surgery is increasing. The skills necessary to perform robotic surgery are unique from those required in open and laparoscopic surgery. A validated laparoscopic surgical skills curriculum (Fundamentals of Laparoscopic Surgery or FLS™) has transformed the way surgeons acquire laparoscopic skills. There is a need for a similar skills training and assessment tool for robotic surgery. Our research group previously developed and validated a robotic training curriculum in a virtual reality (VR) simulator. We hypothesized that novice robotic surgeons could achieve proficiency levels defined by more experienced robotic surgeons on the VR robotic curriculum, and that this would result in improved performance on the actual daVinci Surgical System™. 25 medical students with no prior robotic surgery experience were recruited. Prior to VR training, subjects performed 2 FLS tasks 3 times each (Peg Transfer, Intracorporeal Knot Tying) using the daVinci Surgical System™ docked to a video trainer box. Task performance for the FLS tasks was scored objectively. Subjects then practiced on the VR simulator (daVinci Skills Simulator) until proficiency levels on all 5 tasks were achieved before completing a post-training assessment of the 2 FLS tasks on the daVinci Surgical System™ in the video trainer box. All subjects to complete the study (1 dropped out) reached proficiency levels on all VR tasks in an average of 71 (± 21.7) attempts, accumulating 164.3 (± 55.7) minutes of console training time. There was a significant improvement in performance on the robotic FLS tasks following completion of the VR training curriculum. Novice robotic surgeons are able to attain proficiency levels on a VR simulator. This leads to improved performance in the daVinci surgical platform on simulated tasks. Training to proficiency on a VR robotic surgery simulator is an efficient and viable method for acquiring robotic surgical skills.

Evaluation of skill level between trainees and community orthopaedic surgeons using a virtual reality arthroscopic knee simulator.

PubMed

Cannon, W Dilworth; Nicandri, Gregg T; Reinig, Karl; Mevis, Howard; Wittstein, Jocelyn

2014-04-02

Several virtual reality simulators have been developed to assist orthopaedic surgeons in acquiring the skills necessary to perform arthroscopic surgery. The purpose of this study was to assess the construct validity of the ArthroSim virtual reality arthroscopy simulator by evaluating whether skills acquired through increased experience in the operating room lead to improved performance on the simulator. Using the simulator, six postgraduate year-1 orthopaedic residents were compared with six postgraduate year-5 residents and with six community-based orthopaedic surgeons when performing diagnostic arthroscopy. The time to perform the procedure was recorded. To ensure that subjects did not sacrifice the quality of the procedure to complete the task in a shorter time, the simulator was programmed to provide a completeness score that indicated whether the surgeon accurately performed all of the steps of diagnostic arthroscopy in the correct sequence. The mean time to perform the procedure by each group was 610 seconds for community-based orthopaedic surgeons, 745 seconds for postgraduate year-5 residents, and 1028 seconds for postgraduate year-1 residents. Both the postgraduate year-5 residents and the community-based orthopaedic surgeons performed the procedure in significantly less time (p = 0.006) than the postgraduate year-1 residents. There was a trend toward significance (p = 0.055) in time to complete the procedure when the postgraduate year-5 residents were compared with the community-based orthopaedic surgeons. The mean level of completeness as assigned by the simulator for each group was 85% for the community-based orthopaedic surgeons, 79% for the postgraduate year-5 residents, and 71% for the postgraduate year-1 residents. As expected, these differences were not significant, indicating that the three groups had achieved an acceptable level of consistency in their performance of the procedure. Higher levels of surgeon experience resulted in improved efficiency when performing diagnostic knee arthroscopy on the simulator. Further validation studies utilizing the simulator are currently under way and the additional simulated tasks of arthroscopic meniscectomy, meniscal repair, microfracture, and loose body removal are being developed.

Analysis of English language learner performance on the biology Massachusetts comprehensive assessment system: The impact of english proficiency, first language characteristics, and late-entry ELL status

NASA Astrophysics Data System (ADS)

Mitchell, Mary A.

This study analyzed English language learner (ELL) performance on the June 2012 Biology MCAS, namely on item attributes of domain, cognitive skill, and linguistic complexity. It examined the impact of English proficiency, Latinate first language, first language orthography, and late-entry ELL status. The results indicated that English proficiency was a strong predictor of performance and that ELLs at higher levels of English proficiency overwhelmingly passed. The results further indicated that English proficiency introduced a construct-irrelevant variance on the Biology MCAS and raised validity issues for using this assessment at lower levels of English proficiency. This study also found that ELLs with a Latinate first language consistently had statistically significant lower performance. Late-entry ELL status did not predict Biology MCAS performance.

Clinical Assessment of Risk Management: an INtegrated Approach (CARMINA).

PubMed

Tricarico, Pierfrancesco; Tardivo, Stefano; Sotgiu, Giovanni; Moretti, Francesca; Poletti, Piera; Fiore, Alberto; Monturano, Massimo; Mura, Ida; Privitera, Gaetano; Brusaferro, Silvio

2016-08-08

Purpose - The European Union recommendations for patient safety calls for shared clinical risk management (CRM) safety standards able to guide organizations in CRM implementation. The purpose of this paper is to develop a self-evaluation tool to measure healthcare organization performance on CRM and guide improvements over time. Design/methodology/approach - A multi-step approach was implemented including: a systematic literature review; consensus meetings with an expert panel from eight Italian leader organizations to get to an agreement on the first version; field testing to test instrument feasibility and flexibility; Delphi strategy with a second expert panel for content validation and balanced scoring system development. Findings - The self-assessment tool - Clinical Assessment of Risk Management: an INtegrated Approach includes seven areas (governance, communication, knowledge and skills, safe environment, care processes, adverse event management, learning from experience) and 52 standards. Each standard is evaluated according to four performance levels: minimum; monitoring; outcomes; and improvement actions, which resulted in a feasible, flexible and valid instrument to be used throughout different organizations. Practical implications - This tool allows practitioners to assess their CRM activities compared to minimum levels, monitor performance, benchmarking with other institutions and spreading results to different stakeholders. Originality/value - The multi-step approach allowed us to identify core minimum CRM levels in a field where no consensus has been reached. Most standards may be easily adopted in other countries.

Configuration and validation of a novel prostate disease nomogram predicting prostate biopsy outcome: A prospective study correlating clinical indicators among Filipino adult males with elevated PSA level.

PubMed

Chua, Michael E; Tanseco, Patrick P; Mendoza, Jonathan S; Castillo, Josefino C; Morales, Marcelino L; Luna, Saturnino L

2015-04-01

To configure and validate a novel prostate disease nomogram providing prostate biopsy outcome probabilities from a prospective study correlating clinical indicators and diagnostic parameters among Filipino adult male with elevated serum total prostate specific antigen (PSA) level. All men with an elevated serum total PSA underwent initial prostate biopsy at our institution from January 2011 to August 2014 were included. Clinical indicators, diagnostic parameters, which include PSA level and PSA-derivatives, were collected as predictive factors for biopsy outcome. Multiple logistic-regression analysis involving a backward elimination selection procedure was used to select independent predictors. A nomogram was developed to calculate the probability of the biopsy outcomes. External validation of the nomogram was performed using separate data set from another center for determination of sensitivity and specificity. A receiver-operating characteristic (ROC) curve was used to assess the accuracy in predicting differential biopsy outcome. Total of 552 patients was included. One hundred and ninety-one (34.6%) patients had benign prostatic hyperplasia, and 165 (29.9%) had chronic prostatitis. The remaining 196 (35.5%) patients had prostate adenocarcinoma. The significant independent variables used to predict biopsy outcome were age, family history of prostate cancer, prior antibiotic intake, PSA level, PSA-density, PSA-velocity, echogenic findings on ultrasound, and DRE status. The areas under the receiver-operating characteristic curve for prostate cancer using PSA alone and the nomogram were 0.688 and 0.804, respectively. The nomogram configured based on routinely available clinical parameters, provides high predictive accuracy with good performance characteristics in predicting the prostate biopsy outcome such as presence of prostate cancer, high Gleason prostate cancer, benign prostatic hyperplasia, and chronic prostatitis.

Improvement of web-based data acquisition and management system for GOSAT validation lidar data analysis

NASA Astrophysics Data System (ADS)

Okumura, Hiroshi; Takubo, Shoichiro; Kawasaki, Takeru; Abdullah, Indra Nugraha; Uchino, Osamu; Morino, Isamu; Yokota, Tatsuya; Nagai, Tomohiro; Sakai, Tetsu; Maki, Takashi; Arai, Kohei

2013-01-01

A web-base data acquisition and management system for GOSAT (Greenhouse gases Observation SATellite) validation lidar data-analysis has been developed. The system consists of data acquisition sub-system (DAS) and data management sub-system (DMS). DAS written in Perl language acquires AMeDAS (Automated Meteorological Data Acquisition System) ground-level local meteorological data, GPS Radiosonde upper-air meteorological data, ground-level oxidant data, skyradiometer data, skyview camera images, meteorological satellite IR image data and GOSAT validation lidar data. DMS written in PHP language demonstrates satellite-pass date and all acquired data. In this article, we briefly describe some improvement for higher performance and higher data usability. GPS Radiosonde upper-air meteorological data and U.S. standard atmospheric model in DAS automatically calculate molecule number density profiles. Predicted ozone density prole images above Saga city are also calculated by using Meteorological Research Institute (MRI) chemistry-climate model version 2 for comparison to actual ozone DIAL data.

Anatomy of a physics test: Validation of the physics items on the Texas Assessment of Knowledge and Skills

NASA Astrophysics Data System (ADS)

Marshall, Jill A.; Hagedorn, Eric A.; O'Connor, Jerry

2009-06-01

We report the results of an analysis of the Texas Assessment of Knowledge and Skills (TAKS) designed to determine whether the TAKS is a valid indicator of whether students know and can do physics at the level necessary for success in future coursework, STEM careers, and life in a technological society. We categorized science items from the 2003 and 2004 10th and 11th grade TAKS by content area(s) covered, knowledge and skills required to select the correct answer, and overall quality. We also analyzed a 5000 student sample of item-level results from the 2004 11th grade exam, performing full-information factor analysis, calculating classical test indices, and determining each item's response curve using item response theory. Triangulation of our results revealed strengths and weaknesses of the different methods of analysis. The TAKS was found to be only weakly indicative of physics preparation and we make recommendations for increasing the validity of standardized physics testing.

Evaluating trauma team performance in a Level I trauma center: Validation of the trauma team communication assessment (TTCA-24).

PubMed

DeMoor, Stephanie; Abdel-Rehim, Shady; Olmsted, Richard; Myers, John G; Parker-Raley, Jessica

2017-07-01

Nontechnical skills (NTS), such as team communication, are well-recognized determinants of trauma team performance and good patient care. Measuring these competencies during trauma resuscitations is essential, yet few valid and reliable tools are available. We aimed to demonstrate that the Trauma Team Communication Assessment (TTCA-24) is a valid and reliable instrument that measures communication effectiveness during activations. Two tools with adequate psychometric strength (Trauma Nontechnical Skills Scale [T-NOTECHS], Team Emergency Assessment Measure [TEAM]) were identified during a systematic review of medical literature and compared with TTCA-24. Three coders used each tool to evaluate 35 stable and 35 unstable patient activations (defined according to Advanced Trauma Life Support criteria). Interrater reliability was calculated between coders using the intraclass correlation coefficient. Spearman rank correlation coefficient was used to establish concurrent validity between TTCA-24 and the other two validated tools. Coders achieved an intraclass correlation coefficient of 0.87 for stable patient activations and 0.78 for unstable activations scoring excellent on the interrater agreement guidelines. The median score for each assessment showed good team communication for all 70 videos (TEAM, 39.8 of 54; T-NOTECHS, 17.4 of 25; and TTCA-24, 87.4 of 96). A significant correlation between TTTC-24 and T-NOTECHS was revealed (p = 0.029), but no significant correlation between TTCA-24 and TEAM (p = 0.77). Team communication was rated slightly better across all assessments for stable versus unstable patient activations, but not statistically significant. TTCA-24 correlated with T-NOTECHS, an instrument measuring nontechnical skills for trauma teams, but not TEAM, a tool that assesses communication in generic emergency settings. TTCA-24 is a reliable and valid assessment that can be a useful adjunct when evaluating interpersonal and team communication during trauma activations. Diagnostic tests or criteria, level II.

Construction of a web-based questionnaire for longitudinal investigation of work exposure, musculoskeletal pain and performance impairments in high-performance marine craft populations.

PubMed

Lo Martire, Riccardo; de Alwis, Manudul Pahansen; Äng, Björn Olov; Garme, Karl

2017-07-20

High-performance marine craft personnel (HPMCP) are regularly exposed to vibration and repeated shock (VRS) levels exceeding maximum limitations stated by international legislation. Whereas such exposure reportedly is detrimental to health and performance, the epidemiological data necessary to link these adverse effects causally to VRS are not available in the scientific literature, and no suitable tools for acquiring such data exist. This study therefore constructed a questionnaire for longitudinal investigations in HPMCP. A consensus panel defined content domains, identified relevant items and outlined a questionnaire. The relevance and simplicity of the questionnaire's content were then systematically assessed by expert raters in three consecutive stages, each followed by revisions. An item-level content validity index (I-CVI) was computed as the proportion of experts rating an item as relevant and simple, and a scale-level content validity index (S-CVI/Ave) as the average I-CVI across items. The thresholds for acceptable content validity were 0.78 and 0.90, respectively. Finally, a dynamic web version of the questionnaire was constructed and pilot tested over a 1-month period during a marine exercise in a study population sample of eight subjects, while accelerometers simultaneously quantified VRS exposure. Content domains were defined as work exposure, musculoskeletal pain and human performance, and items were selected to reflect these constructs. Ratings from nine experts yielded S-CVI/Ave of 0.97 and 1.00 for relevance and simplicity, respectively, and the pilot test suggested that responses were sensitive to change in acceleration and that the questionnaire, following some adjustments, was feasible for its intended purpose. A dynamic web-based questionnaire for longitudinal survey of key variables in HPMCP was constructed. Expert ratings supported that the questionnaire content is relevant, simple and sufficiently comprehensive, and the pilot test suggested that the questionnaire is feasible for longitudinal measurements in the study population. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

Performance Assessment and Geometric Calibration of RESOURCESAT-2

NASA Astrophysics Data System (ADS)

Radhadevi, P. V.; Solanki, S. S.; Akilan, A.; Jyothi, M. V.; Nagasubramanian, V.

2016-06-01

Resourcesat-2 (RS-2) has successfully completed five years of operations in its orbit. This satellite has multi-resolution and multi-spectral capabilities in a single platform. A continuous and autonomous co-registration, geo-location and radiometric calibration of image data from different sensors with widely varying view angles and resolution was one of the challenges of RS-2 data processing. On-orbit geometric performance of RS-2 sensors has been widely assessed and calibrated during the initial phase operations. Since then, as an ongoing activity, various geometric performance data are being generated periodically. This is performed with sites of dense ground control points (GCPs). These parameters are correlated to the direct geo-location accuracy of the RS-2 sensors and are monitored and validated to maintain the performance. This paper brings out the geometric accuracy assessment, calibration and validation done for about 500 datasets of RS-2. The objectives of this study are to ensure the best absolute and relative location accuracy of different cameras, location performance with payload steering and co-registration of multiple bands. This is done using a viewing geometry model, given ephemeris and attitude data, precise camera geometry and datum transformation. In the model, the forward and reverse transformations between the coordinate systems associated with the focal plane, payload, body, orbit and ground are rigorously and explicitly defined. System level tests using comparisons to ground check points have validated the operational geo-location accuracy performance and the stability of the calibration parameters.

Risk assessment model for development of advanced age-related macular degeneration.

PubMed

Klein, Michael L; Francis, Peter J; Ferris, Frederick L; Hamon, Sara C; Clemons, Traci E

2011-12-01

To design a risk assessment model for development of advanced age-related macular degeneration (AMD) incorporating phenotypic, demographic, environmental, and genetic risk factors. We evaluated longitudinal data from 2846 participants in the Age-Related Eye Disease Study. At baseline, these individuals had all levels of AMD, ranging from none to unilateral advanced AMD (neovascular or geographic atrophy). Follow-up averaged 9.3 years. We performed a Cox proportional hazards analysis with demographic, environmental, phenotypic, and genetic covariates and constructed a risk assessment model for development of advanced AMD. Performance of the model was evaluated using the C statistic and the Brier score and externally validated in participants in the Complications of Age-Related Macular Degeneration Prevention Trial. The final model included the following independent variables: age, smoking history, family history of AMD (first-degree member), phenotype based on a modified Age-Related Eye Disease Study simple scale score, and genetic variants CFH Y402H and ARMS2 A69S. The model did well on performance measures, with very good discrimination (C statistic = 0.872) and excellent calibration and overall performance (Brier score at 5 years = 0.08). Successful external validation was performed, and a risk assessment tool was designed for use with or without the genetic component. We constructed a risk assessment model for development of advanced AMD. The model performed well on measures of discrimination, calibration, and overall performance and was successfully externally validated. This risk assessment tool is available for online use.

A general method for assessing brain-computer interface performance and its limitations

NASA Astrophysics Data System (ADS)

Hill, N. Jeremy; Häuser, Ann-Katrin; Schalk, Gerwin

2014-04-01

Objective. When researchers evaluate brain-computer interface (BCI) systems, we want quantitative answers to questions such as: How good is the system’s performance? How good does it need to be? and: Is it capable of reaching the desired level in future? In response to the current lack of objective, quantitative, study-independent approaches, we introduce methods that help to address such questions. We identified three challenges: (I) the need for efficient measurement techniques that adapt rapidly and reliably to capture a wide range of performance levels; (II) the need to express results in a way that allows comparison between similar but non-identical tasks; (III) the need to measure the extent to which certain components of a BCI system (e.g. the signal processing pipeline) not only support BCI performance, but also potentially restrict the maximum level it can reach. Approach. For challenge (I), we developed an automatic staircase method that adjusted task difficulty adaptively along a single abstract axis. For challenge (II), we used the rate of information gain between two Bernoulli distributions: one reflecting the observed success rate, the other reflecting chance performance estimated by a matched random-walk method. This measure includes Wolpaw’s information transfer rate as a special case, but addresses the latter’s limitations including its restriction to item-selection tasks. To validate our approach and address challenge (III), we compared four healthy subjects’ performance using an EEG-based BCI, a ‘Direct Controller’ (a high-performance hardware input device), and a ‘Pseudo-BCI Controller’ (the same input device, but with control signals processed by the BCI signal processing pipeline). Main results. Our results confirm the repeatability and validity of our measures, and indicate that our BCI signal processing pipeline reduced attainable performance by about 33% (21 bits min-1). Significance. Our approach provides a flexible basis for evaluating BCI performance and its limitations, across a wide range of tasks and task difficulties.

An intercomparison of a large ensemble of statistical downscaling methods for Europe: Overall results from the VALUE perfect predictor cross-validation experiment

NASA Astrophysics Data System (ADS)

Gutiérrez, Jose Manuel; Maraun, Douglas; Widmann, Martin; Huth, Radan; Hertig, Elke; Benestad, Rasmus; Roessler, Ole; Wibig, Joanna; Wilcke, Renate; Kotlarski, Sven

2016-04-01

VALUE is an open European network to validate and compare downscaling methods for climate change research (http://www.value-cost.eu). A key deliverable of VALUE is the development of a systematic validation framework to enable the assessment and comparison of both dynamical and statistical downscaling methods. This framework is based on a user-focused validation tree, guiding the selection of relevant validation indices and performance measures for different aspects of the validation (marginal, temporal, spatial, multi-variable). Moreover, several experiments have been designed to isolate specific points in the downscaling procedure where problems may occur (assessment of intrinsic performance, effect of errors inherited from the global models, effect of non-stationarity, etc.). The list of downscaling experiments includes 1) cross-validation with perfect predictors, 2) GCM predictors -aligned with EURO-CORDEX experiment- and 3) pseudo reality predictors (see Maraun et al. 2015, Earth's Future, 3, doi:10.1002/2014EF000259, for more details). The results of these experiments are gathered, validated and publicly distributed through the VALUE validation portal, allowing for a comprehensive community-open downscaling intercomparison study. In this contribution we describe the overall results from Experiment 1), consisting of a European wide 5-fold cross-validation (with consecutive 6-year periods from 1979 to 2008) using predictors from ERA-Interim to downscale precipitation and temperatures (minimum and maximum) over a set of 86 ECA&D stations representative of the main geographical and climatic regions in Europe. As a result of the open call for contribution to this experiment (closed in Dec. 2015), over 40 methods representative of the main approaches (MOS and Perfect Prognosis, PP) and techniques (linear scaling, quantile mapping, analogs, weather typing, linear and generalized regression, weather generators, etc.) were submitted, including information both data (downscaled values) and metadata (characterizing different aspects of the downscaling methods). This constitutes the largest and most comprehensive to date intercomparison of statistical downscaling methods. Here, we present an overall validation, analyzing marginal and temporal aspects to assess the intrinsic performance and added value of statistical downscaling methods at both annual and seasonal levels. This validation takes into account the different properties/limitations of different approaches and techniques (as reported in the provided metadata) in order to perform a fair comparison. It is pointed out that this experiment alone is not sufficient to evaluate the limitations of (MOS) bias correction techniques. Moreover, it also does not fully validate PP since we don't learn whether we have the right predictors and whether the PP assumption is valid. These problems will be analyzed in the subsequent community-open VALUE experiments 2) and 3), which will be open for participation along the present year.

Comparison of the goals and MISTELS scores for the evaluation of surgeons on training benches.

PubMed

Wolf, Rémi; Medici, Maud; Fiard, Gaëlle; Long, Jean-Alexandre; Moreau-Gaudry, Alexandre; Cinquin, Philippe; Voros, Sandrine

2018-01-01

Evaluation of surgical technical abilities is a major issue in minimally invasive surgery. Devices such as training benches offer specific scores to evaluate surgeons but cannot transfer in the operating room (OR). A contrario, several scores measure performance in the OR, but have not been evaluated on training benches. Our aim was to demonstrate that the GOALS score, which can effectively grade in the OR the abilities involved in laparoscopy, can be used for evaluation on a laparoscopic testbench (MISTELS). This could lead to training systems that can identify more precisely the skills that have been acquired or must still be worked on. 32 volunteers (surgeons, residents and medical students) performed the 5 tasks of the MISTELS training bench and were simultaneously video-recorded. Their performance was evaluated with the MISTELS score and with the GOALS score based on the review of the recording by two experienced, blinded laparoscopic surgeons. The concurrent validity of the GOALS score was assessed using Pearson and Spearman correlation coefficients with the MISTELS score. The construct validity of the GOALS score was assessed with k-means clustering and accuracy rates. Lastly, abilities explored by each MISTELS task were identified with multiple linear regression. GOALS and MISTELS scores are strongly correlated (Pearson correlation coefficient = 0.85 and Spearman correlation coefficient = 0.82 for the overall score). The GOALS score proves to be valid for construction for the tasks of the training bench, with a better accuracy rate between groups of level after k-means clustering, when compared to the original MISTELS score (accuracy rates, respectively, 0.75 and 0.56). GOALS score is well suited for the evaluation of the performance of surgeons of different levels during the completion of the tasks of the MISTELS training bench.

Development of a virtual reality training curriculum for phacoemulsification surgery.

PubMed

Spiteri, A V; Aggarwal, R; Kersey, T L; Sira, M; Benjamin, L; Darzi, A W; Bloom, P A

2014-01-01

Training within a proficiency-based virtual reality (VR) curriculum may reduce errors during real surgical procedures. This study used a scientific methodology to develop a VR training curriculum for phacoemulsification surgery (PS). Ten novice-(n) (performed <10 cataract operations), 10 intermediate-(i) (50-200), and 10 experienced-(e) (>500) surgeons were recruited. Construct validity was defined as the ability to differentiate between the three levels of experience, based on the simulator-derived metrics for two abstract modules (four tasks) and three procedural modules (five tasks) on a high-fidelity VR simulator. Proficiency measures were based on the performance of experienced surgeons. Abstract modules demonstrated a 'ceiling effect' with construct validity established between groups (n) and (i) but not between groups (i) and (e)-Forceps 1 (46, 87, and 95; P<0.001). Increasing difficulty of task showed significantly reduced performance in (n) but minimal difference for (i) and (e)-Anti-tremor 4 (0, 51, and 59; P<0.001), Forceps 4 (11, 73, and 94; P<0.001). Procedural modules were found to be construct valid between groups (n) and (i) and between groups (i) and (e)-Lens-cracking (0, 22, and 51; P<0.05) and Phaco-quadrants (16, 53, and 87; P<0.05). This was also the case with Capsulorhexis (0, 19, and 63; P<0.05) with the performance decreasing in the (n) and (i) group but improving in the (e) group (0, 55, and 73; P<0.05) and (0, 48, and 76; P<0.05) as task difficulty increased. Experienced/intermediate benchmark skill levels are defined allowing the development of a proficiency-based VR training curriculum for PS for novices using a structured scientific methodology.

Validation of the work and health interview.

PubMed

Stewart, Walter F; Ricci, Judith A; Leotta, Carol; Chee, Elsbeth

2004-01-01

Instruments that measure the impact of illness on work do not usually provide a measure that can be directly translated into lost hours or costs. We describe the validation of the Work and Health Interview (WHI), a questionnaire that provides a measure of lost productive time (LPT) from work absence and reduced performance at work. A sample (n = 67) of inbound phone call agents was recruited for the study. Validity of the WHI was assessed over a 2-week period in reference to workplace data (i.e. absence time, time away from call station and electronic continuous performance) and repeated electronic diary data (n = 48) obtained approximately eight times a day to estimate time not working (i.e. a component of reduced performance). The mean (median) missed work time estimate for any reason was 11 (8.0) and 12.9 (8.0) hours in a 2-week period from the WHI and workplace data, respectively, with a Pearson's (Spearman's) correlation of 0.84 (0.76). The diary-based mean (median) estimate of time not working while at work was 3.9 (2.8) hours compared with the WHI estimate of 5.7 (3.2) hours with a Pearson's (Spearman's) correlation of 0.19 (0.33). The 2-week estimate of total productive time from the diary was 67.2 hours compared with 67.8 hours from the WHI, with a Pearson's (Spearman's) correlation of 0.50 (0.46). At a population level, the WHI provides an accurate estimate of missed time from work and total productive time when compared with workplace and diary estimates. At an individual level, the WHI measure of total missed time, but not reduced performance time, is moderately accurate.

Development and validation of an exercise performance support system for people with lower extremity impairment.

PubMed

Minor, M A; Reid, J C; Griffin, J Z; Pittman, C B; Patrick, T B; Cutts, J H

1998-02-01

To identify innovative strategies to support appropriate, self-directed exercise that increase physical activity levels of people with arthritis. This article reports on one interactive, multimedia exercise performance support system (PSS) for people with lower extremity impairments in strength or flexibility. An interdisciplinary team developed the PSS using self-report of lower extremity musculoskeletal impairments (flexibility and strength) to produce an individualized exercise program with video and print educational materials. Initial evaluation has investigated the validity and reliability of program assessments and recommendations. PSS self-report and professional assessments were similar, with more impairments indicated by self-report. PSS exercise recommendations were similar to those made by 3 expert physical therapists using the same exercise data base. Results of PSS impairment assessments were stable over a 1-week period. PSS exercise recommendations appear to be reliable and a valid reflection of current exercise knowledge in rheumatology. Furthermore, users were able to complete the computer-based program with minimal assistance and reported it to be enjoyable and informative.

Validity of the International Fitness Scale "IFIS" in older adults.

PubMed

Merellano-Navarro, Eugenio; Collado-Mateo, Daniel; García-Rubio, Javier; Gusi, Narcís; Olivares, Pedro R

2017-09-01

To validate the "International Fitness Scale" (IFIS) in older adults. Firstly, cognitive interviews were performed to ensure that the questionnaire was comprehensive for older Chilean adults. After that, a transversal study of 401 institutionalized and non-institutionalized older adults from Maule region in Chile was conducted. A battery of validated fitness tests for this population was used in order to compare the responses obtained in the IFIS with the objectively measured fitness performance (back scratch, chair sit-and-reach, handgrip, 30-s chair stand, timed up-and-go and 6-min walking). Indicated that IFIS presented a high compliance in the comprehension of the items which defined it, and it was able of categorizing older adults according to their measured physical fitness levels. The analysis of covariance ANCOVA adjusted by sex and age showed a concordance between IFIS and the score in physical fitness tests. Based on the results of this study, IFIS questionnaire is a good alternative to assess physical fitness in older adults. Copyright © 2017 Elsevier Inc. All rights reserved.

Colour-cueing in visual search.

PubMed

Laarni, J

2001-02-01

Several studies have shown that people can selectively attend to stimulus colour, e.g., in visual search, and that preknowledge of a target colour can improve response speed/accuracy. The purpose was to use a form-identification task to determine whether valid colour precues can produce benefits and invalid cues costs. The subject had to identify the orientation of a "T"-shaped element in a ring of randomly-oriented "L"s when either two or four of the elements were differently coloured. Contrary to Moore and Egeth's (1998) recent findings, colour-based attention did affect performance under data-limited conditions: Colour cues produced benefits when processing load was high; when the load was reduced, they incurred only costs. Surprisingly, a valid colour cue succeeded in improving performance in the high-load condition even when its validity was reduced to the chance level. Overall, the results suggest that knowledge of a target colour does not facilitate the processing of the target, but makes it possible to prioritize it.

Predicting emergency coronary artery bypass graft following PCI: application of a computational model to refer patients to hospitals with and without onsite surgical backup

PubMed Central

Syed, Zeeshan; Moscucci, Mauro; Share, David; Gurm, Hitinder S

2015-01-01

Background Clinical tools to stratify patients for emergency coronary artery bypass graft (ECABG) after percutaneous coronary intervention (PCI) create the opportunity to selectively assign patients undergoing procedures to hospitals with and without onsite surgical facilities for dealing with potential complications while balancing load across providers. The goal of our study was to investigate the feasibility of a computational model directly optimised for cohort-level performance to predict ECABG in PCI patients for this application. Methods Blue Cross Blue Shield of Michigan Cardiovascular Consortium registry data with 69 pre-procedural and angiographic risk variables from 68 022 PCI procedures in 2004–2007 were used to develop a support vector machine (SVM) model for ECABG. The SVM model was optimised for the area under the receiver operating characteristic curve (AUROC) at the level of the training cohort and validated on 42 310 PCI procedures performed in 2008–2009. Results There were 87 cases of ECABG (0.21%) in the validation cohort. The SVM model achieved an AUROC of 0.81 (95% CI 0.76 to 0.86). Patients in the predicted top decile were at a significantly increased risk relative to the remaining patients (OR 9.74, 95% CI 6.39 to 14.85, p<0.001) for ECABG. The SVM model optimised for the AUROC on the training cohort significantly improved discrimination, net reclassification and calibration over logistic regression and traditional SVM classification optimised for univariate performance. Conclusions Computational risk stratification directly optimising cohort-level performance holds the potential of high levels of discrimination for ECABG following PCI. This approach has value in selectively referring PCI patients to hospitals with and without onsite surgery. PMID:26688738

Examining Construct Validity of the Quantitative Literacy VALUE Rubric in College-Level STEM Assignments

ERIC Educational Resources Information Center

Gray, Julie S.; Brown, Melissa A.; Connolly, John P.

2017-01-01

Data-driven decision making is increasingly viewed as essential in a globally competitive society. Initiatives to augment standardized testing with performance-based assessment have increased as educators progressively respond to mandates for authentic measurement of student attainment. To meet this challenge, multidisciplinary rubrics were…

Key Skills Influencing Student Achievement

ERIC Educational Resources Information Center

Balch, Tonya; Gruenert, Steve

2009-01-01

A predictive, non-experimental, cross-sectional design (Johnson, 2001) was used to conduct a study to determine if elementary administrators' key counseling skills and select demographics predicted state-level student performance indicators in their respective schools. A secondary purpose of this study was to develop a valid and reliable on-line…

Validation of bending tests by nanoindentation for micro-contact analysis of MEMS switches

NASA Astrophysics Data System (ADS)

Broue, Adrien; Fourcade, Thibaut; Dhennin, Jérémie; Courtade, Frédéric; Charvet, Pierre–Louis; Pons, Patrick; Lafontan, Xavier; Plana, Robert

2010-08-01

Research on contact characterization for microelectromechanical system (MEMS) switches has been driven by the necessity to reach a high-reliability level for micro-switch applications. One of the main failures observed during cycling of the devices is the increase of the electrical contact resistance. The key issue is the electromechanical behaviour of the materials used at the contact interface where the current flows through. Metal contact switches have a large and complex set of failure mechanisms according to the current level. This paper demonstrates the validity of a new methodology using a commercial nanoindenter coupled with electrical measurements on test vehicles specially designed to investigate the micro-scale contact physics. Dedicated validation tests and modelling are performed to assess the introduced methodology by analyzing the gold contact interface with 5 µm2 square bumps at various current levels. Contact temperature rise is measured, which affects the mechanical properties of the contact materials and modifies the contact topology. In addition, the data provide a better understanding of micro-contact behaviour related to the impact of current at low- to medium-power levels. This article was originally submitted for the special section 'Selected papers from the 20th Micromechanics Europe Workshop (MME 09) (Toulouse, France, 20-22 September 2009)', Journal of Micromechanics and Microengineering, volume 20, issue 6.

Validity of a combined fibromyalgia (FM) questionnaires to asses physical activity levels in Spanish elderly women: an experimental approach.

PubMed

Cancela, José María; Varela, Silvia; Alvarez, María José; Molina, Antonio; Ayán, Carlos; Martín, Vicente

2011-01-01

Questionnaires designed to assess the level of physical activity among elderly Spanish speaking women usually have problems of reproducibility and are difficult to administer. This study aims to validate a Spanish combined version of two questionnaires originally designed to assess physical activity levels in fibromyalgia women. The leisure time physical activity instrument (LTPAI) and the physical activity at home and work instrument (PAHWI). Both questionnaires were translated to Spanish using translation/back translation methodology, and then were administered to 44 women aged 60-80 twice, with an interval of 2 weeks. During the first administration, participants answered the Yale physical activity questionnaires (YPAS) and performed the 6-min walking test (6MWT). Although the Spanish version of the LTPAI and the PAWHI showed poor test-retest reliability and poor construct validity, the sum of the two questionnaires showed much better associations. The results suggest that the Spanish combined version of LTPAI and PAHWI would seem to be useful tools for assessing the level of physical activity among elderly Spanish speaking women. Nevertheless, such considerations as the cultural adaptation of their content or the link between the intensity of physical activity as perceived and that actually done must be adjusted for greater efficiency. Copyright © 2010 Elsevier Ireland Ltd. All rights reserved.

Simplified Summative Temporal Bone Dissection Scale Demonstrates Equivalence to Existing Measures.

PubMed

Pisa, Justyn; Gousseau, Michael; Mowat, Stephanie; Westerberg, Brian; Unger, Bert; Hochman, Jordan B

2018-01-01

Emphasis on patient safety has created the need for quality assessment of fundamental surgical skills. Existing temporal bone rating scales are laborious, subject to evaluator fatigue, and contain inconsistencies when conferring points. To address these deficiencies, a novel binary assessment tool was designed and validated against a well-established rating scale. Residents completed a mastoidectomy with posterior tympanotomy on identical 3D-printed temporal bone models. Four neurotologists evaluated each specimen using a validated scale (Welling) and a newly developed "CanadaWest" scale, with scoring repeated after a 4-week interval. Nineteen participants were clustered into junior, intermediate, and senior cohorts. An ANOVA found significant differences between performance of the junior-intermediate and junior-senior cohorts for both Welling and CanadaWest scales ( P < .05). Neither scale found a significant difference between intermediate-senior resident performance ( P > .05). Cohen's kappa found strong intrarater reliability (0.711) with a high degree of interrater reliability of (0.858) for the CanadaWest scale, similar to scores on the Welling scale of (0.713) and (0.917), respectively. The CanadaWest scale was facile and delineated performance by experience level with strong intrarater reliability. Comparable to the validated Welling Scale, it distinguished junior from senior trainees but was challenged in differentiating intermediate and senior trainee performance.

Validation of the Malay version of the Modified Dental Anxiety Scale and the prevalence of dental anxiety in a Malaysian population.

PubMed

Sitheeque, Mohaideen; Massoud, Moustafa; Yahya, Suzana; Humphris, Gerry

2015-11-01

The aims of the present study were to evaluate the reliability and validity of the Malay version of the Modified Dental Anxiety Scale (MDAS), and to determine the prevalence of dental anxiety and associated factors in a Malaysian population. A Malay-language questionnaire with questions to elicit demographic and dental care-related information, and the Malay version of the MDAS, were administered to 455 patients at the dental outpatient clinics of the Hospital Universiti Sains Malaysia. Factor analysis and internal consistency statistics were generated. A test-retest of the questionnaire was performed with 30 participants. Cronbach's alpha was 0.854, indicating good internal consistency. Factor analysis yielded results showing good validity. Approximately 3.5% of the participants expressed the highest levels of anxiety. Dental anxiety was significantly higher among females than males. Age correlated inversely with dental anxiety. Individuals seeking dental care only if a problem appeared had significantly more anxiety than regular attendees. Patients who postponed treatment because of fear had significantly higher anxiety levels than those who delayed treatment for other reasons. Past adverse dental experience exacerbated dental anxiety. The Malay version of the MDAS had good reliability and validity. Anxiety levels found in the Malaysians studied were comparable to participants from other countries. © 2014 Wiley Publishing Asia Pty Ltd.

Performance of the Volumetric Diffusive Respirator at Altitude

DTIC Science & Technology

2014-08-18

information if it does not display a currently valid OMB control number. PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. 1. REPORT DATE (DD-MM...increased by 30-40%. Tidal volume remained within 15% of sea level values. Respiratory rate fell, while inspiratory time increased and high frequency...altitude, positive end expiratory pressure and peak inspiratory pressure were increased by 30-40%. Tidal volume remained within 15% of sea level

Multicenter validation of the VITEK MS v2.0 MALDI-TOF mass spectrometry system for the identification of fastidious gram-negative bacteria.

PubMed

Branda, John A; Rychert, Jenna; Burnham, Carey-Ann D; Bythrow, Maureen; Garner, Omai B; Ginocchio, Christine C; Jennemann, Rebecca; Lewinski, Michael A; Manji, Ryhana; Mochon, A Brian; Procop, Gary W; Richter, Sandra S; Sercia, Linda F; Westblade, Lars F; Ferraro, Mary Jane

2014-02-01

The VITEK MS v2.0 MALDI-TOF mass spectrometry system's performance in identifying fastidious gram-negative bacteria was evaluated in a multicenter study. Compared with the reference method (DNA sequencing), the VITEK MS system provided an accurate, species-level identification for 96% of 226 isolates; an additional 1% were accurately identified to the genus level. © 2013.

Operational Test and Evaluation Handbook for Aircrew Training Devices. Volume II. Operational Effectiveness Evaluation

DTIC Science & Technology

1982-02-01

should also convey an understanding of the differ- ences in learning behavior between initial learning activity and later skill maintenance and...refinement might then be, ATTACK MANEUVERS * Pop-up attack # Loft/ LADO type attack * Level/laydown attack Figure 5-4 showe diagrammatically the...sensitive to differ- ences in performance. Severai criteria should be used to guide the selection/development of performance measures, i.e., measure validity

Revalidation of the NASA Ames 11-by 11-Foot Transonic Wind Tunnel with a Commercial Airplane Model

NASA Technical Reports Server (NTRS)

Kmak, Frank J.; Hudgins, M.; Hergert, D.; George, Michael W. (Technical Monitor)

2001-01-01

The 11-By 11-Foot Transonic leg of the Unitary Plan Wind Tunnel (UPWT) was modernized to improve tunnel performance, capability, productivity, and reliability. Wind tunnel tests to demonstrate the readiness of the tunnel for a return to production operations included an Integrated Systems Test (IST), calibration tests, and airplane validation tests. One of the two validation tests was a 0.037-scale Boeing 777 model that was previously tested in the 11-By 11-Foot tunnel in 1991. The objective of the validation tests was to compare pre-modernization and post-modernization results from the same airplane model in order to substantiate the operational readiness of the facility. Evaluation of within-test, test-to-test, and tunnel-to-tunnel data repeatability were made to study the effects of the tunnel modifications. Tunnel productivity was also evaluated to determine the readiness of the facility for production operations. The operation of the facility, including model installation, tunnel operations, and the performance of tunnel systems, was observed and facility deficiency findings generated. The data repeatability studies and tunnel-to-tunnel comparisons demonstrated outstanding data repeatability and a high overall level of data quality. Despite some operational and facility problems, the validation test was successful in demonstrating the readiness of the facility to perform production airplane wind tunnel%, tests.

Validation of highly sensitive simultaneous targeted and untargeted analysis of keto-steroids by Girard P derivatization and stable isotope dilution-liquid chromatography-high resolution mass spectrometry.

PubMed

Frey, Alexander J; Wang, Qingqing; Busch, Christine; Feldman, Daniel; Bottalico, Lisa; Mesaros, Clementina A; Blair, Ian A; Vachani, Anil; Snyder, Nathaniel W

2016-12-01

A multiplexed quantitative method for the analysis of three major unconjugated steroids in human serum by stable isotope dilution liquid chromatography-high resolution mass spectrometry (LC-HRMS) was developed and validated on a Q Exactive Plus hybrid quadrupole/Orbitrap mass spectrometer. This quantification utilized isotope dilution and Girard P derivatization on the keto-groups of testosterone (T), androstenedione (AD) and dehydroepiandrosterone (DHEA) to improve ionization efficiency using electrospray ionization. Major isomeric compounds to T and DHEA; the inactive epimer of testosterone (epiT), and the metabolite of AD, 5α-androstanedione (5α-AD) were completely resolved on a biphenyl column within an 18min method. Inter- and intra-day method validation using LC-HRMS with qualifying product ions was performed and acceptable analytical performance was achieved. The method was further validated by comparing steroid levels from 100μL of serum from young vs older subjects. Since this approach provides high-dimensional HRMS data, untargeted analysis by age group was performed. DHEA and T were detected among the top analytes most significantly different across the two groups after untargeted LC-HRMS analysis, as well as a number of other still unknown metabolites, indicating the potential for combined targeted/untargeted analysis in steroid analysis. Copyright © 2016 Elsevier Inc. All rights reserved.

Evaluation of calibration efficacy under different levels of uncertainty

DOE PAGES

Heo, Yeonsook; Graziano, Diane J.; Guzowski, Leah; ...

2014-06-10

This study examines how calibration performs under different levels of uncertainty in model input data. It specifically assesses the efficacy of Bayesian calibration to enhance the reliability of EnergyPlus model predictions. A Bayesian approach can be used to update uncertain values of parameters, given measured energy-use data, and to quantify the associated uncertainty.We assess the efficacy of Bayesian calibration under a controlled virtual-reality setup, which enables rigorous validation of the accuracy of calibration results in terms of both calibrated parameter values and model predictions. Case studies demonstrate the performance of Bayesian calibration of base models developed from audit data withmore » differing levels of detail in building design, usage, and operation.« less

Development of an interprofessional lean facilitator assessment scale.

PubMed

Bravo-Sanchez, Cindy; Dorazio, Vincent; Denmark, Robert; Heuer, Albert J; Parrott, J Scott

2018-05-01

High reliability is important for optimising quality and safety in healthcare organisations. Reliability efforts include interprofessional collaborative practice (IPCP) and Lean quality/process improvement strategies, which require skilful facilitation. Currently, no validated Lean facilitator assessment tool for interprofessional collaboration exists. This article describes the development and pilot evaluation of such a tool; the Interprofessional Lean Facilitator Assessment Scale (ILFAS), which measures both technical and 'soft' skills, which have not been measured in other instruments. The ILFAS was developed using methodologies and principles from Lean/Shingo, IPCP, metacognition research and Bloom's Taxonomy of Learning Domains. A panel of experts confirmed the initial face validity of the instrument. Researchers independently assessed five facilitators, during six Lean sessions. Analysis included quantitative evaluation of rater agreement. Overall inter-rater agreement of the assessment of facilitator performance was high (92%), and discrepancies in the agreement statistics were analysed. Face and content validity were further established, and usability was evaluated, through primary stakeholder post-pilot feedback, uncovering minor concerns, leading to tool revision. The ILFAS appears comprehensive in the assessment of facilitator knowledge, skills, abilities, and may be useful in the discrimination between facilitators of different skill levels. Further study is needed to explore instrument performance and validity.

Satisfaction with information provided to Danish cancer patients: validation and survey results.

PubMed

Ross, Lone; Petersen, Morten Aagaard; Johnsen, Anna Thit; Lundstrøm, Louise Hyldborg; Groenvold, Mogens

2013-11-01

To validate five items (CPWQ-inf) regarding satisfaction with information provided to cancer patients from health care staff, assess the prevalence of dissatisfaction with this information, and identify factors predicting dissatisfaction. The questionnaire was validated by patient-observer agreement and cognitive interviews. The prevalence of dissatisfaction was assessed in a cross-sectional sample of all cancer patients in contact with hospitals during the past year in three Danish counties. The validation showed that the CPWQ performed well. Between 3 and 23% of the 1490 participating patients were dissatisfied with each of the measured aspects of information. The highest level of dissatisfaction was reported regarding the guidance, support and help provided when the diagnosis was given. Younger patients were consistently more dissatisfied than older patients. The brief CPWQ performs well for survey purposes. The survey depicts the heterogeneous patient population encountered by hospital staff and showed that younger patients probably had higher expectations or a higher need for information and that those with more severe diagnoses/prognoses require extra care in providing information. Four brief questions can efficiently assess information needs. With increasing demands for information, a wide range of innovative initiatives is needed. Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.

Numerical studies and metric development for validation of magnetohydrodynamic models on the HIT-SI experiment

DOE Office of Scientific and Technical Information (OSTI.GOV)

Hansen, C., E-mail: hansec@uw.edu; Columbia University, New York, New York 10027; Victor, B.

We present application of three scalar metrics derived from the Biorthogonal Decomposition (BD) technique to evaluate the level of agreement between macroscopic plasma dynamics in different data sets. BD decomposes large data sets, as produced by distributed diagnostic arrays, into principal mode structures without assumptions on spatial or temporal structure. These metrics have been applied to validation of the Hall-MHD model using experimental data from the Helicity Injected Torus with Steady Inductive helicity injection experiment. Each metric provides a measure of correlation between mode structures extracted from experimental data and simulations for an array of 192 surface-mounted magnetic probes. Numericalmore » validation studies have been performed using the NIMROD code, where the injectors are modeled as boundary conditions on the flux conserver, and the PSI-TET code, where the entire plasma volume is treated. Initial results from a comprehensive validation study of high performance operation with different injector frequencies are presented, illustrating application of the BD method. Using a simplified (constant, uniform density and temperature) Hall-MHD model, simulation results agree with experimental observation for two of the three defined metrics when the injectors are driven with a frequency of 14.5 kHz.« less

An analysis of thrust of a realistic solar sail with focus on a flight validation mission in a geocentric orbit

NASA Astrophysics Data System (ADS)

Campbell, Bruce A.

Several scientifically important space flight missions have been identified that, at this time, can only be practically achieved using a solar sail propulsion system. These missions take advantage of the potentially continuous force on the sail, provided by solar radiation, to produce significant changes in the spacecraft's velocity, in both magnitude and/or direction, without the need for carrying the enormous amount of fuel that conventional propulsion systems would require to provide the same performance. However, to provide thrust levels that would support these missions requires solar sail areas in the (tens of) thousands of square meter sizes. To realize this, many technical areas must be developed further and demonstrated in space before solar sails will be accepted as a viable space mission propulsion system. One of these areas concerns understanding the propulsion performance of a realistic solar sail well enough for mission planning. Without this understanding, solar sail orbits could not be predicted well enough to meet defined mission requirements, such as rendezvous or station-keeping, and solar sail orbit optimization, such as minimizing flight time, could be close to impossible. In most mission studies, either an "ideal" sail's performance is used for mission planning, or some top-level assumptions of certain nonideal sail characteristics are incorporated to give a slightly better estimate of the sail performance. This paper identifies the major sources of solar sail thrust performance uncertainty, and analyzes the most significant ones to provide a more comprehensive understanding of thrust generation by a "realistic" solar sail. With this understanding, mission planners will be able to more confidently and accurately estimate the capabilities of such a system. The first solar sail mission will likely be a system validation mission, using a relatively small sail in a geocentric (Earth-centered) orbit. The author has been involved in conceptual design of such missions, and through this became aware of the current status in solar sail system development, and the need for a better understanding of the thrust performance of a "realistic" solar sail. Such a validation mission is significantly different than most of the "operational" science missions envisioned to utilize a solar sail propulsion system. These future missions will likely use very large, very light sails in heliocentric orbits far away from major gravity fields like planets, have very long mission lifetimes (years), and will conduct relatively minor and slow orbital and attitude control maneuvers. Nonetheless, most of the capabilities of later systems can be gleaned from a small geocentric validation mission. This paper is a significant step toward understanding the thrust characteristics and performance of a realistic solar sail, and provides insight to the methods by which this understanding can be corroborated by a solar sail validation mission.

Batch Effect Confounding Leads to Strong Bias in Performance Estimates Obtained by Cross-Validation

PubMed Central

Delorenzi, Mauro

2014-01-01

Background With the large amount of biological data that is currently publicly available, many investigators combine multiple data sets to increase the sample size and potentially also the power of their analyses. However, technical differences (“batch effects”) as well as differences in sample composition between the data sets may significantly affect the ability to draw generalizable conclusions from such studies. Focus The current study focuses on the construction of classifiers, and the use of cross-validation to estimate their performance. In particular, we investigate the impact of batch effects and differences in sample composition between batches on the accuracy of the classification performance estimate obtained via cross-validation. The focus on estimation bias is a main difference compared to previous studies, which have mostly focused on the predictive performance and how it relates to the presence of batch effects. Data We work on simulated data sets. To have realistic intensity distributions, we use real gene expression data as the basis for our simulation. Random samples from this expression matrix are selected and assigned to group 1 (e.g., ‘control’) or group 2 (e.g., ‘treated’). We introduce batch effects and select some features to be differentially expressed between the two groups. We consider several scenarios for our study, most importantly different levels of confounding between groups and batch effects. Methods We focus on well-known classifiers: logistic regression, Support Vector Machines (SVM), k-nearest neighbors (kNN) and Random Forests (RF). Feature selection is performed with the Wilcoxon test or the lasso. Parameter tuning and feature selection, as well as the estimation of the prediction performance of each classifier, is performed within a nested cross-validation scheme. The estimated classification performance is then compared to what is obtained when applying the classifier to independent data. PMID:24967636

Using the Rasch analysis for the psychometric validation of the Irregular Word Reading Test (TeLPI): A Portuguese test for the assessment of premorbid intelligence.

PubMed

Freitas, Sandra; Prieto, Gerardo; Simões, Mário R; Nogueira, Joana; Santana, Isabel; Martins, Cristina; Alves, Lara

2018-05-03

The present study aims to analyze the psychometric characteristics of the TeLPI (Irregular Words Reading Test), a Portuguese premorbid intelligence test, using the Rasch model for dichotomous items. The results reveal an overall adequacy and a good fit of values regarding both items and persons. A high variability of cognitive performance level and a good quality of the measurements were also found. The TeLPI has proved to be a unidimensional measure with reduced DIF effects. The present findings contribute to overcome an important gap in the psychometric validity of this instrument and provide good evidence of the overall psychometric validity of TeLPI results.

A validation study of the Chinese-Cantonese Addenbrooke’s Cognitive Examination Revised (C-ACER)

PubMed Central

Wong, LL; Chan, CC; Leung, JL; Yung, CY; Wu, KK; Cheung, SYY; Lam, CLM

2013-01-01

Background There is no valid instrument for multidomain cognitive assessment to aid the detection of mild cognitive impairment (MCI) and mild dementia in Hong Kong. This study aimed to validate the Cantonese Addenbrooke’s Cognitive Examination Revised (C-ACER) in the identification of MCI and dementia. Methods 147 participants (Dementia, n = 54; MCI, n = 50; controls, n = 43) aged 60 or above were assessed by a psychiatrist using C-ACER. The C-ACER scores were validated against the expert diagnosis according to DSM-IV criteria for dementia and Petersen criteria for MCI. Statistical analysis was performed using the receiver operating characteristic method and regression analyses. Results The optimal cut-off score for the C-ACER to differentiate MCI from normal controls was 79/80, giving the sensitivity of 0.74, specificity of 0.84 and area under curve (AUC) of 0.84. At the optimal cut-off of 73/74, C-ACER had satisfactory sensitivity (0.93), specificity (0.95) and AUC (0.98) to identify dementia from controls. Performance of C-ACER, as reflected by AUC, was not affected after adjustment of the effect of education level. Total C-ACER scores were significantly correlated with scores of global deterioration scale (Spearman’s rho = −0.73, P < 0.01). Conclusion C-ACER is a sensitive and specific bedside test to assess a broad spectrum of cognitive abilities, and to detect MCI and dementia of different severity. It can be used and interpreted with ease, without the need to adjust for education level in persons aged 60 or above. PMID:23785235

Experimental validation of the influence of white matter anisotropy on the intracranial EEG forward solution.

PubMed

Bangera, Nitin B; Schomer, Donald L; Dehghani, Nima; Ulbert, Istvan; Cash, Sydney; Papavasiliou, Steve; Eisenberg, Solomon R; Dale, Anders M; Halgren, Eric

2010-12-01

Forward solutions with different levels of complexity are employed for localization of current generators, which are responsible for the electric and magnetic fields measured from the human brain. The influence of brain anisotropy on the forward solution is poorly understood. The goal of this study is to validate an anisotropic model for the intracranial electric forward solution by comparing with the directly measured 'gold standard'. Dipolar sources are created at known locations in the brain and intracranial electroencephalogram (EEG) is recorded simultaneously. Isotropic models with increasing level of complexity are generated along with anisotropic models based on Diffusion tensor imaging (DTI). A Finite Element Method based forward solution is calculated and validated using the measured data. Major findings are (1) An anisotropic model with a linear scaling between the eigenvalues of the electrical conductivity tensor and water self-diffusion tensor in brain tissue is validated. The greatest improvement was obtained when the stimulation site is close to a region of high anisotropy. The model with a global anisotropic ratio of 10:1 between the eigenvalues (parallel: tangential to the fiber direction) has the worst performance of all the anisotropic models. (2) Inclusion of cerebrospinal fluid as well as brain anisotropy in the forward model is necessary for an accurate description of the electric field inside the skull. The results indicate that an anisotropic model based on the DTI can be constructed non-invasively and shows an improved performance when compared to the isotropic models for the calculation of the intracranial EEG forward solution.

Are measurements of patient safety culture and adverse events valid and reliable? Results from a cross sectional study.

PubMed

Farup, Per G

2015-05-02

The association between measurements of the patient safety culture and the "true" patient safety has been insufficiently documented, and the validity of the tools used for the measurements has been questioned. This study explored associations between the patient safety culture and adverse events, and evaluated the validity of the tools. In 2008/2009, a survey on patient safety culture was performed with Hospital Survey on Patient Safety Culture (HSOPSC) in two medical departments in two geographically separated hospitals of Innlandet Hospital Trust. Later, a retrospective analysis of adverse events during the same period was performed with the Global Trigger Tool (GTT). The safety culture and adverse events were compared between the departments. 185 employees participated in the study, and 272 patient records were analysed. The HSOPSC scores were lower and adverse events less prevalent in department 1 than in department 2. In departments 1 and 2 the mean HSOPSC scores (SD) were at the unit level 3.62 (0.42) and 3.90 (0.37) (p < 0.001), and at the hospital level 3.35 (1.53) and 3.67 (0.53) (ns, p = 0.19) respectively. The proportion of records with adverse events were 10/135 (7%) and 28/137 (20%) (p = 0.003) respectively. There was an inverse association between the patient safety culture and adverse events. Until the criterion validity of the tools for measuring patient safety culture and tracking of adverse events have been further evaluated, measurement of patient safety culture could not be used as a proxy for the "true" safety.

Development and piloting the Woman Centred Care Scale (WCCS).

PubMed

Brady, Susannah; Bogossian, Fiona; Gibbons, Kristen

2017-06-01

In midwifery we espouse a woman centred care approach to practice, yet in midwifery education no valid instrument exists with which to measure the performance of these behaviours in midwifery students. To develop and validate an instrument to measure woman centred care behaviours in midwifery students. We identified four core concepts; woman's sphere, holism, self-determination and the shared power relationship. We mapped 18 individual descriptive care behaviours (from the Australian National Competency Standards for the Midwife) to these concepts to create an instrument to articulate and measure care behaviours that are specifically woman centred. Review by expert midwifery clinicians ensured face, content and construct validity of the scale and predictive validity and reliability were tested in a simulated learning environment. Midwifery students were video recorded performing a clinical skill and the videos were reviewed and rated by two expert clinicians who assessed the woman centred care behaviours demonstrated by the students (n=69). Test and re-test reliability of the instrument was high for each of the individual raters (Kappa 0.946 and 0.849 respectively p<0.001). However, when raters were compared there were differences between their scores suggesting variation in their expectations of woman centred care behaviours (Kappa 0.470, p<0.001). Midwifery students who had repeated exposures to higher levels of simulation fidelity demonstrated higher levels of woman centred care behaviours. The WCCS has implications for education and the wider midwifery profession in recognising and maintaining practice consistent with the underlying philosophy of woman centred care. Copyright © 2016 Australian College of Midwives. Published by Elsevier Ltd. All rights reserved.

Management of lumbar zygapophysial (facet) joint pain

PubMed Central

Manchikanti, Laxmaiah; Hirsch, Joshua A; Falco, Frank JE; Boswell, Mark V

2016-01-01

AIM: To investigate the diagnostic validity and therapeutic value of lumbar facet joint interventions in managing chronic low back pain. METHODS: The review process applied systematic evidence-based assessment methodology of controlled trials of diagnostic validity and randomized controlled trials of therapeutic efficacy. Inclusion criteria encompassed all facet joint interventions performed in a controlled fashion. The pain relief of greater than 50% was the outcome measure for diagnostic accuracy assessment of the controlled studies with ability to perform previously painful movements, whereas, for randomized controlled therapeutic efficacy studies, the primary outcome was significant pain relief and the secondary outcome was a positive change in functional status. For the inclusion of the diagnostic controlled studies, all studies must have utilized either placebo controlled facet joint blocks or comparative local anesthetic blocks. In assessing therapeutic interventions, short-term and long-term reliefs were defined as either up to 6 mo or greater than 6 mo of relief. The literature search was extensive utilizing various types of electronic search media including PubMed from 1966 onwards, Cochrane library, National Guideline Clearinghouse, clinicaltrials.gov, along with other sources including previous systematic reviews, non-indexed journals, and abstracts until March 2015. Each manuscript included in the assessment was assessed for methodologic quality or risk of bias assessment utilizing the Quality Appraisal of Reliability Studies checklist for diagnostic interventions, and Cochrane review criteria and the Interventional Pain Management Techniques - Quality Appraisal of Reliability and Risk of Bias Assessment tool for therapeutic interventions. Evidence based on the review of the systematic assessment of controlled studies was graded utilizing a modified schema of qualitative evidence with best evidence synthesis, variable from level I to level V. RESULTS: Across all databases, 16 high quality diagnostic accuracy studies were identified. In addition, multiple studies assessed the influence of multiple factors on diagnostic validity. In contrast to diagnostic validity studies, therapeutic efficacy trials were limited to a total of 14 randomized controlled trials, assessing the efficacy of intraarticular injections, facet or zygapophysial joint nerve blocks, and radiofrequency neurotomy of the innervation of the facet joints. The evidence for the diagnostic validity of lumbar facet joint nerve blocks with at least 75% pain relief with ability to perform previously painful movements was level I, based on a range of level I to V derived from a best evidence synthesis. For therapeutic interventions, the evidence was variable from level II to III, with level II evidence for lumbar facet joint nerve blocks and radiofrequency neurotomy for long-term improvement (greater than 6 mo), and level III evidence for lumbosacral zygapophysial joint injections for short-term improvement only. CONCLUSION: This review provides significant evidence for the diagnostic validity of facet joint nerve blocks, and moderate evidence for therapeutic radiofrequency neurotomy and therapeutic facet joint nerve blocks in managing chronic low back pain. PMID:27190760

Technical note: Intercomparison of three AATSR Level 2 (L2) AOD products over China

NASA Astrophysics Data System (ADS)

Che, Yahui; Xue, Yong; Mei, Linlu; Guang, Jie; She, Lu; Guo, Jianping; Hu, Yincui; Xu, Hui; He, Xingwei; Di, Aojie; Fan, Cheng

2016-08-01

One of four main focus areas of the PEEX initiative is to establish and sustain long-term, continuous, and comprehensive ground-based, airborne, and seaborne observation infrastructure together with satellite data. The Advanced Along-Track Scanning Radiometer (AATSR) aboard ENVISAT is used to observe the Earth in dual view. The AATSR data can be used to retrieve aerosol optical depth (AOD) over both land and ocean, which is an important parameter in the characterization of aerosol properties. In recent years, aerosol retrieval algorithms have been developed both over land and ocean, taking advantage of the features of dual view, which can help eliminate the contribution of Earth's surface to top-of-atmosphere (TOA) reflectance. The Aerosol_cci project, as a part of the Climate Change Initiative (CCI), provides users with three AOD retrieval algorithms for AATSR data, including the Swansea algorithm (SU), the ATSR-2ATSR dual-view aerosol retrieval algorithm (ADV), and the Oxford-RAL Retrieval of Aerosol and Cloud algorithm (ORAC). The validation team of the Aerosol-CCI project has validated AOD (both Level 2 and Level 3 products) and AE (Ångström Exponent) (Level 2 product only) against the AERONET data in a round-robin evaluation using the validation tool of the AeroCOM (Aerosol Comparison between Observations and Models) project. For the purpose of evaluating different performances of these three algorithms in calculating AODs over mainland China, we introduce ground-based data from CARSNET (China Aerosol Remote Sensing Network), which was designed for aerosol observations in China. Because China is vast in territory and has great differences in terms of land surfaces, the combination of the AERONET and CARSNET data can validate the L2 AOD products more comprehensively. The validation results show different performances of these products in 2007, 2008, and 2010. The SU algorithm performs very well over sites with different surface conditions in mainland China from March to October, but it slightly underestimates AOD over barren or sparsely vegetated surfaces in western China, with mean bias error (MBE) ranging from 0.05 to 0.10. The ADV product has the same precision with a low root mean square error (RMSE) smaller than 0.2 over most sites and the same error distribution as the SU product. The main limits of the ADV algorithm are underestimation and applicability; underestimation is particularly obvious over the sites of Datong, Lanzhou, and Urumchi, where the dominant land cover is grassland, with an MBE larger than 0.2, and the main aerosol sources are coal combustion and dust. The ORAC algorithm has the ability to retrieve AOD at different ranges, including high AOD (larger than 1.0); however, the stability deceases significantly with increasing AOD, especially when AOD > 1.0. In addition, the ORAC product is consistent with the CARSNET product in winter (December, January, and February), whereas other validation results lack matches during winter.

Mild cognitive impairment as a risk factor for Parkinson's disease dementia.

PubMed

Hoogland, Jeroen; Boel, Judith A; de Bie, Rob M A; Geskus, Ronald B; Schmand, Ben A; Dalrymple-Alford, John C; Marras, Connie; Adler, Charles H; Goldman, Jennifer G; Tröster, Alexander I; Burn, David J; Litvan, Irene; Geurtsen, Gert J

2017-07-01

The International Parkinson and Movement Disorder Society criteria for mild cognitive impairment in PD were recently formulated. The aim of this international study was to evaluate the predictive validity of the comprehensive (level II) version of these criteria by assessment of their contribution to the hazard of PD dementia. Individual patient data were selected from four separate studies on cognition in PD that provided information on demographics, motor examination, depression, neuropsychological examination suitable for application of level II criteria, and longitudinal follow-up for conversion to dementia. Survival analysis evaluated the predictive value of level II criteria for cognitive decline toward dementia as expressed by the relative hazard of dementia. A total of 467 patients were included. The analyses showed a clear contribution of impairment according to level II mild cognitive impairment criteria, age, and severity of PD motor symptoms to the hazard of dementia. There was a trend of increasing hazard of dementia with declining neuropsychological performance. This is the first large international study evaluating the predictive validity of level II mild cognitive impairment criteria for PD. The results showed a clear and unique contribution of classification according to level II criteria to the hazard of PD dementia. This finding supports their predictive validity and shows that they contribute important new information on the hazard of dementia, beyond known demographic and PD-specific factors of influence. © 2017 International Parkinson and Movement Disorder Society. © 2017 International Parkinson and Movement Disorder Society.

Fundamentals of endoscopic surgery: creation and validation of the hands-on test.

PubMed

Vassiliou, Melina C; Dunkin, Brian J; Fried, Gerald M; Mellinger, John D; Trus, Thadeus; Kaneva, Pepa; Lyons, Calvin; Korndorffer, James R; Ujiki, Michael; Velanovich, Vic; Kochman, Michael L; Tsuda, Shawn; Martinez, Jose; Scott, Daniel J; Korus, Gary; Park, Adrian; Marks, Jeffrey M

2014-03-01

The Fundamentals of Endoscopic Surgery™ (FES) program consists of online materials and didactic and skills-based tests. All components were designed to measure the skills and knowledge required to perform safe flexible endoscopy. The purpose of this multicenter study was to evaluate the reliability and validity of the hands-on component of the FES examination, and to establish the pass score. Expert endoscopists identified the critical skill set required for flexible endoscopy. They were then modeled in a virtual reality simulator (GI Mentor™ II, Simbionix™ Ltd., Airport City, Israel) to create five tasks and metrics. Scores were designed to measure both speed and precision. Validity evidence was assessed by correlating performance with self-reported endoscopic experience (surgeons and gastroenterologists [GIs]). Internal consistency of each test task was assessed using Cronbach's alpha. Test-retest reliability was determined by having the same participant perform the test a second time and comparing their scores. Passing scores were determined by a contrasting groups methodology and use of receiver operating characteristic curves. A total of 160 participants (17 % GIs) performed the simulator test. Scores on the five tasks showed good internal consistency reliability and all had significant correlations with endoscopic experience. Total FES scores correlated 0.73, with participants' level of endoscopic experience providing evidence of their validity, and their internal consistency reliability (Cronbach's alpha) was 0.82. Test-retest reliability was assessed in 11 participants, and the intraclass correlation was 0.85. The passing score was determined and is estimated to have a sensitivity (true positive rate) of 0.81 and a 1-specificity (false positive rate) of 0.21. The FES hands-on skills test examines the basic procedural components required to perform safe flexible endoscopy. It meets rigorous standards of reliability and validity required for high-stakes examinations, and, together with the knowledge component, may help contribute to the definition and determination of competence in endoscopy.

Using Android-Based Educational Game for Learning Colloid Material

NASA Astrophysics Data System (ADS)

Sari, S.; Anjani, R.; Farida, I.; Ramdhani, M. A.

2017-09-01

This research is based on the importance of the development of student’s chemical literacy on Colloid material using Android-based educational game media. Educational game products are developed through research and development design. In the analysis phase, material analysis is performed to generate concept maps, determine chemical literacy indicators, game strategies and set game paths. In the design phase, product packaging is carried out, then validation and feasibility test are performed. Research produces educational game based on Android that has the characteristics that is: Colloid material presented in 12 levels of game in the form of questions and challenges, presents visualization of discourse, images and animation contextually to develop the process of thinking and attitude. Based on the analysis of validation and trial results, the product is considered feasible to use.

Development of a reference material for routine performance monitoring of methods measuring polychlorinated dibenzo-p-dioxins, polychlorinated dibenzofurans and dioxin-like polychlorinated biphenyls.

PubMed

Selliah, S S; Cussion, S; MacPherson, K A; Reiner, E J; Toner, D

2001-06-01

Matrix-matched environmental certified reference materials (CRMs) are one of the most useful tools to validate analytical methods, assess analytical laboratory performance and to assist in the resolution of data conflicts between laboratories. This paper describes the development of a lake sediment as a CRM for polychorinated dibenzo-p-dioxins (PCDDs), polychlorinated dibenzofurans (PCDFs) and dioxin-like polychlorinated biphenyls (DLPCBs). The presence of DLPCBs in the environment is of increased concern and analytical methods are being developed internationally for monitoring DLPCBs in the environment. This paper also reports the results of an international interlaboratory study involving thirty-five laboratories from seventeen countries, conducted to characterize and validate levels of a sediment reference material for PCDDs, PCDFs and DLPCBs.

Retina Image Analysis and Ocular Telehealth: The Oak Ridge National Laboratory-Hamilton Eye Institute Case Study

DOE Office of Scientific and Technical Information (OSTI.GOV)

Karnowski, Thomas Paul; Giancardo, Luca; Li, Yaquin

2013-01-01

Automated retina image analysis has reached a high level of maturity in recent years, and thus the question of how validation is performed in these systems is beginning to grow in importance. One application of retina image analysis is in telemedicine, where an automated system could enable the automated detection of diabetic retinopathy and other eye diseases as a low-cost method for broad-based screening. In this work we discuss our experiences in developing a telemedical network for retina image analysis, including our progression from a manual diagnosis network to a more fully automated one. We pay special attention to howmore » validations of our algorithm steps are performed, both using data from the telemedicine network and other public databases.« less

Link performance model for filter bank based multicarrier systems

NASA Astrophysics Data System (ADS)

Petrov, Dmitry; Oborina, Alexandra; Giupponi, Lorenza; Stitz, Tobias Hidalgo

2014-12-01

This paper presents a complete link level abstraction model for link quality estimation on the system level of filter bank multicarrier (FBMC)-based networks. The application of mean mutual information per coded bit (MMIB) approach is validated for the FBMC systems. The considered quality measure of the resource element for the FBMC transmission is the received signal-to-noise-plus-distortion ratio (SNDR). Simulation results of the proposed link abstraction model show that the proposed approach is capable of estimating the block error rate (BLER) accurately, even when the signal is propagated through the channels with deep and frequent fades, as it is the case for the 3GPP Hilly Terrain (3GPP-HT) and Enhanced Typical Urban (ETU) models. The FBMC-related results of link level simulations are compared with cyclic prefix orthogonal frequency division multiplexing (CP-OFDM) analogs. Simulation results are also validated through the comparison to reference publicly available results. Finally, the steps of link level abstraction algorithm for FBMC are formulated and its application for system level simulation of a professional mobile radio (PMR) network is discussed.

Monitoring performance, pituitary-adrenal hormones and mood profiles: how to diagnose non-functional over-reaching in male elite junior soccer players.

PubMed

Schmikli, Sándor L; de Vries, Wouter R; Brink, Michel S; Backx, Frank Jg

2012-11-01

To verify if in male elite junior soccer players a minimum 1-month performance decrease is accompanied by a mood profile and hormone levels typical of non-functional over-reaching (NFOR). A prospective case-control study using a monthly performance monitor with a standardised field test to detect the performance changes. Players with a performance decrease lasting at least 1 month were compared with control players without a performance decrease on mood scores and pre-exercise and postexercise levels of stress hormones. Sporting field and sports medical laboratory. Ninety-four young elite soccer players were monitored during the 2006-2008 seasons. Twenty-one players were invited to the laboratory, seven of whom showed a significant performance decrease. Performance change over time, scores on the profile of mood states and premaximal and postmaximal exercise serum levels of adrenocorticotropic hormone (ACTH), growth hormone (GH) and cortisol. Players with a performance decrease showed psychological and hormonal changes typical of the non-functional state of over-reaching. Scores were higher on depression and anger, whereas the resting GH levels and ACTH levels after maximal exercise were reduced. ACTH and GH were capable of classifying all but one player correctly as either NFOR or control. Performance-related criteria in field tests are capable of identifying players with worsened mood and adaptations of the endocrine system that fit the definition of NFOR. Performance, mood and hormone levels may therefore be considered as valid instruments to diagnose NFOR in young elite soccer players.

Validity of early parathyroid hormone assay as a diagnostic tool for sub-total thyroidectomy related hypocalcaemia.

PubMed

Riaz, Umbreen; Shah, Syed Aslam; Zahoor, Imran; Riaz, Arsalan; Zubair, Muhammad

2014-07-01

To determine the validity of early (one hour postoperatively) parathyroid hormone (PTH) assay (² 10 pg/ml), keeping gold standard as the serum ionic calcium level, for predicting sub-total thyroidectomy-related hypocalcaemia and to calculate the sensitivity and specificity of latent signs of tetany. Cross-sectional validation study. Department of General Surgery, Pakistan Institute of Medical Sciences, Islamabad from August 2008 to August 2010. Patients undergoing sub-total thyroidectomy were included by convenience sampling. PTH assay was performed 1 hour post sub-total thyroidectomy. Serum calcium levels were performed at 24 and 48 hours, 5th day and 2 weeks after surgery. Cases that developed hypocalcaemia were followed-up for a period of 6 months with monthly calcium level estimation to identify cases of permanent hypocalcaemia. Symptoms and signs of hypocalcaemia manifesting in our patients were recorded. Data was analyzed through SPSS version 10. 2 x 2 tables were used to calculate sensitivity and specificity of PTH in detecting post-thyroidectomy hypocalcaemia. Out of a total of 110 patients included in the study, 16.36% (n=18) developed hypocalcaemia including 1.81% (n=2) cases of permanent hypoparathyroidism. The sensitivity of one hour postoperative PTH assay as a predictive tool for post-thyroidectomy related hypocalcaemia was 94.4% while its specificity was 83.6% with 53% positive predictive value and 98.7% negative predictive value. One hour post sub-total thyroidectomy PTH assay can be helpful in predicting post sub-total thyroidectomy hypocalcaemia. Moreover, it can be useful in safe discharge of day-care thyroidectomy patients.

Assessing student engagement and self-regulated learning in a medical gross anatomy course.

PubMed

Pizzimenti, Marc A; Axelson, Rick D

2015-01-01

In courses with large enrollment, faculty members sometimes struggle with an understanding of how individual students are engaging in their courses. Information about the level of student engagement that instructors would likely find most useful can be linked to: (1) the learning strategies that students are using; (2) the barriers to learning that students are encountering; and (3) whether the course materials and activities are yielding the intended learning outcomes. This study drew upon self-regulated learning theory (SRL) to specify relevant information about learning engagement, and how the measures of particular scales might prove useful for student/faculty reflection. We tested the quality of such information as collected via the Motivated Strategies for Learning Questionnaire (MSLQ). MSLQ items were administered through a web-based survey to 150 students in a first-year medical gross anatomy course. The resulting 66 responses (44% response rate) were examined for information quality (internal reliability and predictive validity) and usefulness of the results to the course instructor. Students' final grades in the course were correlated with their MSLQ scale scores to assess the predictive validity of the measures. These results were consistent with the course design and expectations, showing that greater use of learning strategies such as elaboration and critical thinking was associated with higher levels of performance in the course. Motivation subscales for learning were also correlated with the higher levels of performance in the course. The extent to which these scales capture valid and reliable information in other institutional settings and courses needs further investigation. © 2014 American Association of Anatomists.

Validation of a Dry Model for Assessing the Performance of Arthroscopic Hip Labral Repair.

PubMed

Phillips, Lisa; Cheung, Jeffrey J H; Whelan, Daniel B; Murnaghan, Michael Lucas; Chahal, Jas; Theodoropoulos, John; Ogilvie-Harris, Darrell; Macniven, Ian; Dwyer, Tim

2017-07-01

Arthroscopic hip labral repair is a technically challenging and demanding surgical technique with a steep learning curve. Arthroscopic simulation allows trainees to develop these skills in a safe environment. The purpose of this study was to evaluate the use of a combination of assessment ratings for the performance of arthroscopic hip labral repair on a dry model. Cross-sectional study; Level of evidence, 3. A total of 47 participants including orthopaedic surgery residents (n = 37), sports medicine fellows (n = 5), and staff surgeons (n = 5) performed arthroscopic hip labral repair on a dry model. Prior arthroscopic experience was noted. Participants were evaluated by 2 orthopaedic surgeons using a task-specific checklist, the Arthroscopic Surgical Skill Evaluation Tool (ASSET), task completion time, and a final global rating scale. All procedures were video-recorded and scored by an orthopaedic fellow blinded to the level of training of each participant. The internal consistency/reliability (Cronbach alpha) using the total ASSET score for the procedure was high (intraclass correlation coefficient > 0.9). One-way analysis of variance for the total ASSET score demonstrated a difference between participants based on the level of training ( F 3,43 = 27.8, P < .001). A good correlation was seen between the ASSET score and previous exposure to arthroscopic procedures ( r = 0.52-0.73, P < .001). The interrater reliability for the ASSET score was excellent (>0.9). The results of this study demonstrate that the use of dry models to assess the performance of arthroscopic hip labral repair by trainees is both valid and reliable. Further research will be required to demonstrate a correlation with performance on cadaveric specimens or in the operating room.

Validation of the National Aeronautics and Space Administration Task Load Index as a tool to evaluate the learning curve for endoscopy training.

PubMed

Mohamed, Rachid; Raman, Maitreyi; Anderson, John; McLaughlin, Kevin; Rostom, Alaa; Coderre, Sylvain

2014-03-01

Although workplace workload assessments exist in different fields, an endoscopy-specific workload assessment tool is lacking. To validate such a workload tool and use it to map the progression of novice trainees in gastroenterology in performing their first endoscopies. The National Aeronautics and Space Administration Task Load Index (NASA-TLX) workload assessment tool was completed by eight novice trainees in gastroenterology and 10 practicing gastroenterologists⁄surgeons. An exploratory factor analysis was performed to construct a streamlined endoscopy-specific task load index, which was subsequently validated. The 'Endoscopy Task Load Index' was used to monitor progression of trainee exertion and self-assessed performance over their first 40 procedures. From the factor analysis of the NASA-TLX, two principal components emerged: a measure of exertion and a measure of self-efficacy. These items became the components of the newly validated Endoscopy Task Load Index. There was a steady decline in self-perceived exertion over the training period, which was more rapid for gastroscopy than colonoscopy. The self-efficacy scores for gastroscopy rapidly increased over the first few procedures, reaching a plateau after this period of time. For colonoscopy, there was a progressive increase in reported self-efficacy over the first three quartiles of procedures, followed by a drop in self-efficacy scores over the final quartile. The present study validated an Endoscopy Task Load Index that can be completed in <1 min. Practical implications of such a tool in endoscopy education include identifying periods of higher perceived exertion among novice endoscopists, facilitating appropriate levels of guidance from trainers.

Developing and Testing a Model to Predict Outcomes of Organizational Change

PubMed Central

Gustafson, David H; Sainfort, François; Eichler, Mary; Adams, Laura; Bisognano, Maureen; Steudel, Harold

2003-01-01

Objective To test the effectiveness of a Bayesian model employing subjective probability estimates for predicting success and failure of health care improvement projects. Data Sources Experts' subjective assessment data for model development and independent retrospective data on 221 healthcare improvement projects in the United States, Canada, and the Netherlands collected between 1996 and 2000 for validation. Methods A panel of theoretical and practical experts and literature in organizational change were used to identify factors predicting the outcome of improvement efforts. A Bayesian model was developed to estimate probability of successful change using subjective estimates of likelihood ratios and prior odds elicited from the panel of experts. A subsequent retrospective empirical analysis of change efforts in 198 health care organizations was performed to validate the model. Logistic regression and ROC analysis were used to evaluate the model's performance using three alternative definitions of success. Data Collection For the model development, experts' subjective assessments were elicited using an integrative group process. For the validation study, a staff person intimately involved in each improvement project responded to a written survey asking questions about model factors and project outcomes. Results Logistic regression chi-square statistics and areas under the ROC curve demonstrated a high level of model performance in predicting success. Chi-square statistics were significant at the 0.001 level and areas under the ROC curve were greater than 0.84. Conclusions A subjective Bayesian model was effective in predicting the outcome of actual improvement projects. Additional prospective evaluations as well as testing the impact of this model as an intervention are warranted. PMID:12785571

Reliability and validity of the Lithuanian Tinnitus Handicap Inventory.

PubMed

Ulozienė, Ingrida; Balnytė, Renata; Alzbutienė, Giedrė; Arechvo, Irina; Vaitkus, Antanas; Šileikaitė, Milda; Šaferis, Viktoras; Ulozas, Virgilijus

2016-01-01

The aim of this study was to determine the reliability and validity of the Lithuanian version of the Tinnitus Handicap Inventory (THI), a self-report measure of perceived tinnitus handicap. A cross-sectional psychometric validation study was performed in the University Hospital. A total of 248 subjects reporting chronic tinnitus as their primary complaint or secondary to hearing loss were encluded in the study and filled in the Lithuanian version of THI. For assessment of construct validity a subgroup of 55 participants completed the Lithuanian version of the Hospital Anxiety and Depression Scale as a measure of self-perceived levels of anxiety and depression. Test-retest and internal consistency reliability as well as construct validity were calculated. The Lithuanian version of the THI and its subscales showed a robust internal consistency reliability (Cronbach's alpha=0.93) comparable to the original version. Statistically significant correlations were observed between the Lithuanian translation of the THI and the measures of self-perceived levels of anxiety and depression using HADS. Confirmatory factor analysis demonstrated that the three subscales of the THI Lithuanian version corresponded to three different factors, which strongly correlated between themselves. The results suggest that the Lithuanian version of THI maintains its original validity and may serve as reliable and valid measure of general tinnitus related distress that can be used in a clinical setting to quantify the impact of tinnitus on daily living. Copyright © 2016 The Lithuanian University of Health Sciences. Production and hosting by Elsevier Urban & Partner Sp. z o.o. All rights reserved.

[Validation and reliability study of the parent concerns about surgery questionnaire: What worries parents?

PubMed

Gironés Muriel, Alberto; Campos Segovia, Ana; Ríos Gómez, Patricia

2018-01-01

The study of mediating variables and psychological responses to child surgery involves the evaluation of both the patient and the parents as regards different stressors. To have a reliable and reproducible valid evaluation tool that assesses the level of paternal involvement in relation to different stressors in the setting of surgery. A self-report questionnaire study was completed by 123 subjects of both sexes, subdivided into 2populations, due to their relationship with the hospital setting. The items were determined by a group of experts and analysed using the Lawshe validity index to determine a first validity of content. Subsequently, the reliability of the tool was determined by an item-re-item analysis of the 2sub-populations. A factorial analysis was performed to analyse the construct validity with the maximum likelihood and rotation of varimax type factors. A questionnaire of paternal concern was offered, consisting of 21 items with a Cronbach coefficient of 0.97, giving good precision and stability. The posterior factor analysis gives an adequate validity to the questionnaire, with the determination of 10 common stressors that cover 74.08% of the common and non-common variance of the questionnaire. The proposed questionnaire is reliable, valid and easy-to-apply and is developed to assess the level of paternal concern about the surgery of a child and to be able to apply measures and programs through the prior assessment of these elements. Copyright © 2016 Asociación Española de Pediatría. Publicado por Elsevier España, S.L.U. All rights reserved.

Wechsler Adult Intelligence Scale-IV Dyads for Estimating Global Intelligence.

PubMed

Girard, Todd A; Axelrod, Bradley N; Patel, Ronak; Crawford, John R

2015-08-01

All possible two-subtest combinations of the core Wechsler Adult Intelligence Scale-IV (WAIS-IV) subtests were evaluated as possible viable short forms for estimating full-scale IQ (FSIQ). Validity of the dyads was evaluated relative to FSIQ in a large clinical sample (N = 482) referred for neuropsychological assessment. Sample validity measures included correlations, mean discrepancies, and levels of agreement between dyad estimates and FSIQ scores. In addition, reliability and validity coefficients were derived from WAIS-IV standardization data. The Coding + Information dyad had the strongest combination of reliability and validity data. However, several other dyads yielded comparable psychometric performance, albeit with some variability in their particular strengths. We also observed heterogeneity between validity coefficients from the clinical and standardization-based estimates for several dyads. Thus, readers are encouraged to also consider the individual psychometric attributes, their clinical or research goals, and client or sample characteristics when selecting among the dyadic short forms. © The Author(s) 2014.

Development of a Child Abuse Checklist to Evaluate Prehospital Provider Performance.

PubMed

Alphonso, Aimee; Auerbach, Marc; Bechtel, Kirsten; Bilodeau, Kyle; Gawel, Marcie; Koziel, Jeannette; Whitfill, Travis; Tiyyagura, Gunjan Kamdar

2017-01-01

To develop and provide validity evidence for a performance checklist to evaluate the child abuse screening behaviors of prehospital providers. Checklist Development: We developed the first iteration of the checklist after review of the relevant literature and on the basis of the authors' clinical experience. Next, a panel of six content experts participated in three rounds of Delphi review to reach consensus on the final checklist items. Checklist Validation: Twenty-eight emergency medical services (EMS) providers (16 EMT-Basics, 12 EMT-Paramedics) participated in a standardized simulated case of physical child abuse to an infant followed by one-on-one semi-structured qualitative interviews. Three reviewers scored the videotaped performance using the final checklist. Light's kappa and Cronbach's alpha were calculated to assess inter-rater reliability (IRR) and internal consistency, respectively. The correlation of successful child abuse screening with checklist task completion and with participant characteristics were compared using Pearson's chi squared test to gather evidence for construct validity. The Delphi review process resulted in a final checklist that included 24 items classified with trichotomous scoring (done, not done, or not applicable). The overall IRR of the three raters was 0.70 using Light's kappa, indicating substantial agreement. Internal consistency of the checklist was low, with an overall Cronbach's alpha of 0.61. Of 28 participants, only 14 (50%) successfully screened for child abuse in simulation. Participants who successfully screened for child abuse did not differ significantly from those who failed to screen in terms of training level, past experience with child abuse reporting, or self-reported confidence in detecting child abuse (all p > 0.30). Of all 24 tasks, only the task of exposing the infant significantly correlated with successful detection of child abuse (p < 0.05). We developed a child abuse checklist that demonstrated strong content validity and substantial inter-rater reliability, but successful item completion did not correlate with other markers of provider experience. The validated instrument has important potential for training, continuing education, and research for prehospital providers at all levels of training.

PROMIS GH (Patient-Reported Outcomes Measurement Information System Global Health) Scale in Stroke: A Validation Study.

PubMed

Katzan, Irene L; Lapin, Brittany

2018-01-01

The International Consortium for Health Outcomes Measurement recently included the 10-item PROMIS GH (Patient-Reported Outcomes Measurement Information System Global Health) scale as part of their recommended Standard Set of Stroke Outcome Measures. Before collection of PROMIS GH is broadly implemented, it is necessary to assess its performance in the stroke population. The objective of this study was to evaluate the psychometric properties of PROMIS GH in patients with ischemic stroke and intracerebral hemorrhage. PROMIS GH and 6 PROMIS domain scales measuring same/similar constructs were electronically collected on 1102 patients with ischemic and hemorrhagic strokes at various stages of recovery from their stroke who were seen in a cerebrovascular clinic from October 12, 2015, through June 2, 2017. Confirmatory factor analysis was performed to evaluate the adequacy of 2-factor structure of component scores. Test-retest reliability and convergent validity of PROMIS GH items and component scores were assessed. Discriminant validity and responsiveness were compared between PROMIS GH and PROMIS domain scales measuring the same or related constructs. Analyses were repeated stratified by stroke subtype and modified Rankin Scale score <2 versus ≥2. There was moderate internal reliability (ordinal α, 0.82-0.88) and marginal model fit for the 2-factor solution for component scores (root mean square error of approximation, 0.11). Convergent validity was good with significant correlations between all PROMIS GH items and PROMIS domain scales ( P <0.001 for all). There was excellent discrimination for all PROMIS GH items and component scores across modified Rankin Scale levels. Good responsiveness (effect size, >0.5) was demonstrated for 8 of the 10 PROMIS GH items. Reliability and validity remained consistent across stroke subtype and disability level (modified Rankin Scale, <2 versus ≥2). PROMIS GH exhibits acceptable performance in patients with stroke. Our findings support International Consortium for Health Outcomes Measurement recommendation to use PROMIS GH as part of the standard set of outcome measures in stroke. © 2017 American Heart Association, Inc.

Validity and reliability of a health care service evaluation instrument for tuberculosis

PubMed Central

Scatena, Lucia Marina; Wysocki, Anneliese Domingues; Beraldo, Aline Ale; Magnabosco, Gabriela Tavares; Brunello, Maria Eugênia Firmino; Netto, Antonio Ruffino; Nogueira, Jordana de Almeida; Silva, Reinaldo Antonio; Brito, Ewerton William Gomes; Alexandre, Patricia Borges Dias; Monroe, Aline Aparecida; Villa, Tereza Cristina Scatena

2015-01-01

OBJECTIVE To evaluate the validity and reliability of an instrument that evaluates the structure of primary health care units for the treatment of tuberculosis. METHODS This cross-sectional study used simple random sampling and evaluated 1,037 health care professionals from five Brazilian municipalities (Natal, state of Rio Grande do Norte; Cabedelo, state of Paraíba; Foz do Iguaçu, state of Parana; Sao José do Rio Preto, state of Sao Paulo, and Uberaba, state of Minas Gerais) in 2011. Structural indicators were identified and validated, considering different methods of organization of the health care system in the municipalities of different population sizes. Each structure represented the organization of health care services and contained the resources available for the execution of health care services: physical resources (equipment, consumables, and facilities); human resources (number and qualification); and resources for maintenance of the existing infrastructure and technology (deemed as the organization of health care services). The statistical analyses used in the validation process included reliability analysis, exploratory factor analysis, and confirmatory factor analysis. RESULTS The validation process indicated the retention of five factors, with 85.9% of the total variance explained, internal consistency between 0.6460 and 0.7802, and quality of fit of the confirmatory factor analysis of 0.995 using the goodness-of-fit index. The retained factors comprised five structural indicators: professionals involved in the care of tuberculosis patients, training, access to recording instruments, availability of supplies, and coordination of health care services with other levels of care. Availability of supplies had the best performance and the lowest coefficient of variation among the services evaluated. The indicators of assessment of human resources and coordination with other levels of care had satisfactory performance, but the latter showed the highest coefficient of variation. The performance of the indicators “training” and “access to recording instruments” was inferior to that of other indicators. CONCLUSIONS The instrument showed feasibility of application and potential to assess the structure of primary health care units for the treatment of tuberculosis. PMID:25741651

Objective assessment of gynecologic laparoscopic skills using the LapSimGyn virtual reality simulator.

PubMed

Larsen, C R; Grantcharov, T; Aggarwal, R; Tully, A; Sørensen, J L; Dalsgaard, T; Ottesen, B

2006-09-01

Safe realistic training and unbiased quantitative assessment of technical skills are required for laparoscopy. Virtual reality (VR) simulators may be useful tools for training and assessing basic and advanced surgical skills and procedures. This study aimed to investigate the construct validity of the LapSimGyn VR simulator, and to determine the learning curves of gynecologists with different levels of experience. For this study, 32 gynecologic trainees and consultants (juniors or seniors) were allocated into three groups: novices (0 advanced laparoscopic procedures), intermediate level (>20 and <60 procedures), and experts (>100 procedures). All performed 10 sets of simulations consisting of three basic skill tasks and an ectopic pregnancy program. The simulations were carried out on 3 days within a maximum period of 2 weeks. Assessment of skills was based on time, economy of movement, and error parameters measured by the simulator. The data showed that expert gynecologists performed significantly and consistently better than intermediate and novice gynecologists. The learning curves differed significantly between the groups, showing that experts start at a higher level and more rapidly reach the plateau of their learning curve than do intermediate and novice groups of surgeons. The LapSimGyn VR simulator package demonstrates construct validity on both the basic skills module and the procedural gynecologic module for ectopic pregnancy. Learning curves can be obtained, but to reach the maximum performance for the more complex tasks, 10 repetitions do not seem sufficient at the given task level and settings. LapSimGyn also seems to be flexible and widely accepted by the users.

Simultaneous determination of 117 pesticides and 30 mycotoxins in raw coffee, without clean-up, by LC-ESI-MS/MS analysis.

PubMed

Reichert, Bárbara; de Kok, André; Pizzutti, Ionara Regina; Scholten, Jos; Cardoso, Carmem Dickow; Spanjer, Martien

2018-04-03

This paper describes the optimization and validation of an acetonitrile based method for simultaneous extraction of multiple pesticides and mycotoxins from raw coffee beans followed by LC-ESI-MS/MS determination. Before extraction, the raw coffee samples were milled and then slurried with water. The slurried samples were spiked with two separate standard solutions, one containing 131 pesticides and a second with 35 mycotoxins, which were divided into 3 groups of different relative concentration levels. Optimization of the QuEChERS approach included performance tests with acetonitrile acidified with acetic acid or formic acid, with or without buffer and with or without clean-up of the extracts before LC-ESI-MS/MS analysis. For the clean-up step, seven d-SPE sorbents and their various mixtures were evaluated. After method optimization a complete validation study was carried out to ensure adequate performance of the extraction and chromatographic methods. The samples were spiked at 3 concentrations levels with both mycotoxins and pesticides (with 6 replicates at each level, n = 6) and then submitted to the extraction procedure. Before LC-ESI-MS/MS analysis, the acetonitrile extracts were diluted 2-fold with methanol, in order to improve the chromatographic performance of the early-eluting polar analytes. Calibration standard solutions were prepared in organic solvent and in blank coffee extract at 7 concentration levels and analyzed 6 times each. The method was assessed for accuracy (recovery %), precision (RSD%), selectivity, linearity (r 2 ), limit of quantification (LOQ) and matrix effects (%). Copyright © 2017 Elsevier B.V. All rights reserved.

Detecting ecosystem performance anomalies for land management in the upper colorado river basin using satellite observations, climate data, and ecosystem models

USGS Publications Warehouse

Gu, Yingxin; Wylie, B.K.

2010-01-01

This study identifies areas with ecosystem performance anomalies (EPA) within the Upper Colorado River Basin (UCRB) during 2005-2007 using satellite observations, climate data, and ecosystem models. The final EPA maps with 250-m spatial resolution were categorized as normal performance, underperformance, and overperformance (observed performance relative to weather-based predictions) at the 90% level of confidence. The EPA maps were validated using "percentage of bare soil" ground observations. The validation results at locations with comparable site potential showed that regions identified as persistently underperforming (overperforming) tended to have a higher (lower) percentage of bare soil, suggesting that our preliminary EPA maps are reliable and agree with ground-based observations. The 3-year (2005-2007) persistent EPA map from this study provides the first quantitative evaluation of ecosystem performance anomalies within the UCRB and will help the Bureau of Land Management (BLM) identify potentially degraded lands. Results from this study can be used as a prototype by BLM and other land managers for making optimal land management decisions. ?? 2010 by the authors.

Detecting Ecosystem Performance Anomalies for Land Management in the Upper Colorado River Basin Using Satellite Observations, Climate Data, and Ecosystem Models

USGS Publications Warehouse

Gu, Yingxin; Wylie, Bruce K.

2010-01-01

This study identifies areas with ecosystem performance anomalies (EPA) within the Upper Colorado River Basin (UCRB) during 2005–2007 using satellite observations, climate data, and ecosystem models. The final EPA maps with 250-m spatial resolution were categorized as normal performance, underperformance, and overperformance (observed performance relative to weather-based predictions) at the 90% level of confidence. The EPA maps were validated using “percentage of bare soil” ground observations. The validation results at locations with comparable site potential showed that regions identified as persistently underperforming (overperforming) tended to have a higher (lower) percentage of bare soil, suggesting that our preliminary EPA maps are reliable and agree with ground-based observations. The 3-year (2005–2007) persistent EPA map from this study provides the first quantitative evaluation of ecosystem performance anomalies within the UCRB and will help the Bureau of Land Management (BLM) identify potentially degraded lands. Results from this study can be used as a prototype by BLM and other land managers for making optimal land management decisions.

Using DEMATEL approach to develop relationships of performance indicators on sustainable service only supply chain performance measurement

NASA Astrophysics Data System (ADS)

Leksono, EB; Suparno; Vanany, I.

2018-04-01

Service only supply chain (SOSC) concept is service supply chain (SSC) implementation on pure services. The globalization and stakeholder pressure makes operation of SSC should give the attention to the environment effect, community, economic and intangibility assets. SOSC performance measurement (SOSCPM) may be developed for measuring of performance for sustainability aspects and intangibility assets to meet customer satisfaction. This article discusses sustainable SOSCPM based on balanced scorecard (BSC), include sustainability aspects, intangibility and relations between perspectives and indicators. From literature review, it is found 34 performance indicators that must be confirm to expert and SC actors by survey. From survey validation using weighted average and level of consensus, it is found 29 valid indicators for processed by DEMATEL. From DEMATEL, it is found 26 indicators can be used on sustainable SOSCPM. Furthermore, innovation and growth perspective most influence to other, and customer perspective most important. Intangibility indicators incorporated on innovation and growth perspective very related with human resources. Finally, relations between perspectives and indicator used to design of BSC strategy maps.

Validation of a clinical critical thinking skills test in nursing.

PubMed

Shin, Sujin; Jung, Dukyoo; Kim, Sungeun

2015-01-27

The purpose of this study was to develop a revised version of the clinical critical thinking skills test (CCTS) and to subsequently validate its performance. This study is a secondary analysis of the CCTS. Data were obtained from a convenience sample of 284 college students in June 2011. Thirty items were analyzed using item response theory and test reliability was assessed. Test-retest reliability was measured using the results of 20 nursing college and graduate school students in July 2013. The content validity of the revised items was analyzed by calculating the degree of agreement between instrument developer intention in item development and the judgments of six experts. To analyze response process validity, qualitative data related to the response processes of nine nursing college students obtained through cognitive interviews were analyzed. Out of initial 30 items, 11 items were excluded after the analysis of difficulty and discrimination parameter. When the 19 items of the revised version of the CCTS were analyzed, levels of item difficulty were found to be relatively low and levels of discrimination were found to be appropriate or high. The degree of agreement between item developer intention and expert judgments equaled or exceeded 50%. From above results, evidence of the response process validity was demonstrated, indicating that subjects respondeds as intended by the test developer. The revised 19-item CCTS was found to have sufficient reliability and validity and will therefore represents a more convenient measurement of critical thinking ability.

Validation of a clinical critical thinking skills test in nursing

PubMed Central

2015-01-01

Purpose: The purpose of this study was to develop a revised version of the clinical critical thinking skills test (CCTS) and to subsequently validate its performance. Methods: This study is a secondary analysis of the CCTS. Data were obtained from a convenience sample of 284 college students in June 2011. Thirty items were analyzed using item response theory and test reliability was assessed. Test-retest reliability was measured using the results of 20 nursing college and graduate school students in July 2013. The content validity of the revised items was analyzed by calculating the degree of agreement between instrument developer intention in item development and the judgments of six experts. To analyze response process validity, qualitative data related to the response processes of nine nursing college students obtained through cognitive interviews were analyzed. Results: Out of initial 30 items, 11 items were excluded after the analysis of difficulty and discrimination parameter. When the 19 items of the revised version of the CCTS were analyzed, levels of item difficulty were found to be relatively low and levels of discrimination were found to be appropriate or high. The degree of agreement between item developer intention and expert judgments equaled or exceeded 50%. Conclusion: From above results, evidence of the response process validity was demonstrated, indicating that subjects respondeds as intended by the test developer. The revised 19-item CCTS was found to have sufficient reliability and validity and will therefore represents a more convenient measurement of critical thinking ability. PMID:25622716

[Spanish version of the Cancer Worry Scale (CWS). Cross cultural adaptation and validity and reliability analysis].

PubMed

Cabrera, Esther; Zabalegui, Adelaida; Blanco, Ignacio

2011-01-15

The worry for falling ill has been described as a key element in the change of preventive attitudes. Levels of cancer worry not well fitted have been associated with inadequate adherence to preventive strategies. There is not a Spanish validated scale to evaluate the degree of worry for the cancer in our population. The aim of the present study was to perform the cross cultural adaptation and validation of the Cancer Worry Scale described by Lerman. A translation, re-translation of the Cancer Worry Scale to Spanish was done. Validation of the Spanish scale was performed by means of the factorial analysis of principal components with the rotation varimax test in a sample of 200 healthy women with family history of breast cancer. The Escala de Preocupación por el Cáncer (EPC) is the Spanish version of the Cancer Worry Scale and it contains 6 items with a total value ranging from 6 (minimal worry) to 24 (maximum worry). The analysis of content validity demonstrated that the EPC is conceptually equivalent to the original scale. The factorial analysis showed a unique factor that explains 53.07% of the variance confirming the unique dimension. The EPC presented good reliability test - re-test with an Intraclass Correlation Coefficient of 0.777. The Cronbach's alpha was 0.835 for the complete of the scale. The EPC is a validated Spanish scale to measure the cancer worry in healthy individuals, which shows a correct content validity and reliability. Copyright © 2010 Elsevier España, S.L. All rights reserved.

Use of a tibial accelerometer to measure ground reaction force in running: A reliability and validity comparison with force plates.

PubMed

Raper, Damian P; Witchalls, Jeremy; Philips, Elissa J; Knight, Emma; Drew, Michael K; Waddington, Gordon

2018-01-01

The use of microsensor technologies to conduct research and implement interventions in sports and exercise medicine has increased recently. The objective of this paper was to determine the validity and reliability of the ViPerform as a measure of load compared to vertical ground reaction force (GRF) as measured by force plates. Absolute reliability assessment, with concurrent validity. 10 professional triathletes ran 10 trials over force plates with the ViPerform mounted on the mid portion of the medial tibia. Calculated vertical ground reaction force data from the ViPerform was matched to the same stride on the force plate. Bland-Altman (BA) plot of comparative measure of agreement was used to assess the relationship between the calculated load from the accelerometer and the force plates. Reliability was calculated by intra-class correlation coefficients (ICC) with 95% confidence intervals. BA plot indicates minimal agreement between the measures derived from the force plate and ViPerform, with variation at an individual participant plot level. Reliability was excellent (ICC=0.877; 95% CI=0.825-0.917) in calculating the same vertical GRF in a repeated trial. Standard error of measure (SEM) equalled 99.83 units (95% CI=82.10-119.09), which, in turn, gave a minimum detectable change (MDC) value of 276.72 units (95% CI=227.32-330.07). The ViPerform does not calculate absolute values of vertical GRF similar to those measured by a force plate. It does provide a valid and reliable calculation of an athlete's lower limb load at constant velocity. Copyright © 2017 Sports Medicine Australia. Published by Elsevier Ltd. All rights reserved.

Analysis of internal and external validity criteria for a computerized visual search task: A pilot study.

PubMed

Richard's, María M; Introzzi, Isabel; Zamora, Eliana; Vernucci, Santiago

2017-01-01

Inhibition is one of the main executive functions, because of its fundamental role in cognitive and social development. Given the importance of reliable and computerized measurements to assessment inhibitory performance, this research intends to analyze the internal and external criteria of validity of a computerized conjunction search task, to evaluate the role of perceptual inhibition. A sample of 41 children (21 females and 20 males), aged between 6 and 11 years old (M = 8.49, SD = 1.47), intentionally selected from a private management school of Mar del Plata (Argentina), middle socio-economic level were assessed. The Conjunction Search Task from the TAC Battery, Coding and Symbol Search tasks from Wechsler Intelligence Scale for Children were used. Overall, results allow us to confirm that the perceptual inhibition task form TAC presents solid rates of internal and external validity that make a valid measurement instrument of this process.

Validation of Modelled Ice Dynamics of the Greenland Ice Sheet using Historical Forcing

NASA Astrophysics Data System (ADS)

Hoffman, M. J.; Price, S. F.; Howat, I. M.; Bonin, J. A.; Chambers, D. P.; Tezaur, I.; Kennedy, J. H.; Lenaerts, J.; Lipscomb, W. H.; Neumann, T.; Nowicki, S.; Perego, M.; Saba, J. L.; Salinger, A.; Guerber, J. R.

2015-12-01

Although ice sheet models are used for sea level rise projections, the degree to which these models have been validated by observations is fairly limited, due in part to the limited duration of the satellite observation era and the long adjustment time scales of ice sheets. Here we describe a validation framework for the Greenland Ice Sheet applied to the Community Ice Sheet Model by forcing the model annually with flux anomalies at the major outlet glaciers (Enderlin et al., 2014, observed from Landsat/ASTER/Operation IceBridge) and surface mass balance (van Angelen et al., 2013, calculated from RACMO2) for the period 1991-2012. The ice sheet model output is compared to ice surface elevation observations from ICESat and ice sheet mass change observations from GRACE. Early results show promise for assessing the performance of different model configurations. Additionally, we explore the effect of ice sheet model resolution on validation skill.

Validation of a Monte Carlo simulation of the Philips Allegro/GEMINI PET systems using GATE

NASA Astrophysics Data System (ADS)

Lamare, F.; Turzo, A.; Bizais, Y.; Cheze LeRest, C.; Visvikis, D.

2006-02-01

A newly developed simulation toolkit, GATE (Geant4 Application for Tomographic Emission), was used to develop a Monte Carlo simulation of a fully three-dimensional (3D) clinical PET scanner. The Philips Allegro/GEMINI PET systems were simulated in order to (a) allow a detailed study of the parameters affecting the system's performance under various imaging conditions, (b) study the optimization and quantitative accuracy of emission acquisition protocols for dynamic and static imaging, and (c) further validate the potential of GATE for the simulation of clinical PET systems. A model of the detection system and its geometry was developed. The accuracy of the developed detection model was tested through the comparison of simulated and measured results obtained with the Allegro/GEMINI systems for a number of NEMA NU2-2001 performance protocols including spatial resolution, sensitivity and scatter fraction. In addition, an approximate model of the system's dead time at the level of detected single events and coincidences was developed in an attempt to simulate the count rate related performance characteristics of the scanner. The developed dead-time model was assessed under different imaging conditions using the count rate loss and noise equivalent count rates performance protocols of standard and modified NEMA NU2-2001 (whole body imaging conditions) and NEMA NU2-1994 (brain imaging conditions) comparing simulated with experimental measurements obtained with the Allegro/GEMINI PET systems. Finally, a reconstructed image quality protocol was used to assess the overall performance of the developed model. An agreement of <3% was obtained in scatter fraction, with a difference between 4% and 10% in the true and random coincidence count rates respectively, throughout a range of activity concentrations and under various imaging conditions, resulting in <8% differences between simulated and measured noise equivalent count rates performance. Finally, the image quality validation study revealed a good agreement in signal-to-noise ratio and contrast recovery coefficients for a number of different volume spheres and two different (clinical level based) tumour-to-background ratios. In conclusion, these results support the accurate modelling of the Philips Allegro/GEMINI PET systems using GATE in combination with a dead-time model for the signal flow description, which leads to an agreement of <10% in coincidence count rates under different imaging conditions and clinically relevant activity concentration levels.

Estimating energy expenditure from heart rate in older adults: a case for calibration.

PubMed

Schrack, Jennifer A; Zipunnikov, Vadim; Goldsmith, Jeff; Bandeen-Roche, Karen; Crainiceanu, Ciprian M; Ferrucci, Luigi

2014-01-01

Accurate measurement of free-living energy expenditure is vital to understanding changes in energy metabolism with aging. The efficacy of heart rate as a surrogate for energy expenditure is rooted in the assumption of a linear function between heart rate and energy expenditure, but its validity and reliability in older adults remains unclear. To assess the validity and reliability of the linear function between heart rate and energy expenditure in older adults using different levels of calibration. Heart rate and energy expenditure were assessed across five levels of exertion in 290 adults participating in the Baltimore Longitudinal Study of Aging. Correlation and random effects regression analyses assessed the linearity of the relationship between heart rate and energy expenditure and cross-validation models assessed predictive performance. Heart rate and energy expenditure were highly correlated (r=0.98) and linear regardless of age or sex. Intra-person variability was low but inter-person variability was high, with substantial heterogeneity of the random intercept (s.d. =0.372) despite similar slopes. Cross-validation models indicated individual calibration data substantially improves accuracy predictions of energy expenditure from heart rate, reducing the potential for considerable measurement bias. Although using five calibration measures provided the greatest reduction in the standard deviation of prediction errors (1.08 kcals/min), substantial improvement was also noted with two (0.75 kcals/min). These findings indicate standard regression equations may be used to make population-level inferences when estimating energy expenditure from heart rate in older adults but caution should be exercised when making inferences at the individual level without proper calibration.

Reliability and validity of an accele-rometric system for assessing vertical jumping performance.

PubMed

Choukou, M-A; Laffaye, G; Taiar, R

2014-03-01

The validity of an accelerometric system (Myotest©) for assessing vertical jump height, vertical force and power, leg stiffness and reactivity index was examined. 20 healthy males performed 3×"5 hops in place", 3×"1 squat jump" and 3× "1 countermovement jump" during 2 test-retest sessions. The variables were simultaneously assessed using an accelerometer and a force platform at a frequency of 0.5 and 1 kHz, respectively. Both reliability and validity of the accelerometric system were studied. No significant differences between test and retest data were found (p < 0.05), showing a high level of reliability. Besides, moderate to high intraclass correlation coefficients (ICCs) (from 0.74 to 0.96) were obtained for all variables whereas weak to moderate ICCs (from 0.29 to 0.79) were obtained for force and power during the countermovement jump. With regards to validity, the difference between the two devices was not significant for 5 hops in place height (1.8 cm), force during squat (-1.4 N · kg(-1)) and countermovement (0.1 N · kg(-1)) jumps, leg stiffness (7.8 kN · m(-1)) and reactivity index (0.4). So, the measurements of these variables with this accelerometer are valid, which is not the case for the other variables. The main causes of non-validity for velocity, power and contact time assessment are temporal biases of the takeoff and touchdown moments detection.

RELIABILITY AND VALIDITY OF AN ACCELEROMETRIC SYSTEM FOR ASSESSING VERTICAL JUMPING PERFORMANCE

PubMed Central

Laffaye, G.; Taiar, R.

2014-01-01

The validity of an accelerometric system (Myotest©) for assessing vertical jump height, vertical force and power, leg stiffness and reactivity index was examined. 20 healthy males performed 3ד5 hops in place”, 3ד1 squat jump” and 3× “1 countermovement jump” during 2 test-retest sessions. The variables were simultaneously assessed using an accelerometer and a force platform at a frequency of 0.5 and 1 kHz, respectively. Both reliability and validity of the accelerometric system were studied. No significant differences between test and retest data were found (p < 0.05), showing a high level of reliability. Besides, moderate to high intraclass correlation coefficients (ICCs) (from 0.74 to 0.96) were obtained for all variables whereas weak to moderate ICCs (from 0.29 to 0.79) were obtained for force and power during the countermovement jump. With regards to validity, the difference between the two devices was not significant for 5 hops in place height (1.8 cm), force during squat (-1.4 N · kg−1) and countermovement (0.1 N · kg−1) jumps, leg stiffness (7.8 kN · m−1) and reactivity index (0.4). So, the measurements of these variables with this accelerometer are valid, which is not the case for the other variables. The main causes of non-validity for velocity, power and contact time assessment are temporal biases of the takeoff and touchdown moments detection. PMID:24917690

Validation of a Novel Laparoscopic Adjustable Gastric Band Simulator

PubMed Central

Sankaranarayanan, Ganesh; Adair, James D.; Halic, Tansel; Gromski, Mark A.; Lu, Zhonghua; Ahn, Woojin; Jones, Daniel B.; De, Suvranu

2011-01-01

Background Morbid obesity accounts for more than 90,000 deaths per year in the United States. Laparoscopic adjustable gastric banding (LAGB) is the second most common weight loss procedure performed in the US and the most common in Europe and Australia. Simulation in surgical training is a rapidly advancing field that has been adopted by many to prepare surgeons for surgical techniques and procedures. Study Aim The aim of our study was to determine face, construct and content validity for a novel virtual reality laparoscopic adjustable gastric band simulator. Methods Twenty-eight subjects were categorized into two groups (Expert and Novice), determined by their skill level in laparoscopic surgery. Experts consisted of subjects who had at least four years of laparoscopic training and operative experience. Novices consisted of subjects with medical training, but with less than four years of laparoscopic training. The subjects performed the virtual reality laparoscopic adjustable band surgery simulator. They were automatically scored, according to various tasks. The subjects then completed a questionnaire to evaluate face and content validity. Results On a 5-point Likert scale (1 – lowest score, 5 – highest score), the mean score for visual realism was 4.00 ± 0.67 and the mean score for realism of the interface and tool movements was 4.07 ± 0.77 [Face Validity]. There were significant differences in the performance of the two subject groups (Expert and Novice), based on total scores (p<0.001) [Construct Validity]. Mean scores for utility of the simulator, as addressed by the Expert group, was 4.50 ± 0.71 [Content Validity]. Conclusion We created a virtual reality laparoscopic adjustable gastric band simulator. Our initial results demonstrate excellent face, construct and content validity findings. To our knowledge, this is the first virtual reality simulator with haptic feedback for training residents and surgeons in the laparoscopic adjustable gastric banding procedure. PMID:20734069

Predictive validity of the post-enrolment English language assessment tool for commencing undergraduate nursing students.

PubMed

Glew, Paul J; Hillege, Sharon P; Salamonson, Yenna; Dixon, Kathleen; Good, Anthony; Lombardo, Lien

2015-12-01

Nursing students with English as an additional language (EAL) may underperform academically. The post-enrolment English language assessment (PELA) is used in literacy support, but its predictive validity in identifying those at risk of underperformance remains unknown. To validate a PELA, as a predictor of academic performance. Prospective survey design. The study was conducted at a university located in culturally and linguistically diverse areas of western Sydney, Australia. Commencing undergraduate nursing students who were Australian-born (n=1323, 49.6%) and born outside of Australia (n=1346, 50.4%) were recruited for this study. The 2669 (67% of 3957) participants provided consent and completed a first year nursing unit that focussed on developing literacy skills. Between 2010 and 2013, commencing students completed the PELA and English language acculturation scale (ELAS), a previously validated instrument. The grading levels of the PELA tool were: Level 1 (proficient), Level 2 (borderline), and Level 3 (poor, and requiring additional support). Participants with a PELA Level 2 or 3 were more likely to be: a) non-Australian-born (χ(2): 520.6, df: 2, p<0.001); b) spoke a language other than English at home (χ(2): 490.2, df: 2, p<0.001); and c) an international student (χ(2): 225.6, df: 2, p<0.001). There was an inverse relationship between participants' ELAS scores and PELA levels (r=-0.52, p<0.001), and those graded as 'proficient' with a PELA Level 1 were more likely to obtain higher scores in their: i) unit essay assessment (χ(2): 40.2, df: 2, p<0.001); ii) final unit mark (χ(2): 218.6, df: 2, p<0.001), and attain a higher GPA (χ(2): 100.8, df: 2, p<0.001). The PELA is a useful screening tool in identifying commencing nursing students who are at risk of academic underachievement. Crown Copyright © 2015. Published by Elsevier Ltd. All rights reserved.

The Copernicus S5P Mission Performance Centre / Validation Data Analysis Facility for TROPOMI operational atmospheric data products

NASA Astrophysics Data System (ADS)

Compernolle, Steven; Lambert, Jean-Christopher; Langerock, Bavo; Granville, José; Hubert, Daan; Keppens, Arno; Rasson, Olivier; De Mazière, Martine; Fjæraa, Ann Mari; Niemeijer, Sander

2017-04-01

Sentinel-5 Precursor (S5P), to be launched in 2017 as the first atmospheric composition satellite of the Copernicus programme, carries as payload the TROPOspheric Monitoring Instrument (TROPOMI) developed by The Netherlands in close cooperation with ESA. Designed to measure Earth radiance and solar irradiance in the ultraviolet, visible and near infrared, TROPOMI will provide Copernicus with observational data on atmospheric composition at unprecedented geographical resolution. The S5P Mission Performance Center (MPC) provides an operational service-based solution for various QA/QC tasks, including the validation of S5P Level-2 data products and the support to algorithm evolution. Those two tasks are to be accomplished by the MPC Validation Data Analysis Facility (VDAF), one MPC component developed and operated at BIRA-IASB with support from S[&]T and NILU. The routine validation to be ensured by VDAF is complemented by a list of validation AO projects carried out by ESA's S5P Validation Team (S5PVT), with whom interaction is essential. Here we will introduce the general architecture of VDAF, its relation to the other MPC components, the generic and specific validation strategies applied for each of the official TROPOMI data products, and the expected output of the system. The S5P data products to be validated by VDAF are diverse: O3 (vertical profile, total column, tropospheric column), NO2 (total and tropospheric column), HCHO (tropospheric column), SO2 (column), CO (column), CH4 (column), aerosol layer height and clouds (fractional cover, cloud-top pressure and optical thickness). Starting from a generic validation protocol meeting community-agreed standards, a set of specific validation settings is associated with each data product, as well as the appropriate set of Fiducial Reference Measurements (FRM) to which it will be compared. VDAF collects FRMs from ESA's Validation Data Centre (EVDC) and from other sources (e.g., WMO's GAW, NDACC and TCCON). Data manipulations on satellite and FRM data (format conversion, filtering, co-location, regridding and vertical smoothing) are performed by the open source software HARP, while more specific manipulations apply in-house routines. The paper concludes with a short description of expected outputs of the system.

Memory Alteration Test to Detect Amnestic Mild Cognitive Impairment and Early Alzheimer’s Dementia in Population with Low Educational Level

PubMed Central

Custodio, Nilton; Lira, David; Herrera-Perez, Eder; Montesinos, Rosa; Castro-Suarez, Sheila; Cuenca-Alfaro, José; Valeriano-Lorenzo, Lucía

2017-01-01

Background/Aims: Short tests to early detection of the cognitive impairment are necessary in primary care setting, particularly in populations with low educational level. The aim of this study was to assess the performance of Memory Alteration Test (M@T) to discriminate controls, patients with amnestic Mild Cognitive Impairment (aMCI) and patients with early Alzheimer’s Dementia (AD) in a sample of individuals with low level of education. Methods: Cross-sectional study to assess the performance of the M@T (study test), compared to the neuropsychological evaluation (gold standard test) scores in 247 elderly subjects with low education level from Lima-Peru. The cognitive evaluation included three sequential stages: (1) screening (to detect cases with cognitive impairment); (2) nosological diagnosis (to determinate specific disease); and (3) classification (to differentiate disease subtypes). The subjects with negative results for all stages were considered as cognitively normal (controls). The test performance was assessed by means of area under the receiver operating characteristic (ROC) curve. We calculated validity measures (sensitivity, specificity and correctly classified percentage), the internal consistency (Cronbach’s alpha coefficient), and concurrent validity (Pearson’s ratio coefficient between the M@T and Clinical Dementia Rating (CDR) scores). Results: The Cronbach’s alpha coefficient was 0.79 and Pearson’s ratio coefficient was 0.79 (p < 0.01). The AUC of M@T to discriminate between early AD and aMCI was 99.60% (sensitivity = 100.00%, specificity = 97.53% and correctly classified = 98.41%) and to discriminate between aMCI and controls was 99.56% (sensitivity = 99.17%, specificity = 91.11%, and correctly classified = 96.99%). Conclusions: The M@T is a short test with a good performance to discriminate controls, aMCI and early AD in individuals with low level of education from urban settings. PMID:28878665

An Exploration of Academic Reading Proficiency at the University Level: A Cross-Sectional Study of 848 Undergraduates

ERIC Educational Resources Information Center

Gorzycki, Meg; Howard, Pamela; Allen, Diane; Desa, Geoffrey; Rosegard, Erik

2016-01-01

Academic reading proficiently is characterized by the ability to perform cognitive tasks associated with interpreting text. Researchers developed an externally validated Informal Academic Reading Proficiency Test to gauge undergraduates' academic reading proficiency. A cross-sectional study of 23 classes completed the reading test in 2014. This…

Assessment in Immersive Virtual Environments: Cases for Learning, of Learning, and as Learning

ERIC Educational Resources Information Center

Code, Jillianne; Zap, Nick

2017-01-01

The key to education reform lies in exploring alternative forms of assessment. Alternative performance assessments provide a more valid measure than multiple-choice tests of students' conceptual understanding and higher-level skills such as problem solving and inquiry. Advances in game-based and virtual environment technologies are creating new…

Spanish Readability Formulas for Elementary-Level Texts: A Validation Study.

ERIC Educational Resources Information Center

Parker, Richard I.; Hasbrouck, Jan E.; Weaver, Laurie

2001-01-01

Uses two formulas developed for Spanish language text to analyze 9 stories that were read by 36 Spanish-speaking second graders with limited English proficiency. Finds that the Spanish readability formulas only weakly predicted student performance, indicating the need to pursue broader, qualitative indices of difficulty for Spanish text. (SG)

Hybrid and conventional hydrogen engine vehicles that meet EZEV emissions

DOE Office of Scientific and Technical Information (OSTI.GOV)

Aceves, S.M.; Smith, J.R.

In this paper, a time-dependent engine model is used for predicting hydrogen engine efficiency and emissions. The model uses basic thermodynamic equations for the compression and expansion processes, along with an empirical correlation for heat transfer, to predict engine indicated efficiency. A friction correlation and a supercharger/turbocharger model are then used to calculate brake thermal efficiency. The model is validated with many experimental points obtained in a recent evaluation of a hydrogen research engine. A The validated engine model is then used to calculate fuel economy and emissions for three hydrogen-fueled vehicles: a conventional, a parallel hybrid, and a seriesmore » hybrid. All vehicles use liquid hydrogen as a fuel. The hybrid vehicles use a flywheel for energy storage. Comparable ultra capacitor or battery energy storage performance would give similar results. This paper analyzes the engine and flywheel sizing requirements for obtaining a desired level of performance. The results indicate that hydrogen lean-burn spark-ignited engines can provide a high fuel economy and Equivalent Zero Emission Vehicle (EZEV) levels in the three vehicle configurations being analyzed.« less

Time pressure and attention allocation effect on upper limb motion steadiness.

PubMed

Liu, Sicong; Eklund, Robert C; Tenenbaum, Gershon

2015-01-01

Following ironic process theory (IPT), the authors aimed at investigating how attentional allocation affects participants' upper limb motion steadiness under low and high levels of mental load. A secondary purpose was to examine the validity of skin conductance level in measuring perception of pressure. The study consisted of 1 within-participant factor (i.e., phase: baseline, test) and 4 between-participant factors (i.e., gender: male, female; mental load: fake time constraints, no time constraints; attention: positive, suppressive; order: baseline → → → test, test → → baseline). Eighty college students (40 men and 40 women, Mage = 20.20 years, SD(age) = 1.52 years) participated in the study. Gender-stratified random assignment was employed in a 2 × 2 × 2 × 2 × 2 mixed experimental design. The findings generally support IPT but its predictions on motor performance under mental load may not be entirely accurate. Unlike men, women's performance was not susceptible to manipulations of mental load and attention allocation. The validity of skin conductance readings as an index of pressure perception was called into question.

PyMercury: Interactive Python for the Mercury Monte Carlo Particle Transport Code

DOE Office of Scientific and Technical Information (OSTI.GOV)

Iandola, F N; O'Brien, M J; Procassini, R J

2010-11-29

Monte Carlo particle transport applications are often written in low-level languages (C/C++) for optimal performance on clusters and supercomputers. However, this development approach often sacrifices straightforward usability and testing in the interest of fast application performance. To improve usability, some high-performance computing applications employ mixed-language programming with high-level and low-level languages. In this study, we consider the benefits of incorporating an interactive Python interface into a Monte Carlo application. With PyMercury, a new Python extension to the Mercury general-purpose Monte Carlo particle transport code, we improve application usability without diminishing performance. In two case studies, we illustrate how PyMercury improvesmore » usability and simplifies testing and validation in a Monte Carlo application. In short, PyMercury demonstrates the value of interactive Python for Monte Carlo particle transport applications. In the future, we expect interactive Python to play an increasingly significant role in Monte Carlo usage and testing.« less

Handling Qualities of Model Reference Adaptive Controllers with Varying Complexity for Pitch-Roll Coupled Failures

NASA Technical Reports Server (NTRS)

Schaefer, Jacob; Hanson, Curt; Johnson, Marcus A.; Nguyen, Nhan

2011-01-01

Three model reference adaptive controllers (MRAC) with varying levels of complexity were evaluated on a high performance jet aircraft and compared along with a baseline nonlinear dynamic inversion controller. The handling qualities and performance of the controllers were examined during failure conditions that induce coupling between the pitch and roll axes. Results from flight tests showed with a roll to pitch input coupling failure, the handling qualities went from Level 2 with the baseline controller to Level 1 with the most complex MRAC tested. A failure scenario with the left stabilator frozen also showed improvement with the MRAC. Improvement in performance and handling qualities was generally seen as complexity was incrementally added; however, added complexity usually corresponds to increased verification and validation effort required for certification. The tradeoff between complexity and performance is thus important to a controls system designer when implementing an adaptive controller on an aircraft. This paper investigates this relation through flight testing of several controllers of vary complexity.

Balanced scorecard-based performance evaluation of Chinese county hospitals in underdeveloped areas.

PubMed

Gao, Hongda; Chen, He; Feng, Jun; Qin, Xianjing; Wang, Xuan; Liang, Shenglin; Zhao, Jinmin; Feng, Qiming

2018-05-01

Objective Since the Guangxi government implemented public county hospital reform in 2009, there have been no studies of county hospitals in this underdeveloped area of China. This study aimed to establish an evaluation indicator system for Guangxi county hospitals and to generate recommendations for hospital development and policymaking. Methods A performance evaluation indicator system was developed based on balanced scorecard theory. Opinions were elicited from 25 experts from administrative units, universities and hospitals and the Delphi method was used to modify the performance indicators. The indicator system and the Topsis method were used to evaluate the performance of five county hospitals randomly selected from the same batch of 2015 Guangxi reform pilots. Results There were 4 first-level indicators, 9 second-level indicators and 36 third-level indicators in the final performance evaluation indicator system that showed good consistency, validity and reliability. The performance rank of the hospitals was B > E > A > C > D. Conclusions The performance evaluation indicator system established using the balanced scorecard is practical and scientific. Analysis of the results based on this indicator system identified several factors affecting hospital performance, such as resource utilisation efficiency, medical service price, personnel structure and doctor-patient relationships.

Validating a High Performance Liquid Chromatography-Ion Chromatography (HPLC-IC) Method with Conductivity Detection After Chemical Suppression for Water Fluoride Estimation.

PubMed

Bondu, Joseph Dian; Selvakumar, R; Fleming, Jude Joseph

2018-01-01

A variety of methods, including the Ion Selective Electrode (ISE), have been used for estimation of fluoride levels in drinking water. But as these methods suffer many drawbacks, the newer method of IC has replaced many of these methods. The study aimed at (1) validating IC for estimation of fluoride levels in drinking water and (2) to assess drinking water fluoride levels of villages in and around Vellore district using IC. Forty nine paired drinking water samples were measured using ISE and IC method (Metrohm). Water samples from 165 randomly selected villages in and around Vellore district were collected for fluoride estimation over 1 year. Standardization of IC method showed good within run precision, linearity and coefficient of variance with correlation coefficient R 2  = 0.998. The limit of detection was 0.027 ppm and limit of quantification was 0.083 ppm. Among 165 villages, 46.1% of the villages recorded water fluoride levels >1.00 ppm from which 19.4% had levels ranging from 1 to 1.5 ppm, 10.9% had recorded levels 1.5-2 ppm and about 12.7% had levels of 2.0-3.0 ppm. Three percent of villages had more than 3.0 ppm fluoride in the water tested. Most (44.42%) of these villages belonged to Jolarpet taluk with moderate to high (0.86-3.56 ppm) water fluoride levels. Ion Chromatography method has been validated and is therefore a reliable method in assessment of fluoride levels in the drinking water. While the residents of Jolarpet taluk (Vellore distict) are found to be at a high risk of developing dental and skeletal fluorosis.

Examining the validity of the Homework Performance Questionnaire: Multi-informant assessment in elementary and middle school.

PubMed

Power, Thomas J; Watkins, Marley W; Mautone, Jennifer A; Walcott, Christy M; Coutts, Michael J; Sheridan, Susan M

2015-06-01

Methods for measuring homework performance have been limited primarily to parent reports of homework deficits. The Homework Performance Questionnaire (HPQ) was developed to assess the homework functioning of students in Grades 1 to 8 from the perspective of both teachers and parents. The purpose of this study was to examine the factorial validity of teacher and parent versions of this scale, and to evaluate gender and grade-level differences in factor scores. The HPQ was administered in 4 states from varying regions of the United States. The validation sample consisted of students (n = 511) for whom both parent and teacher ratings were obtained (52% female, mean of 9.5 years of age, 79% non-Hispanic, and 78% White). The cross-validation sample included 1,450 parent ratings and 166 teacher ratings with similar demographic characteristics. The results of confirmatory factor analyses demonstrated that the best-fitting model for teachers was a bifactor solution including a general factor and 2 orthogonal factors, referring to student self-regulation and competence. The best-fitting model for parents was also a bifactor solution, including a general factor and 3 orthogonal factors, referring to student self-regulation, student competence, and teacher support of homework. Gender differences were identified for the general and self-regulation factors of both versions. Overall, the findings provide strong support for the HPQ as a multi-informant, multidimensional measure of homework performance that has utility for the assessment of elementary and middle school students. (c) 2015 APA, all rights reserved).

Validation of Digital Microscopy Compared With Light Microscopy for the Diagnosis of Canine Cutaneous Tumors.

PubMed

Bertram, Christof A; Gurtner, Corinne; Dettwiler, Martina; Kershaw, Olivia; Dietert, Kristina; Pieper, Laura; Pischon, Hannah; Gruber, Achim D; Klopfleisch, Robert

2018-07-01

Integration of new technologies, such as digital microscopy, into a highly standardized laboratory routine requires the validation of its performance in terms of reliability, specificity, and sensitivity. However, a validation study of digital microscopy is currently lacking in veterinary pathology. The aim of the current study was to validate the usability of digital microscopy in terms of diagnostic accuracy, speed, and confidence for diagnosing and differentiating common canine cutaneous tumor types and to compare it to classical light microscopy. Therefore, 80 histologic sections including 17 different skin tumor types were examined twice as glass slides and twice as digital whole-slide images by 6 pathologists with different levels of experience at 4 time points. Comparison of both methods found digital microscopy to be noninferior for differentiating individual tumor types within the category epithelial and mesenchymal tumors, but diagnostic concordance was slightly lower for differentiating individual round cell tumor types by digital microscopy. In addition, digital microscopy was associated with significantly shorter diagnostic time, but diagnostic confidence was lower and technical quality was considered inferior for whole-slide images compared with glass slides. Of note, diagnostic performance for whole-slide images scanned at 200× magnification was noninferior in diagnostic performance for slides scanned at 400×. In conclusion, digital microscopy differs only minimally from light microscopy in few aspects of diagnostic performance and overall appears adequate for the diagnosis of individual canine cutaneous tumors with minor limitations for differentiating individual round cell tumor types and grading of mast cell tumors.

The acoustic performance of double-skin facades: A design support tool for architects

NASA Astrophysics Data System (ADS)

Batungbakal, Aireen

This study assesses and validates the influence of measuring sound in the urban environment and the influence of glass facade components in reducing sound transmission to the indoor environment. Among the most reported issues affecting workspaces, increased awareness to minimize noise led building designers to reconsider the design of building envelopes and its site environment. Outdoor sound conditions, such as traffic noise, challenge designers to accurately estimate the capability of glass facades in acquiring an appropriate indoor sound quality. Indicating the density of the urban environment, field-tests acquired existing sound levels in areas of high commercial development, employment, and traffic activity, establishing a baseline for sound levels common in urban work areas. Composed from the direct sound transmission loss of glass facades simulated through INSUL, a sound insulation software, data is utilized as an informative tool correlating the response of glass facade components towards existing outdoor sound levels of a project site in order to achieve desired indoor sound levels. This study progresses to link the disconnection in validating the acoustic performance of glass facades early in a project's design, from conditioned settings such as field-testing and simulations to project completion. Results obtained from the study's facade simulations and facade comparison supports that acoustic comfort is not limited to a singular solution, but multiple design options responsive to its environment.

Reliability and validity of procedure-based assessments in otolaryngology training.

PubMed

Awad, Zaid; Hayden, Lindsay; Robson, Andrew K; Muthuswamy, Keerthini; Tolley, Neil S

2015-06-01

To investigate the reliability and construct validity of procedure-based assessment (PBA) in assessing performance and progress in otolaryngology training. Retrospective database analysis using a national electronic database. We analyzed PBAs of otolaryngology trainees in North London from core trainees (CTs) to specialty trainees (STs). The tool contains six multi-item domains: consent, planning, preparation, exposure/closure, technique, and postoperative care, rated as "satisfactory" or "development required," in addition to an overall performance rating (pS) of 1 to 4. Individual domain score, overall calculated score (cS), and number of "development-required" items were calculated for each PBA. Receiver operating characteristic analysis helped determine sensitivity and specificity. There were 3,152 otolaryngology PBAs from 46 otolaryngology trainees analyzed. PBA reliability was high (Cronbach's α 0.899), and sensitivity approached 99%. cS correlated positively with pS and level in training (rs : +0.681 and +0.324, respectively). ST had higher cS and pS than CT (93% ± 0.6 and 3.2 ± 0.03 vs. 71% ± 3.1 and 2.3 ± 0.08, respectively; P < .001). cS and pS increased from CT1 to ST8 showing construct validity (rs : +0.348 and +0.354, respectively; P < .001). The technical skill domain had the highest utilization (98% of PBAs) and was the best predictor of cS and pS (rs : +0.96 and +0.66, respectively). PBA is reliable and valid for assessing otolaryngology trainees' performance and progress at all levels. It is highly sensitive in identifying competent trainees. The tool is used in a formative and feedback capacity. The technical domain is the best predictor and should be given close attention. NA. © 2014 The American Laryngological, Rhinological and Otological Society, Inc.

Derivation and validation of a two-biomarker panel for diagnosis of ARDS in patients with severe traumatic injuries

PubMed Central

Ware, Lorraine B; Zhao, Zhiguo; Koyama, Tatsuki; Brown, Ryan M; Semler, Matthew W; Janz, David R; May, Addison K; Fremont, Richard D; Matthay, Michael A; Cohen, Mitchell J; Calfee, Carolyn S

2017-01-01

Background Acute respiratory distress syndrome (ARDS) is common after severe traumatic injuries but is underdiagnosed and undertreated. We hypothesized that a panel of plasma biomarkers could be used to diagnose ARDS in severe trauma. To test this hypothesis, we derived and validated a biomarker panel in three independent cohorts and compared the diagnostic performance to clinician recognition of ARDS. Methods Eleven plasma biomarkers of inflammation, lung epithelial and endothelial injury were measured in a derivation cohort of 439 severe trauma patients. ARDS status was analyzed by two-investigator consensus, and cases were required to meet Berlin criteria on intensive care unit (ICU) day 1. Controls were subjects without ARDS during the first 4 days of study enrollment. A multivariable logistic regression model was used to generate probabilities for ARDS. A reduced model with the top two performing markers was then tested in two independent validation cohorts. To assess clinical diagnosis of ARDS, medical records in the derivation cohort were systematically searched for documentation of ARDS diagnosis made by a clinical provider. Results Among 11 biomarkers, the combination of the endothelial injury marker angiopoietin-2 (Ang-2) and the lung epithelial injury marker receptor for advanced glycation endproducts (RAGE) provided good discrimination for ARDS in the derivation cohort (area under the curve (AUC)=0.74 (95% CI 0.67 to 0.80). In the validation cohorts, the AUCs for this model were 0.70 (0.61 to 0.77) and 0.78 (0.71 to 0.84). In contrast, provider assessment demonstrated poor diagnostic accuracy for ARDS, with AUC of 0.55 (0.51 to 0.60). Discussion A two-biomarker panel consisting of Ang-2 and RAGE performed well across multiple patient cohorts and outperformed clinical providers for diagnosing ARDS in severe trauma. Clinical application of this model could improve both diagnosis and treatment of ARDS in patients with severe trauma. Level of evidence Diagnostic study, level II. PMID:29766112

Validating An Analytic Completeness Model for Kepler Target Stars Based on Flux-level Transit Injection Experiments

NASA Astrophysics Data System (ADS)

Catanzarite, Joseph; Burke, Christopher J.; Li, Jie; Seader, Shawn; Haas, Michael R.; Batalha, Natalie; Henze, Christopher; Christiansen, Jessie; Kepler Project, NASA Advanced Supercomputing Division

2016-06-01

The Kepler Mission is developing an Analytic Completeness Model (ACM) to estimate detection completeness contours as a function of exoplanet radius and period for each target star. Accurate completeness contours are necessary for robust estimation of exoplanet occurrence rates.The main components of the ACM for a target star are: detection efficiency as a function of SNR, the window function (WF) and the one-sigma depth function (OSDF). (Ref. Burke et al. 2015). The WF captures the falloff in transit detection probability at long periods that is determined by the observation window (the duration over which the target star has been observed). The OSDF is the transit depth (in parts per million) that yields SNR of unity for the full transit train. It is a function of period, and accounts for the time-varying properties of the noise and for missing or deweighted data.We are performing flux-level transit injection (FLTI) experiments on selected Kepler target stars with the goal of refining and validating the ACM. “Flux-level” injection machinery inserts exoplanet transit signatures directly into the flux time series, as opposed to “pixel-level” injection, which inserts transit signatures into the individual pixels using the pixel response function. See Jie Li's poster: ID #2493668, "Flux-level transit injection experiments with the NASA Pleiades Supercomputer" for details, including performance statistics.Since FLTI is affordable for only a small subset of the Kepler targets, the ACM is designed to apply to most Kepler target stars. We validate this model using “deep” FLTI experiments, with ~500,000 injection realizations on each of a small number of targets and “shallow” FLTI experiments with ~2000 injection realizations on each of many targets. From the results of these experiments, we identify anomalous targets, model their behavior and refine the ACM accordingly.In this presentation, we discuss progress in validating and refining the ACM, and we compare our detection efficiency curves with those derived from the associated pixel-level transit injection experiments.Kepler was selected as the 10th mission of the Discovery Program. Funding for this mission is provided by NASA, Science Mission Directorate.

Optimisation of an analytical method and results from the inter-laboratory comparison of the migration of regulated substances from food packaging into the new mandatory European Union simulant for dry foodstuffs.

PubMed

Jakubowska, Natalia; Beldì, Giorgia; Peychès Bach, Aurélie; Simoneau, Catherine

2014-01-01

This paper presents the outcome of the development, optimisation and validation at European Union level of an analytical method for using poly(2,6-diphenyl phenylene oxide--PPPO), which is stipulated in Regulation (EU) No. 10/2011, as food simulant E for testing specific migration from plastics into dry foodstuffs. Two methods for fortifying respectively PPPO and a low-density polyethylene (LDPE) film with surrogate substances that are relevant to food contact were developed. A protocol for cleaning the PPPO and an efficient analytical method were developed for the quantification of butylhydroxytoluene (BHT), benzophenone (BP), diisobutylphthalate (DiBP), bis(2-ethylhexyl) adipate (DEHA) and 1,2-cyclohexanedicarboxylic acid, diisononyl ester (DINCH) from PPPO. A protocol for a migration test from plastics using small migration cells was also developed. The method was validated by an inter-laboratory comparison (ILC) with 16 national reference laboratories for food contact materials in the European Union. This allowed for the first time data to be obtained on the precision and laboratory performance of both migration and quantification. The results showed that the validation ILC was successful even when taking into account the complexity of the exercise. The results showed that the method performance was 7-9% repeatability standard deviation (rSD) for most substances (regardless of concentration), with 12% rSD for the high level of BHT and for DiBP at very low levels. The reproducibility standard deviation results for the 16 European Union laboratories were in the range of 20-30% for the quantification from PPPO (for the three levels of concentrations of the five substances) and 15-40% from migration experiments from the fortified plastic at 60°C for 10 days and subsequent quantification. Considering the lack of data previously available in the literature, this work has demonstrated that the validation of a method is possible both for migration from a film and for quantification into a corresponding simulant for specific migration.

Constructing and Validating High-Performance MIEC-SVM Models in Virtual Screening for Kinases: A Better Way for Actives Discovery

PubMed Central

Sun, Huiyong; Pan, Peichen; Tian, Sheng; Xu, Lei; Kong, Xiaotian; Li, Youyong; Dan Li; Hou, Tingjun

2016-01-01

The MIEC-SVM approach, which combines molecular interaction energy components (MIEC) derived from free energy decomposition and support vector machine (SVM), has been found effective in capturing the energetic patterns of protein-peptide recognition. However, the performance of this approach in identifying small molecule inhibitors of drug targets has not been well assessed and validated by experiments. Thereafter, by combining different model construction protocols, the issues related to developing best MIEC-SVM models were firstly discussed upon three kinase targets (ABL, ALK, and BRAF). As for the investigated targets, the optimized MIEC-SVM models performed much better than the models based on the default SVM parameters and Autodock for the tested datasets. Then, the proposed strategy was utilized to screen the Specs database for discovering potential inhibitors of the ALK kinase. The experimental results showed that the optimized MIEC-SVM model, which identified 7 actives with IC50 < 10 μM from 50 purchased compounds (namely hit rate of 14%, and 4 in nM level) and performed much better than Autodock (3 actives with IC50 < 10 μM from 50 purchased compounds, namely hit rate of 6%, and 2 in nM level), suggesting that the proposed strategy is a powerful tool in structure-based virtual screening. PMID:27102549

Constructing and Validating High-Performance MIEC-SVM Models in Virtual Screening for Kinases: A Better Way for Actives Discovery.

PubMed

Sun, Huiyong; Pan, Peichen; Tian, Sheng; Xu, Lei; Kong, Xiaotian; Li, Youyong; Dan Li; Hou, Tingjun

2016-04-22

The MIEC-SVM approach, which combines molecular interaction energy components (MIEC) derived from free energy decomposition and support vector machine (SVM), has been found effective in capturing the energetic patterns of protein-peptide recognition. However, the performance of this approach in identifying small molecule inhibitors of drug targets has not been well assessed and validated by experiments. Thereafter, by combining different model construction protocols, the issues related to developing best MIEC-SVM models were firstly discussed upon three kinase targets (ABL, ALK, and BRAF). As for the investigated targets, the optimized MIEC-SVM models performed much better than the models based on the default SVM parameters and Autodock for the tested datasets. Then, the proposed strategy was utilized to screen the Specs database for discovering potential inhibitors of the ALK kinase. The experimental results showed that the optimized MIEC-SVM model, which identified 7 actives with IC50 < 10 μM from 50 purchased compounds (namely hit rate of 14%, and 4 in nM level) and performed much better than Autodock (3 actives with IC50 < 10 μM from 50 purchased compounds, namely hit rate of 6%, and 2 in nM level), suggesting that the proposed strategy is a powerful tool in structure-based virtual screening.

ACCESS - A Science and Engineering Assessment of Space Coronagraph Concepts for the Direct Imaging and Spectroscopy of Exoplanetary Systems

NASA Technical Reports Server (NTRS)

Trauger, John

2008-01-01

Topics include and overview, science objectives, study objectives, coronagraph types, metrics, ACCESS observatory, laboratory validations, and summary. Individual slides examine ACCESS engineering approach, ACCESS gamut of coronagraph types, coronagraph metrics, ACCESS Discovery Space, coronagraph optical layout, wavefront control on the "level playing field", deformable mirror development for HCIT, laboratory testbed demonstrations, high contract imaging with the HCIT, laboratory coronagraph contrast and stability, model validation and performance predictions, HCIT coronagraph optical layout, Lyot coronagraph on the HCIT, pupil mapping (PIAA), shaped pupils, and vortex phase mask experiments on the HCIT.

Overview of calibration and validation activities for the EUMETSAT polar system: second generation (EPS-SG) visible/infrared imager (METimage)

NASA Astrophysics Data System (ADS)

Phillips, P.; Bonsignori, R.; Schlüssel, P.; Schmülling, F.; Spezzi, L.; Watts, P.; Zerfowski, I.

2016-10-01

The EPS-SG Visible/Infrared Imaging (VII) mission is dedicated to supporting the optical imagery user needs for Numerical Weather Prediction (NWP), Nowcasting (NWC) and climate in the timeframe beyond 2020. The VII mission is fulfilled by the METimage instrument, developed by the German Space Agency (DLR) and funded by the German government and EUMETSAT. Following on from an important list of predecessors such as the Advanced Very High Resolution Radiometer (AVHRR) and the Moderate resolution Imaging Spectro-radiometer (MODIS), METimage will fly in the mid-morning orbit of the Joint Polar System, whilst the early-afternoon orbits are served by the JPSS (U.S. Joint Polar Satellite System) Visible Infrared Imager Radiometer Suite (VIIRS). METimage itself is a cross-purpose medium resolution, multi-spectral optical imager, measuring the optical spectrum of radiation emitted and reflected by the Earth from a low-altitude sun synchronous orbit over a minimum swath width of 2700 km. The top of the atmosphere outgoing radiance will be sampled every 500 m (at nadir) with measurements made in 20 spectral channels ranging from 443 nm in the visible up to 13.345 μm in the thermal infrared. The three major objectives of the EPS-SG METimage calibration and validation activities are: • Verification of the instrument performances through continuous in-flight calibration and characterisation, including monitoring of long term stability. • Provision of validated level 1 and level 2 METimage products. • Revision of product processing facilities, i.e. algorithms and auxiliary data sets, to assure that products conform with user requirements, and then, if possible, exceed user expectations. This paper will describe the overall Calibration and Validation (Cal/Val) logic and the methods adopted to ensure that the METimage data products meet performance specifications for the lifetime of the mission. Such methods include inter-comparisons with other missions through simultaneous nadir overpasses and comparisons with ground based observations, analysis of algorithm internal diagnostics to confirm retrieval performance for geophysical products and vicarious calibration to assist with validation of the instrument on-board calibration. Any identified deficiencies in the products will lead to either an update any auxiliary data sets (e.g. calibration key data) that are used to configure the product processors or to a revision of algorithms themselves. The Cal/Val activities are mostly foreseen during commissioning but will inevitably extend to routine operations in order to take on board seasonal variations and ensure long term stability of the calibrated radiances and geophysical products. Pre-requisite to validation of products at scientific level is that the satellite and instrument itself have been verified against their respective specifications both pre-launch and during the satellite in-orbit verification phase.

Design and validation of a questionnaire to assess organizational culture in French hospital wards.

PubMed

Saillour-Glénisson, F; Domecq, S; Kret, M; Sibe, M; Dumond, J P; Michel, P

2016-09-17

Although many organizational culture questionnaires have been developed, there is a lack of any validated multidimensional questionnaire assessing organizational culture at hospital ward level and adapted to health care context. Facing the lack of an appropriate tool, a multidisciplinary team designed and validated a dimensional organizational culture questionnaire for healthcare settings to be administered at ward level. A database of organizational culture items and themes was created after extensive literature review. Items were regrouped into dimensions and subdimensions (classification validated by experts). Pre-test and face validation was conducted with 15 health care professionals. In a stratified cluster random sample of hospitals, the psychometric validation was conducted in three phases on a sample of 859 healthcare professionals from 36 multidisciplinary medicine services: 1) the exploratory phase included a description of responses' saturation levels, factor and correlations analyses and an internal consistency analysis (Cronbach's alpha coefficient); 2) confirmatory phase used the Structural Equation Modeling (SEM); 3) reproducibility was studied by a test-retest. The overall response rate was 80 %; the completion average was 97 %. The metrological results were: a global Cronbach's alpha coefficient of 0.93, higher than 0.70 for 12 sub-dimensions; all Dillon-Goldstein's rho coefficients higher than 0.70; an excellent quality of external model with a Goodness of Fitness (GoF) criterion of 0.99. Seventy percent of the items had a reproducibility ranging from moderate (Intra-Class Coefficient between 50 and 70 % for 25 items) to good (ICC higher than 70 % for 33 items). COMEt (Contexte Organisationnel et Managérial en Etablissement de Santé) questionnaire is a validated multidimensional organizational culture questionnaire made of 6 dimensions, 21 sub-dimensions and 83 items. It is the first dimensional organizational culture questionnaire, specific to healthcare context, for a unit level assessment showing robust psychometric properties (validity and reliability). This tool is suited for research purposes, especially for assessing organizational context in research analysing the effectiveness of hospital quality improvement strategies. Our tool is also suited for an overall assessment of ward culture and could be a powerful trigger to improve management and clinical performance. Its psychometric properties in other health systems need to be tested.

Instrument validation process: a case study using the Paediatric Pain Knowledge and Attitudes Questionnaire.

PubMed

Peirce, Deborah; Brown, Janie; Corkish, Victoria; Lane, Marguerite; Wilson, Sally

2016-06-01

To compare two methods of calculating interrater agreement while determining content validity of the Paediatric Pain Knowledge and Attitudes Questionnaire for use with Australian nurses. Paediatric pain assessment and management documentation was found to be suboptimal revealing a need to assess paediatric nurses' knowledge and attitude to pain. The Paediatric Pain Knowledge and Attitudes Questionnaire was selected as it had been reported as valid and reliable in the United Kingdom with student nurses. The questionnaire required content validity determination prior to use in the Australian context. A two phase process of expert review. Ten paediatric nurses completed a relevancy rating of all 68 questionnaire items. In phase two, five pain experts reviewed the items of the questionnaire that scored an unacceptable item level content validity. Item and scale level content validity indices and intraclass correlation coefficients were calculated. In phase one, 31 items received an item level content validity index <0·78 and the scale level content validity index average was 0·80 which were below levels required for acceptable validity. The intraclass correlation coefficient was 0·47. In phase two, 10 items were amended and four items deleted. The revised questionnaire provided a scale level content validity index average >0·90 and an intraclass correlation coefficient of 0·94 demonstrating excellent agreement between raters therefore acceptable content validity. Equivalent outcomes were achieved using the content validity index and the intraclass correlation coefficient. To assess content validity the content validity index has the advantage of providing an item level score and is a simple calculation. The intraclass correlation coefficient requires statistical knowledge, or support, and has the advantage of accounting for the possibility of chance agreement. © 2016 John Wiley & Sons Ltd.

The impact of pediatric neuropsychological consultation in mild traumatic brain injury: a model for providing feedback after invalid performance.

PubMed

Connery, Amy K; Peterson, Robin L; Baker, David A; Kirkwood, Michael W

2016-05-01

In recent years, pediatric practitioners have increasingly recognized the importance of objectively measuring performance validity during clinical assessments. Yet, no studies have examined the impact of neuropsychological consultation when invalid performance has been identified in pediatric populations and little published guidance exists for clinical management. Here we provide a conceptual model for providing feedback after noncredible performance has been detected. In a pilot study, we examine caregiver satisfaction and postconcussive symptoms following provision of this feedback for patients seen through our concussion program. Participants (N = 70) were 8-17-year-olds with a history of mild traumatic brain injury who underwent an abbreviated neuropsychological evaluation between 2 and 12 months post-injury. We examined postconcussive symptom reduction and caregiver satisfaction after neuropsychological evaluation between groups of patients who were determined to have provided noncredible effort (n = 9) and those for whom no validity concerns were present (n = 61). We found similarly high levels of caregiver satisfaction between groups and greater reduction in self-reported symptoms after feedback was provided using the model with children with noncredible presentations compared to those with credible presentations. The current study lends preliminary support to the idea that the identification and communication of invalid performance can be a beneficial clinical intervention that promotes high levels of caregiver satisfaction and a reduction in self-reported and caregiver-reported symptoms.

The first Latin-American risk stratification system for cardiac surgery: can be used as a graphic pocket-card score.

PubMed

Carosella, Victorio C; Navia, Jose L; Al-Ruzzeh, Sharif; Grancelli, Hugo; Rodriguez, Walter; Cardenas, Cesar; Bilbao, Jorge; Nojek, Carlos

2009-08-01

This study aims to develop the first Latin-American risk model that can be used as a simple, pocket-card graphic score at bedside. The risk model was developed on 2903 patients who underwent cardiac surgery at the Spanish Hospital of Buenos Aires, Argentina, between June 1994 and December 1999. Internal validation was performed on 708 patients between January 2000 and June 2001 at the same center. External validation was performed on 1087 patients between February 2000 and January 2007 at three other centers in Argentina. In the development dataset the area under receiver operating characteristics (ROC) curve was 0.73 and the Hosmer-Lemeshow (HL) test was P=0.88. In the internal validation ROC curve was 0.77. In the external validation ROC curve was 0.81, but imperfect calibration was detected because the observed in-hospital mortality (3.96%) was significantly lower than the development dataset (8.20%) (P<0.0001). Recalibration was done in 2007, showing excellent level of agreement between the observed and predicted mortality rates on all patients (P=0.92). This is the first risk model for cardiac surgery developed in a population of Latin-America with both internal and external validation. A simple graphic pocket-card score allows an easy bedside application with acceptable statistic precision.

The development and psychometric validation of the Ethical Awareness Scale.

PubMed

Milliken, Aimee; Ludlow, Larry; DeSanto-Madeya, Susan; Grace, Pamela

2018-04-19

To develop and psychometrically assess the Ethical Awareness Scale using Rasch measurement principles and a Rasch item response theory model. Critical care nurses must be equipped to provide good (ethical) patient care. This requires ethical awareness, which involves recognizing the ethical implications of all nursing actions. Ethical awareness is imperative in successfully addressing patient needs. Evidence suggests that the ethical import of everyday issues may often go unnoticed by nurses in practice. Assessing nurses' ethical awareness is a necessary first step in preparing nurses to identify and manage ethical issues in the highly dynamic critical care environment. A cross-sectional design was used in two phases of instrument development. Using Rasch principles, an item bank representing nursing actions was developed (33 items). Content validity testing was performed. Eighteen items were selected for face validity testing. Two rounds of operational testing were performed with critical care nurses in Boston between February-April 2017. A Rasch analysis suggests sufficient item invariance across samples and sufficient construct validity. The analysis further demonstrates a progression of items uniformly along a hierarchical continuum; items that match respondent ability levels; response categories that are sufficiently used; and adequate internal consistency. Mean ethical awareness scores were in the low/moderate range. The results suggest the Ethical Awareness Scale is a psychometrically sound, reliable and valid measure of ethical awareness in critical care nurses. © 2018 John Wiley & Sons Ltd.

Model-Based Verification and Validation of Spacecraft Avionics

NASA Technical Reports Server (NTRS)

Khan, M. Omair; Sievers, Michael; Standley, Shaun

2012-01-01

Verification and Validation (V&V) at JPL is traditionally performed on flight or flight-like hardware running flight software. For some time, the complexity of avionics has increased exponentially while the time allocated for system integration and associated V&V testing has remained fixed. There is an increasing need to perform comprehensive system level V&V using modeling and simulation, and to use scarce hardware testing time to validate models; the norm for thermal and structural V&V for some time. Our approach extends model-based V&V to electronics and software through functional and structural models implemented in SysML. We develop component models of electronics and software that are validated by comparison with test results from actual equipment. The models are then simulated enabling a more complete set of test cases than possible on flight hardware. SysML simulations provide access and control of internal nodes that may not be available in physical systems. This is particularly helpful in testing fault protection behaviors when injecting faults is either not possible or potentially damaging to the hardware. We can also model both hardware and software behaviors in SysML, which allows us to simulate hardware and software interactions. With an integrated model and simulation capability we can evaluate the hardware and software interactions and identify problems sooner. The primary missing piece is validating SysML model correctness against hardware; this experiment demonstrated such an approach is possible.

Mindfulness, burnout, and effects on performance evaluations in internal medicine residents

PubMed Central

Braun, Sarah E; Auerbach, Stephen M; Rybarczyk, Bruce; Lee, Bennett; Call, Stephanie

2017-01-01

Purpose Burnout has been documented at high levels in medical residents with negative effects on performance. Some dispositional qualities, like mindfulness, may protect against burnout. The purpose of the present study was to assess burnout prevalence among internal medicine residents at a single institution, examine the relationship between mindfulness and burnout, and provide preliminary findings on the relation between burnout and performance evaluations in internal medicine residents. Methods Residents (n = 38) completed validated measures of burnout at three time points separated by 2 months and a validated measure of dispositional mindfulness at baseline. Program director end-of-year performance evaluations were also obtained on 22 milestones used to evaluate internal medicine resident performance; notably, these milestones have not yet been validated for research purposes; therefore, the investigation here is exploratory. Results Overall, 71.1% (n = 27) of the residents met criteria for burnout during the study. Lower scores on the “acting with awareness” facet of dispositional mindfulness significantly predicted meeting burnout criteria χ2(5) = 11.88, p = 0.04. Lastly, meeting burnout criteria significantly predicted performance on three of the performance milestones, with positive effects on milestones from the “system-based practices” and “professionalism” domains and negative effects on a milestone from the “patient care” domain. Conclusion Burnout rates were high in this sample of internal medicine residents and rates were consistent with other reports of burnout during medical residency. Dispositional mindfulness was supported as a protective factor against burnout. Importantly, results from the exploratory investigation of the relationship between burnout and resident evaluations suggested that burnout may improve performance on some domains of resident evaluations while compromising performance on other domains. Implications and directions for future research are discussed. PMID:28860889

Mindfulness, burnout, and effects on performance evaluations in internal medicine residents.

PubMed

Braun, Sarah E; Auerbach, Stephen M; Rybarczyk, Bruce; Lee, Bennett; Call, Stephanie

2017-01-01

Burnout has been documented at high levels in medical residents with negative effects on performance. Some dispositional qualities, like mindfulness, may protect against burnout. The purpose of the present study was to assess burnout prevalence among internal medicine residents at a single institution, examine the relationship between mindfulness and burnout, and provide preliminary findings on the relation between burnout and performance evaluations in internal medicine residents. Residents (n = 38) completed validated measures of burnout at three time points separated by 2 months and a validated measure of dispositional mindfulness at baseline. Program director end-of-year performance evaluations were also obtained on 22 milestones used to evaluate internal medicine resident performance; notably, these milestones have not yet been validated for research purposes; therefore, the investigation here is exploratory. Overall, 71.1% (n = 27) of the residents met criteria for burnout during the study. Lower scores on the "acting with awareness" facet of dispositional mindfulness significantly predicted meeting burnout criteria χ 2 (5) = 11.88, p = 0.04. Lastly, meeting burnout criteria significantly predicted performance on three of the performance milestones, with positive effects on milestones from the "system-based practices" and "professionalism" domains and negative effects on a milestone from the "patient care" domain. Burnout rates were high in this sample of internal medicine residents and rates were consistent with other reports of burnout during medical residency. Dispositional mindfulness was supported as a protective factor against burnout. Importantly, results from the exploratory investigation of the relationship between burnout and resident evaluations suggested that burnout may improve performance on some domains of resident evaluations while compromising performance on other domains. Implications and directions for future research are discussed.

Physician groups' use of data from patient experience surveys.

PubMed

Friedberg, Mark W; SteelFisher, Gillian K; Karp, Melinda; Schneider, Eric C

2011-05-01

In Massachusetts, physician groups' performance on validated surveys of patient experience has been publicly reported since 2006. Groups also receive detailed reports of their own performance, but little is known about how physician groups have responded to these reports. To examine whether and how physician groups are using patient experience data to improve patient care. During 2008, we conducted semi-structured interviews with the leaders of 72 participating physician groups (out of 117 groups receiving patient experience reports). Based on leaders' responses, we identified three levels of engagement with patient experience reporting: no efforts to improve (level 1), efforts to improve only the performance of low-scoring physicians or practice sites (level 2), and efforts to improve group-wide performance (level 3). Groups' level of engagement and specific efforts to improve patient care. Forty-four group leaders (61%) reported group-wide improvement efforts (level 3), 16 (22%) reported efforts to improve only the performance of low-scoring physicians or practice sites (level 2), and 12 (17%) reported no performance improvement efforts (level 1). Level 3 groups were more likely than others to have an integrated medical group organizational model (84% vs. 31% at level 2 and 33% at level 1; P < 0.005) and to employ the majority of their physicians (69% vs. 25% and 20%; P < 0.05). Among level 3 groups, the most common targets for improvement were access, communication with patients, and customer service. The most commonly reported improvement initiatives were changing office workflow, providing additional training for nonclinical staff, and adopting or enhancing an electronic health record. Despite statewide public reporting, physician groups' use of patient experience data varied widely. Integrated organizational models were associated with greater engagement, and efforts to enhance clinicians' interpersonal skills were uncommon, with groups predominantly focusing on office workflow and support staff.

Prevalence of Invalid Performance on Baseline Testing for Sport-Related Concussion by Age and Validity Indicator.

PubMed

Abeare, Christopher A; Messa, Isabelle; Zuccato, Brandon G; Merker, Bradley; Erdodi, Laszlo

2018-03-12

Estimated base rates of invalid performance on baseline testing (base rates of failure) for the management of sport-related concussion range from 6.1% to 40.0%, depending on the validity indicator used. The instability of this key measure represents a challenge in the clinical interpretation of test results that could undermine the utility of baseline testing. To determine the prevalence of invalid performance on baseline testing and to assess whether the prevalence varies as a function of age and validity indicator. This retrospective, cross-sectional study included data collected between January 1, 2012, and December 31, 2016, from a clinical referral center in the Midwestern United States. Participants included 7897 consecutively tested, equivalently proportioned male and female athletes aged 10 to 21 years, who completed baseline neurocognitive testing for the purpose of concussion management. Baseline assessment was conducted with the Immediate Postconcussion Assessment and Cognitive Testing (ImPACT), a computerized neurocognitive test designed for assessment of concussion. Base rates of failure on published ImPACT validity indicators were compared within and across age groups. Hypotheses were developed after data collection but prior to analyses. Of the 7897 study participants, 4086 (51.7%) were male, mean (SD) age was 14.71 (1.78) years, 7820 (99.0%) were primarily English speaking, and the mean (SD) educational level was 8.79 (1.68) years. The base rate of failure ranged from 6.4% to 47.6% across individual indicators. Most of the sample (55.7%) failed at least 1 of 4 validity indicators. The base rate of failure varied considerably across age groups (117 of 140 [83.6%] for those aged 10 years to 14 of 48 [29.2%] for those aged 21 years), representing a risk ratio of 2.86 (95% CI, 2.60-3.16; P < .001). The results for base rate of failure were surprisingly high overall and varied widely depending on the specific validity indicator and the age of the examinee. The strong age association, with 3 of 4 participants aged 10 to 12 years failing validity indicators, suggests that the clinical interpretation and utility of baseline testing in this age group is questionable. These findings underscore the need for close scrutiny of performance validity indicators on baseline testing across age groups.

Examining the Predictive Validity of a Dynamic Assessment of Decoding to Forecast Response Tier 2 to Intervention

PubMed Central

Cho, Eunsoo; Compton, Donald L.; Fuchs, Doug; Fuchs, Lynn S.; Bouton, Bobette

2013-01-01

The purpose of this study was to examine the role of a dynamic assessment (DA) of decoding in predicting responsiveness to Tier 2 small group tutoring in a response-to-intervention model. First-grade students (n=134) who did not show adequate progress in Tier 1 based on 6 weeks of progress monitoring received Tier 2 small-group tutoring in reading for 14 weeks. Student responsiveness to Tier 2 was assessed weekly with word identification fluency (WIF). A series of conditional individual growth curve analyses were completed that modeled the correlates of WIF growth (final level of performance and growth). Its purpose was to examine the predictive validity of DA in the presence of 3 sets of variables: static decoding measures, Tier 1 responsiveness indicators, and pre-reading variables (phonemic awareness, rapid letter naming, oral vocabulary, and IQ). DA was a significant predictor of final level and growth, uniquely explaining 3% – 13% of the variance in Tier 2 responsiveness depending on the competing predictors in the model and WIF outcome (final level of performance or growth). Although the additional variances explained uniquely by DA were relatively small, results indicate the potential of DA in identifying Tier 2 nonresponders. PMID:23213050

Examining the predictive validity of a dynamic assessment of decoding to forecast response to tier 2 intervention.

PubMed

Cho, Eunsoo; Compton, Donald L; Fuchs, Douglas; Fuchs, Lynn S; Bouton, Bobette

2014-01-01

The purpose of this study was to examine the role of a dynamic assessment (DA) of decoding in predicting responsiveness to Tier 2 small-group tutoring in a response-to-intervention model. First grade students (n = 134) who did not show adequate progress in Tier 1 based on 6 weeks of progress monitoring received Tier 2 small-group tutoring in reading for 14 weeks. Student responsiveness to Tier 2 was assessed weekly with word identification fluency (WIF). A series of conditional individual growth curve analyses were completed that modeled the correlates of WIF growth (final level of performance and growth). Its purpose was to examine the predictive validity of DA in the presence of three sets of variables: static decoding measures, Tier 1 responsiveness indicators, and prereading variables (phonemic awareness, rapid letter naming, oral vocabulary, and IQ). DA was a significant predictor of final level and growth, uniquely explaining 3% to 13% of the variance in Tier 2 responsiveness depending on the competing predictors in the model and WIF outcome (final level of performance or growth). Although the additional variances explained uniquely by DA were relatively small, results indicate the potential of DA in identifying Tier 2 nonresponders. © Hammill Institute on Disabilities 2012.

Commercial Disinfectants During Disinfection Process Validation: More Failures than Success.

PubMed

Chatterjee, Shiv Sekhar; Chumber, Sushil Kumar; Khanduri, Uma

2016-08-01

Disinfection process validation is mandatory before introduction of a new disinfectant in hospital services. Commercial disinfection brands often question existing hospital policy claiming greater efficacy and lack of toxicity of their products. Inadvertent inadequate disinfection leads to morbidity, patient's economic burden, and the risk of mortality. To evaluate commercial disinfectants for high, intermediate and low-level disinfection so as to identify utility for our routine situations. This laboratory based experiment was conducted at St Stephen Hospital, Delhi during July-September 2013. Twelve commercial disinfectants: Sanidex®, Sanocid®, Cidex®, SekuSept Aktiv®, BIB Forte®, Alprojet W®, Desnet®, Sanihygiene®, Incidin®, D125®, Lonzagard®, and Glutishield® were tested. Time-kill assay (suspension test) was performed against six indicator bacteria (Escherichia coli, Staphylococcus aureus, Pseudomonas aeruginosa, Salmonella Typhi, Bacillus cereus, and Mycobacterium fortuitum). Low and high inoculum (final concentrations 1.5X10(6) and 9X10(6) cfu/ml) of the first five bacteria while only low level of M. fortuitum was tested. Cidex® (2.4% Glutaraldehyde) performed best as high level disinfectant while newer quarternary ammonium compounds (QACs) (Incidin®, D125®, and Lonzagard®) were good at low level disinfection. Sanidex® (0.55% Ortho-pthalaldehyde) though mycobactericidal took 10 minutes for sporicidal activity. Older QAC containing BIB Forte® and Desnet® took 20 minutes to fully inhibit P. aeruginosa. All disinfectants effectively reduced S. Typhi to zero counts within 5 minutes. Cidex® is a good high-level disinfectant while newer QACs (Incidin®, D125®, and Lonzagard®) were capable low-level disinfectants.

A novel fuzzy approach for automatic Brunnstrom stage classification using surface electromyography.

PubMed

Liparulo, Luca; Zhang, Zhe; Panella, Massimo; Gu, Xudong; Fang, Qiang

2017-08-01

Clinical assessment plays a major role in post-stroke rehabilitation programs for evaluating impairment level and tracking recovery progress. Conventionally, this process is manually performed by clinicians using chart-based ordinal scales which can be both subjective and inefficient. In this paper, a novel approach based on fuzzy logic is proposed which automatically evaluates stroke patients' impairment level using single-channel surface electromyography (sEMG) signals and generates objective classification results based on the widely used Brunnstrom stages of recovery. The correlation between stroke-induced motor impairment and sEMG features on both time and frequency domain is investigated, and a specifically designed fuzzy kernel classifier based on geometrically unconstrained membership function is introduced in the study to tackle the challenges in discriminating data classes with complex separating surfaces. Experiments using sEMG data collected from stroke patients have been carried out to examine the validity and feasibility of the proposed method. In order to ensure the generalization capability of the classifier, a cross-validation test has been performed. The results, verified using the evaluation decisions provided by an expert panel, have reached a rate of success of the 92.47%. The proposed fuzzy classifier is also compared with other pattern recognition techniques to demonstrate its superior performance in this application.

Experimental and analytical investigation of a modified ring cusp NSTAR engine

NASA Technical Reports Server (NTRS)

Sengupta, Anita

2005-01-01

A series of experimental measurements on a modified laboratory NSTAR engine were used to validate a zero dimensional analytical discharge performance model of a ring cusp ion thruster. The model predicts the discharge performance of a ring cusp NSTAR thruster as a function the magnetic field configuration, thruster geometry, and throttle level. Analytical formalisms for electron and ion confinement are used to predict the ionization efficiency for a given thruster design. Explicit determination of discharge loss and volume averaged plasma parameters are also obtained. The model was used to predict the performance of the nominal and modified three and four ring cusp 30-cm ion thruster configurations operating at the full power (2.3 kW) NSTAR throttle level. Experimental measurements of the modified engine configuration discharge loss compare well with the predicted value for propellant utilizations from 80 to 95%. The theory, as validated by experiment, indicates that increasing the magnetic strength of the minimum closed reduces maxwellian electron diffusion and electrostatically confines the ion population and subsequent loss to the anode wall. The theory also indicates that increasing the cusp strength and minimizing the cusp area improves primary electron confinement increasing the probability of an ionization collision prior to loss at the cusp.

Problems Encountered During the Recertification of the GLORY Solar Array Dual Axis Gimbal Drive Actuators

NASA Technical Reports Server (NTRS)

Saltzman, Marc; Schepis, Jospeh P.; Bruckner, Michael J.

2009-01-01

The Glory observatory is the current incarnation of the Vegetation Canopy Lidar (VCL) mission spacecraft bus. The VCL spacecraft bus, having been cancelled for programmatic reasons in 2000, was nearly integrated when it was put into storage for possible future use. The Glory mission was a suitable candidate for using this spacecraft and in 2006 an effort to recertify the two axis solar array gimbal drive after its extended storage was begun. What was expected to be a simple performance validation of the two dual axis gimbal stepper motors became a serious test, diagnosis and repair task once questions arose on the flight worthiness of the hardware. A significant test program logic flow was developed which identified decisions that could be made based on the results of individual recertification tests. Without disassembling the bi-axial gimbals, beginning with stepper motor threshold voltage measurements and relating these to powered drive torque measurements, both performed at the spacecraft integrator s facility, a confusing picture of the health of the actuators came to light. Tests at the gimbal assembly level and tests of the disassembled actuators were performed by the manufacturer to validate our results and torque discrepancies were noted. Further disassembly to the component level of the actuator revealed the source of the torque loss.

Predicting introductory programming performance: A multi-institutional multivariate study

NASA Astrophysics Data System (ADS)

Bergin, Susan; Reilly, Ronan

2006-12-01

A model for predicting student performance on introductory programming modules is presented. The model uses attributes identified in a study carried out at four third-level institutions in the Republic of Ireland. Four instruments were used to collect the data and over 25 attributes were examined. A data reduction technique was applied and a logistic regression model using 10-fold stratified cross validation was developed. The model used three attributes: Leaving Certificate Mathematics result (final mathematics examination at second level), number of hours playing computer games while taking the module and programming self-esteem. Prediction success was significant with 80% of students correctly classified. The model also works well on a per-institution level. A discussion on the implications of the model is provided and future work is outlined.

Validity and reproducibility of self-reported total physical activity--differences by relative weight.

PubMed

Norman, A; Bellocco, R; Bergström, A; Wolk, A

2001-05-01

Physical activity is hypothesized to reduce the risk of obesity and several other chronic diseases and enhance longevity. However, most of the questionnaires used measure only part of total physical activity, occupational and/or leisure-time activity, which might lead to misclassification of total physical activity level and to dilution of risk estimates. We evaluated the validity and reproducibility of a short self-administered physical activity questionnaire, intended to measure long-term total daily 24 h physical activity. The questionnaire included questions on level of physical activity at work, hours per day of walking/bicycling, home/household work, leisure-time activity/inactivity and sleeping and was sent twice during one year (winter/spring and late summer). Two 7-day activity records, performed 6 months apart, were used as the reference method. One-hundred and eleven men, aged 44-78, completed the questionnaire and one or two activity records. The physical activity levels were measured as metabolic equivalents (MET)xh/day. Spearman correlation coefficient between total daily activity score estimated from the first questionnaire and the records (validity) was 0.56 (deattenuated) and between the first and the second questionnaire (reproducibility) 0.65. Significantly higher validity correlations were observed in men with self-reported body mass index below 26 kg/m(2) than in heavier men (r=0.73 vs r=0.39). This study indicates that the average total daily physical activity scores can be estimated satisfactorily in men using this simple self-administered questionnaire.

Color Trails Test: normative data and criterion validity for the greek adult population.

PubMed

Messinis, Lambros; Malegiannaki, Amaryllis-Chryssi; Christodoulou, Tessa; Panagiotopoulos, Vassillis; Papathanasopoulos, Panagiotis

2011-06-01

The Color Trails Test (CTT) was developed as a culturally fair analog of the Trail Making Test. In the present study, normative data for the CTT were developed for the Greek adult population and further the criterion validity of the CTT was examined in two clinical groups (29 Parkinson's disease [PD] and 25 acute stroke patients). The instrument was applied to 163 healthy participants, aged 19-75. Stepwise linear regression analyses revealed a significant influence of age and education level on completion time in both parts of the CTT (increased age and decreased educational level contributed to slower completion times for both parts), whereas gender did not influence time to completion of part B. Further, the CTT appears to discriminate adequately between the performance of PD and acute stroke patients and matched healthy controls.

Simultaneous determination of multi drug components Theophylline, Etofylline, Guaiphenesine and Ambroxol Hydrochloride by validated RP-HPLC method in liquid dosage form.

PubMed

Jain, Jainendra Kumar; Prakash, M S; Mishra, Rajnish K; Khandhar, Amit P

2008-04-01

The RP-HPLC (reverse phase high performance liquid chromatography) method was developed and validated for simultaneous determination of Multi drug components i.e., Theophylline, Etofylline, Guaiphenesine and Ambroxol Hydrochloride in a liquid dosage form. Chromatographic separation of the four drugs was performed on a Hypersil Phenyl BDS (25cmX4.6mm, 5mm). The mobile phase constituted of triethylamine pH 3.0 buffer: methanol (85:15) v/v was delivered at the flow rate 1.5 mL/min. Detection was performed at 235 nm. The peak purity of Theophylline, Etofylline, Guaiphenesine and Ambroxol Hydrochloride were 0.99970, 0.99979, 0.99986 and 0.99949 respectively. Calibration curves were linear with correlation coefficient between 0.99995 to 0.99997 over a concentration range of 5 to 37 microg/mL for Theophylline, 19 to 140 microg/mL for Etofylline, 20 to 149 microg/mL for Guaiphenesine and 6 to 45 microg/mL for Ambroxol hydrochloride. The relative standard deviation (RSD) was found < 2.0%. The percentage recovery was found between the range of 98.6% and 100.5% at three different levels. Robustness and ruggedness were performed and result found within the RSD of 2%. All the parameters of validation were found in the acceptance range of ICH guideline.

High-Temperature Strain Sensing for Aerospace Applications

NASA Technical Reports Server (NTRS)

Piazza, Anthony; Richards, Lance W.; Hudson, Larry D.

2008-01-01

Thermal protection systems (TPS) and hot structures are utilizing advanced materials that operate at temperatures that exceed abilities to measure structural performance. Robust strain sensors that operate accurately and reliably beyond 1800 F are needed but do not exist. These shortcomings hinder the ability to validate analysis and modeling techniques and hinders the ability to optimize structural designs. This presentation examines high-temperature strain sensing for aerospace applications and, more specifically, seeks to provide strain data for validating finite element models and thermal-structural analyses. Efforts have been made to develop sensor attachment techniques for relevant structural materials at the small test specimen level and to perform laboratory tests to characterize sensor and generate corrections to apply to indicated strains. Areas highlighted in this presentation include sensors, sensor attachment techniques, laboratory evaluation/characterization of strain measurement, and sensor use in large-scale structures.

Novel Composites for Wing and Fuselage Applications. Task 1; Novel Wing Design Concepts

NASA Technical Reports Server (NTRS)

Suarez, J. A.; Buttitta, C.; Flanagan, G.; DeSilva, T.; Egensteiner, W.; Bruno, J.; Mahon, J.; Rutkowski, C.; Collins, R.; Fidnarick, R.;

Severity of anxiety and work-related outcomes of patients with anxiety disorders.

PubMed

Erickson, Steven R; Guthrie, Sally; Vanetten-Lee, Michelle; Himle, Joseph; Hoffman, Jody; Santos, Susana F; Janeck, Amy S; Zivin, Kara; Abelson, James L

2009-01-01

This study examined associations between anxiety and work-related outcomes in an anxiety disorders clinic population, examining both pretreatment links and the impact of anxiety change over 12 weeks of treatment on work outcomes. Four validated instruments were used to also allow examination of their psychometric properties, with the goal of improving measurement of work-related quality of life in this population. Newly enrolled adult patients seeking treatment in a university-based anxiety clinic were administered four work performance measures: Work Limitations Questionnaire (WLQ), Work Productivity and Activity Impairment Questionnaire (WPAI), Endicott Work Productivity Scale (EWPS), and Functional Status Questionnaire Work Performance Scale (WPS). Anxiety severity was determined using the Beck Anxiety Inventory (BAI). The Clinical Global Impressions, Global Improvement Scale (CGI-I) was completed by patients to evaluate symptom change at a 12-week follow-up. Two severity groups (minimal/mild vs. moderate/severe, based on baseline BAI score) were compared to each other on work measures. Eighty-one patients provided complete baseline data. Anxiety severity groups did not differ in job type, time on job, job satisfaction, or job choice. Patients with greater anxiety generally showed lower work performance on all instruments. Job advancement was impaired for the moderate/severe group. The multi-item performance scales demonstrated better validity and internal consistency. The WLQ and the WPAI detected change with symptom improvement. Level of work performance was generally associated with severity of anxiety. Of the instruments tested, the WLQ and the WPAI questionnaire demonstrated acceptable validity and internal reliability.

Three controversies over item disclosure in medical licensure examinations.

PubMed

Park, Yoon Soo; Yang, Eunbae B

2015-01-01

In response to views on public's right to know, there is growing attention to item disclosure - release of items, answer keys, and performance data to the public - in medical licensure examinations and their potential impact on the test's ability to measure competence and select qualified candidates. Recent debates on this issue have sparked legislative action internationally, including South Korea, with prior discussions among North American countries dating over three decades. The purpose of this study is to identify and analyze three issues associated with item disclosure in medical licensure examinations - 1) fairness and validity, 2) impact on passing levels, and 3) utility of item disclosure - by synthesizing existing literature in relation to standards in testing. Historically, the controversy over item disclosure has centered on fairness and validity. Proponents of item disclosure stress test takers' right to know, while opponents argue from a validity perspective. Item disclosure may bias item characteristics, such as difficulty and discrimination, and has consequences on setting passing levels. To date, there has been limited research on the utility of item disclosure for large scale testing. These issues requires ongoing and careful consideration.

The Quantitative Reasoning for College Science (QuaRCS) Assessment: Emerging Themes from 5 Years of Data

NASA Astrophysics Data System (ADS)

Follette, Katherine; Dokter, Erin; Buxner, Sanlyn

2018-01-01

The Quantitative Reasoning for College Science (QuaRCS) Assessment is a validated assessment instrument that was designed to measure changes in students' quantitative reasoning skills, attitudes toward mathematics, and ability to accurately assess their own quantitative abilities. It has been administered to more than 5,000 students at a variety of institutions at the start and end of a semester of general education college science instruction. I will begin by briefly summarizing our published work surrounding validation of the instrument and identification of underlying attitudinal factors (composite variables identified via factor analysis) that predict 50% of the variation in students' scores on the assessment. I will then discuss more recent unpublished work, including: (1) Development and validation of an abbreviated version of the assessment (The QuaRCS Light), which results in marked improvements in students' ability to maintain a high effort level throughout the assessment and has broad implications for quantitative reasoning assessments in general, and (2) Our efforts to revise the attitudinal portion of the assessment to better assess math anxiety level, another key factor in student performance on numerical assessments.

Validation of a liquid chromatography-electrospray ionization tandem mass spectrometric method to determine six polyether ionophores in raw, UHT, pasteurized and powdered milk.

PubMed

Pereira, Mararlene Ulberg; Spisso, Bernardete Ferraz; Jacob, Silvana do Couto; Monteiro, Mychelle Alves; Ferreira, Rosana Gomes; Carlos, Betânia de Souza; da Nóbrega, Armi Wanderley

2016-04-01

This study aimed to validate a method developed for the determination of six antibiotics from the polyether ionophore class (lasalocid, maduramicin, monensin, narasin, salinomycin and semduramicin) at residue levels in raw, UHT, pasteurized and powdered milk using QuEChERS extraction and high performance liquid chromatography coupled to tandem mass spectrometry (HPLC-MS/MS). The validation was conducted under an in-house laboratory protocol that is primarily based on 2002/657/EC Decision, but takes in account the variability of matrix sources. Overall recoveries between 93% and 113% with relative standard deviations up to 16% were obtained under intermediate precision conditions. CCα calculated values did not exceed 20% the Maximum Residue Limit for monensin and 25% the Maximum Levels for all other substances. The method showed to be simple, fast and suitable for verifying the compliance of raw and processed milk samples regarding the limits recommended by Codex Alimentarius and those adopted in European Community for polyether ionophores. Copyright © 2015 Elsevier Ltd. All rights reserved.

Virtual temporal bone dissection system: OSU virtual temporal bone system: development and testing.

PubMed

Wiet, Gregory J; Stredney, Don; Kerwin, Thomas; Hittle, Bradley; Fernandez, Soledad A; Abdel-Rasoul, Mahmoud; Welling, D Bradley

2012-03-01

The objective of this project was to develop a virtual temporal bone dissection system that would provide an enhanced educational experience for the training of otologic surgeons. A randomized, controlled, multi-institutional, single-blinded validation study. The project encompassed four areas of emphasis: structural data acquisition, integration of the system, dissemination of the system, and validation. Structural acquisition was performed on multiple imaging platforms. Integration achieved a cost-effective system. Dissemination was achieved on different levels including casual interest, downloading of software, and full involvement in development and validation studies. A validation study was performed at eight different training institutions across the country using a two-arm randomized trial where study subjects were randomized to a 2-week practice session using either the virtual temporal bone or standard cadaveric temporal bones. Eighty subjects were enrolled and randomized to one of the two treatment arms; 65 completed the study. There was no difference between the two groups using a blinded rating tool to assess performance after training. A virtual temporal bone dissection system has been developed and compared to cadaveric temporal bones for practice using a multicenter trial. There was no statistical difference between practice on the current simulator compared to practice on human cadaveric temporal bones. Further refinements in structural acquisition and interface design have been identified, which can be implemented prior to full incorporation into training programs and used for objective skills assessment. Copyright © 2012 The American Laryngological, Rhinological, and Otological Society, Inc.

Citizen science networks in natural history and the collective validation of biodiversity data.

PubMed

Turnhout, Esther; Lawrence, Anna; Turnhout, Sander

2016-06-01

Biodiversity data are in increasing demand to inform policy and management. A substantial portion of these data is generated in citizen science networks. To ensure the quality of biodiversity data, standards and criteria for validation have been put in place. We used interviews and document analysis from the United Kingdom and The Netherlands to examine how data validation serves as a point of connection between the diverse people and practices in natural history citizen science networks. We found that rather than a unidirectional imposition of standards, validation was performed collectively. Specifically, it was enacted in ongoing circulations of biodiversity records between recorders and validators as they jointly negotiated the biodiversity that was observed and the validity of the records. These collective validation practices contributed to the citizen science character or natural history networks and tied these networks together. However, when biodiversity records were included in biodiversity-information initiatives on different policy levels and scales, the circulation of records diminished. These initiatives took on a more extractive mode of data use. Validation ceased to be collective with important consequences for the natural history networks involved and citizen science more generally. © 2016 The Authors. Conservation Biology published by Wiley Periodicals, Inc. on behalf of Society for Conservation Biology.

Development of a refractive error quality of life scale for Thai adults (the REQ-Thai).

PubMed

Sukhawarn, Roongthip; Wiratchai, Nonglak; Tatsanavivat, Pyatat; Pitiyanuwat, Somwung; Kanato, Manop; Srivannaboon, Sabong; Guyatt, Gordon H

2011-08-01

To develop a scale for measuring refractive error quality of life (QOL) for Thai adults. The full survey comprised 424 respondents from 5 medical centers in Bangkok and from 3 medical centers in Chiangmai, Songkla and KhonKaen provinces. Participants were emmetropes and persons with refractive correction with visual acuity of 20/30 or better An item reduction process was employed by combining 3 methods-expert opinion, impact method and item-total correlation methods. The classical reliability testing and the validity testing including convergent, discriminative and construct validity was performed. The developed questionnaire comprised 87 items in 6 dimensions: 1) quality of vision, 2) visual function, 3) social function, 4) psychological function, 5) symptoms and 6) refractive correction problems. It is the 5-level Likert scale type. The Cronbach's Alpha coefficients of its dimensions ranged from 0.756 to 0. 979. All validity testing were shown to be valid. The construct validity was validated by the confirmatory factor analysis. A short version questionnaire comprised 48 items with good reliability and validity was also developed. This is the first validated instrument for measuring refractive error quality of life for Thai adults that was developed with strong research methodology and large sample size.

From Board to Bedside: How the Application of Financial Structures to Safety and Quality Can Drive Accountability in a Large Health Care System.

PubMed

Austin, J Matthew; Demski, Renee; Callender, Tiffany; Lee, K H Ken; Hoffman, Ann; Allen, Lisa; Radke, Deborah A; Kim, Yungjin; Werthman, Ronald J; Peterson, Ronald R; Pronovost, Peter J

2017-04-01

As the health care system in the United States places greater emphasis on the public reporting of quality and safety data and its use to determine payment, provider organizations must implement structures that ensure discipline and rigor regarding these data. An academic health system, as part of a performance management system, applied four key components of a financial reporting structure to support the goal of top-to-bottom accountability for improving quality and safety. The four components implemented by Johns Hopkins Medicine were governance, accountability, reporting of consolidated quality performance statements, and auditing. Governance is provided by the health system's Patient Safety and Quality Board Committee, which reviews goals and strategy for patient safety and quality, reviews quarterly performance for each entity, and holds organizational leaders accountable for performance. An accountability plan includes escalating levels of review corresponding to the number of months an entity misses the defined performance target for a measure. A consolidated quality statement helps inform the Patient Safety and Quality Board Committee and leadership on key quality and safety issues. An audit evaluates the efficiency and effectiveness of processes for data collection, validation, and storage, as to ensure the accuracy and completeness of quality measure reporting. If hospitals and health systems truly want to prioritize improvements in safety and quality, they will need to create a performance management system that ensures data validity and supports performance accountability. Without valid data, it is difficult to know whether a performance gap is due to data quality or clinical quality. Copyright © 2017 The Joint Commission. Published by Elsevier Inc. All rights reserved.

Sensor Applications and Data Validation

NASA Technical Reports Server (NTRS)

Wiley, John

2008-01-01

The mechanical configuration of automobiles have changed marginally while improvements in sensors and control have dramatically improved engine efficiency, reliability and useful life. The aviation industry has also taken advantage of sensors and control systems to reduce operational costs. Sensors and high fidelity control systems fly planes at levels of performance beyond human capability. Sophisticated environmental controls allow a greater level of comfort and efficiency in our homes. Sensors have given the medical field a better understanding of the human body and the environment in which we live.

Far-Field Acoustic Power Level and Performance Analyses of F31/A31 Open Rotor Model at Simulated Scaled Takeoff, Nominal Takeoff, and Approach Conditions: Technical Report I

NASA Technical Reports Server (NTRS)

Sree, Dave

2015-01-01

Far-field acoustic power level and performance analyses of open rotor model F31/A31 have been performed to determine its noise characteristics at simulated scaled takeoff, nominal takeoff, and approach flight conditions. The nonproprietary parts of the data obtained from experiments in 9- by 15-Foot Low-Speed Wind Tunnel (9?15 LSWT) tests were provided by NASA Glenn Research Center to perform the analyses. The tone and broadband noise components have been separated from raw test data by using a new data analysis tool. Results in terms of sound pressure levels, acoustic power levels, and their variations with rotor speed, angle of attack, thrust, and input shaft power have been presented and discussed. The effect of an upstream pylon on the noise levels of the model has been addressed. Empirical equations relating model's acoustic power level, thrust, and input shaft power have been developed. The far-field acoustic efficiency of the model is also determined for various simulated flight conditions. It is intended that the results presented in this work will serve as a database for comparison and improvement of other open rotor blade designs and also for validating open rotor noise prediction codes.

Validation of simple indexes to assess insulin sensitivity during pregnancy in Wistar and Sprague-Dawley rats.

PubMed

Cacho, J; Sevillano, J; de Castro, J; Herrera, E; Ramos, M P

2008-11-01

Insulin resistance plays a role in the pathogenesis of diabetes, including gestational diabetes. The glucose clamp is considered the gold standard for determining in vivo insulin sensitivity, both in human and in animal models. However, the clamp is laborious, time consuming and, in animals, requires anesthesia and collection of multiple blood samples. In human studies, a number of simple indexes, derived from fasting glucose and insulin levels, have been obtained and validated against the glucose clamp. However, these indexes have not been validated in rats and their accuracy in predicting altered insulin sensitivity remains to be established. In the present study, we have evaluated whether indirect estimates based on fasting glucose and insulin levels are valid predictors of insulin sensitivity in nonpregnant and 20-day-pregnant Wistar and Sprague-Dawley rats. We have analyzed the homeostasis model assessment of insulin resistance (HOMA-IR), the quantitative insulin sensitivity check index (QUICKI), and the fasting glucose-to-insulin ratio (FGIR) by comparing them with the insulin sensitivity (SI(Clamp)) values obtained during the hyperinsulinemic-isoglycemic clamp. We have performed a calibration analysis to evaluate the ability of these indexes to accurately predict insulin sensitivity as determined by the reference glucose clamp. Finally, to assess the reliability of these indexes for the identification of animals with impaired insulin sensitivity, performance of the indexes was analyzed by receiver operating characteristic (ROC) curves in Wistar and Sprague-Dawley rats. We found that HOMA-IR, QUICKI, and FGIR correlated significantly with SI(Clamp), exhibited good sensitivity and specificity, accurately predicted SI(Clamp), and yielded lower insulin sensitivity in pregnant than in nonpregnant rats. Together, our data demonstrate that these indexes provide an easy and accurate measure of insulin sensitivity during pregnancy in the rat.

Experimental validation of the influence of white matter anisotropy on the intracranial EEG forward solution

PubMed Central

Schomer, Donald L.; Dehghani, Nima; Ulbert, Istvan; Cash, Sydney; Papavasiliou, Steve; Eisenberg, Solomon R.; Dale, Anders M.; Halgren, Eric

2010-01-01

Forward solutions with different levels of complexity are employed for localization of current generators, which are responsible for the electric and magnetic fields measured from the human brain. The influence of brain anisotropy on the forward solution is poorly understood. The goal of this study is to validate an anisotropic model for the intracranial electric forward solution by comparing with the directly measured ‘gold standard’. Dipolar sources are created at known locations in the brain and intracranial electroencephalogram (EEG) is recorded simultaneously. Isotropic models with increasing level of complexity are generated along with anisotropic models based on Diffusion tensor imaging (DTI). A Finite Element Method based forward solution is calculated and validated using the measured data. Major findings are (1) An anisotropic model with a linear scaling between the eigenvalues of the electrical conductivity tensor and water self-diffusion tensor in brain tissue is validated. The greatest improvement was obtained when the stimulation site is close to a region of high anisotropy. The model with a global anisotropic ratio of 10:1 between the eigenvalues (parallel: tangential to the fiber direction) has the worst performance of all the anisotropic models. (2) Inclusion of cerebrospinal fluid as well as brain anisotropy in the forward model is necessary for an accurate description of the electric field inside the skull. The results indicate that an anisotropic model based on the DTI can be constructed non-invasively and shows an improved performance when compared to the isotropic models for the calculation of the intracranial EEG forward solution. Electronic supplementary material The online version of this article (doi:10.1007/s10827-009-0205-z) contains supplementary material, which is available to authorized users. PMID:20063051

Performance of basic kinematic thresholds in the identification of crash and near-crash events within naturalistic driving data.

PubMed

Perez, Miguel A; Sudweeks, Jeremy D; Sears, Edie; Antin, Jonathan; Lee, Suzanne; Hankey, Jonathan M; Dingus, Thomas A

2017-06-01

Understanding causal factors for traffic safety-critical events (e.g., crashes and near-crashes) is an important step in reducing their frequency and severity. Naturalistic driving data offers unparalleled insight into these factors, but requires identification of situations where crashes are present within large volumes of data. Sensitivity and specificity of these identification approaches are key to minimizing the resources required to validate candidate crash events. This investigation used data from the Second Strategic Highway Research Program Naturalistic Driving Study (SHRP 2 NDS) and the Canada Naturalistic Driving Study (CNDS) to develop and validate different kinematic thresholds that can be used to detect crash events. Results indicate that the sensitivity of many of these approaches can be quite low, but can be improved by selecting particular threshold levels based on detection performance. Additional improvements in these approaches are possible, and may involve leveraging combinations of different detection approaches, including advanced statistical techniques and artificial intelligence approaches, additional parameter modifications, and automation of validation processes. Copyright © 2017 Elsevier Ltd. All rights reserved.

Validation of a reversed-phase high-performance liquid chromatographic method for the determination of free amino acids in rice using l-theanine as the internal standard.

PubMed

Liyanaarachchi, G V V; Mahanama, K R R; Somasiri, H P P S; Punyasiri, P A N

2018-02-01

The study presents the validation results of the method carried out for analysis of free amino acids (FAAs) in rice using l-theanine as the internal standard (IS) with o-phthalaldehyde (OPA) reagent using high-performance liquid chromatography-fluorescence detection. The detection and quantification limits of the method were in the range 2-16μmol/kg and 3-19μmol/kg respectively. The method had a wide working range from 25 to 600μmol/kg for each individual amino acid, and good linearity with regression coefficients greater than 0.999. Precision measured in terms of repeatability and reproducibility, expressed as percentage relative standard deviation (% RSD) was below 9% for all the amino acids analyzed. The recoveries obtained after fortification at three concentration levels were in the range 75-105%. In comparison to l-norvaline, findings revealed that l-theanine is suitable as an IS and the validated method can be used for FAA determination in rice. Copyright © 2017 Elsevier Ltd. All rights reserved.

Development and validation of an algorithm for laser application in wound treatment 1

PubMed Central

da Cunha, Diequison Rite; Salomé, Geraldo Magela; Massahud, Marcelo Renato; Mendes, Bruno; Ferreira, Lydia Masako

2017-01-01

ABSTRACT Objective: To develop and validate an algorithm for laser wound therapy. Method: Methodological study and literature review. For the development of the algorithm, a review was performed in the Health Sciences databases of the past ten years. The algorithm evaluation was performed by 24 participants, nurses, physiotherapists, and physicians. For data analysis, the Cronbach’s alpha coefficient and the chi-square test for independence was used. The level of significance of the statistical test was established at 5% (p<0.05). Results: The professionals’ responses regarding the facility to read the algorithm indicated: 41.70%, great; 41.70%, good; 16.70%, regular. With regard the algorithm being sufficient for supporting decisions related to wound evaluation and wound cleaning, 87.5% said yes to both questions. Regarding the participants’ opinion that the algorithm contained enough information to support their decision regarding the choice of laser parameters, 91.7% said yes. The questionnaire presented reliability using the Cronbach’s alpha coefficient test (α = 0.962). Conclusion: The developed and validated algorithm showed reliability for evaluation, wound cleaning, and use of laser therapy in wounds. PMID:29211197

Development and validation of a high-performance liquid chromatography method for the quantification of talazoparib in rat plasma: Application to plasma protein binding studies.

PubMed

Hidau, Mahendra Kumar; Kolluru, Srikanth; Palakurthi, Srinath

2018-02-01

A sensitive and selective RP-HPLC method has been developed and validated for the quantification of a highly potent poly ADP ribose polymerase inhibitor talazoparib (TZP) in rat plasma. Chromatographic separation was performed with isocratic elution method. Absorbance for TZP was measured with a UV detector (SPD-20A UV-vis) at a λ max of 227 nm. Protein precipitation was used to extract the drug from plasma samples using methanol-acetonitrile (65:35) as the precipitating solvent. The method proved to be sensitive and reproducible over a 100-2000 ng/mL linearity range with a lower limit of quantification (LLQC) of 100 ng/mL. TZP recovery was found to be >85%. Following analytical method development and validation, it was successfully employed to determine the plasma protein binding of TZP. TZP has a high level of protein binding in rat plasma (95.76 ± 0.38%) as determined by dialysis method. Copyright © 2017 John Wiley & Sons, Ltd.

High-Bandwidth Tactical-Network Data Analysis in a High-Performance-Computing (HPC) Environment: Packet-Level Analysis

DTIC Science & Technology

2015-09-01

with a collection of information if it does not display a currently valid OMB control number. PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS . 1...UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS (ES) Technical and Project Engineering, LLC QED Systems, LLC Alexandria, VA...AND ADDRESS (ES) US Army Research Laboratory ATTN: RDRL-CIH-C Aberdeen Proving Ground, MD 21005 10. SPONSOR/MONITOR’S ACRONYM(S) 11. SPONSOR

Multi-Model Validation in the Chesapeake Bay Region During Frontier Sentinel 2010

DTIC Science & Technology

2012-09-28

which a 72-hr forecast took approximately 1 hr. Identical runs were performed on the DoD Supercomputing Resources Center (DSRC) host “ DaVinci ” at the...performance Navy DSRC host DaVinci . Products of water level and horizontal current maps as well as station time series, identical to those produced by the...forecast meteorological fields. The NCOM simulations were run daily on 128 CPUs at the Navy DSRC host DaVinci and required approximately 5 hrs of wall

High Power Alternator Test Unit (ATU) Electrical System Test

NASA Technical Reports Server (NTRS)

Birchenough, Arthur; Hervol, David

2007-01-01

The Alternator Test Unit (ATU) in the Lunar Power System Facility (LPSF) located at the NASA Glenn Research Center (GRC) in Cleveland, OH was used to simulate the operating conditions and evaluate the performance of the ATU and it s interaction with various LPSF components in accordance with the JIMO AC Power System Requirements. The testing was carried out at the breadboard development level. Results of these tests will be used for the development and validation of analytical models for performance and lifetime prediction.

Impact of the Dayton Tire case.

PubMed

Hart, D L; Isernhagen, S J; Matheson, L N

1998-01-01

Three of the more pertinent legal cases in the United States concerning the performance of ergonomists are summarized. The results of the cited cases have impact on the validity of the NIOSH lifting formulae, the lack of scientific evidence relating performance of jobs with alleged ergonomic stressors with specific medical pathology, and the gold standard for expert witness testimony. The cases, taken together, should act as a catalyst for ergonomists to improve their level of scientific justification for their work and conclusions.

Vented Versus Unvented Chest Seals for Treatment of Pneumothorax and Prevention of Tension Pneumothorax in a Swine Model

DTIC Science & Technology

2013-07-01

person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control ...were obtained. Data Analysis Data are expressed as mean T SEM. Between- group anal- ysis (Halo vs. Bolin) was performed using two-way repeated...measures analysis of variance. Bonferroni post test was used to compare replicate means at each PTx level. Within- group (treat- ment) analysis was performed

Puerto Rican understandings of child disability: methods for the cultural validation of standardized measures of child health.

PubMed

Gannotti, Mary E; Handwerker, W Penn

2002-12-01

Validating the cultural context of health is important for obtaining accurate and useful information from standardized measures of child health adapted for cross-cultural applications. This paper describes the application of ethnographic triangulation for cultural validation of a measure of childhood disability, the Pediatric Evaluation of Disability Inventory (PEDI) for use with children living in Puerto Rico. The key concepts include macro-level forces such as geography, demography, and economics, specific activities children performed and their key social interactions, beliefs, attitudes, emotions, and patterns of behavior surrounding independence in children and childhood disability, as well as the definition of childhood disability. Methods utilize principal components analysis to establish the validity of cultural concepts and multiple regression analysis to identify intracultural variation. Findings suggest culturally specific modifications to the PEDI, provide contextual information for informed interpretation of test scores, and point to the need to re-standardize normative values for use with Puerto Rican children. Without this type of information, Puerto Rican children may appear more disabled than expected for their level of impairment or not to be making improvements in functional status. The methods also allow for cultural boundaries to be quantitatively established, rather than presupposed. Copyright 2002 Elsevier Science Ltd.

Sample size determination for disease prevalence studies with partially validated data.

PubMed

Qiu, Shi-Fang; Poon, Wai-Yin; Tang, Man-Lai

2016-02-01

Disease prevalence is an important topic in medical research, and its study is based on data that are obtained by classifying subjects according to whether a disease has been contracted. Classification can be conducted with high-cost gold standard tests or low-cost screening tests, but the latter are subject to the misclassification of subjects. As a compromise between the two, many research studies use partially validated datasets in which all data points are classified by fallible tests, and some of the data points are validated in the sense that they are also classified by the completely accurate gold-standard test. In this article, we investigate the determination of sample sizes for disease prevalence studies with partially validated data. We use two approaches. The first is to find sample sizes that can achieve a pre-specified power of a statistical test at a chosen significance level, and the second is to find sample sizes that can control the width of a confidence interval with a pre-specified confidence level. Empirical studies have been conducted to demonstrate the performance of various testing procedures with the proposed sample sizes. The applicability of the proposed methods are illustrated by a real-data example. © The Author(s) 2012.

Real-time sensor validation and fusion for distributed autonomous sensors

NASA Astrophysics Data System (ADS)

Yuan, Xiaojing; Li, Xiangshang; Buckles, Bill P.

2004-04-01

Multi-sensor data fusion has found widespread applications in industrial and research sectors. The purpose of real time multi-sensor data fusion is to dynamically estimate an improved system model from a set of different data sources, i.e., sensors. This paper presented a systematic and unified real time sensor validation and fusion framework (RTSVFF) based on distributed autonomous sensors. The RTSVFF is an open architecture which consists of four layers - the transaction layer, the process fusion layer, the control layer, and the planning layer. This paradigm facilitates distribution of intelligence to the sensor level and sharing of information among sensors, controllers, and other devices in the system. The openness of the architecture also provides a platform to test different sensor validation and fusion algorithms and thus facilitates the selection of near optimal algorithms for specific sensor fusion application. In the version of the model presented in this paper, confidence weighted averaging is employed to address the dynamic system state issue noted above. The state is computed using an adaptive estimator and dynamic validation curve for numeric data fusion and a robust diagnostic map for decision level qualitative fusion. The framework is then applied to automatic monitoring of a gas-turbine engine, including a performance comparison of the proposed real-time sensor fusion algorithms and a traditional numerical weighted average.

Hierarchical calibration and validation of computational fluid dynamics models for solid sorbent-based carbon capture

DOE Office of Scientific and Technical Information (OSTI.GOV)

Lai, Canhai; Xu, Zhijie; Pan, Wenxiao

2016-01-01

To quantify the predictive confidence of a solid sorbent-based carbon capture design, a hierarchical validation methodology—consisting of basic unit problems with increasing physical complexity coupled with filtered model-based geometric upscaling has been developed and implemented. This paper describes the computational fluid dynamics (CFD) multi-phase reactive flow simulations and the associated data flows among different unit problems performed within the said hierarchical validation approach. The bench-top experiments used in this calibration and validation effort were carefully designed to follow the desired simple-to-complex unit problem hierarchy, with corresponding data acquisition to support model parameters calibrations at each unit problem level. A Bayesianmore » calibration procedure is employed and the posterior model parameter distributions obtained at one unit-problem level are used as prior distributions for the same parameters in the next-tier simulations. Overall, the results have demonstrated that the multiphase reactive flow models within MFIX can be used to capture the bed pressure, temperature, CO2 capture capacity, and kinetics with quantitative accuracy. The CFD modeling methodology and associated uncertainty quantification techniques presented herein offer a solid framework for estimating the predictive confidence in the virtual scale up of a larger carbon capture device.« less

Ability Testing for Job Selection: Are the Economic Claims Justified?

ERIC Educational Resources Information Center

Levin, Henry M.

The use of ability testing for job selection has become widespread in the Federal Government and in the U.S. Employment Service, which assists private sector employers. The justification for the practice is based largely on research findings claiming a high level of validity for such tests in predicting job performance. More recently, such claims…

An Exploratory Factor Analysis of the Sexual Orientation Counselor Competency Scale: Examining the Variable of Experience

ERIC Educational Resources Information Center

Ali, Shainna; Lambie, Glenn; Bloom, Zachary D.

2017-01-01

The Sexual Orientation Counselor Competency Scale (SOCCS), developed by Bidell in 2005, measures counselors' levels of skills, awareness, and knowledge in assisting lesbian, gay, or bisexual (LGB) clients. In an effort to gain an increased understanding of the construct validity of the SOCCS, researchers performed an exploratory factor analysis on…

25 CFR 900.121 - What happens during the preplanning phase and can an Indian tribe or tribal organization perform...

Code of Federal Regulations, 2010 CFR

2010-04-01

... specific information regarding the type of project to be funded, the objective criteria that will be used... application that least meet the application criteria. (3) Validation. Before final acceptance of a ranked application, the information, such as demographic information, deficiency levels reported in application, the...

Systems Concepts and Computer-Managed Instruction: An Implementation and Validation Study.

ERIC Educational Resources Information Center

Dick, Walter; Gallagher, Paul

The Florida State model of computer-managed instruction (CMI) differs from other such models in that it assumes a student will achieve his maximum performance level by interacting directly with the computer in order to evaluate his learning experience. In this system the computer plays the role of real-time diagnostician and prescriber for the…

Constructing and Evaluating a Validity Argument for the Final-Year Ward Simulation Exercise

ERIC Educational Resources Information Center

Till, Hettie; Ker, Jean; Myford, Carol; Stirling, Kevin; Mires, Gary

2015-01-01

The authors report final-year ward simulation data from the University of Dundee Medical School. Faculty who designed this assessment intend for the final score to represent an individual senior medical student's level of clinical performance. The results are included in each student's portfolio as one source of evidence of the student's…