The use of computers for perioperative simulation in anesthesia, critical care, and pain medicine.

PubMed

Lambden, Simon; Martin, Bruce

2011-09-01

Simulation in perioperative anesthesia training is a field of considerable interest, with an urgent need for tools that reliably train and facilitate objective assessment of performance. This article reviews the available simulation technologies, their evolution, and the current evidence base for their use. The future directions for research in the field and potential applications of simulation technology in anesthesia, critical care, and pain medicine are discussed. Copyright © 2011 Elsevier Inc. All rights reserved.

Excellent reliability of the Hamilton Depression Rating Scale (HDRS-21) in Indonesia after training.

PubMed

Istriana, Erita; Kurnia, Ade; Weijers, Annelies; Hidayat, Teddy; Pinxten, Lucas; de Jong, Cor; Schellekens, Arnt

2013-09-01

The Hamilton Depression Rating Scale (HDRS) is the most widely used depression rating scale worldwide. Reliability of HDRS has been reported mainly from Western countries. The current study tested the reliability of HDRS ratings among psychiatric residents in Indonesia, before and after HDRS training. The hypotheses were that: (i) prior to the training reliability of HDRS ratings is poor; and (ii) HDRS training can improve reliability of HDRS ratings to excellent levels. Furthermore, we explored cultural validity at item level. Videotaped HDRS interviews were rated by 30 psychiatric residents before and after 1 day of HDRS training. Based on a gold standard rating, percentage correct ratings and deviation from the standard were calculated. Correct ratings increased from 83% to 99% at item level and from 70% to 100% for the total rating. The average deviation from the gold standard rating improved from 0.07 to 0.02 at item level and from 2.97 to 0.46 for the total rating. HDRS assessment by psychiatric trainees in Indonesia without prior training is unreliable. A short, evidence-based HDRS training improves reliability to near perfect levels. The outlined training program could serve as a template for HDRS trainings. HDRS items that may be less valid for assessment of depression severity in Indonesia are discussed. Copyright © 2013 Wiley Publishing Asia Pty Ltd.

Maximum Entropy Discrimination Poisson Regression for Software Reliability Modeling.

PubMed

Chatzis, Sotirios P; Andreou, Andreas S

2015-11-01

Reliably predicting software defects is one of the most significant tasks in software engineering. Two of the major components of modern software reliability modeling approaches are: 1) extraction of salient features for software system representation, based on appropriately designed software metrics and 2) development of intricate regression models for count data, to allow effective software reliability data modeling and prediction. Surprisingly, research in the latter frontier of count data regression modeling has been rather limited. More specifically, a lack of simple and efficient algorithms for posterior computation has made the Bayesian approaches appear unattractive, and thus underdeveloped in the context of software reliability modeling. In this paper, we try to address these issues by introducing a novel Bayesian regression model for count data, based on the concept of max-margin data modeling, effected in the context of a fully Bayesian model treatment with simple and efficient posterior distribution updates. Our novel approach yields a more discriminative learning technique, making more effective use of our training data during model inference. In addition, it allows of better handling uncertainty in the modeled data, which can be a significant problem when the training data are limited. We derive elegant inference algorithms for our model under the mean-field paradigm and exhibit its effectiveness using the publicly available benchmark data sets.

Requirements for diagnosis of malaria at different levels of the laboratory network in Africa.

PubMed

Long, Earl G

2009-06-01

The rapid increase of resistance to cheap, reliable antimalarials, the increasing cost of effective drugs, and the low specificity of clinical diagnosis has increased the need for more reliable diagnostic methods for malaria. The most commonly used and most reliable remains microscopic examination of stained blood smears, but this technique requires skilled personnel, precision instruments, and ideally a source of electricity. Microscopy has the advantage of enabling the examiner to identify the species, stage, and density of an infection. An alternative to microscopy is the rapid diagnostic test (RDT), which uses a labeled monoclonal antibody to detect circulating parasitic antigens. This test is most commonly used to detect Plasmodium falciparum infections and is available in a plastic cassette format. Both microscopy and RDTs should be available at all levels of laboratory service in endemic areas, but in peripheral laboratories with minimally trained staff, the RDT may be a more practical diagnostic method.

The Americleft Speech Project: A Training and Reliability Study.

PubMed

Chapman, Kathy L; Baylis, Adriane; Trost-Cardamone, Judith; Cordero, Kelly Nett; Dixon, Angela; Dobbelsteyn, Cindy; Thurmes, Anna; Wilson, Kristina; Harding-Bell, Anne; Sweeney, Triona; Stoddard, Gregory; Sell, Debbie

2016-01-01

To describe the results of two reliability studies and to assess the effect of training on interrater reliability scores. The first study (1) examined interrater and intrarater reliability scores (weighted and unweighted kappas) and (2) compared interrater reliability scores before and after training on the use of the Cleft Audit Protocol for Speech-Augmented (CAPS-A) with British English-speaking children. The second study examined interrater and intrarater reliability on a modified version of the CAPS-A (CAPS-A Americleft Modification) with American and Canadian English-speaking children. Finally, comparisons were made between the interrater and intrarater reliability scores obtained for Study 1 and Study 2. The participants were speech-language pathologists from the Americleft Speech Project. In Study 1, interrater reliability scores improved for 6 of the 13 parameters following training on the CAPS-A protocol. Comparison of the reliability results for the two studies indicated lower scores for Study 2 compared with Study 1. However, this appeared to be an artifact of the kappa statistic that occurred due to insufficient variability in the reliability samples for Study 2. When percent agreement scores were also calculated, the ratings appeared similar across Study 1 and Study 2. The findings of this study suggested that improvements in interrater reliability could be obtained following a program of systematic training. However, improvements were not uniform across all parameters. Acceptable levels of reliability were achieved for those parameters most important for evaluation of velopharyngeal function.

The Americleft Speech Project: A Training and Reliability Study

PubMed Central

Chapman, Kathy L.; Baylis, Adriane; Trost-Cardamone, Judith; Cordero, Kelly Nett; Dixon, Angela; Dobbelsteyn, Cindy; Thurmes, Anna; Wilson, Kristina; Harding-Bell, Anne; Sweeney, Triona; Stoddard, Gregory; Sell, Debbie

2017-01-01

Objective To describe the results of two reliability studies and to assess the effect of training on interrater reliability scores. Design The first study (1) examined interrater and intrarater reliability scores (weighted and unweighted kappas) and (2) compared interrater reliability scores before and after training on the use of the Cleft Audit Protocol for Speech–Augmented (CAPS-A) with British English-speaking children. The second study examined interrater and intrarater reliability on a modified version of the CAPS-A (CAPS-A Americleft Modification) with American and Canadian English-speaking children. Finally, comparisons were made between the interrater and intrarater reliability scores obtained for Study 1 and Study 2. Participants The participants were speech-language pathologists from the Americleft Speech Project. Results In Study 1, interrater reliability scores improved for 6 of the 13 parameters following training on the CAPS-A protocol. Comparison of the reliability results for the two studies indicated lower scores for Study 2 compared with Study 1. However, this appeared to be an artifact of the kappa statistic that occurred due to insufficient variability in the reliability samples for Study 2. When percent agreement scores were also calculated, the ratings appeared similar across Study 1 and Study 2. Conclusion The findings of this study suggested that improvements in interrater reliability could be obtained following a program of systematic training. However, improvements were not uniform across all parameters. Acceptable levels of reliability were achieved for those parameters most important for evaluation of velopharyngeal function. PMID:25531738

Cross-organism learning method to discover new gene functionalities.

PubMed

Domeniconi, Giacomo; Masseroli, Marco; Moro, Gianluca; Pinoli, Pietro

2016-04-01

Knowledge of gene and protein functions is paramount for the understanding of physiological and pathological biological processes, as well as in the development of new drugs and therapies. Analyses for biomedical knowledge discovery greatly benefit from the availability of gene and protein functional feature descriptions expressed through controlled terminologies and ontologies, i.e., of gene and protein biomedical controlled annotations. In the last years, several databases of such annotations have become available; yet, these valuable annotations are incomplete, include errors and only some of them represent highly reliable human curated information. Computational techniques able to reliably predict new gene or protein annotations with an associated likelihood value are thus paramount. Here, we propose a novel cross-organisms learning approach to reliably predict new functionalities for the genes of an organism based on the known controlled annotations of the genes of another, evolutionarily related and better studied, organism. We leverage a new representation of the annotation discovery problem and a random perturbation of the available controlled annotations to allow the application of supervised algorithms to predict with good accuracy unknown gene annotations. Taking advantage of the numerous gene annotations available for a well-studied organism, our cross-organisms learning method creates and trains better prediction models, which can then be applied to predict new gene annotations of a target organism. We tested and compared our method with the equivalent single organism approach on different gene annotation datasets of five evolutionarily related organisms (Homo sapiens, Mus musculus, Bos taurus, Gallus gallus and Dictyostelium discoideum). Results show both the usefulness of the perturbation method of available annotations for better prediction model training and a great improvement of the cross-organism models with respect to the single-organism ones, without influence of the evolutionary distance between the considered organisms. The generated ranked lists of reliably predicted annotations, which describe novel gene functionalities and have an associated likelihood value, are very valuable both to complement available annotations, for better coverage in biomedical knowledge discovery analyses, and to quicken the annotation curation process, by focusing it on the prioritized novel annotations predicted. Copyright © 2015 Elsevier Ireland Ltd. All rights reserved.

Operator adaptation to changes in system reliability under adaptable automation.

PubMed

Chavaillaz, Alain; Sauer, Juergen

2017-09-01

This experiment examined how operators coped with a change in system reliability between training and testing. Forty participants were trained for 3 h on a complex process control simulation modelling six levels of automation (LOA). In training, participants either experienced a high- (100%) or low-reliability system (50%). The impact of training experience on operator behaviour was examined during a 2.5 h testing session, in which participants either experienced a high- (100%) or low-reliability system (60%). The results showed that most operators did not often switch between LOA. Most chose an LOA that relieved them of most tasks but maintained their decision authority. Training experience did not have a strong impact on the outcome measures (e.g. performance, complacency). Low system reliability led to decreased performance and self-confidence. Furthermore, complacency was observed under high system reliability. Overall, the findings suggest benefits of adaptable automation because it accommodates different operator preferences for LOA. Practitioner Summary: The present research shows that operators can adapt to changes in system reliability between training and testing sessions. Furthermore, it provides evidence that each operator has his/her preferred automation level. Since this preference varies strongly between operators, adaptable automation seems to be suitable to accommodate these large differences.

Web-based training and interrater reliability testing for scoring the Hamilton Depression Rating Scale.

PubMed

Rosen, Jules; Mulsant, Benoit H; Marino, Patricia; Groening, Christopher; Young, Robert C; Fox, Debra

2008-10-30

Despite the importance of establishing shared scoring conventions and assessing interrater reliability in clinical trials in psychiatry, these elements are often overlooked. Obstacles to rater training and reliability testing include logistic difficulties in providing live training sessions, or mailing videotapes of patients to multiple sites and collecting the data for analysis. To address some of these obstacles, a web-based interactive video system was developed. It uses actors of diverse ages, gender and race to train raters how to score the Hamilton Depression Rating Scale and to assess interrater reliability. This system was tested with a group of experienced and novice raters within a single site. It was subsequently used to train raters of a federally funded multi-center clinical trial on scoring conventions and to test their interrater reliability. The advantages and limitations of using interactive video technology to improve the quality of clinical trials are discussed.

A Study on the Reliability of Sasang Constitutional Body Trunk Measurement

PubMed Central

Jang, Eunsu; Kim, Jong Yeol; Lee, Haejung; Kim, Honggie; Baek, Younghwa; Lee, Siwoo

2012-01-01

Objective. Body trunk measurement for human plays an important diagnostic role not only in conventional medicine but also in Sasang constitutional medicine (SCM). The Sasang constitutional body trunk measurement (SCBTM) consists of the 5-widths and the 8-circumferences which are standard locations currently employed in the SCM society. This study suggests to what extent a comprehensive training can improve the reliability of the SCBTM. Methods. We recruited 10 male subjects and 5 male observers with no experience of anthropometric measurement. We conducted measurements twice before and after a comprehensive training. Relative technical error of measurement (%TEMs) was produced to assess intra and inter observer reliabilities. Results. Post-training intra-observer %TEMs of the SCBTM were 0.27% to 1.85% reduced from 0.27% to 6.26% in pre-training, respectively. Post-training inter-observer %TEMs of those were 0.56% to 1.66% reduced from 1.00% to 9.60% in pre-training, respectively. Post-training % total TEMs which represent the whole reliability were 0.68% to 2.18% reduced from maximum value of 10.18%. Conclusion. A comprehensive training makes the SCBTM more reliable, hence giving a sufficiently confident diagnostic tool. It is strongly recommended to give a comprehensive training in advance to take the SCBTM. PMID:21822442

Do dogs sense hypoglycaemia?

PubMed

Weber, K S; Roden, M; Müssig, K

2016-07-01

To summarize the current knowledge on the phenomenon of dogs, both trained and untrained, sensing hypoglycaemia and alerting their owners to it. Electronic databases were searched for all types of articles reporting on untrained or trained 'diabetes alert' dogs. Articles published up until December 2014 in the English or German language were included. Several case reports and observational studies provide evidence that animals can perform at a level above that attributable to chance, and may reliably detect low diurnal as well as nocturnal hypoglycaemic episodes. Behavioural changes in untrained dogs were reported during 38-100% of hypoglycaemic events experienced by their owners. The sensitivity and specificity of the performance of trained diabetes alert dogs sensing hypoglycaemia ranged from 22 to 100% and 71 to 90%, respectively. Additionally, 75-81% of patients with diabetes who owned a trained dog reported a subsequent improvement in their quality of life. Nevertheless, the available data are limited and heterogeneous because they rely on low patient numbers and survey-based studies prone to recall bias. Further research is needed to confirm the preliminary data on the reliability and mechanism underlying the dogs' abilities to detect hypoglycaemia, and its impact on patient outcomes. © 2015 Diabetes UK.

Calculus detection calibration among dental hygiene faculty members utilizing dental endoscopy: a pilot study.

PubMed

Partido, Brian B; Jones, Archie A; English, Dana L; Nguyen, Carol A; Jacks, Mary E

2015-02-01

Dental and dental hygiene faculty members often do not provide consistent instruction in the clinical environment, especially in tasks requiring clinical judgment. From previous efforts to calibrate faculty members in calculus detection using typodonts, researchers have suggested using human subjects and emerging technology to improve consistency in clinical instruction. The purpose of this pilot study was to determine if a dental endoscopy-assisted training program would improve intra- and interrater reliability of dental hygiene faculty members in calculus detection. Training included an ODU 11/12 explorer, typodonts, and dental endoscopy. A convenience sample of six participants was recruited from the dental hygiene faculty at a California community college, and a two-group randomized experimental design was utilized. Intra- and interrater reliability was measured before and after calibration training. Pretest and posttest Kappa averages of all participants were compared using repeated measures (split-plot) ANOVA to determine the effectiveness of the calibration training on intra- and interrater reliability. The results showed that both kinds of reliability significantly improved for all participants and the training group improved significantly in interrater reliability from pretest to posttest. Calibration training was beneficial to these dental hygiene faculty members, especially those beginning with less than full agreement. This study suggests that calculus detection calibration training utilizing dental endoscopy can effectively improve interrater reliability of dental and dental hygiene clinical educators. Future studies should include human subjects, involve more participants at multiple locations, and determine whether improved rater reliability can be sustained over time.

DSM-5 field trials in the United States and Canada, Part I: study design, sampling strategy, implementation, and analytic approaches.

PubMed

Clarke, Diana E; Narrow, William E; Regier, Darrel A; Kuramoto, S Janet; Kupfer, David J; Kuhl, Emily A; Greiner, Lisa; Kraemer, Helena C

2013-01-01

This article discusses the design,sampling strategy, implementation,and data analytic processes of the DSM-5 Field Trials. The DSM-5 Field Trials were conducted by using a test-retest reliability design with a stratified sampling approach across six adult and four pediatric sites in the United States and one adult site in Canada. A stratified random sampling approach was used to enhance precision in the estimation of the reliability coefficients. A web-based research electronic data capture system was used for simultaneous data collection from patients and clinicians across sites and for centralized data management.Weighted descriptive analyses, intraclass kappa and intraclass correlation coefficients for stratified samples, and receiver operating curves were computed. The DSM-5 Field Trials capitalized on advances since DSM-III and DSM-IV in statistical measures of reliability (i.e., intraclass kappa for stratified samples) and other recently developed measures to determine confidence intervals around kappa estimates. Diagnostic interviews using DSM-5 criteria were conducted by 279 clinicians of varied disciplines who received training comparable to what would be available to any clinician after publication of DSM-5.Overall, 2,246 patients with various diagnoses and levels of comorbidity were enrolled,of which over 86% were seen for two diagnostic interviews. A range of reliability coefficients were observed for the categorical diagnoses and dimensional measures. Multisite field trials and training comparable to what would be available to any clinician after publication of DSM-5 provided “real-world” testing of DSM-5 proposed diagnoses.

Reliability of anthropometric measurements in European preschool children: the ToyBox-study.

PubMed

De Miguel-Etayo, P; Mesana, M I; Cardon, G; De Bourdeaudhuij, I; Góźdź, M; Socha, P; Lateva, M; Iotova, V; Koletzko, B V; Duvinage, K; Androutsos, O; Manios, Y; Moreno, L A

2014-08-01

The ToyBox-study aims to develop and test an innovative and evidence-based obesity prevention programme for preschoolers in six European countries: Belgium, Bulgaria, Germany, Greece, Poland and Spain. In multicentre studies, anthropometric measurements using standardized procedures that minimize errors in the data collection are essential to maximize reliability of measurements. The aim of this paper is to describe the standardization process and reliability (intra- and inter-observer) of height, weight and waist circumference (WC) measurements in preschoolers. All technical procedures and devices were standardized and centralized training was given to the fieldworkers. At least seven children per country participated in the intra- and inter-observer reliability testing. Intra-observer technical error ranged from 0.00 to 0.03 kg for weight and from 0.07 to 0.20 cm for height, with the overall reliability being above 99%. A second training was organized for WC due to low reliability observed in the first training. Intra-observer technical error for WC ranged from 0.12 to 0.71 cm during the first training and from 0.05 to 1.11 cm during the second training, and reliability above 92% was achieved. Epidemiological surveys need standardized procedures and training of researchers to reduce measurement error. In the ToyBox-study, very good intra- and-inter-observer agreement was achieved for all anthropometric measurements performed. © 2014 World Obesity.

Analyses of inter-rater reliability between professionals, medical students and trained school children as assessors of basic life support skills.

PubMed

Beck, Stefanie; Ruhnke, Bjarne; Issleib, Malte; Daubmann, Anne; Harendza, Sigrid; Zöllner, Christian

2016-10-07

Training of lay-rescuers is essential to improve survival-rates after cardiac arrest. Multiple campaigns emphasise the importance of basic life support (BLS) training for school children. Trainings require a valid assessment to give feedback to school children and to compare the outcomes of different training formats. Considering these requirements, we developed an assessment of BLS skills using MiniAnne and tested the inter-rater reliability between professionals, medical students and trained school children as assessors. Fifteen professional assessors, 10 medical students and 111-trained school children (peers) assessed 1087 school children at the end of a CPR-training event using the new assessment format. Analyses of inter-rater reliability (intraclass correlation coefficient; ICC) were performed. Overall inter-rater reliability of the summative assessment was high (ICC = 0.84, 95 %-CI: 0.84 to 0.86, n = 889). The number of comparisons between peer-peer assessors (n = 303), peer-professional assessors (n = 339), and peer-student assessors (n = 191) was adequate to demonstrate high inter-rater reliability between peer- and professional-assessors (ICC: 0.76), peer- and student-assessors (ICC: 0.88) and peer- and other peer-assessors (ICC: 0.91). Systematic variation in rating of specific items was observed for three items between professional- and peer-assessors. Using this assessment and integrating peers and medical students as assessors gives the opportunity to assess hands-on skills of school children with high reliability.

Training improves interobserver reliability for the diagnosis of scaphoid fracture displacement.

PubMed

Buijze, Geert A; Guitton, Thierry G; van Dijk, C Niek; Ring, David

2012-07-01

The diagnosis of displacement in scaphoid fractures is notorious for poor interobserver reliability. We tested whether training can improve interobserver reliability and sensitivity, specificity, and accuracy for the diagnosis of scaphoid fracture displacement on radiographs and CT scans. Sixty-four orthopaedic surgeons rated a set of radiographs and CT scans of 10 displaced and 10 nondisplaced scaphoid fractures for the presence of displacement, using a web-based rating application. Before rating, observers were randomized to a training group (34 observers) and a nontraining group (30 observers). The training group received an online training module before the rating session, and the nontraining group did not. Interobserver reliability for training and nontraining was assessed by Siegel's multirater kappa and the Z-test was used to test for significance. There was a small, but significant difference in the interobserver reliability for displacement ratings in favor of the training group compared with the nontraining group. Ratings of radiographs and CT scans combined resulted in moderate agreement for both groups. The average sensitivity, specificity, and accuracy of diagnosing displacement of scaphoid fractures were, respectively, 83%, 85%, and 84% for the nontraining group and 87%, 86%, and 87% for the training group. Assuming a 5% prevalence of fracture displacement, the positive predictive value was 0.23 in the nontraining group and 0.25 in the training group. The negative predictive value was 0.99 in both groups. Our results suggest training can improve interobserver reliability and sensitivity, specificity and accuracy for the diagnosis of scaphoid fracture displacement, but the improvements are slight. These findings are encouraging for future research regarding interobserver variation and how to reduce it further.

Issues in development, evaluation, and use of the NASA Preflight Adaptation Trainer (PAT)

NASA Technical Reports Server (NTRS)

Lane, Norman E.; Kennedy, Robert S.

1988-01-01

The Preflight Adaptation Trainer (PAT) is intended to reduce or alleviate space adaptation syndrome by providing opportunities for portions of that adaptation to occur under normal gravity conditions prior to space flight. Since the adaptation aspects of the PAT objectives involve modification not only of the behavior of the trainee, but also of sensiomotor skills which underly the behavioral generation, the defining of training objectives of the PAT utilizes four mechanisms: familiarization, demonstration, training and adaptation. These mechanisms serve as structural reference points for evaluation, drive the content and organization of the training procedures, and help to define the roles of the PAT instructors and operators. It was determined that three psychomotor properties are most critical for PAT evaluation: reliability; sensitivity; and relevance. It is cause for concern that the number of measures available to examine PAT effects exceed those that can be properly studied with the available sample sizes; special attention will be required in selection of the candidate measure set. The issues in PAT use and application within a training system context are addressed through linking the three training related mechanisms of familiarization, demonstration and training to the fourth mechanism, adaptation.

Training and quality assurance with the Structured Clinical Interview for DSM-IV (SCID-I/P).

PubMed

Ventura, J; Liberman, R P; Green, M F; Shaner, A; Mintz, J

1998-06-15

Accuracy in psychiatric diagnosis is critical for evaluating the suitability of the subjects for entry into research protocols and for establishing comparability of findings across study sites. However, training programs in the use of diagnostic instruments for research projects are not well systematized. Furthermore, little information has been published on the maintenance of interrater reliability of diagnostic assessments. At the UCLA Research Center for Major Mental Illnesses, a Training and Quality Assurance Program for SCID interviewers was used to evaluate interrater reliability and diagnostic accuracy. Although clinically experienced interviewers achieved better interrater reliability and overall diagnostic accuracy than neophyte interviewers, both groups were able to achieve and maintain high levels of interrater reliability, diagnostic accuracy, and interviewer skill. At the first quality assurance check after training, there were no significant differences between experienced and neophyte interviewers in interrater reliability or diagnostic accuracy. Standardization of training and quality assurance procedures within and across research projects may make research findings from study sites more comparable.

Genomic prediction using imputed whole-genome sequence data in Holstein Friesian cattle.

PubMed

van Binsbergen, Rianne; Calus, Mario P L; Bink, Marco C A M; van Eeuwijk, Fred A; Schrooten, Chris; Veerkamp, Roel F

2015-09-17

In contrast to currently used single nucleotide polymorphism (SNP) panels, the use of whole-genome sequence data is expected to enable the direct estimation of the effects of causal mutations on a given trait. This could lead to higher reliabilities of genomic predictions compared to those based on SNP genotypes. Also, at each generation of selection, recombination events between a SNP and a mutation can cause decay in reliability of genomic predictions based on markers rather than on the causal variants. Our objective was to investigate the use of imputed whole-genome sequence genotypes versus high-density SNP genotypes on (the persistency of) the reliability of genomic predictions using real cattle data. Highly accurate phenotypes based on daughter performance and Illumina BovineHD Beadchip genotypes were available for 5503 Holstein Friesian bulls. The BovineHD genotypes (631,428 SNPs) of each bull were used to impute whole-genome sequence genotypes (12,590,056 SNPs) using the Beagle software. Imputation was done using a multi-breed reference panel of 429 sequenced individuals. Genomic estimated breeding values for three traits were predicted using a Bayesian stochastic search variable selection (BSSVS) model and a genome-enabled best linear unbiased prediction model (GBLUP). Reliabilities of predictions were based on 2087 validation bulls, while the other 3416 bulls were used for training. Prediction reliabilities ranged from 0.37 to 0.52. BSSVS performed better than GBLUP in all cases. Reliabilities of genomic predictions were slightly lower with imputed sequence data than with BovineHD chip data. Also, the reliabilities tended to be lower for both sequence data and BovineHD chip data when relationships between training animals were low. No increase in persistency of prediction reliability using imputed sequence data was observed. Compared to BovineHD genotype data, using imputed sequence data for genomic prediction produced no advantage. To investigate the putative advantage of genomic prediction using (imputed) sequence data, a training set with a larger number of individuals that are distantly related to each other and genomic prediction models that incorporate biological information on the SNPs or that apply stricter SNP pre-selection should be considered.

New Resources for Collecting Psychological Conditions Information

DTIC Science & Technology

2007-12-01

about personnel security investigations, and presents additional recommendations to assist investigators who must collect psychological conditions...After it appears in the Monitor, add the article by the American Psychological Association’s ethics director to the investigator training program. 3...person concept. Available, reliable information about the person, past and present , favorable and unfavorable, should be considered in reaching a

Training induces scapular dyskinesis in pain-free competitive swimmers: a reliability and observational study.

PubMed

Madsen, Pernille H; Bak, Klaus; Jensen, Susanne; Welter, Ulrik

2011-03-01

Scapular dyskinesis is a major etiological factor in overhead athletes' shoulder problems. Our hypotheses were to evaluate if (1) visual observation of scapular dyskinesis during scaption has substantial interobserver reliability, and (2) scapular dyskinesis may be induced by swim training in pain-free swimmers. A reliability and observational study. Bachelor project at a college institution and at a private sports orthopedic hospital. Seventy-eight competitive swimmers with no history of shoulder pain were included in the study. Fourteen swimmers were evaluated regarding reliability. Inclusion criteria were competitive swimmers with high training volume who previously had no shoulder pain. Observations of scapular dyskinesis (yes/no) during simple scaption. The interobserver reliability of scaption and wall push-up was evaluated in 14 swimmers using kappa analysis. Prevalence of scapular dyskinesis at 4 time intervals during a swim training session. The scaption test resulted in a weighted kappa value of 0.75. Scapular dyskinesis was seen in 29 shoulders (37%) after the first time interval, in another 24 (cumulated prevalence 68%) after one-half of the training session, and in an additional 4 swimmers (cumulated prevalence 73%) after three-quarters of the training session. During the last quarter of the training session, another 7 swimmers had dyskinesis, resulting in a cumulated prevalence of 82%. The prevalence of abnormal scapular kinesis during a normal training session is high in previously pain-free swimmers. The prevalence increases with more training and occurs early during the training session.

Reliability of Interaural Time Difference-Based Localization Training in Elderly Individuals with Speech-in-Noise Perception Disorder.

PubMed

Delphi, Maryam; Lotfi, M-Yones; Moossavi, Abdollah; Bakhshi, Enayatollah; Banimostafa, Maryam

2017-09-01

Previous studies have shown that interaural-time-difference (ITD) training can improve localization ability. Surprisingly little is, however, known about localization training vis-à-vis speech perception in noise based on interaural time difference in the envelope (ITD ENV). We sought to investigate the reliability of an ITD ENV-based training program in speech-in-noise perception among elderly individuals with normal hearing and speech-in-noise disorder. The present interventional study was performed during 2016. Sixteen elderly men between 55 and 65 years of age with the clinical diagnosis of normal hearing up to 2000 Hz and speech-in-noise perception disorder participated in this study. The training localization program was based on changes in ITD ENV. In order to evaluate the reliability of the training program, we performed speech-in-noise tests before the training program, immediately afterward, and then at 2 months' follow-up. The reliability of the training program was analyzed using the Friedman test and the SPSS software. Significant statistical differences were shown in the mean scores of speech-in-noise perception between the 3 time points (P=0.001). The results also indicated no difference in the mean scores of speech-in-noise perception between the 2 time points of immediately after the training program and 2 months' follow-up (P=0.212). The present study showed the reliability of an ITD ENV-based localization training in elderly individuals with speech-in-noise perception disorder.

Human Reliability Analysis in Support of Risk Assessment for Positive Train Control

DOT National Transportation Integrated Search

2003-06-01

This report describes an approach to evaluating the reliability of human actions that are modeled in a probabilistic risk assessment : (PRA) of train control operations. This approach to human reliability analysis (HRA) has been applied in the case o...

Reliability of Single-Leg Balance and Landing Tests in Rugby Union; Prospect of Using Postural Control to Monitor Fatigue.

PubMed

Troester, Jordan C; Jasmin, Jason G; Duffield, Rob

2018-06-01

The present study examined the inter-trial (within test) and inter-test (between test) reliability of single-leg balance and single-leg landing measures performed on a force plate in professional rugby union players using commercially available software (SpartaMARS, Menlo Park, USA). Twenty-four players undertook test - re-test measures on two occasions (7 days apart) on the first training day of two respective pre-season weeks following 48h rest and similar weekly training loads. Two 20s single-leg balance trials were performed on a force plate with eyes closed. Three single-leg landing trials were performed by jumping off two feet and landing on one foot in the middle of a force plate 1m from the starting position. Single-leg balance results demonstrated acceptable inter-trial reliability (ICC = 0.60-0.81, CV = 11-13%) for sway velocity, anterior-posterior sway velocity, and mediolateral sway velocity variables. Acceptable inter-test reliability (ICC = 0.61-0.89, CV = 7-13%) was evident for all variables except mediolateral sway velocity on the dominant leg (ICC = 0.41, CV = 15%). Single-leg landing results only demonstrated acceptable inter-trial reliability for force based measures of relative peak landing force and impulse (ICC = 0.54-0.72, CV = 9-15%). Inter-test results indicate improved reliability through the averaging of three trials with force based measures again demonstrating acceptable reliability (ICC = 0.58-0.71, CV = 7-14%). Of the variables investigated here, total sway velocity and relative landing impulse are the most reliable measures of single-leg balance and landing performance, respectively. These measures should be considered for monitoring potential changes in postural control in professional rugby union.

The influence of applying insurance medicine guidelines for depression on disability assessments

PubMed Central

2013-01-01

Background In the current study we report on the effects of an implementation strategy in the form of a training programme on the assessed work limitations of a client with depression by insurance physicians (IPs) participating in a RCT. These assessed work limitations of a client were in the form of scores on the List of Functional Abilities (LFA). Method We conducted a randomised controlled trial (RCT) for IPs in which we compared the intervention of a specially developed training programme with the usual methods of implementation and training currently used. The outcome was the mean sum score and the inter-rater reliability (Intraclass Correlation Coefficient, ICC) of the LFA scores. These LFA scores were scored by the IPs participating in the RCT for the work limitations of the cases presented in different videos, two videos before the training and two after the training of the intervention group. Results At baseline, the intervention group (IG) consisted of 21 IPs and the control group (CG) of 19. For one participant of the IG and for one of the CG the LFAs of the two case reports after training were not available. Before training the sum scores for the first case report did not differ significantly between the groups, while the mean sum score was higher in the IG than in the CG for the second case report. For both case reports after training a higher score was found in the IG than in the CG. The inter-rater reliability measured for the two case reports before training was about the same in the IG and the CG: 0.64 and 0.65, respectively. For the two case reports after training, the ICC was higher in the IG than in the CG: 0.69 and 0.54, respectively. This difference was not significant however. Conclusion It would appear that the implementation of a specially designed training programme on guidelines for depression may lead to greater inter-rater reliability in the assessments by insurance physicians of the work limitations of clients with depression. It is, however, important to note that insurance physicians who receive training may find more work limitations than those who do not. Trial registration Netherlands’ Trial Register NTR1863 PMID:23758690

The reliability of in-training assessment when performance improvement is taken into account.

PubMed

van Lohuizen, Mirjam T; Kuks, Jan B M; van Hell, Elisabeth A; Raat, A N; Stewart, Roy E; Cohen-Schotanus, Janke

2010-12-01

During in-training assessment students are frequently assessed over a longer period of time and therefore it can be expected that their performance will improve. We studied whether there really is a measurable performance improvement when students are assessed over an extended period of time and how this improvement affects the reliability of the overall judgement. In-training assessment results were obtained from 104 students on rotation at our university hospital or at one of the six affiliated hospitals. Generalisability theory was used in combination with multilevel analysis to obtain reliability coefficients and to estimate the number of assessments needed for reliable overall judgement, both including and excluding performance improvement. Students' clinical performance ratings improved significantly from a mean of 7.6 at the start to a mean of 7.8 at the end of their clerkship. When taking performance improvement into account, reliability coefficients were higher. The number of assessments needed to achieve a reliability of 0.80 or higher decreased from 17 to 11. Therefore, when studying reliability of in-training assessment, performance improvement should be considered.

Grant Peer Review: Improving Inter-Rater Reliability with Training

DOE PAGES

Sattler, David N.; McKnight, Patrick E.; Naney, Linda; ...

2015-06-15

In this study, we developed and evaluated a brief training program for grant reviewers that aimed to increase inter-rater reliability, rating scale knowledge, and effort to read the grant review criteria. Enhancing reviewer training may improve the reliability and accuracy of research grant proposal scoring and funding recommendations. Seventy-five Public Health professors from U.S. research universities watched the training video we produced and assigned scores to the National Institutes of Health scoring criteria proposal summary descriptions. For both novice and experienced reviewers, the training video increased scoring accuracy (the percentage of scores that reflect the true rating scale values), inter-ratermore » reliability, and the amount of time reading the review criteria compared to the no video condition. The increase in reliability for experienced reviewers is notable because it is commonly assumed that reviewers—especially those with experience—have good understanding of the grant review rating scale. Our findings suggest that both experienced and novice reviewers who had not received the type of training developed in this study may not have appropriate understanding of the definitions and meaning for each value of the rating scale and that experienced reviewers may overestimate their knowledge of the rating scale. Lastly, the results underscore the benefits of and need for specialized peer reviewer training.« less

Grant Peer Review: Improving Inter-Rater Reliability with Training.

PubMed

Sattler, David N; McKnight, Patrick E; Naney, Linda; Mathis, Randy

2015-01-01

This study developed and evaluated a brief training program for grant reviewers that aimed to increase inter-rater reliability, rating scale knowledge, and effort to read the grant review criteria. Enhancing reviewer training may improve the reliability and accuracy of research grant proposal scoring and funding recommendations. Seventy-five Public Health professors from U.S. research universities watched the training video we produced and assigned scores to the National Institutes of Health scoring criteria proposal summary descriptions. For both novice and experienced reviewers, the training video increased scoring accuracy (the percentage of scores that reflect the true rating scale values), inter-rater reliability, and the amount of time reading the review criteria compared to the no video condition. The increase in reliability for experienced reviewers is notable because it is commonly assumed that reviewers--especially those with experience--have good understanding of the grant review rating scale. The findings suggest that both experienced and novice reviewers who had not received the type of training developed in this study may not have appropriate understanding of the definitions and meaning for each value of the rating scale and that experienced reviewers may overestimate their knowledge of the rating scale. The results underscore the benefits of and need for specialized peer reviewer training.

75 FR 35689 - System Personnel Training Reliability Standards

Federal Register 2010, 2011, 2012, 2013, 2014

2010-06-23

... using realistic simulations.\\14\\ \\13\\ Id. P 1331. \\14\\ Reliability Standard PER-002-0. 9. In Order No... development process to: (1) Include formal training requirements for reliability coordinators similar to those... simulation technology such as a simulator, virtual technology, or other technology in their emergency...

[Multiple mini interviews before the occupation of main training posts in paediatrics].

PubMed

Hertel, Niels Thomas; Bjerager, Mia; Boas, Malene; Boisen, Kirsten A; Børch, Klaus; Frederiksen, Marianne Sjølin; Holm, Kirsten; Grum-Nymann, Anette; Johnsen, Martin M; Whitehouse, Stine; Balslev, Thomas

2013-09-09

Interviews are mandatory in Denmark when selecting doctors for training positions. We used multiple mini interviews (MMI) at four recruitment rounds for the main training posts in paediatrics. In total, 125 candidates were evaluated and assessed by CV and MMI (4-5 stations). Reliability for individual stations in MMI assessed by Cronbach's alpha was adequate (0.63-0.92). The overall reliability assessed by G-theory was lower, suggesting that different skills were tested. The acceptability was high. Our experiences with MMI suggest good feasibility and reliability. An increasing number of stations may improve the overall reliability.

The relative reliability of actively participating and passively observing raters in a simulation-based assessment for selection to specialty training in anaesthesia.

PubMed

Roberts, M J; Gale, T C E; Sice, P J A; Anderson, I R

2013-06-01

Selection to specialty training is a high-stakes assessment demanding valuable consultant time. In one initial entry level and two higher level anaesthesia selection centres, we investigated the feasibility of using staff participating in simulation scenarios, rather than observing consultants, to rate candidate performance. We compared participant and observer scores using four different outcomes: inter-rater reliability; score distributions; correlation of candidate rankings; and percentage of candidates whose selection might be affected by substituting participants' for observers' ratings. Inter-rater reliability between observers was good (correlation coefficient 0.73-0.96) but lower between participants (correlation coefficient 0.39-0.92), particularly at higher level where participants also rated candidates more favourably than did observers. Station rank orderings were strongly correlated between the rater groups at entry level (rho 0.81, p < 0.001) but weaker at the two higher level centres (rho 0.52, p = 0.018; rho 0.58, p = 0.001). Substituting participants' for observers' ratings had less effect once scores were combined with those from other selection centre stations. Selection decisions for 0-20% of candidates could have changed, depending on the numbers of training posts available. We conclude that using participating raters is feasible at initial entry level only. Anaesthesia © 2013 The Association of Anaesthetists of Great Britain and Ireland.

Evaluating information skills training in health libraries: a systematic review.

PubMed

Brettle, Alison

2007-12-01

Systematic reviews have shown that there is limited evidence to demonstrate that the information literacy training health librarians provide is effective in improving clinicians' information skills or has an impact on patient care. Studies lack measures which demonstrate validity and reliability in evaluating the impact of training. To determine what measures have been used; the extent to which they are valid and reliable; to provide guidance for health librarians who wish to evaluate the impact of their information skills training. Systematic review methodology involved searching seven databases, and personal files. Studies were included if they were about information skills training, used an objective measure to assess outcomes, and occurred in a health setting. Fifty-four studies were included in the review. Most outcome measures used in the studies were not tested for the key criteria of validity and reliability. Three tested for validity and reliability are described in more detail. Selecting an appropriate measure to evaluate the impact of training is a key factor in carrying out any evaluation. This systematic review provides guidance to health librarians by highlighting measures used in various circumstances, and those that demonstrate validity and reliability.

[Training of interviewers in the utilization of standardized questionnaires in psychiatry: studies realized with the Present State Examination (PSE)].

PubMed

Lesage, A D; Cyr, M; Toupin, J; Cormier, H; Valiquette, C

1991-01-01

Interview questionnaires offer more validity than self-administered format in exploring psychopathological or psychosocial phenomena of interest in psychiatric research. If used, special care needs to be paid to interviewers' training and ensuring that they maintain their reliability. No widespread training standards exist and each schedule may carry its own procedure. Our aims are to indicate how we trained interviewers with the French version of the Present State Examination (Wing, Cooper and Sartorius, 1974) and how we checked and kept acceptable interraters reliability during one study. We will provide data on the interraters reliability during the training and the study, as well as the test-retest reliability. These results will be used to support some guidelines when using this sort of psychiatric research questionnaires in order to ensure comparability both within the study and between studies.

Reliability of pelvic floor measurements on three- and four-dimensional ultrasound during and after first pregnancy: implications for training.

PubMed

van Veelen, G A; Schweitzer, K J; van der Vaart, C H

2013-11-01

To evaluate the reliability of measurements of the levator hiatus and levator-urethra gap (LUG) using three/four-dimensional (3D/4D) transperineal ultrasound in women during their first pregnancy and 6 months postpartum, and to assess the learning process for these measurements. An inexperienced observer was taught to perform measurements of the levator hiatus and LUG by an experienced observer. After training, 3D/4D ultrasound volume datasets of 40 women in the first trimester were analyzed by these two observers. Another training session then took place and both observers repeated the analyses of the same volume datasets. Finally, analyses of 40 volume datasets of the women 6 months postpartum were performed by both observers. Intra- and interobserver reliability were determined by intraclass correlation coefficients (ICC) with 95% CIs. For levator hiatal measurements, in the women during their first pregnancy the interobserver reliability was substantial to almost perfect after both the first and second training session (ICC, 0.62-0.83 and 0.71-0.89, respectively, for anteroposterior diameter, transverse diameter and area at rest, on contraction and on Valsalva) and the intraobserver reliability was substantial to almost perfect for both observers. For these measurements performed once the women had delivered, interobserver reliability was moderate to almost perfect. For LUG measurements performed during pregnancy, interobserver reliability was slight to moderate after the first training session (ICC, 0.14-0.54), but improved after the second training session (ICC, 0.38-0.71), and intraobserver reliability was moderate to substantial for the experienced observer and slight to moderate for the inexperienced observer. For these measurements performed when the women had delivered, interobserver reliability was fair to moderate. The levator hiatus and LUG can be measured reliably using 3D/4D ultrasound in primigravid and primiparous women. The technique to measure dimensions of the levator hiatus requires limited teaching, but LUG measurements are more difficult and require more extensive training. Copyright © 2013 ISUOG. Published by John Wiley & Sons Ltd.

Employment-Based Training in Japanese Firms in Japan and in the United States: Experiences of Automobile Manufacturers.

ERIC Educational Resources Information Center

Hashimoto, Masanori

An economic theory of training holds that training in technical skills and training in employment relations (namely, information reliability or the ability to quickly and reliably disseminate information among the members of the firm) reinforce each other. This theory is an organizing framework for understanding some practices at Japanese firms in…

Roland: A Case for or Against NATO Standardization?

DTIC Science & Technology

1980-05-01

with often competing, even opposing, objectives in testing, financial auditing , cost estimating, reliability, value engineering, maintenance, training...supposedly mature system. Multilocation tests, early in the program when test beds and spare parts availability would be at a minimum, would require...Similar institutionalized conflicts resided in the audit community, which, under the Armed Services Procurement Regulation, was required to audit and

Machine Learning Algorithms for Automatic Classification of Marmoset Vocalizations

PubMed Central

Ribeiro, Sidarta; Pereira, Danillo R.; Papa, João P.; de Albuquerque, Victor Hugo C.

2016-01-01

Automatic classification of vocalization type could potentially become a useful tool for acoustic the monitoring of captive colonies of highly vocal primates. However, for classification to be useful in practice, a reliable algorithm that can be successfully trained on small datasets is necessary. In this work, we consider seven different classification algorithms with the goal of finding a robust classifier that can be successfully trained on small datasets. We found good classification performance (accuracy > 0.83 and F1-score > 0.84) using the Optimum Path Forest classifier. Dataset and algorithms are made publicly available. PMID:27654941

Effect of rater training on reliability and accuracy of mini-CEX scores: a randomized, controlled trial.

PubMed

Cook, David A; Dupras, Denise M; Beckman, Thomas J; Thomas, Kris G; Pankratz, V Shane

2009-01-01

Mini-CEX scores assess resident competence. Rater training might improve mini-CEX score interrater reliability, but evidence is lacking. Evaluate a rater training workshop using interrater reliability and accuracy. Randomized trial (immediate versus delayed workshop) and single-group pre/post study (randomized groups combined). Academic medical center. Fifty-two internal medicine clinic preceptors (31 randomized and 21 additional workshop attendees). The workshop included rater error training, performance dimension training, behavioral observation training, and frame of reference training using lecture, video, and facilitated discussion. Delayed group received no intervention until after posttest. Mini-CEX ratings at baseline (just before workshop for workshop group), and four weeks later using videotaped resident-patient encounters; mini-CEX ratings of live resident-patient encounters one year preceding and one year following the workshop; rater confidence using mini-CEX. Among 31 randomized participants, interrater reliabilities in the delayed group (baseline intraclass correlation coefficient [ICC] 0.43, follow-up 0.53) and workshop group (baseline 0.40, follow-up 0.43) were not significantly different (p = 0.19). Mean ratings were similar at baseline (delayed 4.9 [95% confidence interval 4.6-5.2], workshop 4.8 [4.5-5.1]) and follow-up (delayed 5.4 [5.0-5.7], workshop 5.3 [5.0-5.6]; p = 0.88 for interaction). For the entire cohort, rater confidence (1 = not confident, 6 = very confident) improved from mean (SD) 3.8 (1.4) to 4.4 (1.0), p = 0.018. Interrater reliability for ratings of live encounters (entire cohort) was higher after the workshop (ICC 0.34) than before (ICC 0.18) but the standard error of measurement was similar for both periods. Rater training did not improve interrater reliability or accuracy of mini-CEX scores. clinicaltrials.gov identifier NCT00667940

Evaluating training of screening, brief intervention, and referral to treatment (SBIRT) for substance use: Reliability of the MD3 SBIRT Coding Scale.

PubMed

DiClemente, Carlo C; Crouch, Taylor Berens; Norwood, Amber E Q; Delahanty, Janine; Welsh, Christopher

2015-03-01

Screening, brief intervention, and referral to treatment (SBIRT) has become an empirically supported and widely implemented approach in primary and specialty care for addressing substance misuse. Accordingly, training of providers in SBIRT has increased exponentially in recent years. However, the quality and fidelity of training programs and subsequent interventions are largely unknown because of the lack of SBIRT-specific evaluation tools. The purpose of this study was to create a coding scale to assess quality and fidelity of SBIRT interactions addressing alcohol, tobacco, illicit drugs, and prescription medication misuse. The scale was developed to evaluate performance in an SBIRT residency training program. Scale development was based on training protocol and competencies with consultation from Motivational Interviewing coding experts. Trained medical residents practiced SBIRT with standardized patients during 10- to 15-min videotaped interactions. This study included 25 tapes from the Family Medicine program coded by 3 unique coder pairs with varying levels of coding experience. Interrater reliability was assessed for overall scale components and individual items via intraclass correlation coefficients. Coder pair-specific reliability was also assessed. Interrater reliability was excellent overall for the scale components (>.85) and nearly all items. Reliability was higher for more experienced coders, though still adequate for the trained coder pair. Descriptive data demonstrated a broad range of adherence and skills. Subscale correlations supported concurrent and discriminant validity. Data provide evidence that the MD3 SBIRT Coding Scale is a psychometrically reliable coding system for evaluating SBIRT interactions and can be used to evaluate implementation skills for fidelity, training, assessment, and research. Recommendations for refinement and further testing of the measure are discussed. (PsycINFO Database Record (c) 2015 APA, all rights reserved).

The ESRC: A Web-based Environmental Satellite Resource Center

NASA Astrophysics Data System (ADS)

Abshire, W. E.; Guarente, B.; Dills, P. N.

2009-12-01

The COMET® Program has developed an Environmental Satellite Resource Center (known as the ESRC), a searchable, database-driven Website that provides easy access to a wide range of useful information training materials on polar-orbiting and geostationary satellites. Primarily sponsored by the NPOESS Program and NOAA, the ESRC is a tool for users seeking reliable sources of satellite information, training, and data. First published in September 2008, and upgraded in April 2009, the site is freely available at: http://www.meted.ucar.edu/esrc. Additional contributions to the ESRC are sought and made on an ongoing basis. The ESRC was created in response to a broad community request first made in May 2006. The COMET Program was asked to develop the site to consolidate and simplify access to reliable, current, and diverse information, training materials, and data associated with environmental satellites. The ESRC currently includes over 400 significant resources from NRL, CIMSS, CIRA, NASA, VISIT, NESDIS, and EUMETSAT, and improves access to the numerous satellite resources available from COMET’s MetEd Website. The ESRC is designed as a community site where organizations and individuals around the globe can easily submit their resources via online forms by providing a small set of metadata. The ESRC supports languages other than English and multi-lingual character sets have been tested. COMET’s role is threefold: 1) maintain the site, 2) populate it with our own materials, including smaller, focused learning objects derived from our larger training modules, and 3) provide the necessary quality assurance and monitoring to ensure that all resources are appropriate and well described before being made available. Our presentation will demonstrate many of the features and functionality of searching for resources using the ESRC, and will outline the steps for users to make their own submissions. For the site to reach its full potential, submissions representing diverse interests and intended for diverse audiences are strongly encouraged.

Developing a method for specifying the components of behavior change interventions in practice: the example of smoking cessation.

PubMed

Lorencatto, Fabiana; West, Robert; Seymour, Natalie; Michie, Susan

2013-06-01

There is a difference between interventions as planned and as delivered in practice. Unless we know what was actually delivered, we cannot understand "what worked" in effective interventions. This study aimed to (a) assess whether an established taxonomy of 53 smoking cessation behavior change techniques (BCTs) may be applied or adapted as a method for reliably specifying the content of smoking cessation behavioral support consultations and (b) develop an effective method for training researchers and practitioners in the reliable application of the taxonomy. Fifteen transcripts of audio-recorded consultations delivered by England's Stop Smoking Services were coded into component BCTs using the taxonomy. Interrater reliability and potential adaptations to the taxonomy to improve coding were discussed following 3 coding waves. A coding training manual was developed through expert consensus and piloted on 10 trainees, assessing coding reliability and self-perceived competence before and after training. An average of 33 BCTs from the taxonomy were identified at least once across sessions and coding waves. Consultations contained on average 12 BCTs (range = 8-31). Average interrater reliability was high (88% agreement). The taxonomy was adapted to simplify coding by merging co-occurring BCTs and refining BCT definitions. Coding reliability and self-perceived competence significantly improved posttraining for all trainees. It is possible to apply a taxonomy to reliably identify and classify BCTs in smoking cessation behavioral support delivered in practice, and train inexperienced coders to do so reliably. This method can be used to investigate variability in provision of behavioral support across services, monitor fidelity of delivery, and identify training needs.

Large-screen display technology assessment for military applications

NASA Astrophysics Data System (ADS)

Blaha, Richard J.

1990-08-01

Full-color, large screen display systems can enhance military applications that require group presentation, coordinated decisions, or interaction between decision makers. The technology already plays an important role in operations centers, simulation facilities, conference rooms, and training centers. Some applications display situational, status, or briefing information, while others portray instructional material for procedural training or depict realistic panoramic scenes that are used in simulators. While each specific application requires unique values of luminance, resolution, response time, reliability, and the video interface, suitable performance can be achieved with available commercial large screen displays. Advances in the technology of large screen displays are driven by the commercial applications because the military applications do not provide the significant market share enjoyed by high definition television (HDTV), entertainment, advertisement, training, and industrial applications. This paper reviews the status of full-color, large screen display technologies and includes the performance and cost metrics of available systems. For this discussion, performance data is based upon either measurements made by our personnel or extractions from vendors' data sheets.

A rater training protocol to assess team performance.

PubMed

Eppich, Walter; Nannicelli, Anna P; Seivert, Nicholas P; Sohn, Min-Woong; Rozenfeld, Ranna; Woods, Donna M; Holl, Jane L

2015-01-01

Simulation-based methodologies are increasingly used to assess teamwork and communication skills and provide team training. Formative feedback regarding team performance is an essential component. While effective use of simulation for assessment or training requires accurate rating of team performance, examples of rater-training programs in health care are scarce. We describe our rater training program and report interrater reliability during phases of training and independent rating. We selected an assessment tool shown to yield valid and reliable results and developed a rater training protocol with an accompanying rater training handbook. The rater training program was modeled after previously described high-stakes assessments in the setting of 3 facilitated training sessions. Adjacent agreement was used to measure interrater reliability between raters. Nine raters with a background in health care and/or patient safety evaluated team performance of 42 in-situ simulations using post-hoc video review. Adjacent agreement increased from the second training session (83.6%) to the third training session (85.6%) when evaluating the same video segments. Adjacent agreement for the rating of overall team performance was 78.3%, which was added for the third training session. Adjacent agreement was 97% 4 weeks posttraining and 90.6% at the end of independent rating of all simulation videos. Rater training is an important element in team performance assessment, and providing examples of rater training programs is essential. Articulating key rating anchors promotes adequate interrater reliability. In addition, using adjacent agreement as a measure allows differentiation between high- and low-performing teams on video review. © 2015 The Alliance for Continuing Education in the Health Professions, the Society for Academic Continuing Medical Education, and the Council on Continuing Medical Education, Association for Hospital Medical Education.

Recommendations for Assessment of the Reliability, Sensitivity, and Validity of Data Provided by Wearable Sensors Designed for Monitoring Physical Activity

PubMed Central

2018-01-01

Although it is becoming increasingly popular to monitor parameters related to training, recovery, and health with wearable sensor technology (wearables), scientific evaluation of the reliability, sensitivity, and validity of such data is limited and, where available, has involved a wide variety of approaches. To improve the trustworthiness of data collected by wearables and facilitate comparisons, we have outlined recommendations for standardized evaluation. We discuss the wearable devices themselves, as well as experimental and statistical considerations. Adherence to these recommendations should be beneficial not only for the individual, but also for regulatory organizations and insurance companies. PMID:29712629

Interrater reliability of a Pilates movement-based classification system.

PubMed

Yu, Kwan Kenny; Tulloch, Evelyn; Hendrick, Paul

2015-01-01

To determine the interrater reliability for identification of a specific movement pattern using a Pilates Classification system. Videos of 5 subjects performing specific movement tasks were sent to raters trained in the DMA-CP classification system. Ninety-six raters completed the survey. Interrater reliability for the detection of a directional bias was excellent (Pi = 0.92, and K(free) = 0.89). Interrater reliability for classifying an individual into a specific subgroup was moderate (Pi = 0.64, K(free) = 0.55) however raters who had completed levels 1-4 of the DMA-CP training and reported using the assessment daily demonstrated excellent reliability (Pi = 0.89 and K(free) = 0.87). The reliability of the classification system demonstrated almost perfect agreement in determining the existence of a specific movement pattern and classifying into a subgroup for experienced raters. There was a trend for greater reliability associated with increased levels of training and experience of the raters. Copyright © 2014 Elsevier Ltd. All rights reserved.

An Examination of the Reliability of a New Observation Measure for Autism Spectrum Disorders: The Autism Spectrum Disorder Observation for Children

ERIC Educational Resources Information Center

Neal, Daniene; Matson, Johnny L.; Belva, Brian C.

2013-01-01

The "autism spectrum disorder observation for children" ("ASD-OC") is a newly created 54-item observation measure for autism spectrum disorders (ASD). Due to the fact that many of the ASD observation measures currently available do not have established psychometric properties and require extensive time and training to administer, the "ASD-OC"…

Training less-experienced faculty improves reliability of skills assessment in cardiac surgery.

PubMed

Lou, Xiaoying; Lee, Richard; Feins, Richard H; Enter, Daniel; Hicks, George L; Verrier, Edward D; Fann, James I

2014-12-01

Previous work has demonstrated high inter-rater reliability in the objective assessment of simulated anastomoses among experienced educators. We evaluated the inter-rater reliability of less-experienced educators and the impact of focused training with a video-embedded coronary anastomosis assessment tool. Nine less-experienced cardiothoracic surgery faculty members from different institutions evaluated 2 videos of simulated coronary anastomoses (1 by a medical student and 1 by a resident) at the Thoracic Surgery Directors Association Boot Camp. They then underwent a 30-minute training session using an assessment tool with embedded videos to anchor rating scores for 10 components of coronary artery anastomosis. Afterward, they evaluated 2 videos of a different student and resident performing the task. Components were scored on a 1 to 5 Likert scale, yielding an average composite score. Inter-rater reliabilities of component and composite scores were assessed using intraclass correlation coefficients (ICCs) and overall pass/fail ratings with kappa. All components of the assessment tool exhibited improvement in reliability, with 4 (bite, needle holder use, needle angles, and hand mechanics) improving the most from poor (ICC range, 0.09-0.48) to strong (ICC range, 0.80-0.90) agreement. After training, inter-rater reliabilities for composite scores improved from moderate (ICC, 0.76) to strong (ICC, 0.90) agreement, and for overall pass/fail ratings, from poor (kappa = 0.20) to moderate (kappa = 0.78) agreement. Focused, video-based anchor training facilitates greater inter-rater reliability in the objective assessment of simulated coronary anastomoses. Among raters with less teaching experience, such training may be needed before objective evaluation of technical skills. Published by Elsevier Inc.

Does a web-based feedback training program result in improved reliability in clinicians' ratings of the Global Assessment of Functioning (GAF) Scale?

PubMed

Støre-Valen, Jakob; Ryum, Truls; Pedersen, Geir A F; Pripp, Are H; Jose, Paul E; Karterud, Sigmund

2015-09-01

The Global Assessment of Functioning (GAF) Scale is used in routine clinical practice and research to estimate symptom and functional severity and longitudinal change. Concerns about poor interrater reliability have been raised, and the present study evaluated the effect of a Web-based GAF training program designed to improve interrater reliability in routine clinical practice. Clinicians rated up to 20 vignettes online, and received deviation scores as immediate feedback (i.e., own scores compared with expert raters) after each rating. Growth curves of absolute SD scores across the vignettes were modeled. A linear mixed effects model, using the clinician's deviation scores from expert raters as the dependent variable, indicated an improvement in reliability during training. Moderation by content of scale (symptoms; functioning), scale range (average; extreme), previous experience with GAF rating, profession, and postgraduate training were assessed. Training reduced deviation scores for inexperienced GAF raters, for individuals in clinical professions other than nursing and medicine, and for individuals with no postgraduate specialization. In addition, training was most beneficial for cases with average severity of symptoms compared with cases with extreme severity. The results support the use of Web-based training with feedback routines as a means to improve the reliability of GAF ratings performed by clinicians in mental health practice. These results especially pertain to clinicians in mental health practice who do not have a masters or doctoral degree. (c) 2015 APA, all rights reserved.

Reliability of joint count assessment in rheumatoid arthritis: a systematic literature review.

PubMed

Cheung, Peter P; Gossec, Laure; Mak, Anselm; March, Lyn

2014-06-01

Joint counts are central to the assessment of rheumatoid arthritis (RA) but reliability is an issue. To evaluate the reliability and agreement of joint counts (intra-observer and inter-observer) by health care professionals (physicians, nurses, and metrologists) and patients in RA, and the impact of training and standardization on joint count reliability through a systematic literature review. Articles reporting joint count reliability or agreement in RA in PubMed, EMBase, and the Cochrane library between 1960 and 2012 were selected. Data were extracted regarding tender joint counts (TJCs) and swollen joint counts (SJCs) derived by physicians, metrologists, or patients for intra-observer and inter-observer reliability. In addition, methods and effects of training or standardization were extracted. Statistics expressing reliability such as intraclass correlation coefficients (ICCs) were extracted. Data analysis was primarily descriptive due to high heterogeneity. Twenty-eight studies on health care professionals (HCP) and 20 studies on patients were included. Intra-observer reliability for TJCs and SJCs was good for HCPs and patients (range of ICC: 0.49-0.98). Inter-observer reliability between HCPs for TJCs was higher than for SJCs (range of ICC: 0.64-0.88 vs. 0.29-0.98). Patient inter-observer reliability with HCPs as comparators was better for TJCs (range of ICC: 0.31-0.91) compared to SJCs (0.16-0.64). Nine studies (7 with HCPs and 2 with patients) evaluated consensus or training, with improvement in reliability of TJCs but conflicting evidence for SJCs. Intra- and inter-observer reliability was high for TJCs for HCPs and patients: among all groups, reliability was better for TJCs than SJCs. Inter-observer reliability of SJCs was poorer for patients than HCPs. Data were inconclusive regarding the potential for training to improve SJC reliability. Overall, the results support further evaluation for patient-reported joint counts as an outcome measure. © 2013 Published by Elsevier Inc.

[Strategic measures for patient safety in the National Health System: on-line training resources and access to scientific knowledge].

PubMed

Novillo-Ortíz, D; Agra, Y; Fernández-Maíllo, M M; del Peso, P; Terol, E

2008-12-01

Patient safety (PS) is a priority strategy included in the Quality Plan for the Spanish National Health System and its first objective is to promote PS culture among professionals and patients. The Internet is playing a key role in the access to clinical evidence and in the training of health professionals. A multidisciplinary working group was created, who defined the criteria to help improve clinical practice in the field of patient safety, by making available and using web-based patient safety training resources and information. Taking advantage of the possibilities offered by the Internet in terms of training, two online self-training tutorials were developed on risk management, patient safety and adverse event prevention. A Newsletter was also launched, together with two specific patient safety Supplements. Moreover, to extend the reach of the PS Strategy, a patient safety web page and weblog were created, in addition to a collaborative (internal) working group tool. Excelenciaclinica.net was also developed; a meta-search engine specialized in evidence-based information for health professionals, to make it easier to access reliable and valuable information. Health professionals were also allowed to consult, free of charge, reliable health information resources, such as the GuiaSalud platform, the Cochrane Library Plus and the resources of the Joanna Briggs Institute. The involvement of health professionals in these measures and the role that these measures may be expected to play in the development of a premium-quality health service.

Examining the interrater reliability of the Hare Psychopathy Checklist-Revised across a large sample of trained raters.

PubMed

Blais, Julie; Forth, Adelle E; Hare, Robert D

2017-06-01

The goal of the current study was to assess the interrater reliability of the Psychopathy Checklist-Revised (PCL-R) among a large sample of trained raters (N = 280). All raters completed PCL-R training at some point between 1989 and 2012 and subsequently provided complete coding for the same 6 practice cases. Overall, 3 major conclusions can be drawn from the results: (a) reliability of individual PCL-R items largely fell below any appropriate standards while the estimates for Total PCL-R scores and factor scores were good (but not excellent); (b) the cases representing individuals with high psychopathy scores showed better reliability than did the cases of individuals in the moderate to low PCL-R score range; and (c) there was a high degree of variability among raters; however, rater specific differences had no consistent effect on scoring the PCL-R. Therefore, despite low reliability estimates for individual items, Total scores and factor scores can be reliably scored among trained raters. We temper these conclusions by noting that scoring standardized videotaped case studies does not allow the rater to interact directly with the offender. Real-world PCL-R assessments typically involve a face-to-face interview and much more extensive collateral information. We offer recommendations for new web-based training procedures. (PsycINFO Database Record (c) 2017 APA, all rights reserved).

Effect of training on the reliability of satiety evaluation and use of trained panellists to determine the satiety effect of dietary fibre: a randomised controlled trial.

PubMed

Solah, Vicky A; Meng, Xingqiong; Wood, Simon; Gahler, Roland J; Kerr, Deborah A; James, Anthony P; Pal, Sebely; Fenton, Haelee K; Johnson, Stuart K

2015-01-01

The assessment of satiety effects on foods is commonly performed by untrained volunteers marking their perceived hunger or fullness on line scales, marked with pre-set descriptors. The lack of reproducibility of satiety measurement using this approach however results in the tool being unable to distinguish between foods that have small, but possibly important, differences in their satiety effects. An alternate approach is used in sensory evaluation; panellists can be trained in the correct use of the assessment line-scale and brought to consensus on the meanings of descriptors used for food quality attributes to improve the panel reliability. The effect of training on the reliability of a satiety panel has not previously been reported. In a randomised controlled parallel intervention, the effect of training in the correct use of a satiety labelled magnitude scale (LMS) was assessed versus no-training. The test-retest precision and reliability of two hour postprandial satiety evaluation after consumption of a standard breakfast was compared. The trained panel then compared the satiety effect of two breakfast meals containing either a viscous or a non-viscous dietary fibre in a crossover trial. A subgroup of the 23 panellists (n = 5) improved their test re-test precision after training. Panel satiety area under the curve, "after the training" intervention was significantly different to "before training" (p < 0.001). Reliability of the panel determined by intraclass correlation (ICC) of test and retest showed improved strength of the correlation from 0.70 pre-intervention to 0.95 post intervention. The trained "satiety expert panel" determined that a standard breakfast with 5g of viscous fibre gave significantly higher satiety than with 5g non-viscous fibre (area under curve (AUC) of 478.2, 334.4 respectively) (p ≤ 0.002). Training reduced between panellist variability. The improved strength of test-retest ICC as a result of the training intervention suggests that training satiety panellists can improve the discriminating power of satiety evaluation.

Reliability Modeling of Microelectromechanical Systems Using Neural Networks

NASA Technical Reports Server (NTRS)

Perera. J. Sebastian

2000-01-01

Microelectromechanical systems (MEMS) are a broad and rapidly expanding field that is currently receiving a great deal of attention because of the potential to significantly improve the ability to sense, analyze, and control a variety of processes, such as heating and ventilation systems, automobiles, medicine, aeronautical flight, military surveillance, weather forecasting, and space exploration. MEMS are very small and are a blend of electrical and mechanical components, with electrical and mechanical systems on one chip. This research establishes reliability estimation and prediction for MEMS devices at the conceptual design phase using neural networks. At the conceptual design phase, before devices are built and tested, traditional methods of quantifying reliability are inadequate because the device is not in existence and cannot be tested to establish the reliability distributions. A novel approach using neural networks is created to predict the overall reliability of a MEMS device based on its components and each component's attributes. The methodology begins with collecting attribute data (fabrication process, physical specifications, operating environment, property characteristics, packaging, etc.) and reliability data for many types of microengines. The data are partitioned into training data (the majority) and validation data (the remainder). A neural network is applied to the training data (both attribute and reliability); the attributes become the system inputs and reliability data (cycles to failure), the system output. After the neural network is trained with sufficient data. the validation data are used to verify the neural networks provided accurate reliability estimates. Now, the reliability of a new proposed MEMS device can be estimated by using the appropriate trained neural networks developed in this work.

Smartphone Apps for Cardiopulmonary Resuscitation Training and Real Incident Support: A Mixed-Methods Evaluation Study

PubMed Central

Felzen, Marc; Rossaint, Rolf; Tabuenca, Bernardo; Specht, Marcus; Skorning, Max

2014-01-01

Background No systematic evaluation of smartphone/mobile apps for resuscitation training and real incident support is available to date. To provide medical, usability, and additional quality criteria for the development of apps, we conducted a mixed-methods sequential evaluation combining the perspective of medical experts and end-users. Objective The study aims to assess the quality of current mobile apps for cardiopulmonary resuscitation (CPR) training and real incident support from expert as well as end-user perspective. Methods Two independent medical experts evaluated the medical content of CPR apps from the Google Play store and the Apple App store. The evaluation was based on pre-defined minimum medical content requirements according to current Basic Life Support (BLS) guidelines. In a second phase, non-medical end-users tested usability and appeal of the apps that had at least met the minimum requirements. Usability was assessed with the System Usability Scale (SUS); appeal was measured with the self-developed ReactionDeck toolkit. Results Out of 61 apps, 46 were included in the experts’ evaluation. A consolidated list of 13 apps resulted for the following layperson evaluation. The interrater reliability was substantial (kappa=.61). Layperson end-users (n=14) had a high interrater reliability (intraclass correlation 1 [ICC1]=.83, P<.001, 95% CI 0.75-0.882 and ICC2=.79, P<.001, 95% CI 0.695-0.869). Their evaluation resulted in a list of 5 recommendable apps. Conclusions Although several apps for resuscitation training and real incident support are available, very few are designed according to current BLS guidelines and offer an acceptable level of usability and hedonic quality for laypersons. The results of this study are intended to optimize the development of CPR mobile apps. The app ranking supports the informed selection of mobile apps for training situations and CPR campaigns as well as for real incident support. PMID:24647361

Smartphone apps for cardiopulmonary resuscitation training and real incident support: a mixed-methods evaluation study.

PubMed

Kalz, Marco; Lenssen, Niklas; Felzen, Marc; Rossaint, Rolf; Tabuenca, Bernardo; Specht, Marcus; Skorning, Max

2014-03-19

No systematic evaluation of smartphone/mobile apps for resuscitation training and real incident support is available to date. To provide medical, usability, and additional quality criteria for the development of apps, we conducted a mixed-methods sequential evaluation combining the perspective of medical experts and end-users. The study aims to assess the quality of current mobile apps for cardiopulmonary resuscitation (CPR) training and real incident support from expert as well as end-user perspective. Two independent medical experts evaluated the medical content of CPR apps from the Google Play store and the Apple App store. The evaluation was based on pre-defined minimum medical content requirements according to current Basic Life Support (BLS) guidelines. In a second phase, non-medical end-users tested usability and appeal of the apps that had at least met the minimum requirements. Usability was assessed with the System Usability Scale (SUS); appeal was measured with the self-developed ReactionDeck toolkit. Out of 61 apps, 46 were included in the experts' evaluation. A consolidated list of 13 apps resulted for the following layperson evaluation. The interrater reliability was substantial (kappa=.61). Layperson end-users (n=14) had a high interrater reliability (intraclass correlation 1 [ICC1]=.83, P<.001, 95% CI 0.75-0.882 and ICC2=.79, P<.001, 95% CI 0.695-0.869). Their evaluation resulted in a list of 5 recommendable apps. Although several apps for resuscitation training and real incident support are available, very few are designed according to current BLS guidelines and offer an acceptable level of usability and hedonic quality for laypersons. The results of this study are intended to optimize the development of CPR mobile apps. The app ranking supports the informed selection of mobile apps for training situations and CPR campaigns as well as for real incident support.

Reliability of Single-Leg Balance and Landing Tests in Rugby Union; Prospect of Using Postural Control to Monitor Fatigue

PubMed Central

Troester, Jordan C.; Jasmin, Jason G.; Duffield, Rob

2018-01-01

The present study examined the inter-trial (within test) and inter-test (between test) reliability of single-leg balance and single-leg landing measures performed on a force plate in professional rugby union players using commercially available software (SpartaMARS, Menlo Park, USA). Twenty-four players undertook test – re-test measures on two occasions (7 days apart) on the first training day of two respective pre-season weeks following 48h rest and similar weekly training loads. Two 20s single-leg balance trials were performed on a force plate with eyes closed. Three single-leg landing trials were performed by jumping off two feet and landing on one foot in the middle of a force plate 1m from the starting position. Single-leg balance results demonstrated acceptable inter-trial reliability (ICC = 0.60-0.81, CV = 11-13%) for sway velocity, anterior-posterior sway velocity, and mediolateral sway velocity variables. Acceptable inter-test reliability (ICC = 0.61-0.89, CV = 7-13%) was evident for all variables except mediolateral sway velocity on the dominant leg (ICC = 0.41, CV = 15%). Single-leg landing results only demonstrated acceptable inter-trial reliability for force based measures of relative peak landing force and impulse (ICC = 0.54-0.72, CV = 9-15%). Inter-test results indicate improved reliability through the averaging of three trials with force based measures again demonstrating acceptable reliability (ICC = 0.58-0.71, CV = 7-14%). Of the variables investigated here, total sway velocity and relative landing impulse are the most reliable measures of single-leg balance and landing performance, respectively. These measures should be considered for monitoring potential changes in postural control in professional rugby union. Key points Single-leg balance demonstrated acceptable inter-trial and inter-test reliability. Single-leg landing demonstrated good inter-trial and inter-test reliability for measures of relative peak landing force and relative impulse, but not time to stabilization. Of the variables investigated, sway velocity and relative landing impulse are the most reliable measures of single-leg balance and landing respectively, and should considered for monitoring changes in postural control. PMID:29769817

Complexity of GPs' explanations about mental health problems: development, reliability, and validity of a measure

PubMed Central

Cape, John; Morris, Elena; Burd, Mary; Buszewicz, Marta

2008-01-01

Background How GPs understand mental health problems determines their treatment choices; however, measures describing GPs' thinking about such problems are not currently available. Aim To develop a measure of the complexity of GP explanations of common mental health problems and to pilot its reliability and validity. Design of study A qualitative development of the measure, followed by inter-rater reliability and validation pilot studies. Setting General practices in North London. Method Vignettes of simulated consultations with patients with mental health problems were videotaped, and an anchored measure of complexity of psychosocial explanation in response to these vignettes was developed. Six GPs, four psychologists, and two lay people viewed the vignettes. Their responses were rated for complexity, both using the anchored measure and independently by two experts in primary care mental health. In a second reliability and revalidation study, responses of 50 GPs to two vignettes were rated for complexity. The GPs also completed a questionnaire to determine their interest and training in mental health, and they completed the Depression Attitudes Questionnaire. Results Inter-rater reliability of the measure of complexity of explanation in both pilot studies was satisfactory (intraclass correlation coefficient = 0.78 and 0.72). The measure correlated with expert opinion as to what constitutes a complex explanation, and the responses of psychologists, GPs, and lay people differed in measured complexity. GPs with higher complexity scores had greater interest, more training in mental health, and more positive attitudes to depression. Conclusion Results suggest that the complexity of GPs' psychosocial explanations about common mental health problems can be reliably and validly assessed by this new standardised measure. PMID:18505616

Development and pilot testing of an online module for ethics education based on the Nigerian National Code for Health Research Ethics.

PubMed

Ogunrin, Olubunmi A; Ogundiran, Temidayo O; Adebamowo, Clement

2013-01-02

The formulation and implementation of national ethical regulations to protect research participants is fundamental to ethical conduct of research. Ethics education and capacity are inadequate in developing African countries. This study was designed to develop a module for online training in research ethics based on the Nigerian National Code of Health Research Ethics and assess its ease of use and reliability among biomedical researchers in Nigeria. This was a three-phased evaluation study. Phase one involved development of an online training module based on the Nigerian Code of Health Research Ethics (NCHRE) and uploading it to the Collaborative Institutional Training Initiative (CITI) website while the second phase entailed the evaluation of the module for comprehensibility, readability and ease of use by 45 Nigerian biomedical researchers. The third phase involved modification and re-evaluation of the module by 30 Nigerian biomedical researchers and determination of test-retest reliability of the module using Cronbach's alpha. The online module was easily accessible and comprehensible to 95% of study participants. There were significant differences in the pretest and posttest scores of study participants during the evaluation of the online module (p = 0.001) with correlation coefficients of 0.9 and 0.8 for the pretest and posttest scores respectively. The module also demonstrated excellent test-retest reliability and internal consistency as shown by Cronbach's alpha coefficients of 0.92 and 0.84 for the pretest and posttest respectively. The module based on the Nigerian Code was developed, tested and made available online as a valuable tool for training in cultural and societal relevant ethical principles to orient national and international biomedical researchers working in Nigeria. It would complement other general research ethics and Good Clinical Practice modules. Participants suggested that awareness of the online module should be increased through seminars, advertisement on government websites and portals used by Nigerian biomedical researchers, and incorporation of the Code into the undergraduate medical training curriculum.

Reliability of the Spanish version of the wheelchair skills test 4.2 for manual wheelchair users with spinal cord injury.

PubMed

Passuni, Diego; Dalzotto, Elisa; F Gath, Christian; Buffetti, Eliana; Elizalde, Milagros; Jarmoluk, Verónica; Russo, Maria J; Intruvini, Silvia; Olmos, Lisandro E; Freixes, Orestes

2018-05-03

Cross-sectional. The majority of people with a spinal cord injury (SCI) are dependent on wheelchair for their mobility. Approximately, 36% of wheelchair users reported that obstacles such as curbs, uneven terrain, flooring surfaces and thresholds were barriers to mobility. Several studies have shown that assessment and training of wheelchair skills leads to improvements in those skills. The purpose of our study was to translate the Wheelchair Skill Test (4.2) and its report form into Spanish and then determine the inter-rater reliability of the WST 4.2 for manual wheelchairs operated by their users. Rehabilitation Unit, FLENI Institute, Buenos Aires, Argentina. The translation was performed by a physical therapist with advanced English language skills and specialized in the treatment of SCI subjects. We administrated and video-recorded the WST 4.2 manual Spanish version in 11 SCI subjects. Two physical therapists received specific training for administering the test and scoring the record. The reliability of the total percentage WST score were statistically quantified by intraclass correlation coefficient (ICC). ICC values for Interrater were 0.998 (p < .0001). 17 out of the 32 skills had a 100 percentage of agreement. Percentage of agreement in the three skills that presented less rating agreement was 73%, 81 % and 82 %, respectively. The results show that the Spanish version of WST 4.2 is a reliable assessment tool to evaluate the skills capacity of spinal cord manual wheelchair users. Implications for rehabilitation Wheelchair users require a proficient management of various wheelchair skills to achieve maximum independence in daily life. Determining which wheelchair skills should be addressed during the rehabilitation process is of great importance for their correct training. The WST 4.2 is an appropriate assessment tool to determine the functional capacity of wheelchair users. Making available the WST 4.2 in the Spanish language and demonstrating its reliability in this language allows its use in the Spanish-speaking world. A reliable wheelchair skills test is needed in the Spanish language.

Human Reliability and the Cost of Doing Business

NASA Technical Reports Server (NTRS)

DeMott, Diana

2014-01-01

Most businesses recognize that people will make mistakes and assume errors are just part of the cost of doing business, but does it need to be? Companies with high risk, or major consequences, should consider the effect of human error. In a variety of industries, Human Errors have caused costly failures and workplace injuries. These have included: airline mishaps, medical malpractice, administration of medication and major oil spills have all been blamed on human error. A technique to mitigate or even eliminate some of these costly human errors is the use of Human Reliability Analysis (HRA). Various methodologies are available to perform Human Reliability Assessments that range from identifying the most likely areas for concern to detailed assessments with human error failure probabilities calculated. Which methodology to use would be based on a variety of factors that would include: 1) how people react and act in different industries, and differing expectations based on industries standards, 2) factors that influence how the human errors could occur such as tasks, tools, environment, workplace, support, training and procedure, 3) type and availability of data and 4) how the industry views risk & reliability influences ( types of emergencies, contingencies and routine tasks versus cost based concerns). The Human Reliability Assessments should be the first step to reduce, mitigate or eliminate the costly mistakes or catastrophic failures. Using Human Reliability techniques to identify and classify human error risks allows a company more opportunities to mitigate or eliminate these risks and prevent costly failures.

Simulated training in colonoscopic stenting of colonic strictures: validation of a cadaver model.

PubMed

Iordache, F; Bucobo, J C; Devlin, D; You, K; Bergamaschi, R

2015-07-01

There are currently no available simulation models for training in colonoscopic stent deployment. The aim of this study was to validate a cadaver model for simulation training in colonoscopy with stent deployment for colonic strictures. This was a prospective study enrolling surgeons at a single institution. Participants performed colonoscopic stenting on a cadaver model. Their performance was assessed by two independent observers. Measurements were performed for quantitative analysis (time to identify stenosis, time for deployment, accuracy) and a weighted score was devised for assessment. The Mann-Whitney U-test and Student's t-test were used for nonparametric and parametric data, respectively. Cohen's kappa coefficient was used for reliability. Twenty participants performed a colonoscopy with deployment of a self-expandable metallic stent in two cadavers (groups A and B) with 20 strictures overall. The median time was 206 s. The model was able to differentiate between experts and novices (P = 0. 013). The results showed a good consensus estimate of reliability, with kappa = 0.571 (P < 0.0001). The cadaver model described in this study has content, construct and concurrent validity for simulation training in colonoscopic deployment of self-expandable stents for colonic strictures. Further studies are needed to evaluate the predictive validity of this model in terms of skill transfer to clinical practice. Colorectal Disease © 2014 The Association of Coloproctology of Great Britain and Ireland.

Using Ensemble Decisions and Active Selection to Improve Low-Cost Labeling for Multi-View Data

NASA Technical Reports Server (NTRS)

Rebbapragada, Umaa; Wagstaff, Kiri L.

2011-01-01

This paper seeks to improve low-cost labeling in terms of training set reliability (the fraction of correctly labeled training items) and test set performance for multi-view learning methods. Co-training is a popular multiview learning method that combines high-confidence example selection with low-cost (self) labeling. However, co-training with certain base learning algorithms significantly reduces training set reliability, causing an associated drop in prediction accuracy. We propose the use of ensemble labeling to improve reliability in such cases. We also discuss and show promising results on combining low-cost ensemble labeling with active (low-confidence) example selection. We unify these example selection and labeling strategies under collaborative learning, a family of techniques for multi-view learning that we are developing for distributed, sensor-network environments.

The Influence of Resistance Training Experience on the Between-Day Reliability of Commonly Used Strength Measures in Male Youth Athletes.

PubMed

Weakley, Jonathon J S; Till, Kevin; Darrall-Jones, Joshua; Roe, Gregory A B; Phibbs, Padraic J; Read, Dale B; Jones, Ben L

2017-07-01

Weakley, JJS, Till, K, Darrall-Jones, J, Roe, GAB, Phibbs, PJ, Read, DB, and Jones, BL. The influence of resistance training experience on the between-day reliability of commonly used strength measures in male youth athletes. J Strength Cond Res 31(7): 2005-2010, 2017-The purpose of this study was to determine the between-day reliability of commonly used strength measures in male youth athletes while considering resistance training experience. Data were collected on 25 male athletes over 2 testing sessions, with 72 hours rest between, for the 3 repetition maximum (3RM) front squat, chin-up, and bench press. Subjects were initially categorized by resistance training experience (inexperienced; 6-12 months, experienced; >2 years). The assessment of the between-day reliability (coefficient of variation [CV%]) showed that the front squat (experienced: 2.90%; inexperienced: 1.90%), chin-up (experienced: 1.70%; inexperienced: 1.90%), and bench press (experienced: 4.50%; inexperienced: 2.40%) were all reliable measures of strength in both groups. Comparison between groups for the error of measurement for each exercise showed trivial differences. When both groups were combined, the CV% for the front squat, bench press, and chin-up were 2.50, 1.80, and 3.70%, respectively. This study provides scientists and practitioners with the between-day reliability reference data to determine real and practical changes for strength in male youth athletes with different resistance training experience. Furthermore, this study demonstrates that 3RM front squat, chin-up, and bench press are reliable exercises to quantify strength in male youth athletes.

Effects of consensus training on the reliability of auditory perceptual ratings of voice quality.

PubMed

Iwarsson, Jenny; Reinholt Petersen, Niels

2012-05-01

This study investigates the effect of consensus training of listeners on intrarater and interrater reliability and agreement of perceptual voice analysis. The use of such training, including a reference voice sample, could be assumed to make the internal standards held in memory common and more robust, which is of great importance to reduce the variability of auditory perceptual ratings. A prospective design with testing before and after training. Thirteen students of audiologopedics served as listening subjects. The ratings were made using a multidimensional protocol with four-point equal-appearing interval scales. The stimuli consisted of text reading by authentic dysphonic patients. The consensus training for each perceptual voice parameter included (1) definition, (2) underlying physiology, (3) presentation of carefully selected sound examples representing the parameter in three different grades followed by group discussions of perceived characteristics, and (4) practical exercises including imitation to make use of the listeners' proprioception. Intrarater reliability and agreement showed a marked improvement for intermittent aphonia but not for vocal fry. Interrater reliability was high for most parameters before training with a slight increase after training. Interrater agreement showed marked increases for most voice quality parameters as a result of the training. The results support the recommendation of specific consensus training, including use of a reference voice sample material, to calibrate, equalize, and stabilize the internal standards held in memory by the listeners. Copyright © 2012 The Voice Foundation. Published by Mosby, Inc. All rights reserved.

Effect of standardized training on the reliability of the Cochrane risk of bias assessment tool: a prospective study.

PubMed

da Costa, Bruno R; Beckett, Brooke; Diaz, Alison; Resta, Nina M; Johnston, Bradley C; Egger, Matthias; Jüni, Peter; Armijo-Olivo, Susan

2017-03-03

The Cochrane risk of bias tool is commonly criticized for having a low reliability. We aimed to investigate whether training of raters, with objective and standardized instructions on how to assess risk of bias, can improve the reliability of the Cochrane risk of bias tool. In this pilot study, four raters inexperienced in risk of bias assessment were randomly allocated to minimal or intensive standardized training for risk of bias assessment of randomized trials of physical therapy treatments for patients with knee osteoarthritis pain. Two raters were experienced risk of bias assessors who served as reference. The primary outcome of our study was between-group reliability, defined as the agreement of the risk of bias assessments of inexperienced raters with the reference assessments of experienced raters. Consensus-based assessments were used for this purpose. The secondary outcome was within-group reliability, defined as the agreement of assessments within pairs of inexperienced raters. We calculated the chance-corrected weighted Kappa to quantify agreement within and between groups of raters for each of the domains of the risk of bias tool. A total of 56 trials were included in our analysis. The Kappa for the agreement of inexperienced raters with reference across items of the risk of bias tool ranged from 0.10 to 0.81 for the minimal training group and from 0.41 to 0.90 for the standardized training group. The Kappa values for the agreement within pairs of inexperienced raters across the items of the risk of bias tool ranged from 0 to 0.38 for the minimal training group and from 0.93 to 1 for the standardized training group. Between-group differences in Kappa for the agreement of inexperienced raters with reference always favored the standardized training group and was most pronounced for incomplete outcome data (difference in Kappa 0.52, p < 0.001) and allocation concealment (difference in Kappa 0.30, p = 0.004). Intensive, standardized training on risk of bias assessment may significantly improve the reliability of the Cochrane risk of bias tool.

Effect of Training on the Reliability of Satiety Evaluation and Use of Trained Panellists to Determine the Satiety Effect of Dietary Fibre: A Randomised Controlled Trial

PubMed Central

Solah, Vicky A.; Meng, Xingqiong; Wood, Simon; Gahler, Roland J.; Kerr, Deborah A.; James, Anthony P.; Pal, Sebely; Fenton, Haelee K.; Johnson, Stuart K.

2015-01-01

Background The assessment of satiety effects on foods is commonly performed by untrained volunteers marking their perceived hunger or fullness on line scales, marked with pre-set descriptors. The lack of reproducibility of satiety measurement using this approach however results in the tool being unable to distinguish between foods that have small, but possibly important, differences in their satiety effects. An alternate approach is used in sensory evaluation; panellists can be trained in the correct use of the assessment line-scale and brought to consensus on the meanings of descriptors used for food quality attributes to improve the panel reliability. The effect of training on the reliability of a satiety panel has not previously been reported. Method In a randomised controlled parallel intervention, the effect of training in the correct use of a satiety labelled magnitude scale (LMS) was assessed versus no-training. The test-retest precision and reliability of two hour postprandial satiety evaluation after consumption of a standard breakfast was compared. The trained panel then compared the satiety effect of two breakfast meals containing either a viscous or a non-viscous dietary fibre in a crossover trial. Results A subgroup of the 23 panellists (n = 5) improved their test re-test precision after training. Panel satiety area under the curve, “after the training” intervention was significantly different to “before training” (p < 0.001). Reliability of the panel determined by intraclass correlation (ICC) of test and retest showed improved strength of the correlation from 0.70 pre-intervention to 0.95 post intervention. The trained “satiety expert panel” determined that a standard breakfast with 5g of viscous fibre gave significantly higher satiety than with 5g non-viscous fibre (area under curve (AUC) of 478.2, 334.4 respectively) (p ≤ 0.002). Conclusion Training reduced between panellist variability. The improved strength of test-retest ICC as a result of the training intervention suggests that training satiety panellists can improve the discriminating power of satiety evaluation. PMID:25978321

Reliability of SNOMED-CT Coding by Three Physicians using Two Terminology Browsers

PubMed Central

Chiang, Michael F.; Hwang, John C.; Yu, Alexander C.; Casper, Daniel S.; Cimino, James J.; Starren, Justin

2006-01-01

SNOMED-CT has been promoted as a reference terminology for electronic health record (EHR) systems. Many important EHR functions are based on the assumption that medical concepts will be coded consistently by different users. This study is designed to measure agreement among three physicians using two SNOMED-CT terminology browsers to encode 242 concepts from five ophthalmology case presentations in a publicly-available clinical journal. Inter-coder reliability, based on exact coding match by each physician, was 44% using one browser and 53% using the other. Intra-coder reliability testing revealed that a different SNOMED-CT code was obtained up to 55% of the time when the two browsers were used by one user to encode the same concept. These results suggest that the reliability of SNOMED-CT coding is imperfect, and may be a function of browsing methodology. A combination of physician training, terminology refinement, and browser improvement may help increase the reproducibility of SNOMED-CT coding. PMID:17238317

An evaluation of pediatric dental patient education materials using contemporary health literacy measures.

PubMed

Kang, Edith; Fields, Henry W; Cornett, Sandy; Beck, F Michael

2005-01-01

The purpose of this study was to determine the appropriateness of nationally available dental information materials according to the suitability assessment of materials (SAM) method. Clinically related, professionally produced patient dental health education materials (N=22) provided by the American Academy of Pediatric Dentistry (AAPD) were evaluated using the SAM method that had previously been judged valid and reliable. A rater was trained by an experienced health literacy evaluator to establish validity. The rater then rated all materials for 5 categories of assessment (content, literacy demand, graphics, layout and typography, and learning stimulation/motivation) and an overall assessment, and repeated 5 materials to establish intrarater reliability. When compared to the experienced rater, the validity was K=0.43. The reliability was established for all ratings as K=0.52. The consistently weakest categories were content, graphics, and learning stimulation, while reading level as part of literacy demand was often not suitable. The overall suitability of the AAPD materials was generally classified as superior. Reliable and valid evaluation of available dental patient information materials can be accomplished. The materials were largely superior. There is great variability within the categories of evaluation. The categories of content, graphics, and learning stimulation require attention and could raise the overall quality of the materials.

ALCHEMY: a reliable method for automated SNP genotype calling for small batch sizes and highly homozygous populations

PubMed Central

Wright, Mark H.; Tung, Chih-Wei; Zhao, Keyan; Reynolds, Andy; McCouch, Susan R.; Bustamante, Carlos D.

2010-01-01

Motivation: The development of new high-throughput genotyping products requires a significant investment in testing and training samples to evaluate and optimize the product before it can be used reliably on new samples. One reason for this is current methods for automated calling of genotypes are based on clustering approaches which require a large number of samples to be analyzed simultaneously, or an extensive training dataset to seed clusters. In systems where inbred samples are of primary interest, current clustering approaches perform poorly due to the inability to clearly identify a heterozygote cluster. Results: As part of the development of two custom single nucleotide polymorphism genotyping products for Oryza sativa (domestic rice), we have developed a new genotype calling algorithm called ‘ALCHEMY’ based on statistical modeling of the raw intensity data rather than modelless clustering. A novel feature of the model is the ability to estimate and incorporate inbreeding information on a per sample basis allowing accurate genotyping of both inbred and heterozygous samples even when analyzed simultaneously. Since clustering is not used explicitly, ALCHEMY performs well on small sample sizes with accuracy exceeding 99% with as few as 18 samples. Availability: ALCHEMY is available for both commercial and academic use free of charge and distributed under the GNU General Public License at http://alchemy.sourceforge.net/ Contact: mhw6@cornell.edu Supplementary information: Supplementary data are available at Bioinformatics online. PMID:20926420

Effects of measurement method and transcript availability on inexperienced raters' stuttering frequency scores.

PubMed

Chakraborty, Nalanda; Logan, Kenneth J

To examine the effects of measurement method and transcript availability on the accuracy, reliability, and efficiency of inexperienced raters' stuttering frequency measurements. 44 adults, all inexperienced at evaluating stuttered speech, underwent 20 min of preliminary training in stuttering measurement and then analyzed a series of sentences, with and without access to transcripts of sentence stimuli, using either a syllable-based analysis (SBA) or an utterance-based analysis (UBA). Participants' analyses were compared between groups and to a composite analysis from two experienced evaluators. Stuttering frequency scores from the SBA and UBA groups differed significantly from the experienced evaluators' scores; however, UBA scores were significantly closer to the experienced evaluators' scores and were completed significantly faster than the SBA scores. Transcript availability facilitated scoring accuracy and efficiency in both groups. The internal reliability of stuttering frequency scores was acceptable for the SBA and UBA groups; however, the SBA group demonstrated only modest point-by-point agreement with ratings from the experienced evaluators. Given its accuracy and efficiency advantages over syllable-based analysis, utterance-based fluency analysis appears to be an appropriate context for introducing stuttering frequency measurement to raters who have limited experience in stuttering measurement. To address accuracy gaps between experienced and inexperienced raters, however, use of either analysis must be supplemented with training activities that expose inexperienced raters to the decision-making processes used by experienced raters when identifying stuttered syllables. Copyright © 2018 Elsevier Inc. All rights reserved.

Nutrition screening tools: an analysis of the evidence.

PubMed

Skipper, Annalynn; Ferguson, Maree; Thompson, Kyle; Castellanos, Victoria H; Porcari, Judy

2012-05-01

In response to questions about tools for nutrition screening, an evidence analysis project was developed to identify the most valid and reliable nutrition screening tools for use in acute care and hospital-based ambulatory care settings. An oversight group defined nutrition screening and literature search criteria. A trained analyst conducted structured searches of the literature for studies of nutrition screening tools according to predetermined criteria. Eleven nutrition screening tools designed to detect undernutrition in patients in acute care and hospital-based ambulatory care were identified. Trained analysts evaluated articles for quality using criteria specified by the American Dietetic Association's Evidence Analysis Library. Members of the oversight group assigned quality grades to the tools based on the quality of the supporting evidence, including reliability and validity data. One tool, the NRS-2002, received a grade I, and 4 tools-the Simple Two-Part Tool, the Mini-Nutritional Assessment-Short Form (MNA-SF), the Malnutrition Screening Tool (MST), and Malnutrition Universal Screening Tool (MUST)-received a grade II. The MST was the only tool shown to be both valid and reliable for identifying undernutrition in the settings studied. Thus, validated nutrition screening tools that are simple and easy to use are available for application in acute care and hospital-based ambulatory care settings.

Reliability of a Market Basket Assessment Tool (MBAT) for Use in SNAP-Ed Healthy Retail Initiatives.

PubMed

Misyak, Sarah A; Hedrick, Valisa E; Pudney, Ellen; Serrano, Elena L; Farris, Alisha R

2018-05-01

To evaluate the reliability of the Market Basket Assessment Tool (MBAT) for assessing the availability of fruits and vegetables, low-fat or nonfat dairy and eggs, lean meats, whole-grain products, and seeds, beans, and nuts in Supplemental Nutrition Assistance Program-authorized retail environments. Different trained raters used the MBAT simultaneously at 14 retail environments to measure interrater reliability. Raters returned to 12 retail environments (85.7%) 1 week later to measure test-retest reliability. Data were analyzed using paired-sample t tests and correlations. No significant differences were found for interrater reliability or test-retest reliability for individual categories (mean differences, 0.0 to 0.3 ± 0.2 points) or total score (mean difference, 0.5 ± 0.4 points and (mean differences, 0.0 to 0.3 ± 0.3 points) or total score (mean difference, 0.8 ± 0.4 points), respectively. Future steps include validation of the MBAT. A low-burden tool can facilitate evaluation of efforts to promote healthful foods in retail environments. Copyright © 2018 Society for Nutrition Education and Behavior. Published by Elsevier Inc. All rights reserved.

Effect of standardized training on the reliability of the Cochrane risk of bias assessment tool: a study protocol.

PubMed

da Costa, Bruno R; Resta, Nina M; Beckett, Brooke; Israel-Stahre, Nicholas; Diaz, Alison; Johnston, Bradley C; Egger, Matthias; Jüni, Peter; Armijo-Olivo, Susan

2014-12-13

The Cochrane risk of bias (RoB) tool has been widely embraced by the systematic review community, but several studies have reported that its reliability is low. We aim to investigate whether training of raters, including objective and standardized instructions on how to assess risk of bias, can improve the reliability of this tool. We describe the methods that will be used in this investigation and present an intensive standardized training package for risk of bias assessment that could be used by contributors to the Cochrane Collaboration and other reviewers. This is a pilot study. We will first perform a systematic literature review to identify randomized clinical trials (RCTs) that will be used for risk of bias assessment. Using the identified RCTs, we will then do a randomized experiment, where raters will be allocated to two different training schemes: minimal training and intensive standardized training. We will calculate the chance-corrected weighted Kappa with 95% confidence intervals to quantify within- and between-group Kappa agreement for each of the domains of the risk of bias tool. To calculate between-group Kappa agreement, we will use risk of bias assessments from pairs of raters after resolution of disagreements. Between-group Kappa agreement will quantify the agreement between the risk of bias assessment of raters in the training groups and the risk of bias assessment of experienced raters. To compare agreement of raters under different training conditions, we will calculate differences between Kappa values with 95% confidence intervals. This study will investigate whether the reliability of the risk of bias tool can be improved by training raters using standardized instructions for risk of bias assessment. One group of inexperienced raters will receive intensive training on risk of bias assessment and the other will receive minimal training. By including a control group with minimal training, we will attempt to mimic what many review authors commonly have to do, that is-conduct risk of bias assessment in RCTs without much formal training or standardized instructions. If our results indicate that an intense standardized training does improve the reliability of the RoB tool, our study is likely to help improve the quality of risk of bias assessments, which is a central component of evidence synthesis.

Transcultural Adaptation of GRID Hamilton Rating Scale For Depression (GRID-HAMD) to Brazilian Portuguese and Evaluation of the Impact of Training Upon Inter-Rater Reliability.

PubMed

Henrique-Araújo, Ricardo; Osório, Flávia L; Gonçalves Ribeiro, Mônica; Soares Monteiro, Ivandro; Williams, Janet B W; Kalali, Amir; Alexandre Crippa, José; Oliveira, Irismar Reis De

2014-07-01

GRID-HAMD is a semi-structured interview guide developed to overcome flaws in HAM-D, and has been incorporated into an increasing number of studies. Carry out the transcultural adaptation of GRID-HAMD into the Brazilian Portuguese language, evaluate the inter-rater reliability of this instrument and the training impact upon this measure, and verify the raters' opinions of said instrument. The transcultural adaptation was conducted by appropriate methodology. The measurement of inter-rater reliability was done by way of videos that were evaluated by 85 professionals before and after training for the use of this instrument. The intraclass correlation coefficient (ICC) remained between 0.76 and 0.90 for GRID-HAMD-21 and between 0.72 and 0.91 for GRID-HAMD-17. The training did not have an impact on the ICC, except for a few groups of participants with a lower level of experience. Most of the participants showed high acceptance of GRID-HAMD, when compared to other versions of HAM-D. The scale presented adequate inter-rater reliability even before training began. Training did not have an impact on this measure, except for a few groups with less experience. GRID-HAMD received favorable opinions from most of the participants.

Infant polysomnography: reliability and validity of infant arousal assessment.

PubMed

Crowell, David H; Kulp, Thomas D; Kapuniai, Linda E; Hunt, Carl E; Brooks, Lee J; Weese-Mayer, Debra E; Silvestri, Jean; Ward, Sally Davidson; Corwin, Michael; Tinsley, Larry; Peucker, Mark

2002-10-01

Infant arousal scoring based on the Atlas Task Force definition of transient EEG arousal was evaluated to determine (1). whether transient arousals can be identified and assessed reliably in infants and (2). whether arousal and no-arousal epochs scored previously by trained raters can be validated reliably by independent sleep experts. Phase I for inter- and intrarater reliability scoring was based on two datasets of sleep epochs selected randomly from nocturnal polysomnograms of healthy full-term, preterm, idiopathic apparent life-threatening event cases, and siblings of Sudden Infant Death Syndrome infants of 35 to 64 weeks postconceptional age. After training, test set 1 reliability was assessed and discrepancies identified. After retraining, test set 2 was scored by the same raters to determine interrater reliability. Later, three raters from the trained group rescored test set 2 to assess inter- and intrarater reliabilities. Interrater and intrarater reliability kappa's, with 95% confidence intervals, ranged from substantial to almost perfect levels of agreement. Interrater reliabilities for spontaneous arousals were initially moderate and then substantial. During the validation phase, 315 previously scored epochs were presented to four sleep experts to rate as containing arousal or no-arousal events. Interrater expert agreements were diverse and considered as noninterpretable. Concordance in sleep experts' agreements, based on identification of the previously sampled arousal and no-arousal epochs, was used as a secondary evaluative technique. Results showed agreement by two or more experts on 86% of the Collaborative Home Infant Monitoring Evaluation Study arousal scored events. Conversely, only 1% of the Collaborative Home Infant Monitoring Evaluation Study-scored no-arousal epochs were rated as an arousal. In summary, this study presents an empirically tested model with procedures and criteria for attaining improved reliability in transient EEG arousal assessments in infants using the modified Atlas Task Force standards. With training based on specific criteria, substantial inter- and intrarater agreement in identifying infant arousals was demonstrated. Corroborative validation results were too disparate for meaningful interpretation. Alternate evaluation based on concordance agreements supports reliance on infant EEG criteria for assessment. Results mandate additional confirmatory validation studies with specific training on infant EEG arousal assessment criteria.

Simulation: Moving from Technology Challenge to Human Factors Success

DOE Office of Scientific and Technical Information (OSTI.GOV)

Gould, Derek A., E-mail: dgould@liv.ac.uk; Chalmers, Nicholas; Johnson, Sheena J.

2012-06-15

Recognition of the many limitations of traditional apprenticeship training is driving new approaches to learning medical procedural skills. Among simulation technologies and methods available today, computer-based systems are topical and bring the benefits of automated, repeatable, and reliable performance assessments. Human factors research is central to simulator model development that is relevant to real-world imaging-guided interventional tasks and to the credentialing programs in which it would be used.

The new GRID Hamilton Rating Scale for Depression demonstrates excellent inter-rater reliability for inexperienced and experienced raters before and after training.

PubMed

Tabuse, Hideaki; Kalali, Amir; Azuma, Hideki; Ozaki, Norio; Iwata, Nakao; Naitoh, Hiroshi; Higuchi, Teruhiko; Kanba, Shigenobu; Shioe, Kunihiko; Akechi, Tatsuo; Furukawa, Toshi A

2007-09-30

The Hamilton Rating Scale for Depression (HAMD) is the de facto international gold standard for the assessment of depression. There are some criticisms, however, especially with regard to its inter-rater reliability, due to the lack of standardized questions or explicit scoring procedures. The GRID-HAMD was developed to provide standardized explicit scoring conventions and a structured interview guide for administration and scoring of the HAMD. We developed the Japanese version of the GRID-HAMD and examined its inter-rater reliability among experienced and inexperienced clinicians (n=70), how rater characteristics may affect it, and how training can improve it in the course of a model training program using videotaped interviews. The results showed that the inter-rater reliability of the GRID-HAMD total score was excellent to almost perfect and those of most individual items were also satisfactory to excellent, both with experienced and inexperienced raters, and both before and after the training. With its standardized definitions, questions and detailed scoring conventions, the GRID-HAMD appears to be the best achievable set of interview guides for the HAMD and can provide a solid tool for highly reliable assessment of depression severity.

Development and interrater reliability testing of a telephone interview training programme for Australian nurse interviewers.

PubMed

Ahern, Tracey; Gardner, Anne; Gardner, Glenn; Middleton, Sandy; Della, Phillip

2013-05-01

The final phase of a three phase study analysing the implementation and impact of the nurse practitioner role in Australia (the Australian Nurse Practitioner Project or AUSPRAC) was undertaken in 2009, requiring nurse telephone interviewers to gather information about health outcomes directly from patients and their treating nurse practitioners. A team of several registered nurses was recruited and trained as telephone interviewers. The aim of this paper is to report on development and evaluation of the training process for telephone interviewers. The training process involved planning the content and methods to be used in the training session; delivering the session; testing skills and understanding of interviewers post-training; collecting and analysing data to determine the degree to which the training process was successful in meeting objectives and post-training follow-up. All aspects of the training process were informed by established educational principles. Interrater reliability between interviewers was high for well-validated sections of the survey instrument resulting in 100% agreement between interviewers. Other sections with unvalidated questions showed lower agreement (between 75% and 90%). Overall the agreement between interviewers was 92%. Each interviewer was also measured against a specifically developed master script or gold standard and for this each interviewer achieved a percentage of correct answers of 94.7% or better. This equated to a Kappa value of 0.92 or better. The telephone interviewer training process was very effective and achieved high interrater reliability. We argue that the high reliability was due to the use of well validated instruments and the carefully planned programme based on established educational principles. There is limited published literature on how to successfully operationalise educational principles and tailor them for specific research studies; this report addresses this knowledge gap. Copyright © 2012 Elsevier Ltd. All rights reserved.

76 FR 35031 - Notice of Quarterly Report (January 1, 2011-March 31, 2011)

Federal Register 2010, 2011, 2012, 2013, 2014

2011-06-15

... Number of farmers Profitability of trained in Commercial cultivation, Agriculture. services to Number of.... Knowledge of good hygiene practices. Households with reliable water services. Enterprises with reliable... by public outreach efforts. Personnel trained. Number of Services Fonciers Ruraux (rural land service...

Test-retest reliability of the Military Pre-training Questionnaire.

PubMed

Robinson, M; Stokes, K; Bilzon, J; Standage, M; Brown, P; Thompson, D

2010-09-01

Musculoskeletal injuries are a significant cause of morbidity during military training. A brief, inexpensive and user-friendly tool that demonstrates reliability and validity is warranted to effectively monitor the relationship between multiple predictor variables and injury incidence in military populations. To examine the test-retest reliability of the Military Pre-training Questionnaire (MPQ), designed specifically to assess risk factors for injury among military trainees across five domains (physical activity, injury history, diet, alcohol and smoking). Analyses were based on a convenience sample of 58 male British Army trainees. Kappa (kappa), weighted kappa (kappa(w)) and intraclass correlation coefficients (ICC) were used to evaluate the 2-week test-retest reliability of the MPQ. For index measures constituting the assessment of a given construct, internal consistency was assessed by Cronbach's alpha (alpha) coefficients. Reliability of individual items ranged from poor to almost perfect (kappa range = 0.45-0.86; kappa(w) range = 0.11-0.91; ICC range = 0.34-0.86) with most items demonstrating moderate reliability. Overall scores related to physical activity, diet, alcohol and smoking constructs were reliable between both administrations (ICC = 0.63-0.85). Support for the internal consistency of the incorporated alcohol (alpha = 0.78) and cigarette (alpha = 0.75) scales was also provided. The MPQ is a reliable self-report instrument for assessing multiple injury-related risk factors during initial military training. Further assessment of the psychometric properties of the MPQ (e.g. different types of validity) with military populations/samples will support its interpretation and use in future surveillance and epidemiological studies.

An evidence-based decision assistance model for predicting training outcome in juvenile guide dogs.

PubMed

Harvey, Naomi D; Craigon, Peter J; Blythe, Simon A; England, Gary C W; Asher, Lucy

2017-01-01

Working dog organisations, such as Guide Dogs, need to regularly assess the behaviour of the dogs they train. In this study we developed a questionnaire-style behaviour assessment completed by training supervisors of juvenile guide dogs aged 5, 8 and 12 months old (n = 1,401), and evaluated aspects of its reliability and validity. Specifically, internal reliability, temporal consistency, construct validity, predictive criterion validity (comparing against later training outcome) and concurrent criterion validity (comparing against a standardised behaviour test) were evaluated. Thirty-nine questions were sourced either from previously published literature or created to meet requirements identified via Guide Dogs staff surveys and staff feedback. Internal reliability analyses revealed seven reliable and interpretable trait scales named according to the questions within them as: Adaptability; Body Sensitivity; Distractibility; Excitability; General Anxiety; Trainability and Stair Anxiety. Intra-individual temporal consistency of the scale scores between 5-8, 8-12 and 5-12 months was high. All scales excepting Body Sensitivity showed some degree of concurrent criterion validity. Predictive criterion validity was supported for all seven scales, since associations were found with training outcome, at at-least one age. Thresholds of z-scores on the scales were identified that were able to distinguish later training outcome by identifying 8.4% of all dogs withdrawn for behaviour and 8.5% of all qualified dogs, with 84% and 85% specificity. The questionnaire assessment was reliable and could detect traits that are consistent within individuals over time, despite juvenile dogs undergoing development during the study period. By applying thresholds to scores produced from the questionnaire this assessment could prove to be a highly valuable decision-making tool for Guide Dogs. This is the first questionnaire-style assessment of juvenile dogs that has shown value in predicting the training outcome of individual working dogs.

Learning to rank image tags with limited training examples.

PubMed

Songhe Feng; Zheyun Feng; Rong Jin

2015-04-01

With an increasing number of images that are available in social media, image annotation has emerged as an important research topic due to its application in image matching and retrieval. Most studies cast image annotation into a multilabel classification problem. The main shortcoming of this approach is that it requires a large number of training images with clean and complete annotations in order to learn a reliable model for tag prediction. We address this limitation by developing a novel approach that combines the strength of tag ranking with the power of matrix recovery. Instead of having to make a binary decision for each tag, our approach ranks tags in the descending order of their relevance to the given image, significantly simplifying the problem. In addition, the proposed method aggregates the prediction models for different tags into a matrix, and casts tag ranking into a matrix recovery problem. It introduces the matrix trace norm to explicitly control the model complexity, so that a reliable prediction model can be learned for tag ranking even when the tag space is large and the number of training images is limited. Experiments on multiple well-known image data sets demonstrate the effectiveness of the proposed framework for tag ranking compared with the state-of-the-art approaches for image annotation and tag ranking.

Cross-view gait recognition using joint Bayesian

NASA Astrophysics Data System (ADS)

Li, Chao; Sun, Shouqian; Chen, Xiaoyu; Min, Xin

2017-07-01

Human gait, as a soft biometric, helps to recognize people by walking. To further improve the recognition performance under cross-view condition, we propose Joint Bayesian to model the view variance. We evaluated our prosed method with the largest population (OULP) dataset which makes our result reliable in a statically way. As a result, we confirmed our proposed method significantly outperformed state-of-the-art approaches for both identification and verification tasks. Finally, sensitivity analysis on the number of training subjects was conducted, we find Joint Bayesian could achieve competitive results even with a small subset of training subjects (100 subjects). For further comparison, experimental results, learning models, and test codes are available.

Reliability in fiber optic cable harness manufacturing

NASA Astrophysics Data System (ADS)

McCoy, Bruce M.

Key aspects of manufacturing cable harnesses for aircraft and spacecraft that incorporate optical fiber/cables along with traditional wiring are discussed. Issues regarding feasibility of automation of assembly processes, manual assembly, testing, installation, quality assurance, reliability and maintainability are addressed. Training procedures, formal training programs, and their results are reviewed.

Improving the Cost Efficiency and Readiness of MC-130 Aircrew Training: A Case Study

DTIC Science & Technology

2015-01-01

51 Jiang, Changbing, "A reliable solver of Euclidean traveling salesman problems with Microsoft excel add-in tools for small-size systems...DisplayPage.aspx?DocType=Reference&ItemId=+++1 343364&Pubabbrev=JAWA 124 Jiang, Changbing, "A Reliable Solver of Euclidean Traveling Salesman Problems with...49 Figure 4.5 Training Resources Locations Traveling Salesperson Problem In order to participate in training, aircrews must fly to the

Development, test-retest reliability, and construct validity of the resistance training skills battery.

PubMed

Lubans, David R; Smith, Jordan J; Harries, Simon K; Barnett, Lisa M; Faigenbaum, Avery D

2014-05-01

The aim of this study was to describe the development and assess test-retest reliability and construct validity of the Resistance Training Skills Battery (RTSB) for adolescents. The RTSB provides an assessment of resistance training skill competency and includes 6 exercises (i.e., body weight squat, push-up, lunge, suspended row, standing overhead press, and front support with chest touches). Scoring for each skill is based on the number of performance criteria successfully demonstrated. An overall resistance training skill quotient (RTSQ) is created by adding participants' scores for the 6 skills. Participants (44 boys and 19 girls, mean age = 14.5 ± 1.2 years) completed the RTSB on 2 occasions separated by 7 days. Participants also completed the following fitness tests, which were used to create a muscular fitness score (MFS): handgrip strength, timed push-up, and standing long jump tests. Intraclass correlation (ICC), paired samples t-tests, and typical error were used to assess test-retest reliability. To assess construct validity, gender and RTSQ were entered into a regression model predicting MFS. The rank order repeatability of the RTSQ was high (ICC = 0.88). The model explained 39% of the variance in MFS (p ≤ 0.001) and RTSQ (r = 0.40, p ≤ 0.001) was a significant predictor. This study has demonstrated the construct validity and test-retest reliability of the RTSB in a sample of adolescents. The RTSB can reliably rank participants in regards to their resistance training competency and has the necessary sensitivity to detect small changes in resistance training skill proficiency.

Strategic assessment of the availability of pediatric trauma care equipment, technology and supplies in Ghana.

PubMed

Ankomah, James; Stewart, Barclay T; Oppong-Nketia, Victor; Koranteng, Adofo; Gyedu, Adam; Quansah, Robert; Donkor, Peter; Abantanga, Francis; Mock, Charles

2015-11-01

This study aimed to assess the availability of pediatric trauma care items (i.e. equipment, supplies, technology) and factors contributing to deficiencies in Ghana. Ten universal and 9 pediatric-sized items were selected from the World Health Organization's Guidelines for Essential Trauma Care. Direct inspection and structured interviews with administrative, clinical and biomedical engineering staff were used to assess item availability at 40 purposively sampled district, regional and tertiary hospitals in Ghana. Hospital assessments demonstrated marked deficiencies for a number of essential items (e.g. basic airway supplies, chest tubes, blood pressure cuffs, electrolyte determination, portable X-ray). Lack of pediatric-sized items resulting from equipment absence, lack of training, frequent stock-outs and technology breakage were common. Pediatric items were consistently less available than adult-sized items at each hospital level. This study identified several successes and problems with pediatric trauma care item availability in Ghana. Item availability could be improved, both affordably and reliably, by better organization and planning (e.g. regular assessment of demand and inventory, reliable financing for essential trauma care items). In addition, technology items were often broken. Developing local service and biomedical engineering capability was highlighted as a priority to avoid long periods of equipment breakage. Copyright © 2015 Elsevier Inc. All rights reserved.

Strategic assessment of the availability of pediatric trauma care equipment, technology and supplies in Ghana

PubMed Central

Ankomah, James; Stewart, Barclay T; Oppong-Nketia, Victor; Koranteng, Adofo; Gyedu, Adam; Quansah, Robert; Donkor, Peter; Abantanga, Francis; Mock, Charles

2015-01-01

Background This study aimed to assess the availability of pediatric trauma care items (i.e. equipment, supplies, technology) and factors contributing to deficiencies in Ghana. Methods Ten universal and 9 pediatric-sized items were selected from the World Health Organization’s Guidelines for Essential Trauma Care. Direct inspection and structured interviews with administrative, clinical and biomedical engineering staff were used to assess item availability at 40 purposively sampled district, regional and tertiary hospitals in Ghana. Results Hospital assessments demonstrated marked deficiencies for a number of essential items (e.g. basic airway supplies, chest tubes, blood pressure cuffs, electrolyte determination, portable Xray). Lack of pediatric-sized items resulting from equipment absence, lack of training, frequent stock-outs and technology breakage were common. Pediatric items were consistently less available than adult-sized items at each hospital level. Conclusion This study identified several successes and problems with pediatric trauma care item availability in Ghana. Item availability could be improved, both affordably and reliably, by better organization and planning (e.g. regular assessment of demand and inventory, reliable financing for essential trauma care items). In addition, technology items were often broken. Developing local service and biomedical engineering capability was highlighted as a priority to avoid long periods of equipment breakage. PMID:25841284

Home food and activity assessment. Development and validation of an instrument for diverse families of young children.

PubMed

Boles, Richard E; Burdell, Alexandra; Johnson, Susan L; Gavin, William J; Davies, Patricia L; Bellows, Laura L

2014-09-01

The purpose of this study was to refine and psychometrically test an instrument measuring the home food and activity environment of geographically and economically diverse families of preschool aged children. Caregivers of preschool aged children (n = 83) completed a modified self-report questionnaire. Reliably trained researchers conducted independent observations on 25 randomly selected homes. Agreement statistics were conducted at the item level (154 total items) to determine reliability. Frequency counts were calculated to identify item availability. Results showed Kappa statistics were high (.67-1.00) between independent researchers but varied between researchers and parents resulting in 85 items achieving criterion validity (Kappa >.60). Analyses of reliable items revealed the presence in the home of a high frequency of unhealthy snack foods, high fat milk and low frequency of availability of fruits/vegetables and low fat milk. Fifty-two percent of the homes were arranged with a television in the preschool child's bedroom. Physical Activity devices also were found to have high frequency availability. Families reporting lower education reported higher levels of sugar sweetened beverages and less low-fat dairy (p < .05) compared with higher education families. Low-income families (<$27K per year) reported significantly fewer Physical Activity devices (p < .001) compared with higher income families. Hispanic families reported significantly higher numbers of Sedentary Devices (p < .05) compared with non-Hispanic families. There were no significant differences between demographic comparisons on available fruits/vegetables, meats, whole grains, and regular fat dairy. A modified home food and activity instrument was found to reliably identify foods and activity devices with geographically and economically diverse families. Copyright © 2014 Elsevier Ltd. All rights reserved.

Lessons learned developing a diagnostic tool for HIV-associated dementia feasible to implement in resource-limited settings: pilot testing in Kenya.

PubMed

Kwasa, Judith; Cettomai, Deanna; Lwanya, Edwin; Osiemo, Dennis; Oyaro, Patrick; Birbeck, Gretchen L; Price, Richard W; Bukusi, Elizabeth A; Cohen, Craig R; Meyer, Ana-Claire L

2012-01-01

To conduct a preliminary evaluation of the utility and reliability of a diagnostic tool for HIV-associated dementia (HAD) for use by primary health care workers (HCW) which would be feasible to implement in resource-limited settings. In resource-limited settings, HAD is an indication for anti-retroviral therapy regardless of CD4 T-cell count. Anti-retroviral therapy, the treatment for HAD, is now increasingly available in resource-limited settings. Nonetheless, HAD remains under-diagnosed likely because of limited clinical expertise and availability of diagnostic tests. Thus, a simple diagnostic tool which is practical to implement in resource-limited settings is an urgent need. A convenience sample of 30 HIV-infected outpatients was enrolled in Western Kenya. We assessed the sensitivity and specificity of a diagnostic tool for HAD as administered by a primary HCW. This was compared to an expert clinical assessment which included examination by a physician, neuropsychological testing, and in selected cases, brain imaging. Agreement between HCW and an expert examiner on certain tool components was measured using Kappa statistic. The sample was 57% male, mean age was 38.6 years, mean CD4 T-cell count was 323 cells/µL, and 54% had less than a secondary school education. Six (20%) of the subjects were diagnosed with HAD by expert clinical assessment. The diagnostic tool was 63% sensitive and 67% specific for HAD. Agreement between HCW and expert examiners was poor for many individual items of the diagnostic tool (K = .03-.65). This diagnostic tool had moderate sensitivity and specificity for HAD. However, reliability was poor, suggesting that substantial training and formal evaluations of training adequacy will be critical to enable HCW to reliably administer a brief diagnostic tool for HAD.

Frame-of-Reference Training: Establishing Reliable Assessment of Teaching Effectiveness.

PubMed

Newman, Lori R; Brodsky, Dara; Jones, Richard N; Schwartzstein, Richard M; Atkins, Katharyn Meredith; Roberts, David H

2016-01-01

Frame-of-reference (FOR) training has been used successfully to teach faculty how to produce accurate and reliable workplace-based ratings when assessing a performance. We engaged 21 Harvard Medical School faculty members in our pilot and implementation studies to determine the effectiveness of using FOR training to assess health professionals' teaching performances. All faculty were novices at rating their peers' teaching effectiveness. Before FOR training, we asked participants to evaluate a recorded lecture using a criterion-based peer assessment of medical lecturing instrument. At the start of training, we discussed the instrument and emphasized its precise behavioral standards. During training, participants practiced rating lectures and received immediate feedback on how well they categorized and scored performances as compared with expert-derived scores of the same lectures. At the conclusion of the training, we asked participants to rate a post-training recorded lecture to determine agreement with the experts' scores. Participants and experts had greater rating agreement for the post-training lecture compared with the pretraining lecture. Through this investigation, we determined that FOR training is a feasible method to teach faculty how to accurately and reliably assess medical lectures. Medical school instructors and continuing education presenters should have the opportunity to be observed and receive feedback from trained peer observers. Our results show that it is possible to use FOR rater training to teach peer observers how to accurately rate medical lectures. The process is time efficient and offers the prospect for assessment and feedback beyond traditional learner evaluation of instruction.

Automated MRI Cerebellar Size Measurements Using Active Appearance Modeling

PubMed Central

Price, Mathew; Cardenas, Valerie A.; Fein, George

2014-01-01

Although the human cerebellum has been increasingly identified as an important hub that shows potential for helping in the diagnosis of a large spectrum of disorders, such as alcoholism, autism, and fetal alcohol spectrum disorder, the high costs associated with manual segmentation, and low availability of reliable automated cerebellar segmentation tools, has resulted in a limited focus on cerebellar measurement in human neuroimaging studies. We present here the CATK (Cerebellar Analysis Toolkit), which is based on the Bayesian framework implemented in FMRIB’s FIRST. This approach involves training Active Appearance Models (AAM) using hand-delineated examples. CATK can currently delineate the cerebellar hemispheres and three vermal groups (lobules I–V, VI–VII, and VIII–X). Linear registration with the low-resolution MNI152 template is used to provide initial alignment, and Point Distribution Models (PDM) are parameterized using stellar sampling. The Bayesian approach models the relationship between shape and texture through computation of conditionals in the training set. Our method varies from the FIRST framework in that initial fitting is driven by 1D intensity profile matching, and the conditional likelihood function is subsequently used to refine fitting. The method was developed using T1-weighted images from 63 subjects that were imaged and manually labeled: 43 subjects were scanned once and were used for training models, and 20 subjects were imaged twice (with manual labeling applied to both runs) and used to assess reliability and validity. Intraclass correlation analysis shows that CATK is highly reliable (average test-retest ICCs of 0.96), and offers excellent agreement with the gold standard (average validity ICC of 0.87 against manual labels). Comparisons against an alternative atlas-based approach, SUIT (Spatially Unbiased Infratentorial Template), that registers images with a high-resolution template of the cerebellum, show that our AAM approach offers superior reliability and validity. Extensions of CATK to cerebellar hemisphere parcels is envisioned. PMID:25192657

Frame-of-reference training for simulation-based intraoperative communication assessment.

PubMed

Gardner, Aimee K; Russo, Michael A; Jabbour, Ibrahim I; Kosemund, Matthew; Scott, Daniel J

2016-09-01

The purpose of this study was to examine the impact of frame-of-reference (FOR) training on assessments of intraoperative communication skills and identify areas of need to inform curricular efforts. Simulation instructors (M.D., Ph.D., Research Fellow, Simulation Technician) underwent a 2-hour FOR training session with the operating room communication instrument. They then independently rated communication skills of 19 PGY1s who participated in a team-based simulation. Residents completed self-assessments via video review of the scenario. Intraclass correlation coefficients were used to examine inter-rater reliability. Relationships between trained raters and resident scores were assessed with Pearson correlation coefficients and paired sample t tests. Inter-reliability after FOR training was .91. The correlation between trained rater scores and resident evaluations was nonsignificant. Residents significantly underestimated their intraoperative communication skills (P < .05). Use of names, closed loop communication, and sharing information with team members demonstrated consistently low ratings among all residents. These findings reveal that a number of individuals can be trained to reliably rate resident intraoperative communication performance and that residents tend to under-rate their communication skills. Copyright © 2016 Elsevier Inc. All rights reserved.

Vocational training and employability: Evaluation evidence from Romania.

PubMed

Popescu, Madalina Ecaterina; Roman, Monica

2018-04-01

This study evaluates the direct effects of vocational training, which is a popular active labour market policy in a European developing country such as Romania. Since the available official statistical microdata were insufficient to conduct reliable impact evaluations, the main findings were obtained through a counterfactual impact evaluation using newly produced micro survey data. Moreover, the research provides a heterogeneity analysis of groups of trainees, in order to identify the categories for which the programme performs best. The main results reveal that the training measure has a positive, but modest impact upon employability in Romania: participation increases employment chances properly controlled by 15%. It is most successful for women and for people living in urban areas. Measures for increasing the impacts of the vocational training programme in Romania are identified in terms of better targeting and profiling the trainees and closer adjustment of the programme to the specific needs of the labour market. Copyright © 2017 Elsevier Ltd. All rights reserved.

Can training improve the quality of inferences made by raters in competency modeling? A quasi-experiment.

PubMed

Lievens, Filip; Sanchez, Juan I

2007-05-01

A quasi-experiment was conducted to investigate the effects of frame-of-reference training on the quality of competency modeling ratings made by consultants. Human resources consultants from a large consulting firm were randomly assigned to either a training or a control condition. The discriminant validity, interrater reliability, and accuracy of the competency ratings were significantly higher in the training group than in the control group. Further, the discriminant validity and interrater reliability of competency inferences were highest among an additional group of trained consultants who also had competency modeling experience. Together, these results suggest that procedural interventions such as rater training can significantly enhance the quality of competency modeling. 2007 APA, all rights reserved

Feature reliability determines specificity and transfer of perceptual learning in orientation search.

PubMed

Yashar, Amit; Denison, Rachel N

2017-12-01

Training can modify the visual system to produce a substantial improvement on perceptual tasks and therefore has applications for treating visual deficits. Visual perceptual learning (VPL) is often specific to the trained feature, which gives insight into processes underlying brain plasticity, but limits VPL's effectiveness in rehabilitation. Under what circumstances VPL transfers to untrained stimuli is poorly understood. Here we report a qualitatively new phenomenon: intrinsic variation in the representation of features determines the transfer of VPL. Orientations around cardinal are represented more reliably than orientations around oblique in V1, which has been linked to behavioral consequences such as visual search asymmetries. We studied VPL for visual search of near-cardinal or oblique targets among distractors of the other orientation while controlling for other display and task attributes, including task precision, task difficulty, and stimulus exposure. Learning was the same in all training conditions; however, transfer depended on the orientation of the target, with full transfer of learning from near-cardinal to oblique targets but not the reverse. To evaluate the idea that representational reliability was the key difference between the orientations in determining VPL transfer, we created a model that combined orientation-dependent reliability, improvement of reliability with learning, and an optimal search strategy. Modeling suggested that not only search asymmetries but also the asymmetric transfer of VPL depended on preexisting differences between the reliability of near-cardinal and oblique representations. Transfer asymmetries in model behavior also depended on having different learning rates for targets and distractors, such that greater learning for low-reliability distractors facilitated transfer. These findings suggest that training on sensory features with intrinsically low reliability may maximize the generalizability of learning in complex visual environments.

Reliability of real-time ultrasound measurement of transversus abdominis thickness in healthy trained subjects.

PubMed

Gnat, Rafael; Saulicz, Edward; Miądowicz, Barbara

2012-08-01

To investigate intra- and inter-rater reliability of the ultrasound measurement of transversus abdominis (TrA) thickness and thickness change (difference between thickness at rest and during contraction) in asymptomatic, trained subjects. To define the number of repeated measurements that provide acceptable level of reliability. To investigate variability of the measurements over time of 5 days and the reliability of duplicate analysis of images. A single-group repeated-measures design was used to assess reliability. Healthy volunteers (n = 10) were subjected to 1-week training in voluntary activation of TrA. Real-time ultrasound imaging and subsequent measurement of the TrA thickness at rest and during voluntary contraction were repeated on Monday, Wednesday and Friday of the next week. Using a single repeated measurement, intraclass correlation coefficients (ICCs) for TrA thickness were: 0.86-0.95 (intra-rater), 0.86-0.92 (inter-rater); and for TrA thickness change: 0.34-0.56 (intra-rater), 0.47-0.61 (inter-rater). Using the mean of three repeated measurements respective values were: 0.97, 0.96-0.98; and 0.81-0.84, 0.80-0.90. No significant differences were found between mean values of TrA thickness as well as thickness change obtained on three consecutive measurement days. Duplicate analysis of the images was highly reliable with ICCs of 0.89-0.99. Two repeated measurements for TrA thickness and at least three measurements for TrA thickness change are needed to achieve acceptable levels of intra- and inter-rater reliability. In healthy trained volunteers TrA thickness and thickness change are relatively stable parameters over a 5-day period. Duplicate analysis of the same images by two blinded observers is reliable.

Feature reliability determines specificity and transfer of perceptual learning in orientation search

PubMed Central

2017-01-01

Training can modify the visual system to produce a substantial improvement on perceptual tasks and therefore has applications for treating visual deficits. Visual perceptual learning (VPL) is often specific to the trained feature, which gives insight into processes underlying brain plasticity, but limits VPL’s effectiveness in rehabilitation. Under what circumstances VPL transfers to untrained stimuli is poorly understood. Here we report a qualitatively new phenomenon: intrinsic variation in the representation of features determines the transfer of VPL. Orientations around cardinal are represented more reliably than orientations around oblique in V1, which has been linked to behavioral consequences such as visual search asymmetries. We studied VPL for visual search of near-cardinal or oblique targets among distractors of the other orientation while controlling for other display and task attributes, including task precision, task difficulty, and stimulus exposure. Learning was the same in all training conditions; however, transfer depended on the orientation of the target, with full transfer of learning from near-cardinal to oblique targets but not the reverse. To evaluate the idea that representational reliability was the key difference between the orientations in determining VPL transfer, we created a model that combined orientation-dependent reliability, improvement of reliability with learning, and an optimal search strategy. Modeling suggested that not only search asymmetries but also the asymmetric transfer of VPL depended on preexisting differences between the reliability of near-cardinal and oblique representations. Transfer asymmetries in model behavior also depended on having different learning rates for targets and distractors, such that greater learning for low-reliability distractors facilitated transfer. These findings suggest that training on sensory features with intrinsically low reliability may maximize the generalizability of learning in complex visual environments. PMID:29240813

Training Raters to Assess Adult ADHD: Reliability of Ratings

ERIC Educational Resources Information Center

Adler, Lenard A.; Spencer, Thomas; Faraone, Stephen V.; Reimherr, Fred W.; Kelsey, Douglas; Michelson, David; Biederman, Joseph

2005-01-01

The standardization of ADHD ratings in adults is important given their differing symptom presentation. The authors investigated the agreement and reliability of rater standardization in a large-scale trial of atomoxetine in adults with ADHD. Training of 91 raters for the investigator-administered ADHD Rating Scale (ADHDRS-IV-Inv) occurred prior to…

The Effect of Instrument-Specific Rater Training on Interrater Reliability and Counseling Skills Performance Differentiation

ERIC Educational Resources Information Center

Meacham, Paul Douglas, Jr.

2013-01-01

The purpose of this study was to explore the effect of instrument-specific rater training on interrater reliability (IRR) and counseling skills performance differentiation. Strong IRR is of primary concern to effective program evaluation (McCullough, Kuhn, Andrews, Valen, Hatch, & Osimo, 2003; Schanche, Nielsen, McCullough, Valen, &…

Standardized Patients Provide a Reliable Assessment of Athletic Training Students' Clinical Skills

ERIC Educational Resources Information Center

Armstrong, Kirk J.; Jarriel, Amanda J.

2016-01-01

Context: Providing students reliable objective feedback regarding their clinical performance is of great value for ongoing clinical skill assessment. Since a standardized patient (SP) is trained to consistently portray the case, students can be assessed and receive immediate feedback within the same clinical encounter; however, no research, to our…

TOXNET and Beyond - Using the NLMs Environmental Health and Toxicology Portal-February

DOE Office of Scientific and Technical Information (OSTI.GOV)

Templin-Branner, W.

2010-02-24

The purpose of this training is to familiarize participants with reliable online environmental health and toxicology information, from the National Library of Medicine and other reliable sources. Skills and knowledge acquired in this training class will enable participants to access, utilize, and refer others to environmental health and toxicology information.

Reliable assessment of general surgeons' non-technical skills based on video-recordings of patient simulated scenarios.

PubMed

Spanager, Lene; Beier-Holgersen, Randi; Dieckmann, Peter; Konge, Lars; Rosenberg, Jacob; Oestergaard, Doris

2013-11-01

Nontechnical skills are essential for safe and efficient surgery. The aim of this study was to evaluate the reliability of an assessment tool for surgeons' nontechnical skills, Non-Technical Skills for Surgeons dk (NOTSSdk), and the effect of rater training. A 1-day course was conducted for 15 general surgeons in which they rated surgeons' nontechnical skills in 9 video recordings of scenarios simulating real intraoperative situations. Data were gathered from 2 sessions separated by a 4-hour training session. Interrater reliability was high for both pretraining ratings (Cronbach's α = .97) and posttraining ratings (Cronbach's α = .98). There was no statistically significant development in assessment skills. The D study showed that 2 untrained raters or 1 trained rater was needed to obtain generalizability coefficients >.80. The high pretraining interrater reliability indicates that videos were easy to rate and Non-Technical Skills for Surgeons dk easy to use. This implies that Non-Technical Skills for Surgeons dk (NOTSSdk) could be an important tool in surgical training, potentially improving safety and quality for surgical patients. Copyright © 2013 Elsevier Inc. All rights reserved.

A study on the real-time reliability of on-board equipment of train control system

NASA Astrophysics Data System (ADS)

Zhang, Yong; Li, Shiwei

2018-05-01

Real-time reliability evaluation is conducive to establishing a condition based maintenance system for the purpose of guaranteeing continuous train operation. According to the inherent characteristics of the on-board equipment, the connotation of reliability evaluation of on-board equipment is defined and the evaluation index of real-time reliability is provided in this paper. From the perspective of methodology and practical application, the real-time reliability of the on-board equipment is discussed in detail, and the method of evaluating the realtime reliability of on-board equipment at component level based on Hidden Markov Model (HMM) is proposed. In this method the performance degradation data is used directly to realize the accurate perception of the hidden state transition process of on-board equipment, which can achieve a better description of the real-time reliability of the equipment.

Inter-rater reliability of data elements from a prototype of the Paul Coverdell National Acute Stroke Registry

PubMed Central

Reeves, Mathew J; Mullard, Andrew J; Wehner, Susan

2008-01-01

Background The Paul Coverdell National Acute Stroke Registry (PCNASR) is a U.S. based national registry designed to monitor and improve the quality of acute stroke care delivered by hospitals. The registry monitors care through specific performance measures, the accuracy of which depends in part on the reliability of the individual data elements used to construct them. This study describes the inter-rater reliability of data elements collected in Michigan's state-based prototype of the PCNASR. Methods Over a 6-month period, 15 hospitals participating in the Michigan PCNASR prototype submitted data on 2566 acute stroke admissions. Trained hospital staff prospectively identified acute stroke admissions, abstracted chart information, and submitted data to the registry. At each hospital 8 randomly selected cases were re-abstracted by an experienced research nurse. Inter-rater reliability was estimated by the kappa statistic for nominal variables, and intraclass correlation coefficient (ICC) for ordinal and continuous variables. Factors that can negatively impact the kappa statistic (i.e., trait prevalence and rater bias) were also evaluated. Results A total of 104 charts were available for re-abstraction. Excellent reliability (kappa or ICC > 0.75) was observed for many registry variables including age, gender, black race, hemorrhagic stroke, discharge medications, and modified Rankin Score. Agreement was at least moderate (i.e., 0.75 > kappa ≥; 0.40) for ischemic stroke, TIA, white race, non-ambulance arrival, hospital transfer and direct admit. However, several variables had poor reliability (kappa < 0.40) including stroke onset time, stroke team consultation, time of initial brain imaging, and discharge destination. There were marked systematic differences between hospital abstractors and the audit abstractor (i.e., rater bias) for many of the data elements recorded in the emergency department. Conclusion The excellent reliability of many of the data elements supports the use of the PCNASR to monitor and improve care. However, the poor reliability for several variables, particularly time-related events in the emergency department, indicates the need for concerted efforts to improve the quality of data collection. Specific recommendations include improvements to data definitions, abstractor training, and the development of ED-based real-time data collection systems. PMID:18547421

Familiarization, validity and smallest detectable difference of the isometric squat test in evaluating maximal strength.

PubMed

Drake, David; Kennedy, Rodney; Wallace, Eric

2018-02-06

Isometric multi-joint tests are considered reliable and have strong relationships with 1RM performance. However, limited evidence is available for the isometric squat in terms of effects of familiarization and reliability. This study aimed to assess, the effect of familiarization, stability reliability, determine the smallest detectible difference, and the correlation of the isometric squat test with 1RM squat performance. Thirty-six strength-trained participants volunteered to take part in this study. Following three familiarization sessions, test-retest reliability was evaluated with a 48-hour window between each time point. Isometric squat peak, net and relative force were assessed. Results showed three familiarizations were required, isometric squat had a high level of stability reliability and smallest detectible difference of 11% for peak and relative force. Isometric strength at a knee angle of ninety degrees had a strong significant relationship with 1RM squat performance. In conclusion, the isometric squat is a valid test to assess multi-joint strength and can discriminate between strong and weak 1RM squat performance. Changes greater than 11% in peak and relative isometric squat performance should be considered as meaningful in participants who are familiar with the test.

Inter-rater reliability of an observation-based ergonomics assessment checklist for office workers.

PubMed

Pereira, Michelle Jessica; Straker, Leon Melville; Comans, Tracy Anne; Johnston, Venerina

2016-12-01

To establish the inter-rater reliability of an observation-based ergonomics assessment checklist for computer workers. A 37-item (38-item if a laptop was part of the workstation) comprehensive observational ergonomics assessment checklist comparable to government guidelines and up to date with empirical evidence was developed. Two trained practitioners assessed full-time office workers performing their usual computer-based work and evaluated the suitability of workstations used. Practitioners assessed each participant consecutively. The order of assessors was randomised, and the second assessor was blinded to the findings of the first. Unadjusted kappa coefficients between the raters were obtained for the overall checklist and subsections that were formed from question-items relevant to specific workstation equipment. Twenty-seven office workers were recruited. The inter-rater reliability between two trained practitioners achieved moderate to good reliability for all except one checklist component. This checklist has mostly moderate to good reliability between two trained practitioners. Practitioner Summary: This reliable ergonomics assessment checklist for computer workers was designed using accessible government guidelines and supplemented with up-to-date evidence. Employers in Queensland (Australia) can fulfil legislative requirements by using this reliable checklist to identify and subsequently address potential risk factors for work-related injury to provide a safe working environment.

Development and pilot testing of an online module for ethics education based on the Nigerian National Code for Health Research Ethics

PubMed Central

2013-01-01

Background The formulation and implementation of national ethical regulations to protect research participants is fundamental to ethical conduct of research. Ethics education and capacity are inadequate in developing African countries. This study was designed to develop a module for online training in research ethics based on the Nigerian National Code of Health Research Ethics and assess its ease of use and reliability among biomedical researchers in Nigeria. Methodology This was a three-phased evaluation study. Phase one involved development of an online training module based on the Nigerian Code of Health Research Ethics (NCHRE) and uploading it to the Collaborative Institutional Training Initiative (CITI) website while the second phase entailed the evaluation of the module for comprehensibility, readability and ease of use by 45 Nigerian biomedical researchers. The third phase involved modification and re-evaluation of the module by 30 Nigerian biomedical researchers and determination of test-retest reliability of the module using Cronbach’s alpha. Results The online module was easily accessible and comprehensible to 95% of study participants. There were significant differences in the pretest and posttest scores of study participants during the evaluation of the online module (p = 0.001) with correlation coefficients of 0.9 and 0.8 for the pretest and posttest scores respectively. The module also demonstrated excellent test-retest reliability and internal consistency as shown by Cronbach’s alpha coefficients of 0.92 and 0.84 for the pretest and posttest respectively. Conclusion The module based on the Nigerian Code was developed, tested and made available online as a valuable tool for training in cultural and societal relevant ethical principles to orient national and international biomedical researchers working in Nigeria. It would complement other general research ethics and Good Clinical Practice modules. Participants suggested that awareness of the online module should be increased through seminars, advertisement on government websites and portals used by Nigerian biomedical researchers, and incorporation of the Code into the undergraduate medical training curriculum. PMID:23281968

Electronic device for endosurgical skills training (EDEST): study of reliability.

PubMed

Pagador, J B; Uson, J; Sánchez, M A; Moyano, J L; Moreno, J; Bustos, P; Mateos, J; Sánchez-Margallo, F M

2011-05-01

Minimally Invasive Surgery procedures are commonly used in many surgical practices, but surgeons need specific training models and devices due to its difficulty and complexity. In this paper, an innovative electronic device for endosurgical skills training (EDEST) is presented. A study on reliability for this device was performed. Different electronic components were used to compose this new training device. The EDEST was focused on two basic laparoscopic tasks: triangulation and coordination manoeuvres. A configuration and statistical software was developed to complement the functionality of the device. A calibration method was used to assure the proper work of the device. A total of 35 subjects (8 experts and 27 novices) were used to check the reliability of the system using the MTBF analysis. Configuration values for triangulation and coordination exercises were calculated as 0.5 s limit threshold and 800-11,000 lux range of light intensity, respectively. Zero errors in 1,050 executions (0%) for triangulation and 21 errors in 5,670 executions (0.37%) for coordination were obtained. A MTBF of 2.97 h was obtained. The results show that the reliability of the EDEST device is acceptable when used under previously defined light conditions. These results along with previous work could demonstrate that the EDEST device can help surgeons during first training stages.

Reliability testing of two classification systems for osteoarthritis and post-traumatic arthritis of the elbow.

PubMed

Amini, Michael H; Sykes, Joshua B; Olson, Stephen T; Smith, Richard A; Mauck, Benjamin M; Azar, Frederick M; Throckmorton, Thomas W

2015-03-01

The severity of elbow arthritis is one of many factors that surgeons must evaluate when considering treatment options for a given patient. Elbow surgeons have historically used the Broberg and Morrey (BM) and Hastings and Rettig (HR) classification systems to radiographically stage the severity of post-traumatic arthritis (PTA) and primary osteoarthritis (OA). We proposed to compare the intraobserver and interobserver reliability between systems for patients with either PTA or OA. The radiographs of 45 patients were evaluated at least 2 weeks apart by 6 evaluators of different levels of training. Intraobserver and interobserver reliability were calculated by Spearman correlation coefficients with 95% confidence intervals. Agreement was considered almost perfect for coefficients >0.80 and substantial for coefficients of 0.61 to 0.80. In patients with both PTA and OA, intraobserver reliability and interobserver reliability were substantial, with no difference between classification systems. There were no significant differences in intraobserver or interobserver reliability between attending physicians and trainees for either classification system (all P > .10). The presence of fracture implants did not affect reliability in the BM system but did substantially worsen reliability in the HR system (intraobserver P = .04 and interobserver P = .001). The BM and HR classifications both showed substantial intraobserver and interobserver reliability for PTA and OA. Training level differences did not affect reliability for either system. Both trainees and fellowship-trained surgeons may easily and reliably apply each classification system to the evaluation of primary elbow OA and PTA, although the HR system was less reliable in the presence of fracture implants. Copyright © 2015 Journal of Shoulder and Elbow Surgery Board of Trustees. Published by Elsevier Inc. All rights reserved.

Reliability testing of a portfolio assessment tool for postgraduate family medicine training in South Africa

PubMed Central

Mash, Bob; Derese, Anselme

2013-01-01

Abstract Background Competency-based education and the validity and reliability of workplace-based assessment of postgraduate trainees have received increasing attention worldwide. Family medicine was recognised as a speciality in South Africa six years ago and a satisfactory portfolio of learning is a prerequisite to sit the national exit exam. A massive scaling up of the number of family physicians is needed in order to meet the health needs of the country. Aim The aim of this study was to develop a reliable, robust and feasible portfolio assessment tool (PAT) for South Africa. Methods Six raters each rated nine portfolios from the Stellenbosch University programme, using the PAT, to test for inter-rater reliability. This rating was repeated three months later to determine test–retest reliability. Following initial analysis and feedback the PAT was modified and the inter-rater reliability again assessed on nine new portfolios. An acceptable intra-class correlation was considered to be > 0.80. Results The total score was found to be reliable, with a coefficient of 0.92. For test–retest reliability, the difference in mean total score was 1.7%, which was not statistically significant. Amongst the subsections, only assessment of the educational meetings and the logbook showed reliability coefficients > 0.80. Conclusion This was the first attempt to develop a reliable, robust and feasible national portfolio assessment tool to assess postgraduate family medicine training in the South African context. The tool was reliable for the total score, but the low reliability of several sections in the PAT helped us to develop 12 recommendations regarding the use of the portfolio, the design of the PAT and the training of raters.

Chaos-induced modulation of reliability boosts output firing rate in downstream cortical areas.

PubMed

Tiesinga, P H E

2004-03-01

The reproducibility of neural spike train responses to an identical stimulus across different presentations (trials) has been studied extensively. Reliability, the degree of reproducibility of spike trains, was found to depend in part on the amplitude and frequency content of the stimulus [J. Hunter and J. Milton, J. Neurophysiol. 90, 387 (2003)]. The responses across different trials can sometimes be interpreted as the response of an ensemble of similar neurons to a single stimulus presentation. How does the reliability of the activity of neural ensembles affect information transmission between different cortical areas? We studied a model neural system consisting of two ensembles of neurons with Hodgkin-Huxley-type channels. The first ensemble was driven by an injected sinusoidal current that oscillated in the gamma-frequency range (40 Hz) and its output spike trains in turn drove the second ensemble by fast excitatory synaptic potentials with short term depression. We determined the relationship between the reliability of the first ensemble and the response of the second ensemble. In our paradigm the neurons in the first ensemble were initially in a chaotic state with unreliable and imprecise spike trains. The neurons became entrained to the oscillation and responded reliably when the stimulus power was increased by less than 10%. The firing rate of the first ensemble increased by 30%, whereas that of the second ensemble could increase by an order of magnitude. We also determined the response of the second ensemble when its input spike trains, which had non-Poisson statistics, were replaced by an equivalent ensemble of Poisson spike trains. The resulting output spike trains were significantly different from the original response, as assessed by the metric introduced by Victor and Purpura [J. Neurophysiol. 76, 1310 (1996)]. These results are a proof of principle that weak temporal modulations in the power of gamma-frequency oscillations in a given cortical area can strongly affect firing rate responses downstream by way of reliability in spite of rather modest changes in firing rate in the originating area.

Validity and reliability of the session-RPE method for quantifying training in Australian football: a comparison of the CR10 and CR100 scales.

PubMed

Scott, Tannath J; Black, Cameron R; Quinn, John; Coutts, Aaron J

2013-01-01

The purpose of this study was to examine and compare the criterion validity and test-retest reliability of the CR10 and CR100 rating of perceived exertion (RPE) scales for team sport athletes that undertake high-intensity, intermittent exercise. Twenty-one male Australian football (AF) players (age: 19.0 ± 1.8 years, body mass: 83.92 ± 7.88 kg) participated the first part (part A) of this study, which examined the construct validity of the session-RPE (sRPE) method for quantifying training load in AF. Ten male athletes (age: 16.1 ± 0.5 years) participated in the second part of the study (part B), which compared the test-retest reliability of the CR10 and CR100 RPE scales. In part A, the validity of the sRPE method was assessed by examining the relationships between sRPE, and objective measures of internal (i.e., heart rate) and external training load (i.e., distance traveled), collected from AF training sessions. Part B of the study assessed the reliability of sRPE through examining the test-retest reliability of sRPE during 3 different intensities of controlled intermittent running (10, 11.5, and 13 km·h(-1)). Results from part A demonstrated strong correlations for CR10- and CR100-derived sRPE with measures of internal training load (Banisters TRIMP and Edwards TRIMP) (CR10: r = 0.83 and 0.83, and CR100: r = 0.80 and 0.81, p < 0.05). Correlations between sRPE and external training load (distance, higher speed running and player load) for both the CR10 (r = 0.81, 0.71, and 0.83) and CR100 (r = 0.78, 0.69, and 0.80) were significant (p < 0.05). Results from part B demonstrated poor reliability for both the CR10 (31.9% CV) and CR100 (38.6% CV) RPE scales after short bouts of intermittent running. Collectively, these results suggest both CR10- and CR100-derived sRPE methods have good construct validity for assessing training load in AF. The poor levels of reliability revealed under field testing indicate that the sRPE method may not be sensible to detecting small changes in exercise intensity during brief intermittent running bouts. Despite this limitation, the sRPE remains a valid method to quantify training loads in high-intensity, intermittent team sport.

Pediatric laryngeal simulator using 3D printed models: A novel technique.

PubMed

Kavanagh, Katherine R; Cote, Valerie; Tsui, Yvonne; Kudernatsch, Simon; Peterson, Donald R; Valdez, Tulio A

2017-04-01

Simulation to acquire and test technical skills is an essential component of medical education and residency training in both surgical and nonsurgical specialties. High-quality simulation education relies on the availability, accessibility, and reliability of models. The objective of this work was to describe a practical pediatric laryngeal model for use in otolaryngology residency training. Ideally, this model would be low-cost, have tactile properties resembling human tissue, and be reliably reproducible. Pediatric laryngeal models were developed using two manufacturing methods: direct three-dimensional (3D) printing of anatomical models and casted anatomical models using 3D-printed molds. Polylactic acid, acrylonitrile butadiene styrene, and high-impact polystyrene (HIPS) were used for the directly printed models, whereas a silicone elastomer (SE) was used for the casted models. The models were evaluated for anatomic quality, ease of manipulation, hardness, and cost of production. A tissue likeness scale was created to validate the simulation model. Fleiss' Kappa rating was performed to evaluate interrater agreement, and analysis of variance was performed to evaluate differences among the materials. The SE provided the most anatomically accurate models, with the tactile properties allowing for surgical manipulation of the larynx. Direct 3D printing was more cost-effective than the SE casting method but did not possess the material properties and tissue likeness necessary for surgical simulation. The SE models of the pediatric larynx created from a casting method demonstrated high quality anatomy, tactile properties comparable to human tissue, and easy manipulation with standard surgical instruments. Their use in a reliable, low-cost, accessible, modular simulation system provides a valuable training resource for otolaryngology residents. N/A. Laryngoscope, 127:E132-E137, 2017. © 2016 The American Laryngological, Rhinological and Otological Society, Inc.

[Upper limb functional assessment scale for children with Duchenne muscular dystrophy and Spinal muscular atrophy].

PubMed

Escobar, Raúl G; Lucero, Nayadet; Solares, Carmen; Espinoza, Victoria; Moscoso, Odalie; Olguín, Polín; Muñoz, Karin T; Rosas, Ricardo

2016-08-16

Duchenne muscular dystrophy (DMD) and Spinal muscular atrophy (SMA) causes significant disability and progressive functional impairment. Readily available instruments that assess functionality, especially in advanced stages of the disease, are required to monitor the progress of the disease and the impact of therapeutic interventions. To describe the development of a scale to evaluate upper limb function (UL) in patients with DMD and SMA, and describe its validation process, which includes self-training for evaluators. The development of the scale included a review of published scales, an exploratory application of a pilot scale in healthy children and those with DMD, self-training of evaluators in applying the scale using a handbook and video tutorial, and assessment of a group of children with DMD and SMA using the final scale. Reliability was assessed using Cronbach and Kendall concordance and with intra and inter-rater test-retest, and validity with concordance and factorial analysis. A high level of reliability was observed, with high internal consistency (Cronbach α=0.97), and inter-rater (Kendall W=0.96) and intra-rater concordance (r=0.97 to 0.99). The validity was demonstrated by the absence of significant differences between results by different evaluators with an expert evaluator (F=0.023, P>.5), and by the factor analysis that showed that four factors account for 85.44% of total variance. This scale is a reliable and valid tool for assessing UL functionality in children with DMD and SMA. It is also easily implementable due to the possibility of self-training and the use of simple and inexpensive materials. Copyright © 2016 Sociedad Chilena de Pediatría. Publicado por Elsevier España, S.L.U. All rights reserved.

Managing and monitoring tuberculosis using web-based tools in combination with traditional approaches

PubMed Central

Chapman, Ann LN; Darton, Thomas C; Foster, Rachel A

2013-01-01

Tuberculosis (TB) remains a global health emergency. Ongoing challenges include the coordination of national and international control programs, high levels of drug resistance in many parts of the world, and availability of accurate and rapid diagnostic tests. The increasing availability and reliability of Internet access throughout both affluent and resource-limited countries brings new opportunities to improve TB management and control through the integration of web-based technologies with traditional approaches. In this review, we explore current and potential future use of web-based tools in the areas of TB diagnosis, treatment, epidemiology, service monitoring, and teaching and training. PMID:24294008

The Reliability of In-Training Assessment when Performance Improvement Is Taken into Account

ERIC Educational Resources Information Center

van Lohuizen, Mirjam T.; Kuks, Jan B. M.; van Hell, Elisabeth A.; Raat, A. N.; Stewart, Roy E.; Cohen-Schotanus, Janke

2010-01-01

During in-training assessment students are frequently assessed over a longer period of time and therefore it can be expected that their performance will improve. We studied whether there really is a measurable performance improvement when students are assessed over an extended period of time and how this improvement affects the reliability of the…

Replicability of sight word training and phonics training in poor readers: a randomised controlled trial

PubMed Central

Kohnen, S; Jones, K; Eve, P; Banales, E; Larsen, L; Castles, A

2015-01-01

Given the importance of effective treatments for children with reading impairment, paired with growing concern about the lack of scientific replication in psychological science, the aim of this study was to replicate a quasi-randomised trial of sight word and phonics training using a randomised controlled trial (RCT) design. One group of poor readers (N = 41) did 8 weeks of phonics training (i.e., phonological decoding) and then 8 weeks of sight word training (i.e., whole-word recognition). A second group did the reverse order of training. Sight word and phonics training each had a large and significant valid treatment effect on trained irregular words and word reading fluency. In addition, combined sight word and phonics training had a moderate and significant valid treatment effect on nonword reading accuracy and fluency. These findings demonstrate the reliability of both phonics and sight word training in treating poor readers in an era where the importance of scientific reliability is under close scrutiny. PMID:26019992

Clinical indicators for routine use in the evaluation of early psychosis intervention: development, training support and inter-rater reliability.

PubMed

Catts, Stanley V; Frost, Aaron D J; O'Toole, Brian I; Carr, Vaughan J; Lewin, Terry; Neil, Amanda L; Harris, Meredith G; Evans, Russell W; Crissman, Belinda R; Eadie, Kathy

2011-01-01

Clinical practice improvement carried out in a quality assurance framework relies on routinely collected data using clinical indicators. Herein we describe the development, minimum training requirements, and inter-rater agreement of indicators that were used in an Australian multi-site evaluation of the effectiveness of early psychosis (EP) teams. Surveys of clinician opinion and face-to-face consensus-building meetings were used to select and conceptually define indicators. Operationalization of definitions was achieved by iterative refinement until clinicians could be quickly trained to code indicators reliably. Calculation of percentage agreement with expert consensus coding was based on ratings of paper-based clinical vignettes embedded in a 2-h clinician training package. Consensually agreed upon conceptual definitions for seven clinical indicators judged most relevant to evaluating EP teams were operationalized for ease-of-training. Brief training enabled typical clinicians to code indicators with acceptable percentage agreement (60% to 86%). For indicators of suicide risk, psychosocial function, and family functioning this level of agreement was only possible with less precise 'broad range' expert consensus scores. Estimated kappa values indicated fair to good inter-rater reliability (kappa > 0.65). Inspection of contingency tables (coding category by health service) and modal scores across services suggested consistent, unbiased coding across services. Clinicians are able to agree upon what information is essential to routinely evaluate clinical practice. Simple indicators of this information can be designed and coding rules can be reliably applied to written vignettes after brief training. The real world feasibility of the indicators remains to be tested in field trials.

A new tool to evaluate postgraduate training posts: the Job Evaluation Survey Tool (JEST).

PubMed

Wall, David; Goodyear, Helen; Singh, Baldev; Whitehouse, Andrew; Hughes, Elizabeth; Howes, Jonathan

2014-10-02

Three reports in 2013 about healthcare and patient safety in the UK, namely Berwick, Francis and Keogh have highlighted the need for junior doctors' views about their training experience to be heard. In the UK, the General Medical Council (GMC) quality assures medical training programmes and requires postgraduate deaneries to undertake quality management and monitoring of all training posts in their area. The aim of this study was to develop a simple trainee questionnaire for evaluation of postgraduate training posts based on the GMC, UK standards and to look at the reliability and validity including comparison with a well-established and internationally validated tool, the Postgraduate Hospital Educational Environment Measure (PHEEM). The Job Evaluation Survey Tool (JEST), a fifteen item job evaluation questionnaire was drawn up in 2006, piloted with Foundation doctors (2007), field tested with specialist paediatric registrars (2008) and used over a three year period (2008-11) by Foundation Doctors. Statistical analyses including descriptives, reliability, correlation and factor analysis were undertaken and JEST compared with PHEEM. The JEST had a reliability of 0.91 in the pilot study of 76 Foundation doctors, 0.88 in field testing of 173 Paediatric specialist registrars and 0.91 in three years of general use in foundation training with 3367 doctors completing JEST. Correlation of JEST with PHEEM was 0.80 (p < 0.001). Factor analysis showed two factors, a teaching factor and a social and lifestyle one. The JEST has proved to be a simple, valid and reliable evaluation tool in the monitoring and evaluation of postgraduate hospital training posts.

An evidence-based decision assistance model for predicting training outcome in juvenile guide dogs

PubMed Central

Craigon, Peter J.; Blythe, Simon A.; England, Gary C. W.; Asher, Lucy

2017-01-01

Working dog organisations, such as Guide Dogs, need to regularly assess the behaviour of the dogs they train. In this study we developed a questionnaire-style behaviour assessment completed by training supervisors of juvenile guide dogs aged 5, 8 and 12 months old (n = 1,401), and evaluated aspects of its reliability and validity. Specifically, internal reliability, temporal consistency, construct validity, predictive criterion validity (comparing against later training outcome) and concurrent criterion validity (comparing against a standardised behaviour test) were evaluated. Thirty-nine questions were sourced either from previously published literature or created to meet requirements identified via Guide Dogs staff surveys and staff feedback. Internal reliability analyses revealed seven reliable and interpretable trait scales named according to the questions within them as: Adaptability; Body Sensitivity; Distractibility; Excitability; General Anxiety; Trainability and Stair Anxiety. Intra-individual temporal consistency of the scale scores between 5–8, 8–12 and 5–12 months was high. All scales excepting Body Sensitivity showed some degree of concurrent criterion validity. Predictive criterion validity was supported for all seven scales, since associations were found with training outcome, at at-least one age. Thresholds of z-scores on the scales were identified that were able to distinguish later training outcome by identifying 8.4% of all dogs withdrawn for behaviour and 8.5% of all qualified dogs, with 84% and 85% specificity. The questionnaire assessment was reliable and could detect traits that are consistent within individuals over time, despite juvenile dogs undergoing development during the study period. By applying thresholds to scores produced from the questionnaire this assessment could prove to be a highly valuable decision-making tool for Guide Dogs. This is the first questionnaire-style assessment of juvenile dogs that has shown value in predicting the training outcome of individual working dogs. PMID:28614347

A Mixed-methods Study to Assess Interrater Reliability and Nurse Perception of the Braden Scale in a Tertiary Acute Care Setting.

PubMed

Ho, Chester H; Cheung, Amanda; Southern, Danielle; Ocampo, Wrechelle; Kaufman, Jaime; Hogan, David B; Baylis, Barry; Conly, John M; Stelfox, Henry T; Ghali, William A

2016-12-01

Research regarding the reliability of the Braden Scale and nurses' perspectives on the instrument for predicting pressure ulcer (PU) risk in acute care settings is limited. A mixed-methods study was conducted in a tertiary acute care facility to examine interrater reliability (IRR) of the Braden Scale and its subscales, and a qualitative survey using semi-structured interviews was conducted among nurses caring for patients in acute care units to gain nurse perspective regarding scale usability. Data were extracted from a previous retrospective, randomized, controlled trial involving adult patients with compromised mobility receiving care in a tertiary acute care hospital in Canada. One-way, intraclass correlation coefficients (ICCs) were calculated on item and total scores, and kappa statistics were used to determine reliability of categorizing patients on their risk. Interview results were categorized by common themes. Reliability was assessed on 64 patients, where nurses and research staff independently assessed enrolled participants at baseline and after 72 hours using the Braden Scale as it appeared on an electronic medical record. IRR for the total score was high (ICC = 0.807). The friction and shear item had the lowest reliability (ICC = 0.266). Reliability of categorizing patients' level of risk had moderate agreement (κ = 0.408). Three (3) major and 12 subthemes emerged from the 14 nurse interviews; nurses were aware of the scale's purpose but were uncertain of its effectiveness, some items were difficult to rate, and questions were raised as to whether using the scale enhanced patient care. Aspects identified by nurses to enhance usability included: 1) changes to the electronic version (incorporating the scale into daily assessment documents with readily available item descriptions), 2) additional training, and 3) easily available resource material to improve reliability and usability of scale. These findings need to be considered when using the Braden Scale in clinical practice. Further study of the value of the total Braden Scale and its subscales is warranted.

Human Reliability Assessments: Using the Past (Shuttle) to Predict the Future (Orion)

NASA Technical Reports Server (NTRS)

DeMott, Diana L.; Bigler, Mark A.

2017-01-01

NASA (National Aeronautics and Space Administration) Johnson Space Center (JSC) Safety and Mission Assurance (S&MA) uses two human reliability analysis (HRA) methodologies. The first is a simplified method which is based on how much time is available to complete the action, with consideration included for environmental and personal factors that could influence the human's reliability. This method is expected to provide a conservative value or placeholder as a preliminary estimate. This preliminary estimate or screening value is used to determine which placeholder needs a more detailed assessment. The second methodology is used to develop a more detailed human reliability assessment on the performance of critical human actions. This assessment needs to consider more than the time available, this would include factors such as: the importance of the action, the context, environmental factors, potential human stresses, previous experience, training, physical design interfaces, available procedures/checklists and internal human stresses. The more detailed assessment is expected to be more realistic than that based primarily on time available. When performing an HRA on a system or process that has an operational history, we have information specific to the task based on this history and experience. In the case of a Probabilistic Risk Assessment (PRA) that is based on a new design and has no operational history, providing a "reasonable" assessment of potential crew actions becomes more challenging. To determine what is expected of future operational parameters, the experience from individuals who had relevant experience and were familiar with the system and process previously implemented by NASA was used to provide the "best" available data. Personnel from Flight Operations, Flight Directors, Launch Test Directors, Control Room Console Operators, and Astronauts were all interviewed to provide a comprehensive picture of previous NASA operations. Verification of the assumptions and expectations expressed in the assessments will be needed when the procedures, flight rules, and operational requirements are developed and then finalized.

Human Reliability Assessments: Using the Past (Shuttle) to Predict the Future (Orion)

NASA Technical Reports Server (NTRS)

DeMott, Diana; Bigler, Mark

2016-01-01

NASA (National Aeronautics and Space Administration) Johnson Space Center (JSC) Safety and Mission Assurance (S&MA) uses two human reliability analysis (HRA) methodologies. The first is a simplified method which is based on how much time is available to complete the action, with consideration included for environmental and personal factors that could influence the human's reliability. This method is expected to provide a conservative value or placeholder as a preliminary estimate. This preliminary estimate or screening value is used to determine which placeholder needs a more detailed assessment. The second methodology is used to develop a more detailed human reliability assessment on the performance of critical human actions. This assessment needs to consider more than the time available, this would include factors such as: the importance of the action, the context, environmental factors, potential human stresses, previous experience, training, physical design interfaces, available procedures/checklists and internal human stresses. The more detailed assessment is expected to be more realistic than that based primarily on time available. When performing an HRA on a system or process that has an operational history, we have information specific to the task based on this history and experience. In the case of a Probabilistic Risk Assessment (PRA) that is based on a new design and has no operational history, providing a "reasonable" assessment of potential crew actions becomes more challenging. In order to determine what is expected of future operational parameters, the experience from individuals who had relevant experience and were familiar with the system and process previously implemented by NASA was used to provide the "best" available data. Personnel from Flight Operations, Flight Directors, Launch Test Directors, Control Room Console Operators and Astronauts were all interviewed to provide a comprehensive picture of previous NASA operations. Verification of the assumptions and expectations expressed in the assessments will be needed when the procedures, flight rules and operational requirements are developed and then finalized.

A High Volume Stack Sampler

NASA Technical Reports Server (NTRS)

Boubel, Richard W.

1971-01-01

The stack sampler described in this paper has been developed to overcome the difficulties of particulate sampling with presently available equipment. Its use on emissions from hog fuel fired boilers, back-fired incinerators, wigwam burners, asphalt plants, and seed cleaning cyclones is reported. The results indicate that the sampler is rapid and reliable in its use. It is relatively simple and inexpensive to operate. For most sources it should be considered over the more complicated and expensive sampling trains being used and specified.

Determination of Minimum Training Sample Size for Microarray-Based Cancer Outcome Prediction–An Empirical Assessment

PubMed Central

Cheng, Ningtao; Wu, Leihong; Cheng, Yiyu

2013-01-01

The promise of microarray technology in providing prediction classifiers for cancer outcome estimation has been confirmed by a number of demonstrable successes. However, the reliability of prediction results relies heavily on the accuracy of statistical parameters involved in classifiers. It cannot be reliably estimated with only a small number of training samples. Therefore, it is of vital importance to determine the minimum number of training samples and to ensure the clinical value of microarrays in cancer outcome prediction. We evaluated the impact of training sample size on model performance extensively based on 3 large-scale cancer microarray datasets provided by the second phase of MicroArray Quality Control project (MAQC-II). An SSNR-based (scale of signal-to-noise ratio) protocol was proposed in this study for minimum training sample size determination. External validation results based on another 3 cancer datasets confirmed that the SSNR-based approach could not only determine the minimum number of training samples efficiently, but also provide a valuable strategy for estimating the underlying performance of classifiers in advance. Once translated into clinical routine applications, the SSNR-based protocol would provide great convenience in microarray-based cancer outcome prediction in improving classifier reliability. PMID:23861920

Body mass changes during training in elite rugby union: Is a single test of hydration indices reliable?

PubMed

Black, Katherine Elizabeth; Black, Alistair David; Baker, Dane; Fairbairn, Kirsty

2018-05-28

There is limited research studying fluid and electrolyte balance in rugby union players, and a paucity of information regarding the test-retest reliability. This study describes the fluid balance of elite rugby union players across multiple squads and the reliability of fluid balance measures between two equivalent training sessions. Sixty-one elite rugby players completed a single fluid balance testing session during a game simulation training session. A subsample of 21 players completed a second fluid balance testing session during an equivalent training session. Players were weighed in minimal clothing before and after each training session. Each player was provided with their own drinks which were weighed before and after each training session. More players gained body weight (9 (14.8%)) during training than lost greater than 2% of their initial body mass (1 (1.6%)). Pre-training body mass and rate of fluid loss were significantly associated (r = 0.318, p = .013). There was a significant correlation between rate of fluid loss in sessions 1 (1.74 ± 0.32 L h -1 ) and 2 (1.10 ± 0.31 L. h -1 ), (r = 0.470, p = .032). This could be useful for nutritionists working with rugby squads to identify players with high sweat losses.

Optimal reinforcement of training datasets in semi-supervised landmark-based segmentation

NASA Astrophysics Data System (ADS)

Ibragimov, Bulat; Likar, Boštjan; Pernuš, Franjo; Vrtovec, Tomaž

2015-03-01

During the last couple of decades, the development of computerized image segmentation shifted from unsupervised to supervised methods, which made segmentation results more accurate and robust. However, the main disadvantage of supervised segmentation is a need for manual image annotation that is time-consuming and subjected to human error. To reduce the need for manual annotation, we propose a novel learning approach for training dataset reinforcement in the area of landmark-based segmentation, where newly detected landmarks are optimally combined with reference landmarks from the training dataset and therefore enriches the training process. The approach is formulated as a nonlinear optimization problem, where the solution is a vector of weighting factors that measures how reliable are the detected landmarks. The detected landmarks that are found to be more reliable are included into the training procedure with higher weighting factors, whereas the detected landmarks that are found to be less reliable are included with lower weighting factors. The approach is integrated into the landmark-based game-theoretic segmentation framework and validated against the problem of lung field segmentation from chest radiographs.

Recommendations for Assessment of the Reliability, Sensitivity, and Validity of Data Provided by Wearable Sensors Designed for Monitoring Physical Activity.

PubMed

Düking, Peter; Fuss, Franz Konstantin; Holmberg, Hans-Christer; Sperlich, Billy

2018-04-30

Although it is becoming increasingly popular to monitor parameters related to training, recovery, and health with wearable sensor technology (wearables), scientific evaluation of the reliability, sensitivity, and validity of such data is limited and, where available, has involved a wide variety of approaches. To improve the trustworthiness of data collected by wearables and facilitate comparisons, we have outlined recommendations for standardized evaluation. We discuss the wearable devices themselves, as well as experimental and statistical considerations. Adherence to these recommendations should be beneficial not only for the individual, but also for regulatory organizations and insurance companies. ©Peter Düking, Franz Konstantin Fuss, Hans-Christer Holmberg, Billy Sperlich. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 30.04.2018.

Dental examiners consistency in applying the ICDAS criteria for a caries prevention community trial.

PubMed

Nelson, S; Eggertsson, H; Powell, B; Mandelaris, J; Ntragatakis, M; Richardson, T; Ferretti, G

2011-09-01

To examine dental examiners' one-year consistency in utilizing the International Caries Detection and Assessment System (ICDAS) criteria after baseline training and calibration. A total of three examiners received baseline training/calibration by a "gold standard" examiner, and one year later re-calibration was conducted. For the baseline training/calibration, subjects aged 8-16 years, and for the re-calibration subjects aged five to six years were recruited for the study. The ICDAS criteria were used to classify visual caries lesion severity (0-6 scale), lesion activity (active/inactive), and presence of filling material (0-9 scale) of all available tooth surfaces of permanent and primary teeth. The examination used a clinical light, mirror and air syringe. Kappa (weighted: Wkappa, unweighted: Kappa) statistics were used to determine inter-and intra-examiner reliability at baseline and re-calibration. For lesion severity and filling criteria, the baseline calibration on 35 subjects indicated an inter-rater Wkappa ranging from 0.69-0.92 and intra-rater Wkappa ranging from 0.81-0.92. Re-calibration on 22 subjects indicated an inter-rater Wkappa of 0.77-0.98 and intra-rater Wkappa ranged from 0.93-1.00. The Wkappa for filling was consistently in the excellent range, while lesion severity was in the good to excellent range. Activity kappa was in the poor to good range. All examiners improved with time. The baseline training/calibration in ICDAS was crucial to maintain the stability of the examiners reliability over a one year period. The ICDAS can be an effective assessment tool for community-based clinical trials.

Teaching DSM-III to clinicians. Some problems of the DSM-III system reducing reliability, using the diagnosis and classification of depressive disorders as an example.

PubMed

Malt, U F

1986-01-01

Experiences from teaching DSM-III to more than three hundred Norwegian psychiatrists and clinical psychologists suggest that reliable DSM-III diagnoses can be achieved within a few hours training with reference to the decision trees and the diagnostic criteria only. The diagnoses provided are more reliable than the corresponding ICD diagnoses which the participants were more familiar with. The three main sources of reduced reliability of the DSM-III diagnoses are related to: poor knowledge of the criteria which often is connected with failure of obtaining diagnostic key information during the clinical interview; unfamiliar concepts and vague or ambiguous criteria. The two first issues are related to the quality of the teaching of DSM-III. The third source of reduced reliability reflects unsolved validity issues. By using the classification of five affective case stories as examples, these sources of diagnostic pitfalls, reducing reliability and ways to overcome these problems when teaching the DSM-III system, are discussed. It is concluded that the DSM-III system of classification is easy to teach and that the system is superior to other classification systems available from a reliability point of view. The current version of the DSM-III system, however, partly owes a high degree of reliability to broad and heterogeneous diagnostic categories like the concept major depression, which may have questionable validity. Thus, the future revisions of the DSM-III system should, above all, address the issue of validity.

Identifying reliable independent components via split-half comparisons

PubMed Central

Groppe, David M.; Makeig, Scott; Kutas, Marta

2011-01-01

Independent component analysis (ICA) is a family of unsupervised learning algorithms that have proven useful for the analysis of the electroencephalogram (EEG) and magnetoencephalogram (MEG). ICA decomposes an EEG/MEG data set into a basis of maximally temporally independent components (ICs) that are learned from the data. As with any statistic, a concern with using ICA is the degree to which the estimated ICs are reliable. An IC may not be reliable if ICA was trained on insufficient data, if ICA training was stopped prematurely or at a local minimum (for some algorithms), or if multiple global minima were present. Consequently, evidence of ICA reliability is critical for the credibility of ICA results. In this paper, we present a new algorithm for assessing the reliability of ICs based on applying ICA separately to split-halves of a data set. This algorithm improves upon existing methods in that it considers both IC scalp topographies and activations, uses a probabilistically interpretable threshold for accepting ICs as reliable, and requires applying ICA only three times per data set. As evidence of the method’s validity, we show that the method can perform comparably to more time intensive bootstrap resampling and depends in a reasonable manner on the amount of training data. Finally, using the method we illustrate the importance of checking the reliability of ICs by demonstrating that IC reliability is dramatically increased by removing the mean EEG at each channel for each epoch of data rather than the mean EEG in a prestimulus baseline. PMID:19162199

Fuel Cell Balance-of-Plant Reliability Testbed Project

DOE Office of Scientific and Technical Information (OSTI.GOV)

Sproat, Vern; LaHurd, Debbie

Reliability of the fuel cell system balance-of-plant (BoP) components is a critical factor that needs to be addressed prior to fuel cells becoming fully commercialized. Failure or performance degradation of BoP components has been identified as a life-limiting factor in fuel cell systems.1 The goal of this project is to develop a series of test beds that will test system components such as pumps, valves, sensors, fittings, etc., under operating conditions anticipated in real Polymer Electrolyte Membrane (PEM) fuel cell systems. Results will be made generally available to begin removing reliability as a roadblock to the growth of the PEMmore » fuel cell industry. Stark State College students participating in the project, in conjunction with their coursework, have been exposed to technical knowledge and training in the handling and maintenance of hydrogen, fuel cells and system components as well as component failure modes and mechanisms. Three test beds were constructed. Testing was completed on gas flow pumps, tubing, and pressure and temperature sensors and valves.« less

The Training Evaluation Inventory (TEI)--Evaluation of Training Design and Measurement of Training Outcomes for Predicting Training Success

ERIC Educational Resources Information Center

Ritzmann, Sandrina; Hagemann, Vera; Kluge, Annette

2014-01-01

Training evaluation in research and organisational contexts is vital to ensure informed decisions regarding the value of training. The present study describes the development of a valid and reliable training evaluation inventory (TEI), as it does not exist so far. The objectives were a) to construct an instrument that is theoretically and…

Dental hygiene faculty calibration in the evaluation of calculus detection.

PubMed

Garland, Kandis V; Newell, Kathleen J

2009-03-01

The purpose of this pilot study was to explore the impact of faculty calibration training on intra- and interrater reliability regarding calculus detection. After IRB approval, twelve dental hygiene faculty members were recruited from a pool of twenty-two for voluntary participation and randomized into two groups. All subjects provided two pre- and two posttest scorings of calculus deposits on each of three typodonts by recording yes or no indicating if they detected calculus. Accuracy and consistency of calculus detection were evaluated using an answer key. The experimental group received three two-hour training sessions to practice a prescribed exploring sequence and technique for calculus detection. Participants immediately corrected their answers, received feedback from the trainer, and reconciled missed areas. Intra- and interrater reliability (pre- and posttest) was determined using Cohen's Kappa and compared between groups using repeated measures (split-plot) ANOVA. The groups did not differ from pre- to posttraining (intrarater reliability p=0.64; interrater reliability p=0.20). Training had no effect on reliability levels for simulated calculus detection in this study. Recommendations for future studies of faculty calibration when evaluating students include using patients for assessing rater reliability, employing larger samples at multiple sites, and assessing the impact on students' attitudes and learning outcomes.

Reliability testing of the Larsen and Sharp classifications for rheumatoid arthritis of the elbow.

PubMed

Jew, Nicholas B; Hollins, Anthony M; Mauck, Benjamin M; Smith, Richard A; Azar, Frederick M; Miller, Robert H; Throckmorton, Thomas W

2017-01-01

Two popular systems for classifying rheumatoid arthritis affecting the elbow are the Larsen and Sharp schemes. To our knowledge, no study has investigated the reliability of these 2 systems. We compared the intraobserver and interobserver agreement of the 2 systems to determine whether one is more reliable than the other. The radiographs of 45 patients diagnosed with rheumatoid arthritis affecting the elbow were evaluated. Anteroposterior and lateral radiographs were deidentified and distributed to 6 evaluators (4 fellowship-trained upper extremity surgeons and 2 orthopedic trainees). Each evaluator graded all 45 radiographs according to the Larsen and Sharp scoring methods on 2 occasions, at least 2 weeks apart. Overall intraobserver reliability was 0.93 (95% confidence interval [CI], 0.90-0.95) for the Larsen system and 0.92 (95% CI, 0.86-0.96) for the Sharp classification, both indicating substantial agreement. Overall interobserver reliability was 0.70 (95% CI, 0.60-0.80) for the Larsen classification and 0.68 (95% CI, 0.54-0.81) for the Sharp system, both indicating good agreement. There were no significant differences in the intraobserver or interobserver reliability of the systems overall and no significant differences in reliability between attending surgeons and trainees for either classification system. The Larsen and Sharp systems both show substantial intraobserver reliability and good interobserver agreement for the radiographic classification of rheumatoid arthritis affecting the elbow. Differences in training level did not result in substantial variances in reliability for either system. We conclude that both systems can be reliably used to evaluate rheumatoid arthritis of the elbow by observers of varying training levels. Copyright © 2017 Journal of Shoulder and Elbow Surgery Board of Trustees. Published by Elsevier Inc. All rights reserved.

Reliability of Trained Dogs to Alert to Hypoglycemia in Patients With Type 1 Diabetes

PubMed Central

Los, Evan A.; Ramsey, Katrina L.; Guttmann-Bauman, Ines; Ahmann, Andrew J.

2016-01-01

Background: We examined the reliability of trained dogs to alert to hypoglycemia in individuals with type 1 diabetes. Methods: Patients with type 1 diabetes who currently used diabetes alert dogs participated in this exploratory study. Subjects reported satisfaction, perceived dog glucose sensing ability and reasons for obtaining a trained dog. Reliability of dog alerts was assessed using capillary blood glucose (CBG) and blinded continuous glucose monitoring (CGM) as comparators in 8 subjects (age 4-48). Hypoglycemia was defined as CBG or CGM <70 mg/dL. Results: Dog users were very satisfied (8.9/10 on a Likert-type scale) and largely confident (7.9/10) in their dog’s ability to detect hypoglycemia. Detection of hypoglycemia was the primary reason for obtaining a trained dog. During hypoglycemia, spontaneous dog alerts occurred at a rate 3.2 (2.0-5.2, 95% CI) times higher than during euglycemia (70-179 mg/dL). Dogs provided timely alerts in 36% (sensitivity) of all hypoglycemia events (n = 45). Due to inappropriate alerts, the PPV of a dog alert for hypoglycemia was 12%. When there was concurrence of a hypoglycemic event between the dog alert and CGM (n = 30), CGM would have alerted prior to the dog in 73% of events (median 22-minute difference). Conclusions: This is the first study evaluating reliability of trained dogs to alert to hypoglycemia under real-life conditions. Trained dogs often alert a human companion to otherwise unknown hypoglycemia; however due to high false-positive rate, a dog alert alone is unlikely to be helpful in differentiating hypo-/hyper-/euglycemia. CGM often detects hypoglycemia before a trained dog by a clinically significant margin. PMID:27573791

Reliability of Trained Dogs to Alert to Hypoglycemia in Patients With Type 1 Diabetes.

PubMed

Los, Evan A; Ramsey, Katrina L; Guttmann-Bauman, Ines; Ahmann, Andrew J

2017-05-01

We examined the reliability of trained dogs to alert to hypoglycemia in individuals with type 1 diabetes. Patients with type 1 diabetes who currently used diabetes alert dogs participated in this exploratory study. Subjects reported satisfaction, perceived dog glucose sensing ability and reasons for obtaining a trained dog. Reliability of dog alerts was assessed using capillary blood glucose (CBG) and blinded continuous glucose monitoring (CGM) as comparators in 8 subjects (age 4-48). Hypoglycemia was defined as CBG or CGM <70 mg/dL. Dog users were very satisfied (8.9/10 on a Likert-type scale) and largely confident (7.9/10) in their dog's ability to detect hypoglycemia. Detection of hypoglycemia was the primary reason for obtaining a trained dog. During hypoglycemia, spontaneous dog alerts occurred at a rate 3.2 (2.0-5.2, 95% CI) times higher than during euglycemia (70-179 mg/dL). Dogs provided timely alerts in 36% (sensitivity) of all hypoglycemia events (n = 45). Due to inappropriate alerts, the PPV of a dog alert for hypoglycemia was 12%. When there was concurrence of a hypoglycemic event between the dog alert and CGM (n = 30), CGM would have alerted prior to the dog in 73% of events (median 22-minute difference). This is the first study evaluating reliability of trained dogs to alert to hypoglycemia under real-life conditions. Trained dogs often alert a human companion to otherwise unknown hypoglycemia; however due to high false-positive rate, a dog alert alone is unlikely to be helpful in differentiating hypo-/hyper-/euglycemia. CGM often detects hypoglycemia before a trained dog by a clinically significant margin.

Human Reliability Assessments: Using the Past (Shuttle) to Predict the Future (ORION)

NASA Technical Reports Server (NTRS)

Mott, Diana L.; Bigler, Mark A.

2017-01-01

NASA uses two HRA assessment methodologies. The first is a simplified method which is based on how much time is available to complete the action, with consideration included for environmental and personal factors that could influence the human's reliability. This method is expected to provide a conservative value or placeholder as a preliminary estimate. This preliminary estimate is used to determine which placeholder needs a more detailed assessment. The second methodology is used to develop a more detailed human reliability assessment on the performance of critical human actions. This assessment needs to consider more than the time available, this would include factors such as: the importance of the action, the context, environmental factors, potential human stresses, previous experience, training, physical design interfaces, available procedures/checklists and internal human stresses. The more detailed assessment is still expected to be more realistic than that based primarily on time available. When performing an HRA on a system or process that has an operational history, we have information specific to the task based on this history and experience. In the case of a PRA model that is based on a new design and has no operational history, providing a "reasonable" assessment of potential crew actions becomes more problematic. In order to determine what is expected of future operational parameters, the experience from individuals who had relevant experience and were familiar with the system and process previously implemented by NASA was used to provide the "best" available data. Personnel from Flight Operations, Flight Directors, Launch Test Directors, Control Room Console Operators and Astronauts were all interviewed to provide a comprehensive picture of previous NASA operations. Verification of the assumptions and expectations expressed in the assessments will be needed when the procedures, flight rules and operational requirements are developed and then finalized.

Validity and reliability of the session-RPE method for quantifying training load in karate athletes.

PubMed

Tabben, M; Tourny, C; Haddad, M; Chaabane, H; Chamari, K; Coquart, J B

2015-04-24

To test the construct validity and reliability of the session rating of perceived exertion (sRPE) method by examining the relationship between RPE and physiological parameters (heart rate: HR and blood lactate concentration: [La --] ) and the correlations between sRPE and two HR--based methods for quantifying internal training load (Banister's method and Edwards's method) during karate training camp. Eighteen elite karate athletes: ten men (age: 24.2 ± 2.3 y, body mass: 71.2 ± 9.0 kg, body fat: 8.2 ± 1.3% and height: 178 ± 7 cm) and eight women (age: 22.6 ± 1.2 y, body mass: 59.8 ± 8.4 kg, body fat: 20.2 ± 4.4%, height: 169 ± 4 cm) were included in the study. During training camp, subjects participated in eight karate--training sessions including three training modes (4 tactical--technical, 2 technical--development, and 2 randori training), during which RPE, HR, and [La -- ] were recorded. Significant correlations were found between RPE and physiological parameters (percentage of maximal HR: r = 0.75, 95% CI = 0.64--0.86; [La --] : r = 0.62, 95% CI = 0.49--0.75; P < 0.001). Moreover, individual sRPE was significantly correlated with two HR--based methods for quantifying internal training load ( r = 0.65--0.95; P < 0.001). The sRPE method showed the high reliability of the same intensity across training sessions (Cronbach's α = 0.81, 95% CI = 0.61--0.92). This study demonstrates that the sRPE method is valid for quantifying internal training load and intensity in karate.

Is the Scale for Measuring Motivational Interviewing Skills a valid and reliable instrument for measuring the primary care professionals motivational skills?: EVEM study protocol.

PubMed

Pérula, Luis Á; Campiñez, Manuel; Bosch, Josep M; Barragán Brun, Nieves; Arboniés, Juan C; Bóveda Fontán, Julia; Martín Alvarez, Remedios; Prados, Jose A; Martín-Rioboó, Enrique; Massons, Josep; Criado, Margarita; Fernández, José Á; Parras, Juan M; Ruiz-Moral, Roger; Novo, Jesús M

2012-11-22

Lifestyle is one of the main determinants of people's health. It is essential to find the most effective prevention strategies to be used to encourage behavioral changes in their patients. Many theories are available that explain change or adherence to specific health behaviors in subjects. In this sense the named Motivational Interviewing has increasingly gained relevance. Few well-validated instruments are available for measuring doctors' communication skills, and more specifically the Motivational Interviewing. The hypothesis of this study is that the Scale for Measuring Motivational Interviewing Skills (EVEM questionnaire) is a valid and reliable instrument for measuring the primary care professionals skills to get behavior change in patients. To test the hypothesis we have designed a prospective, observational, multi-center study to validate a measuring instrument. - Thirty-two primary care centers in Spain. -Sampling and Size: a) face and consensual validity: A group composed of 15 experts in Motivational Interviewing. b) Assessment of the psychometric properties of the scale; 50 physician- patient encounters will be videoed; a total of 162 interviews will be conducted with six standardized patients, and another 200 interviews will be conducted with 50 real patients (n=362). Four physicians will be specially trained to assess 30 interviews randomly selected to test the scale reproducibility. -Measurements for to test the hypothesis: a) Face validity: development of a draft questionnaire based on a theoretical model, by using Delphi-type methodology with experts. b) Scale psychometric properties: intraobservers will evaluate video recorded interviews: content-scalability validity (Exploratory Factor Analysis), internal consistency (Cronbach alpha), intra-/inter-observer reliability (Kappa index, intraclass correlation coefficient, Bland & Altman methodology), generalizability, construct validity and sensitivity to change (Pearson product-moment correlation coefficient). The verification of the hypothesis that EVEM is a valid and reliable tool for assessing motivational interviewing would be a major breakthrough in the current theoretical and practical knowledge, as it could be used to assess if the providers put into practice a patient centered communication style and can be used both for training or researching purposes. TRIALS REGISTRATION Dislip-EM study: NCT01282190 (ClinicalTrials.gov).

Transfer learning for biomedical named entity recognition with neural networks.

PubMed

Giorgi, John M; Bader, Gary D

2018-06-01

The explosive increase of biomedical literature has made information extraction an increasingly important tool for biomedical research. A fundamental task is the recognition of biomedical named entities in text (BNER) such as genes/proteins, diseases, and species. Recently, a domain-independent method based on deep learning and statistical word embeddings, called long short-term memory network-conditional random field (LSTM-CRF), has been shown to outperform state-of-the-art entity-specific BNER tools. However, this method is dependent on gold-standard corpora (GSCs) consisting of hand-labeled entities, which tend to be small but highly reliable. An alternative to GSCs are silver-standard corpora (SSCs), which are generated by harmonizing the annotations made by several automatic annotation systems. SSCs typically contain more noise than GSCs but have the advantage of containing many more training examples. Ideally, these corpora could be combined to achieve the benefits of both, which is an opportunity for transfer learning. In this work, we analyze to what extent transfer learning improves upon state-of-the-art results for BNER. We demonstrate that transferring a deep neural network (DNN) trained on a large, noisy SSC to a smaller, but more reliable GSC significantly improves upon state-of-the-art results for BNER. Compared to a state-of-the-art baseline evaluated on 23 GSCs covering four different entity classes, transfer learning results in an average reduction in error of approximately 11%. We found transfer learning to be especially beneficial for target data sets with a small number of labels (approximately 6000 or less). Source code for the LSTM-CRF is available athttps://github.com/Franck-Dernoncourt/NeuroNER/ and links to the corpora are available athttps://github.com/BaderLab/Transfer-Learning-BNER-Bioinformatics-2018/. john.giorgi@utoronto.ca. Supplementary data are available at Bioinformatics online.

iPhone otoscopes: Currently available, but reliable for tele-otoscopy in the hands of parents?

PubMed

Shah, Manan Udayan; Sohal, Maheep; Valdez, Tulio A; Grindle, Christopher R

2018-03-01

Tele-otoscopy has been validated for tympanostomy surveillance and remote diagnosis when images are recorded by trained professionals. The CellScope iPhone Otoscope is a device that may be used for tele-otoscopy and it enables parents to record their children's ear examinations and send the films for remote physician diagnosis. This study aims to determine the ability to diagnose, and the reliability of the diagnosis when utilizing video exams obtained by a parent versus video exams obtained by an otolaryngologist. Parents of children ages 17 years or younger attempted recordings of the tympanic membrane of their children with the CellScope after a video tutorial; a physician subsequently used the device to record the same ear. Recordings occurred prior to standard pediatric otolaryngology office evaluation. Later, a remote pediatric otolaryngologist attempted diagnosis solely based on the videos, blinded to whether the examination was filmed by a parent or physician. Interrater reliability between video diagnosis and original diagnosis on pneumatic otoscopy was measured, and objective tympanic membrane landmarks visualized on the films were recorded. Eighty ears were enrolled and recorded. There was low interrater agreement (k = 0.42) between diagnosis based on parent videos as compared with pneumatic otoscopy. There was high agreement (k = 0.71) between diagnosis based on physician videos and pneumatic otoscopy. Physician videos and parent videos had only slight agreement on objective landmarks identified (k = 0.087). iPhone otoscopy provides reliable tele-otoscopy images in when used by trained professionals but, currently, images obtained by parents are not suitable for use in diagnosis. Copyright © 2018 Elsevier B.V. All rights reserved.

Home-based anthropometric, blood pressure and pulse measurements in young children by trained data collectors in the National Children's Study.

PubMed

Zimowski, Michele; Moye, Jack; Dugoni, Bernard; Heim Viox, Melissa; Cohen, Hildie; Winfrey, Krishna

2017-02-01

The current study assessed whether home-based data collection by trained data collectors can produce high-quality physical measurement data in young children. The study assessed the quality of intra-examiner measurements of blood pressure, pulse rate and anthropometric dimensions using intra-examiner reliability and intra-examiner technical error of measurement (TEM). Non-clinical, primarily private homes of National Children's Study participants in twenty-two study locations across the USA. Children in four age groups: 5-7 months (n 91), 11-16 months (n 393), 23-28 months (n 1410) and 35-40 months (n 800). Absolute TEM ranged in value from 0·09 to 16·21, varying widely by age group and measure, as expected. Relative TEM spanned from 0·27 to 13·71 across age groups and physical measures. Reliabilities for anthropometric measurements by age group and measure ranged from 0·46 to >0·99 with most exceeding 0·90, suggesting that the large majority of anthropometric measures can be collected in a home-based setting on young children by trained data collectors. Reliabilities for blood pressure and pulse rate measurements by age group ranged from 0·21 to 0·74, implying these are less reliably measured with young children when taken in the data collection context described here. Reliability estimates >0·95 for weight, length, height, and thigh, waist and head circumference, and >0·90 for triceps and subscapular skinfolds, indicate that these measures can be collected in the field by trained data collectors without compromising data quality. These estimates can be used for interim evaluations of data collector training and measurement protocols.

Examiner Training and Reliability in Two Randomized Clinical Trials of Adult Dental Caries

PubMed Central

Banting, David W.; Amaechi, Bennett T.; Bader, James D.; Blanchard, Peter; Gilbert, Gregg H.; Gullion, Christina M.; Holland, Jan Carlton; Makhija, Sonia K.; Papas, Athena; Ritter, André V.; Singh, Mabi L.; Vollmer, William M.

2013-01-01

Objectives This report describes the training of dental examiners participating in two dental caries clinical trials and reports the inter- and intra- examiner reliability scores from the initial standardization sessions. Methods Study examiners were trained to use a modified ICDAS-II system to detect the visual signs of non-cavitated and cavitated dental caries in adult subjects. Dental caries was classified as no caries (S), non-cavitated caries (D1), enamel caries (D2) and dentine caries (D3). Three standardization sessions involving 60 subjects and 3604 tooth surface calls were used to calculate several measures of examiner reliability. Results The prevalence of dental caries observed in the standardization sessions ranged from 1.4% to 13.5% of the coronal tooth surfaces examined. Overall agreement between pairs of examiners ranged from 0.88 to 0.99. An intra-class coefficient threshold of 0.60 was surpassed for all but one examiner. Inter-examiner unweighted kappa values were low (0.23– 0.35) but weighted kappas and the ratio of observed to maximum kappas were more encouraging (0.42– 0.83). The highest kappa values occurred for the S/D1 vs. D2/D3 two-level classification of dental caries, for which seven of the eight examiners achieved observed to maximum kappa values over 0.90.Intra-examiner reliability was notably higher than inter-examiner reliability for all measures and dental caries classification systems employed. Conclusion The methods and results for the initial examiner training and standardization sessions for two large clinical trials are reported. Recommendations for others planning examiner training and standardization sessions are offered. PMID:22320292

Examiner training and reliability in two randomized clinical trials of adult dental caries.

PubMed

Banting, David W; Amaechi, Bennett T; Bader, James D; Blanchard, Peter; Gilbert, Gregg H; Gullion, Christina M; Holland, Jan Carlton; Makhija, Sonia K; Papas, Athena; Ritter, André V; Singh, Mabi L; Vollmer, William M

2011-01-01

This report describes the training of dental examiners participating in two dental caries clinical trials and reports the inter- and intra-examiner reliability scores from the initial standardization sessions. Study examiners were trained to use a modified International Caries Detection and Assessment System II system to detect the visual signs of non-cavitated and cavitated dental caries in adult subjects. Dental caries was classified as no caries (S), non-cavitated caries (D1), enamel caries (D2), and dentine caries (D3). Three standardization sessions involving 60 subjects and 3,604 tooth surface calls were used to calculate several measures of examiner reliability. The prevalence of dental caries observed in the standardization sessions ranged from 1.4 percent to 13.5 percent of the coronal tooth surfaces examined. Overall agreement between pairs of examiners ranged from 0.88 to 0.99. An intra-class coefficient threshold of 0.60 was surpassed for all but one examiner. Inter-examiner unweighted kappa values were low (0.23-0.35), but weighted kappas and the ratio of observed to maximum kappas were more encouraging (0.42-0.83). The highest kappa values occurred for the S/D1 versus D2/D3 two-level classification of dental caries, for which seven of the eight examiners achieved observed to maximum kappa values over 0.90. Intra-examiner reliability was notably higher than inter-examiner reliability for all measures and dental caries classifications employed. The methods and results for the initial examiner training and standardization sessions for two large clinical trials are reported. Recommendations for others planning examiner training and standardization sessions are offered. © 2011 American Association of Public Health Dentistry.

Some Factors Influencing Transfer of Simulator Training.

ERIC Educational Resources Information Center

Caro, Paul W.

Studies of transfer of training may be used to determine whether simulator training improves pilot performance in an aircraft. Some approaches to determining simulator training effectiveness, such as surveys of pilot and instructor opinions, are not considered particularly reliable. Several other approaches have also been suggested. One factor…

Development, initial reliability and validity testing of an observational tool for assessing technical skills of operating room nurses.

PubMed

Sevdalis, Nick; Undre, Shabnam; Henry, Janet; Sydney, Elaine; Koutantji, Mary; Darzi, Ara; Vincent, Charles A

2009-09-01

The recent emergence of the Systems Approach to the safety and quality of surgical care has triggered individual and team skills training modules for surgeons and anaesthetists and relevant observational assessment tools have been developed. To develop an observational tool that captures operating room (OR) nurses' technical skill and can be used for assessment and training. The Imperial College Assessment of Technical Skills for Nurses (ICATS-N) assesses (i) gowning and gloving, (ii) setting up instrumentation, (iii) draping, and (iv) maintaining sterility. Three to five observable behaviours have been identified for each skill and are rated on 1-6 scales. Feasibility and aspects of reliability and validity were assessed in 20 simulation-based crisis management training modules for trainee nurses and doctors, carried out in a Simulated Operating Room. The tool was feasible to use in the context of simulation-based training. Satisfactory reliability (Cronbach alpha) was obtained across trainers' and trainees' scores (analysed jointly and separately). Moreover, trainer nurse's ratings of the four skills correlated positively, thus indicating adequate content validity. Trainer's and trainees' ratings did not correlate. Assessment of OR nurses' technical skill is becoming a training priority. The present evidence suggests that the ICATS-N could be considered for use as an assessment/training tool for junior OR nurses.

Reliability in Cross-National Content Analysis.

ERIC Educational Resources Information Center

Peter, Jochen; Lauf, Edmund

2002-01-01

Investigates how coder characteristics such as language skills, political knowledge, coding experience, and coding certainty affected inter-coder and coder-training reliability. Shows that language skills influenced both reliability types. Suggests that cross-national researchers should pay more attention to cross-national assessments of…

Development and Validation of an Instrument for Measuring the Quality of Teamwork in Teaching Teams in Postgraduate Medical Training (TeamQ)

PubMed Central

Slootweg, Irene A.; Lombarts, Kiki M. J. M. H.; Boerebach, Benjamin C. M.; Heineman, Maas Jan; Scherpbier, Albert J. J. A.; van der Vleuten, Cees P. M.

2014-01-01

Background Teamwork between clinical teachers is a challenge in postgraduate medical training. Although there are several instruments available for measuring teamwork in health care, none of them are appropriate for teaching teams. The aim of this study is to develop an instrument (TeamQ) for measuring teamwork, to investigate its psychometric properties and to explore how clinical teachers assess their teamwork. Method To select the items to be included in the TeamQ questionnaire, we conducted a content validation in 2011, using a Delphi procedure in which 40 experts were invited. Next, for pilot testing the preliminary tool, 1446 clinical teachers from 116 teaching teams were requested to complete the TeamQ questionnaire. For data analyses we used statistical strategies: principal component analysis, internal consistency reliability coefficient, and the number of evaluations needed to obtain reliable estimates. Lastly, the median TeamQ scores were calculated for teams to explore the levels of teamwork. Results In total, 31 experts participated in the Delphi study. In total, 114 teams participated in the TeamQ pilot. The median team response was 7 evaluations per team. The principal component analysis revealed 11 factors; 8 were included. The reliability coefficients of the TeamQ scales ranged from 0.75 to 0.93. The generalizability analysis revealed that 5 to 7 evaluations were needed to obtain internal reliability coefficients of 0.70. In terms of teamwork, the clinical teachers scored residents' empowerment as the highest TeamQ scale and feedback culture as the area that would most benefit from improvement. Conclusions This study provides initial evidence of the validity of an instrument for measuring teamwork in teaching teams. The high response rates and the low number of evaluations needed for reliably measuring teamwork indicate that TeamQ is feasible for use by teaching teams. Future research could explore the effectiveness of feedback on teamwork in follow up measurements. PMID:25393006

Development and validation of an instrument for measuring the quality of teamwork in teaching teams in postgraduate medical training (TeamQ).

PubMed

Slootweg, Irene A; Lombarts, Kiki M J M H; Boerebach, Benjamin C M; Heineman, Maas Jan; Scherpbier, Albert J J A; van der Vleuten, Cees P M

2014-01-01

Teamwork between clinical teachers is a challenge in postgraduate medical training. Although there are several instruments available for measuring teamwork in health care, none of them are appropriate for teaching teams. The aim of this study is to develop an instrument (TeamQ) for measuring teamwork, to investigate its psychometric properties and to explore how clinical teachers assess their teamwork. To select the items to be included in the TeamQ questionnaire, we conducted a content validation in 2011, using a Delphi procedure in which 40 experts were invited. Next, for pilot testing the preliminary tool, 1446 clinical teachers from 116 teaching teams were requested to complete the TeamQ questionnaire. For data analyses we used statistical strategies: principal component analysis, internal consistency reliability coefficient, and the number of evaluations needed to obtain reliable estimates. Lastly, the median TeamQ scores were calculated for teams to explore the levels of teamwork. In total, 31 experts participated in the Delphi study. In total, 114 teams participated in the TeamQ pilot. The median team response was 7 evaluations per team. The principal component analysis revealed 11 factors; 8 were included. The reliability coefficients of the TeamQ scales ranged from 0.75 to 0.93. The generalizability analysis revealed that 5 to 7 evaluations were needed to obtain internal reliability coefficients of 0.70. In terms of teamwork, the clinical teachers scored residents' empowerment as the highest TeamQ scale and feedback culture as the area that would most benefit from improvement. This study provides initial evidence of the validity of an instrument for measuring teamwork in teaching teams. The high response rates and the low number of evaluations needed for reliably measuring teamwork indicate that TeamQ is feasible for use by teaching teams. Future research could explore the effectiveness of feedback on teamwork in follow up measurements.

Interrater Reliability in Large-Scale Assessments--Can Teachers Score National Tests Reliably without External Controls?

ERIC Educational Resources Information Center

Pantzare, Anna Lind

2015-01-01

In most large-scale assessment systems a set of rather expensive external quality controls are implemented in order to guarantee the quality of interrater reliability. This study empirically examines if teachers' ratings of national tests in mathematics can be reliable without using monitoring, training, or other methods of external quality…

The Effect of Cognitive Load and Outcome Congruency on the Learned Predictiveness Effect in Human Predictive Learning

ERIC Educational Resources Information Center

Jorge A. Pinto,; Vogel, Edgar H.; Núñez, Daniel E.

2017-01-01

The learned predictiveness effect or LPE is the finding that when people learn that certain cues are reliable predictors of an outcome in an initial stage of training (phase 1), they exhibit a learning bias in favor of these cues in a subsequent training involving new outcomes (phase 2) despite all cues being equally reliable in phase 2. In…

Using Mental Computation Training to Improve Complex Mathematical Performance

ERIC Educational Resources Information Center

Liu, Allison S.; Kallai, Arava Y.; Schunn, Christian D.; Fiez, Julie A.

2015-01-01

Mathematical fluency is important for academic and mathematical success. Fluency training programs have typically focused on fostering retrieval, which leads to math performance that does not reliably transfer to non-trained problems. More recent studies have focused on training number understanding and representational precision, but few have…

A Standardized System of Training Intensity Guidelines for the Sports of Track and Field and Cross Country

ERIC Educational Resources Information Center

Belcher, Christopher P.; Pemberton, Cynthia Lee A.

2012-01-01

Accurate quantification of training intensity is an essential component of a training program (Rowbottom, 2000). A training program designed to optimize athlete performance abilities cannot be practically planned or implemented without a valid and reliable indication of training intensity and its effect on the physiological mechanisms of the human…

How to survive the medical misinformation mess.

PubMed

Ioannidis, John P A; Stuart, Michael E; Brownlee, Shannon; Strite, Sheri A

2017-11-01

Most physicians and other healthcare professionals are unaware of the pervasiveness of poor quality clinical evidence that contributes considerably to overuse, underuse, avoidable adverse events, missed opportunities for right care and wasted healthcare resources. The Medical Misinformation Mess comprises four key problems. First, much published medical research is not reliable or is of uncertain reliability, offers no benefit to patients, or is not useful to decision makers. Second, most healthcare professionals are not aware of this problem. Third, they also lack the skills necessary to evaluate the reliability and usefulness of medical evidence. Finally, patients and families frequently lack relevant, accurate medical evidence and skilled guidance at the time of medical decision-making. Increasing the reliability of available, published evidence may not be an imminently reachable goal. Therefore, efforts should focus on making healthcare professionals, more sensitive to the limitations of the evidence, training them to do critical appraisal, and enhancing their communication skills so that they can effectively summarize and discuss medical evidence with patients to improve decision-making. Similar efforts may need to target also patients, journalists, policy makers, the lay public and other healthcare stakeholders. © 2017 Stichting European Society for Clinical Investigation Journal Foundation.

Developing a reliable method for signal wire attachment : [research results].

DOT National Transportation Integrated Search

2013-03-01

Railroad signaling systems detect trains on the track, identify track fractures, prevent derailments, and alert signal crossing stations when trains approach. These systems are vital to safe train operation; therefore, each component of this system h...

Portfolio Assessment: Increasing Reliability and Validity.

ERIC Educational Resources Information Center

Griffee, Dale

2002-01-01

Addresses the traditional understanding of reliability as it pertains to writing portfolio assessments. Offers a list of practical actions that can be taken to increase assessment reliability, including explicit definitions of what a portfolio holds, rater training, rater burnout, and consistent rating procedures. (Contains 26 references.) (NB)

Establishing Inter- and Intrarater Reliability for High-Stakes Testing Using Simulation.

PubMed

Kardong-Edgren, Suzan; Oermann, Marilyn H; Rizzolo, Mary Anne; Odom-Maryon, Tamara

This article reports one method to develop a standardized training method to establish the inter- and intrarater reliability of a group of raters for high-stakes testing. Simulation is used increasingly for high-stakes testing, but without research into the development of inter- and intrarater reliability for raters. Eleven raters were trained using a standardized methodology. Raters scored 28 student videos over a six-week period. Raters then rescored all videos over a two-day period to establish both intra- and interrater reliability. One rater demonstrated poor intrarater reliability; a second rater failed all students. Kappa statistics improved from the moderate to substantial agreement range with the exclusion of the two outlier raters' scores. There may be faculty who, for different reasons, should not be included in high-stakes testing evaluations. All faculty are content experts, but not all are expert evaluators.

Reliability and Maintainability (RAM) Training

NASA Technical Reports Server (NTRS)

Lalli, Vincent R. (Editor); Malec, Henry A. (Editor); Packard, Michael H. (Editor)

2000-01-01

The theme of this manual is failure physics-the study of how products, hardware, software, and systems fail and what can be done about it. The intent is to impart useful information, to extend the limits of production capability, and to assist in achieving low-cost reliable products. In a broader sense the manual should do more. It should underscore the urgent need CI for mature attitudes toward reliability. Five of the chapters were originally presented as a classroom course to over 1000 Martin Marietta engineers and technicians. Another four chapters and three appendixes have been added, We begin with a view of reliability from the years 1940 to 2000. Chapter 2 starts the training material with a review of mathematics and a description of what elements contribute to product failures. The remaining chapters elucidate basic reliability theory and the disciplines that allow us to control and eliminate failures.

Time to competency, reliability of flexible transnasal laryngoscopy by training level: a pilot study.

PubMed

Brook, Christopher D; Platt, Michael P; Russell, Kimberly; Grillone, Gregory A; Aliphas, Avner; Noordzij, J Pieter

2015-05-01

To determine the progression of flexible transnasal laryngoscopy reliability and competency in otolaryngology residency training. Prospective case control study. Academic otolaryngology department. Medical students, otolaryngology residents, and otolaryngology attending physicians. Fourteen otolaryngology residents from PGY-1 to PGY-5 and 3 attending otolaryngologists viewed 25 selected and digitally recorded flexible transnasal laryngoscopies. The evaluators were asked to rate 13 items relating to abnormalities in the oropharynx, hypopharynx, larynx, and subglottis. The level of concern and level of comfort with the diagnosis were assessed. Intraclass correlations were calculated for each topic and by level of training to determine reliability within each class and compare competency versus attending interpretations. Intraclass correlation of residents compared to attending physicians demonstrated significant improvements by year for left and right vocal fold immobility, subglottic stenosis, laryngeal mass, left and right vocal cord abnormalities, and level of concern. Additionally, pooled vocal cord mobility and pooled results in categories with good attending reliability demonstrated stepwise improvement as well. For these categories, resident reliability was found to be statistically similar to attending physicians in all categories by PGY-3. There were no trends for base of tongue abnormalities, pharyngeal abnormalities, and pharyngeal and hypopharyngeal masses. Resident competency for flexible transnasal laryngoscopy progresses during residency to reliability with attending otolaryngologists by the PGY-3 year over key facets of the examination. © American Academy of Otolaryngology-Head and Neck Surgery Foundation 2015.

Auditory Perceptual Learning for Speech Perception Can be Enhanced by Audiovisual Training.

PubMed

Bernstein, Lynne E; Auer, Edward T; Eberhardt, Silvio P; Jiang, Jintao

2013-01-01

Speech perception under audiovisual (AV) conditions is well known to confer benefits to perception such as increased speed and accuracy. Here, we investigated how AV training might benefit or impede auditory perceptual learning of speech degraded by vocoding. In Experiments 1 and 3, participants learned paired associations between vocoded spoken nonsense words and nonsense pictures. In Experiment 1, paired-associates (PA) AV training of one group of participants was compared with audio-only (AO) training of another group. When tested under AO conditions, the AV-trained group was significantly more accurate than the AO-trained group. In addition, pre- and post-training AO forced-choice consonant identification with untrained nonsense words showed that AV-trained participants had learned significantly more than AO participants. The pattern of results pointed to their having learned at the level of the auditory phonetic features of the vocoded stimuli. Experiment 2, a no-training control with testing and re-testing on the AO consonant identification, showed that the controls were as accurate as the AO-trained participants in Experiment 1 but less accurate than the AV-trained participants. In Experiment 3, PA training alternated AV and AO conditions on a list-by-list basis within participants, and training was to criterion (92% correct). PA training with AO stimuli was reliably more effective than training with AV stimuli. We explain these discrepant results in terms of the so-called "reverse hierarchy theory" of perceptual learning and in terms of the diverse multisensory and unisensory processing resources available to speech perception. We propose that early AV speech integration can potentially impede auditory perceptual learning; but visual top-down access to relevant auditory features can promote auditory perceptual learning.

Auditory Perceptual Learning for Speech Perception Can be Enhanced by Audiovisual Training

PubMed Central

Bernstein, Lynne E.; Auer, Edward T.; Eberhardt, Silvio P.; Jiang, Jintao

2013-01-01

Speech perception under audiovisual (AV) conditions is well known to confer benefits to perception such as increased speed and accuracy. Here, we investigated how AV training might benefit or impede auditory perceptual learning of speech degraded by vocoding. In Experiments 1 and 3, participants learned paired associations between vocoded spoken nonsense words and nonsense pictures. In Experiment 1, paired-associates (PA) AV training of one group of participants was compared with audio-only (AO) training of another group. When tested under AO conditions, the AV-trained group was significantly more accurate than the AO-trained group. In addition, pre- and post-training AO forced-choice consonant identification with untrained nonsense words showed that AV-trained participants had learned significantly more than AO participants. The pattern of results pointed to their having learned at the level of the auditory phonetic features of the vocoded stimuli. Experiment 2, a no-training control with testing and re-testing on the AO consonant identification, showed that the controls were as accurate as the AO-trained participants in Experiment 1 but less accurate than the AV-trained participants. In Experiment 3, PA training alternated AV and AO conditions on a list-by-list basis within participants, and training was to criterion (92% correct). PA training with AO stimuli was reliably more effective than training with AV stimuli. We explain these discrepant results in terms of the so-called “reverse hierarchy theory” of perceptual learning and in terms of the diverse multisensory and unisensory processing resources available to speech perception. We propose that early AV speech integration can potentially impede auditory perceptual learning; but visual top-down access to relevant auditory features can promote auditory perceptual learning. PMID:23515520

Equal Access Initiative HIV/AIDS Information Resources from NLM

DOE Office of Scientific and Technical Information (OSTI.GOV)

Templin-Branner W. and N. Dancy

The Equal Access Initiative: HIV/AIDS Information Resources from the National Library of Medicine training is designed specifically for the National Minority AIDS Council 2010 Equal Access Initiative (EAI) Computer Grants Program awardees to provide valuable health information resources from the National Library of Medicine and other reliable sources to increase awareness of the wealth of treatment information and educational materials that are available on the Internet and to improve prevention and treatment education for their clients. These resources will also meet the needs of community-based

HIV/AIDS Information Resources from the National Library of Medicine-STOP

DOE Office of Scientific and Technical Information (OSTI.GOV)

Templin-Branner, W. and N. Dancy

2010-06-15

The HIV/AIDS Information Resources from the National Library of Medicine training is designed specifically for the UNCFSP HBCU Screening, Testing, Outreach, and Prevention (STOP) HIV/AIDS Program project members to provide valuable health information resources from the National Library of Medicine and other reliable sources to increase awareness of the wealth of treatment information and educational materials that are available on the Internet and to improve prevention and treatment education for their clients. These resources will also meet the needs of community-based organizations

Portable Load Measurement Device for Use During ARED Exercise on ISS

NASA Technical Reports Server (NTRS)

Hanson, A.; Peters, B.; Caldwell, E.; Sinka, J.; Kreutzburg, G.; Ploutz-Snyder, L.

2014-01-01

The Advanced Resistive Exercise Device (ARED) (Fig.1) is unique countermeasure hardware available to crewmembers aboard the International Space Station (ISS) used for resistance exercise training to protect against bone and muscle loss during long duration space missions. ARED instrumentation system was designed to measure and record exercise load data, but: - Reliably accurate data has not been available due to a defective force platform. - No ARED data has been recorded since mid-2011 due to failures in the instrumentation power system. ARED load data supports on-going HRP funded research, and is available to extramural researchers through LSDA-Repository. Astronaut Strength, Conditioning, and Rehabilitation specialists (ASCRs) use ARED data to track training progress and advance exercise prescriptions. ARED load data is necessary to fulfill medical requirements. HRP directed task intends to reduce to program risk (HRP IRMA Risk 1735), and evaluate the XSENS ForceShoeTM as a means of obtaining ARED load data during exercise sessions. The XSENS ForceShoes"TM" will fly as a hardware demonstration to ISS in May 2014 (39S). Additional portable load monitoring devices (PLMDs) are under evaluation in the ExPC Lab. PLMDs are favored over platform redesign as they support future exploration needs.

The Fellowship Council: a decade of impact on surgical training.

PubMed

Fowler, Dennis L; Hogle, Nancy J

2013-10-01

The objective of this project is to document the history of the Fellowship Council (FC) and report its current impact on surgical training. The need for advanced training in laparoscopic surgery resulted in the rapid development of fellowships for which there was no oversight. Fellowship program directors began meeting in the 1990s and formally created the FC in 2004 to provide that oversight. To obtain information with which to create a narrative of the history of the FC, the authors performed a detailed review of all available minutes from the meetings of the various iterations of the council and its committees between 2001 and 2012. Information about fellowships and meetings of the directors of fellowships prior to 2001 are based on information included in minutes of meetings after 2001. Minimally invasive surgery fellowship program directors in collaboration with surgical societies created the FC to bring order to the application process for residents and program directors. It has evolved into an organization with mature, reliable processes for application, matching, curriculum development, accreditation, and reporting. It now receives applications from more than 30 % of graduating chief residents in general surgery. It has 223 accredited fellowship positions in the following disciplines: Minimally invasive surgery, bariatric/metabolic surgery, Flexible endoscopy, hepato-pancreato-biliary Surgery, colorectal surgery, and Thoracic surgery. The FC provides a reliable, fair process for matching residents with fellowship programs and has successfully expanded its oversight of such programs with mature processes for accreditation, curriculum development, and reporting.

The validity of a behavioural multiple-mini-interview within an assessment centre for selection into specialty training

PubMed Central

2014-01-01

Background Entry into specialty training was determined by a National Assessment Centre (NAC) approach using a combination of a behavioural Multiple-Mini-Interview (MMI) and a written Situational Judgement Test (SJT). We wanted to know if interviewers could make reliable and valid decisions about the non-cognitive characteristics of candidates with the purpose of selecting them into general practice specialty training using the MMI. Second, we explored the concurrent validity of the MMI with the SJT. Methods A variance components analysis estimated the reliability and sources of measurement error. Further modelling estimated the optimal configurations for future MMI iterations. We calculated the relationship of the MMI with the SJT. Results Data were available from 1382 candidates, 254 interviewers, six MMI questions, five alternate forms of a 50-item SJT, and 11 assessment centres. For a single MMI question and one assessor, 28% of the variance between scores was due to candidate-to-candidate variation. Interviewer subjectivity, in particular the varying views that interviewer had for particular candidates accounted for 40% of the variance in scores. The generalisability co-efficient for a six question MMI was 0.7; to achieve 0.8 would require ten questions. A disattenuated correlation with the SJT (r = 0.35), and in particular a raw score correlation with the subdomain related to clinical knowledge (r = 0.25) demonstrated evidence for construct and concurrent validity. Less than two per cent of candidates would have failed the MMI. Conclusion The MMI is a moderately reliable method of assessment in the context of a National Assessment Centre approach. The largest source of error relates to aspects of interviewer subjectivity, suggesting enhanced interviewer training would be beneficial. MMIs need to be sufficiently long for precise comparison for ranking purposes. In order to justify long term sustainable use of the MMI in a postgraduate assessment centre approach, more theoretical work is required to understand how written and performance based test of non-cognitive attributes can be combined, in a way that achieves acceptable generalizability, and has validity. PMID:25123968

Health Information Needs and Reliability of Sources Among Nondegree Health Sciences Students: A Prerequisite for Designing eHealth Literacy.

PubMed

Haruna, Hussein; Tshuma, Ndumiso; Hu, Xiao

Understanding health information needs and health-seeking behavior is a prerequisite for developing an electronic health information literacy (EHIL) or eHealth literacy program for nondegree health sciences students. At present, interest in researching health information needs and reliable sources paradigms has gained momentum in many countries. However, most studies focus on health professionals and students in higher education institutions. The present study was aimed at providing new insight and filling the existing gap by examining health information needs and reliability of sources among nondegree health sciences students in Tanzania. A cross-sectional study was conducted in 15 conveniently selected health training institutions, where 403 health sciences students were participated. Thirty health sciences students were both purposely and conveniently chosen from each health-training institution. The selected students were pursuing nursing and midwifery, clinical medicine, dentistry, environmental health sciences, pharmacy, and medical laboratory sciences courses. Involved students were either in their first year, second year, or third year of study. Health sciences students' health information needs focus on their educational requirements, clinical practice, and personal information. They use print, human, and electronic health information. They lack eHealth research skills in navigating health information resources and have insufficient facilities for accessing eHealth information, a lack of specialists in health information, high costs for subscription electronic information, and unawareness of the availability of free Internet and other online health-related databases. This study found that nondegree health sciences students have limited skills in EHIL. Thus, designing and incorporating EHIL skills programs into the curriculum of nondegree health sciences students is vital. EHIL is a requirement common to all health settings, learning environments, and levels of study. Our future intention is to design EHIL to support nondegree health sciences students to retrieve and use available health information resources on the Internet. Copyright © 2017 Icahn School of Medicine at Mount Sinai. Published by Elsevier Inc. All rights reserved.

Using Neuroplasticity-Based Auditory Training to Improve Verbal Memory in Schizophrenia

PubMed Central

Fisher, Melissa; Holland, Christine; Merzenich, Michael M.; Vinogradov, Sophia

2009-01-01

Objective Impaired verbal memory in schizophrenia is a key rate-limiting factor for functional outcome, does not respond to currently available medications, and shows only modest improvement after conventional behavioral remediation. The authors investigated an innovative approach to the remediation of verbal memory in schizophrenia, based on principles derived from the basic neuroscience of learning-induced neuroplasticity. The authors report interim findings in this ongoing study. Method Fifty-five clinically stable schizophrenia subjects were randomly assigned to either 50 hours of computerized auditory training or a control condition using computer games. Those receiving auditory training engaged in daily computerized exercises that placed implicit, increasing demands on auditory perception through progressively more difficult auditory-verbal working memory and verbal learning tasks. Results Relative to the control group, subjects who received active training showed significant gains in global cognition, verbal working memory, and verbal learning and memory. They also showed reliable and significant improvement in auditory psychophysical performance; this improvement was significantly correlated with gains in verbal working memory and global cognition. Conclusions Intensive training in early auditory processes and auditory-verbal learning results in substantial gains in verbal cognitive processes relevant to psychosocial functioning in schizophrenia. These gains may be due to a training method that addresses the early perceptual impairments in the illness, that exploits intact mechanisms of repetitive practice in schizophrenia, and that uses an intensive, adaptive training approach. PMID:19448187

Rating the methodological quality of single-subject designs and n-of-1 trials: introducing the Single-Case Experimental Design (SCED) Scale.

PubMed

Tate, Robyn L; McDonald, Skye; Perdices, Michael; Togher, Leanne; Schultz, Regina; Savage, Sharon

2008-08-01

Rating scales that assess methodological quality of clinical trials provide a means to critically appraise the literature. Scales are currently available to rate randomised and non-randomised controlled trials, but there are none that assess single-subject designs. The Single-Case Experimental Design (SCED) Scale was developed for this purpose and evaluated for reliability. Six clinical researchers who were trained and experienced in rating methodological quality of clinical trials developed the scale and participated in reliability studies. The SCED Scale is an 11-item rating scale for single-subject designs, of which 10 items are used to assess methodological quality and use of statistical analysis. The scale was developed and refined over a 3-year period. Content validity was addressed by identifying items to reduce the main sources of bias in single-case methodology as stipulated by authorities in the field, which were empirically tested against 85 published reports. Inter-rater reliability was assessed using a random sample of 20/312 single-subject reports archived in the Psychological Database of Brain Impairment Treatment Efficacy (PsycBITE). Inter-rater reliability for the total score was excellent, both for individual raters (overall ICC = 0.84; 95% confidence interval 0.73-0.92) and for consensus ratings between pairs of raters (overall ICC = 0.88; 95% confidence interval 0.78-0.95). Item reliability was fair to excellent for consensus ratings between pairs of raters (range k = 0.48 to 1.00). The results were replicated with two independent novice raters who were trained in the use of the scale (ICC = 0.88, 95% confidence interval 0.73-0.95). The SCED Scale thus provides a brief and valid evaluation of methodological quality of single-subject designs, with the total score demonstrating excellent inter-rater reliability using both individual and consensus ratings. Items from the scale can also be used as a checklist in the design, reporting and critical appraisal of single-subject designs, thereby assisting to improve standards of single-case methodology.

Using Tri-Axial Accelerometry in Daily Elite Swim Training Practice.

PubMed

Ganzevles, Sander; Vullings, Rik; Beek, Peter Jan; Daanen, Hein; Truijens, Martin

2017-04-29

Background : Coaches in elite swimming carefully design the training programs of their swimmers and are keen on achieving strict adherence to those programs by their athletes. At present, coaches usually monitor the compliance of their swimmers to the training program with a stopwatch. However, this measurement clearly limits the monitoring possibilities and is subject to human error. Therefore, the present study was designed to examine the reliability and practical usefulness of tri-axial accelerometers for monitoring lap time, stroke count and stroke rate in swimming. Methods : In the first part of the study, a 1200 m warm-up swimming routine was measured in 13 elite swimmers using tri-axial accelerometers and synchronized video recordings. Reliability was determined using the typical error of measurement (TEM) as well as a Bland-Altman analysis. In the second part, training compliance both within and between carefully prescribed training sessions was assessed in four swimmers in order to determine the practical usefulness of the adopted accelerometric approach. In these sessions, targets were set for lap time and stroke count by the coach. Results : The results indicated high reliability for lap time (TEM = 0.26 s, bias = 0.74 [0.56 0.91] with limits of agreement (LoA) from -1.20 [-1.50 -0.90] to 2.70 [2.40 3.00]), stroke count (TEM 0.73 strokes, bias = 0.46 [0.32 0.60] with LoA from -1.70 [-1.94 -1.46] to 2.60 [2.36 2.84]) and stroke rate (TEM 0.72 str∙min -1 , bias = -0.13 [-0.20 -0.06] with LoA from -2.20 [-2.32 -2.08] to 1.90 [1.78 2.02]), while the results for the monitoring of training compliance demonstrated the practical usefulness of our approach in daily swimming training. Conclusions : The daily training of elite swimmers can be accurately and reliably monitored using tri-axial accelerometers. They provide the coach with more useful information to guide and control the training process than hand-clocked times.

Using Tri-Axial Accelerometry in Daily Elite Swim Training Practice

PubMed Central

Ganzevles, Sander; Vullings, Rik; Beek, Peter Jan; Daanen, Hein; Truijens, Martin

2017-01-01

Background: Coaches in elite swimming carefully design the training programs of their swimmers and are keen on achieving strict adherence to those programs by their athletes. At present, coaches usually monitor the compliance of their swimmers to the training program with a stopwatch. However, this measurement clearly limits the monitoring possibilities and is subject to human error. Therefore, the present study was designed to examine the reliability and practical usefulness of tri-axial accelerometers for monitoring lap time, stroke count and stroke rate in swimming. Methods: In the first part of the study, a 1200 m warm-up swimming routine was measured in 13 elite swimmers using tri-axial accelerometers and synchronized video recordings. Reliability was determined using the typical error of measurement (TEM) as well as a Bland-Altman analysis. In the second part, training compliance both within and between carefully prescribed training sessions was assessed in four swimmers in order to determine the practical usefulness of the adopted accelerometric approach. In these sessions, targets were set for lap time and stroke count by the coach. Results: The results indicated high reliability for lap time (TEM = 0.26 s, bias = 0.74 [0.56 0.91] with limits of agreement (LoA) from −1.20 [−1.50 −0.90] to 2.70 [2.40 3.00]), stroke count (TEM 0.73 strokes, bias = 0.46 [0.32 0.60] with LoA from −1.70 [−1.94 −1.46] to 2.60 [2.36 2.84]) and stroke rate (TEM 0.72 str∙min−1, bias = −0.13 [−0.20 −0.06] with LoA from −2.20 [−2.32 −2.08] to 1.90 [1.78 2.02]), while the results for the monitoring of training compliance demonstrated the practical usefulness of our approach in daily swimming training. Conclusions: The daily training of elite swimmers can be accurately and reliably monitored using tri-axial accelerometers. They provide the coach with more useful information to guide and control the training process than hand-clocked times. PMID:28468255

The validation of a swimming turn wall-contact-time measurement system: a touchpad application reliability study.

PubMed

Brackley, Victoria; Ball, Kevin; Tor, Elaine

2018-05-12

The effectiveness of the swimming turn is highly influential to overall performance in competitive swimming. The push-off or wall contact, within the turn phase, is directly involved in determining the speed the swimmer leaves the wall. Therefore, it is paramount to develop reliable methods to measure the wall-contact-time during the turn phase for training and research purposes. The aim of this study was to determine the concurrent validity and reliability of the Pool Pad App to measure wall-contact-time during the freestyle and backstroke tumble turn. The wall-contact-times of nine elite and sub-elite participants were recorded during their regular training sessions. Concurrent validity statistics included the standardised typical error estimate, linear analysis and effect sizes while the intraclass correlating coefficient (ICC) was used for the reliability statistics. The standardised typical error estimate resulted in a moderate Cohen's d effect size with an R 2 value of 0.80 and the ICC between the Pool Pad and 2D video footage was 0.89. Despite these measurement differences, the results from this concurrent validity and reliability analyses demonstrated that the Pool Pad is suitable for measuring wall-contact-time during the freestyle and backstroke tumble turn within a training environment.

Strategies for Tobacco Control in India: A Systematic Review

PubMed Central

McKay, Ailsa J.; Patel, Raju K. K.; Majeed, Azeem

2015-01-01

Background Tobacco control needs in India are large and complex. Evaluation of outcomes to date has been limited. Aim To review the extent of tobacco control measures, and the outcomes of associated trialled interventions, in India. Methods Information was identified via database searches, journal hand-searches, reference and citation searching, and contact with experts. Studies of any population resident in India were included. Studies where outcomes were not yet available, not directly related to tobacco use, or not specific to India, were excluded. Pre-tested proformas were used for data extraction and quality assessment. Studies with reliability concerns were excluded from some aspects of analysis. The Framework Convention on Tobacco Control (FCTC) was use as a framework for synthesis. Heterogeneity limited meta-analysis options. Synthesis was therefore predominantly narrative. Results Additional to the Global Tobacco Surveillance System data, 80 studies were identified, 45 without reliability concerns. Most related to education (FCTC Article 12) and tobacco-use cessation (Article 14). They indicated widespread understanding of tobacco-related harm, but less knowledge about specific consequences of use. Healthcare professionals reported low confidence in cessation assistance, in keeping with low levels of training. Training for schoolteachers also appeared suboptimal. Educational and cessation assistance interventions demonstrated positive impact on tobacco use. Studies relating to smoke-free policies (Article 8), tobacco advertisements and availability (Articles 13 and 16) indicated increasingly widespread smoke-free policies, but persistence of high levels of SHS exposure, tobacco promotions and availability—including to minors. Data relating to taxation/pricing and packaging (Articles 6 and 11) were limited. We did not identify any studies of product regulation, alternative employment strategies, or illicit trade (Articles 9, 10, 15 and 17). Conclusions Tobacco-use outcomes could be improved by school/community-based and adult education interventions, and cessation assistance, facilitated by training for health professionals and schoolteachers. Additional tobacco control measures should be assessed. PMID:25856462

Video training and certification program improves reliability of postischemic neurologic deficit measurement in the rat.

PubMed

Taninishi, Hideki; Pearlstein, Molly; Sheng, Huaxin; Izutsu, Miwa; Chaparro, Rafael E; Goldstein, Larry B; Warner, David S

2016-12-01

Scoring systems are used to measure behavioral deficits in stroke research. Video-assisted training is used to standardize stroke-related neurologic deficit scoring in humans. We hypothesized that a video-assisted training and certification program can improve inter-rater reliability in assessing neurologic function after middle cerebral artery occlusion in rats. Three expert raters scored neurologic deficits in post-middle cerebral artery occlusion rats using three published systems having different complexity levels (3, 18, or 48 points). The system having the highest point estimate for the correlation between neurologic score and infarct size was selected to create a video-assisted training and certification program. Eight trainee raters completed the video-assisted training and certification program. Inter-rater agreement ( Κ: score) and agreement with expert consensus scores were measured before and after video-assisted training and certification program completion. The 48-point system correlated best with infarct size. Video-assisted training and certification improved agreement with expert consensus scores (pretraining = 65 ± 10, posttraining = 87 ± 14, 112 possible scores, P < 0.0001), median number of trainee raters with scores within ±2 points of the expert consensus score (pretraining = 4, posttraining = 6.5, P < 0.01), categories with Κ:  > 0.4 (pretraining = 4, posttraining = 9), and number of categories with an improvement in the Κ: score from pretraining to posttraining (n = 6). Video-assisted training and certification improved trainee inter-rater reliability and agreement with expert consensus behavioral scores in rats after middle cerebral artery occlusion. Video-assisted training and certification may be useful in multilaboratory preclinical studies. © The Author(s) 2015.

The Effect of Anchors and Training on the Reliability of Voice Quality Ratings for Different Types of Speech Stimuli.

PubMed

Brinca, Lilia; Batista, Ana Paula; Tavares, Ana Inês; Pinto, Patrícia N; Araújo, Lara

2015-11-01

The main objective of the present study was to investigate if the type of voice stimuli-sustained vowel, oral reading, and connected speech-results in good intrarater and interrater agreement/reliability. A short-term panel study was performed. Voice samples from 30 native European Portuguese speakers were used in the present study. The speech materials used were (1) the sustained vowel /a/, (2) oral reading of the European Portuguese version of "The Story of Arthur the Rat," and (3) connected speech. After an extensive training with textual and auditory anchors, the judges were asked to rate the severity of dysphonic voice stimuli using the phonation dimensions G, R, and B from the GRBAS scale. The voice samples were judged 6 months and 1 year after the training. Intrarater agreement and reliability were generally very good for all the phonation dimensions and voice stimuli. The highest interrater reliability was obtained using the oral reading stimulus, particularly for phonation dimensions grade (G) and breathiness (B). Roughness (R) was the voice quality that was the most difficult to evaluate, leading to interrater unreliability in all voice quality ratings. Extensive training using textual and auditory anchors and the use of anchors during the voice evaluations appear to be good methods for auditory-perceptual evaluation of dysphonic voices. The best results of interrater reliability were obtained when the oral reading stimulus was used. Breathiness appears to be a voice quality that is easier to evaluate than roughness. Copyright © 2015 The Voice Foundation. Published by Elsevier Inc. All rights reserved.

Automatic training and reliability estimation for 3D ASM applied to cardiac MRI segmentation

NASA Astrophysics Data System (ADS)

Tobon-Gomez, Catalina; Sukno, Federico M.; Butakoff, Constantine; Huguet, Marina; Frangi, Alejandro F.

2012-07-01

Training active shape models requires collecting manual ground-truth meshes in a large image database. While shape information can be reused across multiple imaging modalities, intensity information needs to be imaging modality and protocol specific. In this context, this study has two main purposes: (1) to test the potential of using intensity models learned from MRI simulated datasets and (2) to test the potential of including a measure of reliability during the matching process to increase robustness. We used a population of 400 virtual subjects (XCAT phantom), and two clinical populations of 40 and 45 subjects. Virtual subjects were used to generate simulated datasets (MRISIM simulator). Intensity models were trained both on simulated and real datasets. The trained models were used to segment the left ventricle (LV) and right ventricle (RV) from real datasets. Segmentations were also obtained with and without reliability information. Performance was evaluated with point-to-surface and volume errors. Simulated intensity models obtained average accuracy comparable to inter-observer variability for LV segmentation. The inclusion of reliability information reduced volume errors in hypertrophic patients (EF errors from 17 ± 57% to 10 ± 18% LV MASS errors from -27 ± 22 g to -14 ± 25 g), and in heart failure patients (EF errors from -8 ± 42% to -5 ± 14%). The RV model of the simulated images needs further improvement to better resemble image intensities around the myocardial edges. Both for real and simulated models, reliability information increased segmentation robustness without penalizing accuracy.

Automatic training and reliability estimation for 3D ASM applied to cardiac MRI segmentation.

PubMed

Tobon-Gomez, Catalina; Sukno, Federico M; Butakoff, Constantine; Huguet, Marina; Frangi, Alejandro F

2012-07-07

Training active shape models requires collecting manual ground-truth meshes in a large image database. While shape information can be reused across multiple imaging modalities, intensity information needs to be imaging modality and protocol specific. In this context, this study has two main purposes: (1) to test the potential of using intensity models learned from MRI simulated datasets and (2) to test the potential of including a measure of reliability during the matching process to increase robustness. We used a population of 400 virtual subjects (XCAT phantom), and two clinical populations of 40 and 45 subjects. Virtual subjects were used to generate simulated datasets (MRISIM simulator). Intensity models were trained both on simulated and real datasets. The trained models were used to segment the left ventricle (LV) and right ventricle (RV) from real datasets. Segmentations were also obtained with and without reliability information. Performance was evaluated with point-to-surface and volume errors. Simulated intensity models obtained average accuracy comparable to inter-observer variability for LV segmentation. The inclusion of reliability information reduced volume errors in hypertrophic patients (EF errors from 17 ± 57% to 10 ± 18%; LV MASS errors from -27 ± 22 g to -14 ± 25 g), and in heart failure patients (EF errors from -8 ± 42% to -5 ± 14%). The RV model of the simulated images needs further improvement to better resemble image intensities around the myocardial edges. Both for real and simulated models, reliability information increased segmentation robustness without penalizing accuracy.

The Chinese-version of the CARE Measure reliably differentiates between doctors in primary care: a cross-sectional study in Hong Kong

PubMed Central

2011-01-01

Background The Consultation and Relational Empathy (CARE) Measure is a widely used patient-rated experience measure which has recently been translated into Chinese and has undergone preliminary qualitative and quantitative validation. The objective of this study was to determine the reliability of the Chinese-version of the CARE Measure in reliably differentiating between doctors in a primary care setting in Hong Kong Methods Data were collected from 984 primary care patients attending 20 doctors with differing levels of training in family medicine in 5 public clinics in Hong Kong. The acceptability of the Chinese-CARE measure to patients was assessed. The reliability of the measure in discriminating effectively between doctors was analysed by Generalisability-theory (G-Theory) Results The items in the Chinese-CARE measure were regarded as important by patients and there were few 'not applicable' responses. The measure showed high internal reliability (coefficient 0.95) and effectively differentiated between doctors with only 15-20 patient ratings per doctor (inter-rater reliability > 0.8). Doctors' mean CARE measure scores varied widely, ranging from 24.1 to 45.9 (maximum possible score 50) with a mean of 34.6. CARE Measure scores were positively correlated with level of training in family medicine (Spearman's rho 0.493, p < 0.05). Conclusion These data demonstrate the acceptability, feasibility and reliability of using the Chinese-CARE Measure in primary care in Hong Kong to differentiate between doctors interpersonal competencies. Training in family medicine appears to enhance these key interpersonal skills. PMID:21631927

Finding Reliable Health Information Online

MedlinePlus

Skip to main content Finding Reliable Health Information Online Enter Search Term(s): Español Research Funding An Overview Bioinformatics Current Grants Education and Training Funding Extramural Research News Features Funding Divisions Funding ...

Artificial neural network prediction of aircraft aeroelastic behavior

NASA Astrophysics Data System (ADS)

Pesonen, Urpo Juhani

An Artificial Neural Network that predicts aeroelastic behavior of aircraft is presented. The neural net was designed to predict the shape of a flexible wing in static flight conditions using results from a structural analysis and an aerodynamic analysis performed with traditional computational tools. To generate reliable training and testing data for the network, an aeroelastic analysis code using these tools as components was designed and validated. To demonstrate the advantages and reliability of Artificial Neural Networks, a network was also designed and trained to predict airfoil maximum lift at low Reynolds numbers where wind tunnel data was used for the training. Finally, a neural net was designed and trained to predict the static aeroelastic behavior of a wing without the need to iterate between the structural and aerodynamic solvers.

Validity and reliability of an in-training evaluation report to measure the CanMEDS roles in emergency medicine residents.

PubMed

Kassam, Aliya; Donnon, Tyrone; Rigby, Ian

2014-03-01

There is a question of whether a single assessment tool can assess the key competencies of residents as mandated by the Royal College of Physicians and Surgeons of Canada CanMEDS roles framework. The objective of the present study was to investigate the reliability and validity of an emergency medicine (EM) in-training evaluation report (ITER). ITER data from 2009 to 2011 were combined for residents across the 5 years of the EM residency training program. An exploratory factor analysis with varimax rotation was used to explore the construct validity of the ITER. A total of 172 ITERs were completed on residents across their first to fifth year of training. A combined, 24-item ITER yielded a five-factor solution measuring the CanMEDs role Medical Expert/Scholar, Communicator/Collaborator, Professional, Health Advocate and Manager subscales. The factor solution accounted for 79% of the variance, and reliability coefficients (Cronbach alpha) ranged from α  =  0.90 to 0.95 for each subscale and α  =  0.97 overall. The combined, 24-item ITER used to assess residents' competencies in the EM residency program showed strong reliability and evidence of construct validity for assessment of the CanMEDS roles. Further research is needed to develop and test ITER items that will differentiate each CanMEDS role exclusively.

The effect on reliability and sensitivity to level of training of combining analytic and holistic rating scales for assessing communication skills in an internal medicine resident OSCE.

PubMed

Daniels, Vijay John; Harley, Dwight

2017-07-01

Although previous research has compared checklists to rating scales for assessing communication, the purpose of this study was to compare the effect on reliability and sensitivity to level of training of an analytic, a holistic, and a combined analytic-holistic rating scale in assessing communication skills. The University of Alberta Internal Medicine Residency runs OSCEs for postgraduate year (PGY) 1 and 2 residents and another for PGY-4 residents. Communication stations were scored with an analytic scale (empathy, non-verbal skills, verbal skills, and coherence subscales) and a holistic scale. Authors analyzed reliability of individual and combined scales using generalizability theory and evaluated each scale's sensitivity to level of training. For analytic, holistic, and combined scales, 12, 12, and 11 stations respectively yielded a Phi of 0.8 for the PGY-1,2 cohort, and 16, 16, and 14 stations yielded a Phi of 0.8 for the PGY-4 cohort. PGY-4 residents scored higher on the combined scale, the analytic rating scale, and the non-verbal and coherence subscales. A combined analytic-holistic rating scale increased score reliability and was sensitive to level of training. Given increased validity evidence, OSCE developers should consider combining analytic and holistic scales when assessing communication skills. Copyright © 2017 Elsevier B.V. All rights reserved.

Reliability and variability of day-to-day vault training measures in artistic gymnastics.

PubMed

Bradshaw, Elizabeth; Hume, Patria; Calton, Mark; Aisbett, Brad

2010-06-01

Inter-day training reliability and variability in artistic gymnastics vaulting was determined using a customised infra-red timing gate and contact mat timing system. Thirteen Australian high performance gymnasts (eight males and five females) aged 11-23 years were assessed during two consecutive days of normal training. Each gymnast completed a number of vault repetitions per daily session. Inter-day variability of vault run-up velocities (at -18 to -12 m, -12 to -6 m, -6 to -2 m, and -2 to 0 m from the nearest edge of the beat board), and board contact, pre-flight, and table contact times were determined using mixed modelling statistics to account for random (within-subject variability) and fixed effects (gender, number of subjects, number of trials). The difference in the mean (Mdiff) and Cohen's effect sizes for reliability assessment and intra-class correlation coefficients, and the coefficient of variation percentage (CV%) were calculated for variability assessment. Approach velocity (-18 to -2m, CV = 2.4-7.8%) and board contact time (CV = 3.5%) were less variable measures when accounting for day-to-day performance differences, than pre-flight time (CV = 17.7%) and table contact time (CV = 20.5%). While pre-flight and table contact times are relevant training measures, approach velocity and board contact time are more reliable when quantifying vaulting performance.

Structured On-the-Job Training: Unleashing Employee Experience in the Workplace. Second Edition.

ERIC Educational Resources Information Center

Jacobs, Ronald L.

This book presents detailed guidelines for conducting structured on-the-job training (S-OJT), which is training based on the premise that OJT can be structured and planned so as to produce more predictable and reliable training outcomes. Together, the book's 15 chapters systematically examine the topics of meeting the demand for employee…

Looking for underlying features in automatic and reviewed seismic bulletins through a neural network

NASA Astrophysics Data System (ADS)

Carluccio, R.; Console, R.; Chiappini, M.; Chiappini, S.

2009-12-01

SEL1 bulletins are, among all IDC products, a fundamental tool for NDCs in their task of national assessment of compliance with the CTBT. This is because SEL1s are expected to be disseminated within 2 hours from the occurrence of any detected waveform event, and the National Authorities are supposed to take a political decision in nearly real time, especially in the case when the event could triggers the request for an on site inspection. In this context not only the rapidity, but also the reliability of the SEL1 is a fundamental requirement. Our last years experience gained in the comparison between SEL1 and Italian Seismic Bulletin events has shown that SEL1s usually contain a big fraction of bogus events (sometimes close to 50%). This is due to many factors, all related to the availability of processing data and to the fast automatic algorithms involved. On the other hand, REBs are much more reliable as proved by our experience. Therefore, in spite of their relevant time delay by which they are distributed, which prevents their real-time use, REBs can be still useful in a retrospective way as reference information for comparison with SEL1s. This study tries to set up a sort of logical filter on the SEL1s that, while maintaining the rapidity requirements, improves their reliability. Our idea is based on the assumption that the SEL1s are produced by systematic algorithm of phase association and therefore some patterns among the input and output data could exist and be recognized. Our approach was initially based on a set of rules suggested by human experts on their personal experience, and its application on large datasets on a global scale. Other approaches not involving human interaction (data mining techniques) do exist. This study refers specifically to a semi-automatic approach: fitting of multi-parametric relationships hidden in the data set, through the application of neural networks by an algorithm of supervised learning. Full SEL1 and REB bulletins from Jan 2005 to Oct 2008 have been inserted in a database, together with IMS stations availability information. Part of these data have been used to create two sets of independent data (learning and verifying) used to train a "feed-forward" supervised neural network. A network supervised training algorithm using "confirmation flag" values has been used. In order to optimize network training input a significant, not redundant subset of input parameters has been looked for with the help of a genetic algorithm search tool. A suitable 12 input subset has been found and a network architecture of 12-20-1 has thus been chosen and trained on a 15094 records data set. Different runs of training sequences have been conducted, all showing CCR (Correct Classification Rate) values of the order of 75% - 80%. The trained network behavior is shown in term of ROC curve and input-out success-error matrices. The results of the analysis on our testing and validating data groups appear promising.

Accuracy and reliability of peer assessment of athletic training psychomotor laboratory skills.

PubMed

Marty, Melissa C; Henning, Jolene M; Willse, John T

2010-01-01

Peer assessment is defined as students judging the level or quality of a fellow student's understanding. No researchers have yet demonstrated the accuracy or reliability of peer assessment in athletic training education. To determine the accuracy and reliability of peer assessment of athletic training students' psychomotor skills. Cross-sectional study. Entry-level master's athletic training education program. First-year (n  =  5) and second-year (n  =  8) students. Participants evaluated 10 videos of a peer performing 3 psychomotor skills (middle deltoid manual muscle test, Faber test, and Slocum drawer test) on 2 separate occasions using a valid assessment tool. Accuracy of each peer-assessment score was examined through percentage correct scores. We used a generalizability study to determine how reliable athletic training students were in assessing a peer performing the aforementioned skills. Decision studies using generalizability theory demonstrated how the peer-assessment scores were affected by the number of participants and number of occasions. Participants had a high percentage of correct scores: 96.84% for the middle deltoid manual muscle test, 94.83% for the Faber test, and 97.13% for the Slocum drawer test. They were not able to reliably assess a peer performing any of the psychomotor skills on only 1 occasion. However, the φ increased (exceeding the 0.70 minimal standard) when 2 participants assessed the skill on 3 occasions (φ  =  0.79) for the Faber test, with 1 participant on 2 occasions (φ  =  0.76) for the Slocum drawer test, and with 3 participants on 2 occasions for the middle deltoid manual muscle test (φ  =  0.72). Although students did not detect all errors, they assessed their peers with an average of 96% accuracy. Having only 1 student assess a peer performing certain psychomotor skills was less reliable than having more than 1 student assess those skills on more than 1 occasion. Peer assessment of psychomotor skills could be an important part of the learning process and a tool to supplement instructor assessment.

Curriculum Development and Implementation of a National Interprofessional Fellowship in Patient Safety.

PubMed

Watts, Bradley V; Williams, Linda; Mills, Peter D; Paull, Douglas E; Cully, Jeffrey A; Gilman, Stuart C; Hemphill, Robin R

2018-06-15

Developing a workforce skilled in improving the safety of medical care has often been cited as an important means to achieve safer care. Although some educational programs geared toward patient safety have been developed, few advanced training programs have been described in the literature. We describe the development of a patient safety fellowship program. We describe the development and curriculum of an Interprofessional Fellowship in Patient Safety. The 1-year in residence fellowship focuses on domains such as leadership, spreading innovations, medical improvement, patient safety culture, reliability science, and understanding errors. Specific training in patient safety is available and has been delivered to 48 fellows from a wide range of backgrounds. Fellows have accomplished much in terms of improvement projects, educational innovations, and publications. After completing the fellowship program, fellows are obtaining positions within health-care quality and safety and are likely to make long-term contributions. We offer a curriculum and fellowship design for the topic of patient safety. Available evidence suggests that the fellowship results in the development of patient safety professionals.

77 FR 47831 - Combined Notice of Filings #1

Federal Register 2010, 2011, 2012, 2013, 2014

2012-08-10

... to Reliability Standard CIP-002-4--Critical Cyber Asset Identification. Filed Date: 8/1/12. Accession... Corporation for Approval of an Interpretation to Reliability Standard CIP-004-4--Personnel and Training. Filed...

Evaluation of Human Reliability in Selected Activities in the Railway Industry

NASA Astrophysics Data System (ADS)

Sujová, Erika; Čierna, Helena; Molenda, Michał

2016-09-01

The article focuses on evaluation of human reliability in the human - machine system in the railway industry. Based on a survey of a train dispatcher and of selected activities, we have identified risk factors affecting the dispatcher`s work and the evaluated risk level of their influence on the reliability and safety of preformed activities. The research took place at the authors` work place between 2012-2013. A survey method was used. With its help, authors were able to identify selected work activities of train dispatcher's risk factors that affect his/her work and the evaluated seriousness of its influence on the reliability and safety of performed activities. Amongst the most important finding fall expressions of unclear and complicated internal regulations and work processes, a feeling of being overworked, fear for one's safety at small, insufficiently protected stations.

Sources of Free and Open Source Spatial Data for Natural Disasters and Principles for Use in Developing Country Contexts

NASA Astrophysics Data System (ADS)

Taylor, Faith E.; Malamud, Bruce D.; Millington, James D. A.

2016-04-01

Access to reliable spatial and quantitative datasets (e.g., infrastructure maps, historical observations, environmental variables) at regional and site specific scales can be a limiting factor for understanding hazards and risks in developing country settings. Here we present a 'living database' of >75 freely available data sources relevant to hazard and risk in Africa (and more globally). Data sources include national scientific foundations, non-governmental bodies, crowd-sourced efforts, academic projects, special interest groups and others. The database is available at http://tinyurl.com/africa-datasets and is continually being updated, particularly in the context of broader natural hazards research we are doing in the context of Malawi and Kenya. For each data source, we review the spatiotemporal resolution and extent and make our own assessments of reliability and usability of datasets. Although such freely available datasets are sometimes presented as a panacea to improving our understanding of hazards and risk in developing countries, there are both pitfalls and opportunities unique to using this type of freely available data. These include factors such as resolution, homogeneity, uncertainty, access to metadata and training for usage. Based on our experience, use in the field and grey/peer-review literature, we present a suggested set of guidelines for using these free and open source data in developing country contexts.

Voltage control on a train system

DOEpatents

Gordon, Susanna P.; Evans, John A.

2004-01-20

The present invention provides methods for preventing low train voltages and managing interference, thereby improving the efficiency, reliability, and passenger comfort associated with commuter trains. An algorithm implementing neural network technology is used to predict low voltages before they occur. Once voltages are predicted, then multiple trains can be controlled to prevent low voltage events. Further, algorithms for managing inference are presented in the present invention. Different types of interference problems are addressed in the present invention such as "Interference During Acceleration", "Interference Near Station Stops", and "Interference During Delay Recovery." Managing such interference avoids unnecessary brake/acceleration cycles during acceleration, immediately before station stops, and after substantial delays. Algorithms are demonstrated to avoid oscillatory brake/acceleration cycles due to interference and to smooth the trajectories of closely following trains. This is achieved by maintaining sufficient following distances to avoid unnecessary braking/accelerating. These methods generate smooth train trajectories, making for a more comfortable ride, and improve train motor reliability by avoiding unnecessary mode-changes between propulsion and braking. These algorithms can also have a favorable impact on traction power system requirements and energy consumption.

Method of managing interference during delay recovery on a train system

DOEpatents

Gordon, Susanna P.; Evans, John A.

2005-12-27

The present invention provides methods for preventing low train voltages and managing interference, thereby improving the efficiency, reliability, and passenger comfort associated with commuter trains. An algorithm implementing neural network technology is used to predict low voltages before they occur. Once voltages are predicted, then multiple trains can be controlled to prevent low voltage events. Further, algorithms for managing inference are presented in the present invention. Different types of interference problems are addressed in the present invention such as "Interference During Acceleration", "Interference Near Station Stops", and "Interference During Delay Recovery." Managing such interference avoids unnecessary brake/acceleration cycles during acceleration, immediately before station stops, and after substantial delays. Algorithms are demonstrated to avoid oscillatory brake/acceleration cycles due to interference and to smooth the trajectories of closely following trains. This is achieved by maintaining sufficient following distances to avoid unnecessary braking/accelerating. These methods generate smooth train trajectories, making for a more comfortable ride, and improve train motor reliability by avoiding unnecessary mode-changes between propulsion and braking. These algorithms can also have a favorable impact on traction power system requirements and energy consumption.

Efficient high density train operations

DOEpatents

Gordon, Susanna P.; Evans, John A.

2001-01-01

The present invention provides methods for preventing low train voltages and managing interference, thereby improving the efficiency, reliability, and passenger comfort associated with commuter trains. An algorithm implementing neural network technology is used to predict low voltages before they occur. Once voltages are predicted, then multiple trains can be controlled to prevent low voltage events. Further, algorithms for managing inference are presented in the present invention. Different types of interference problems are addressed in the present invention such as "Interference. During Acceleration", "Interference Near Station Stops", and "Interference During Delay Recovery." Managing such interference avoids unnecessary brake/acceleration cycles during acceleration, immediately before station stops, and after substantial delays. Algorithms are demonstrated to avoid oscillatory brake/acceleration cycles due to interference and to smooth the trajectories of closely following trains. This is achieved by maintaining sufficient following distances to avoid unnecessary braking/accelerating. These methods generate smooth train trajectories, making for a more comfortable ride, and improve train motor reliability by avoiding unnecessary mode-changes between propulsion and braking. These algorithms can also have a favorable impact on traction power system requirements and energy consumption.

The Internet for neurosurgeons: current resources and future challenges.

PubMed

Hughes, Mark A; Brennan, Paul M

2011-06-01

Our professional and personal lives depend increasingly on access to information via the Internet. As an open access resource, the Internet is on the whole unbridled by censorship and can facilitate the rapid propagation of ideas and discoveries. At the same time, this liberty in sharing information, being unregulated and often free from external validation, can be oppressive; overloading the user and hindering effective decision-making. It can be difficult, if not impossible, to reliably ascertain the provenance of data and opinion. We must, therefore, discern what is useful, relevant, and above all reliable if we are to harness the Internet's potential to improve training, delivery of care, research, and provision of patient information. This article profiles the resources currently available to neurosurgeons, asks how we can sort the informational wheat from the chaff, and explores where future developments might further influence neurosurgical practice.

The probability estimation of the electronic lesson implementation taking into account software reliability

NASA Astrophysics Data System (ADS)

Gurov, V. V.

2017-01-01

Software tools for educational purposes, such as e-lessons, computer-based testing system, from the point of view of reliability, have a number of features. The main ones among them are the need to ensure a sufficiently high probability of their faultless operation for a specified time, as well as the impossibility of their rapid recovery by the way of replacing it with a similar running program during the classes. The article considers the peculiarities of reliability evaluation of programs in contrast to assessments of hardware reliability. The basic requirements to reliability of software used for carrying out practical and laboratory classes in the form of computer-based training programs are given. The essential requirements applicable to the reliability of software used for conducting the practical and laboratory studies in the form of computer-based teaching programs are also described. The mathematical tool based on Markov chains, which allows to determine the degree of debugging of the training program for use in the educational process by means of applying the graph of the software modules interaction, is presented.

An Australian hospital's training program and referral pathway within a multi-disciplinary health-justice partnership addressing family violence.

PubMed

Forsdike, Kirsty; Humphreys, Cathy; Diemer, Kristin; Ross, Stuart; Gyorki, Linda; Maher, Helena; Vye, Penelope; Llewelyn, Fleur; Hegarty, Kelsey

2018-06-01

An innovative health-justice partnership was established to deliver legal assistance to women experiencing family violence who attended an Australian hospital. This paper reports on a multifaceted response to build capacity and willingness of health professionals to identify signs of family violence and engage with referral pathways to on-site legal assistance. A Realistic Evaluation analysed health professionals' knowledge and attitudes towards identification, response and referral for family violence before and after training; and use of referral pathways. Of 123 health professionals participating in training, 67 completed baseline and follow-up surveys. Training improved health professionals' self-reported knowledge of, and confidence in, responding to family violence and understanding of lawyers' roles in hospitals. Belief that patients should be referred to on-site legal services increased. Training did not correspond to actual increased referrals to legal assistance. The program built capacity and willingness of health professionals to identify signs of, and respond to, family violence. Increase in referral rates to legal assistance was not shown. Potential improvements include better data capture and greater availability of legal services. Implications for public health: Strong hospital system supports and reliable recording of family violence referrals need to be in place before introducing such partnerships to other hospitals. © 2017 The Authors.

Symptom-based categorization of in-flight passenger medical incidents.

PubMed

Mahony, Paul H; Myers, Julia A; Larsen, Peter D; Powell, David M C; Griffiths, Robin F

2011-12-01

The majority of in-flight passenger medical events are managed by cabin crew. Our study aimed to evaluate the reliability of cabin crew reports of in-flight medical events and to develop a symptom-based categorization system. All cabin crew in-flight passenger medical incident reports for an airline over a 9-yr period were examined retrospectively. Validation of incident descriptions were undertaken on a sample of 162 cabin crew reports where medically trained persons' reports were available for comparison using a three Round Delphi technique and testing concordance using Cohen's Kappa. A hierarchical symptom-based categorization system was designed and validated. The rate was 159 incidents per 106 passengers carried, or 70.4/113.3 incidents per 106 revenue passenger kilometres/miles, respectively. Concordance between cabin crew and medical reports was 96%, with a high validity rating (mean 4.6 on a 1-5 scale) and high Cohen's Kappa (0.94). The most common in-flight medical events were transient loss of consciousness (41%), nausea/vomiting/diarrhea (19.5%), and breathing difficulty (16%). Cabin crew records provide reliable data regarding in-flight passenger medical incidents, complementary to diagnosis-based systems, and allow the use of currently underutilized data. The categorization system provides a means for tracking passenger medical incidents internationally and an evidence base for cabin crew first aid training.

Next-generation healthcare: a strategic appraisal.

PubMed

Montague, Terrence

2009-01-01

Successful next-generation healthcare must deliver timely access and quality for an aging population, while simultaneously promoting disease prevention and managing costs. The key factors for sustained success are a culture with aligned goals and values; coordinated team care that especially engages with physicians and patients; practical information that is collected and communicated reliably; and education in the theory and methods of collaboration, measurement and leadership. Currently, optimal population health is challenged by a high prevalence of chronic disease, with large gaps between best and usual care, a scarcity of health human resources - particularly with the skills, attitudes and training for coordinated team care - and the absence of flexible, reliable clinical measurement systems. However, to make things better, institutional models and supporting technologies are available. In the short term, a first step is to enhance the awareness of the practical opportunities to improve, including the expansion of proven community-based disease management programs that communicate knowledge, competencies and clinical measurements among professional and patient partners, leading to reduced care gaps and improved clinical and economic outcomes. Longer-term success requires two additional steps. One is formal inter-professional training to provide, on an ongoing basis, the polyvalent human resource skills and foster the culture of working with others to improve the care of whole populations. The other is the adoption of reliable information systems, including electronic health records, to allow useful and timely measurement and effective communication of clinical information in real-world settings. A better health future can commence immediately, within existing resources, and be sustained with feasible innovations in provider and patient education and information systems. The future is now.

Structured assessment of current mental state in clinical practice: an international study of the reliability and validity of the Current Psychiatric State interview, CPS-50.

PubMed

Falloon, I R H; Mizuno, M; Murakami, M; Roncone, R; Unoka, Z; Harangozo, J; Pullman, J; Gedye, R; Held, T; Hager, B; Erickson, D; Burnett, K

2005-01-01

To develop a reliable standardized assessment of psychiatric symptoms for use in clinical practice. A 50-item interview, the Current Psychiatric State 50 (CPS-50), was used to assess 237 patients with a range of psychiatric diagnoses. Ratings were made by interviewers after a 2-day training. Comparisons of inter-rater reliability on each item and on eight clinical subscales were made across four international centres and between psychiatrists and non-psychiatrists. A principal components analysis was used to validate these clinical scales. Acceptable inter-rater reliability (intra-class coefficient > 0.80) was found for 46 of the 50 items, and for all eight subscales. There was no difference between centres or between psychiatrists and non-psychiatrists. The principal components analysis factors were similar to the clinical scales. The CPS-50 is a reliable standardized assessment of current mental status that can be used in clinical practice by all mental health professionals after brief training. Blackwell Munksgaard 2004

Chemical name extraction based on automatic training data generation and rich feature set.

PubMed

Yan, Su; Spangler, W Scott; Chen, Ying

2013-01-01

The automation of extracting chemical names from text has significant value to biomedical and life science research. A major barrier in this task is the difficulty of getting a sizable and good quality data to train a reliable entity extraction model. Another difficulty is the selection of informative features of chemical names, since comprehensive domain knowledge on chemistry nomenclature is required. Leveraging random text generation techniques, we explore the idea of automatically creating training sets for the task of chemical name extraction. Assuming the availability of an incomplete list of chemical names, called a dictionary, we are able to generate well-controlled, random, yet realistic chemical-like training documents. We statistically analyze the construction of chemical names based on the incomplete dictionary, and propose a series of new features, without relying on any domain knowledge. Compared to state-of-the-art models learned from manually labeled data and domain knowledge, our solution shows better or comparable results in annotating real-world data with less human effort. Moreover, we report an interesting observation about the language for chemical names. That is, both the structural and semantic components of chemical names follow a Zipfian distribution, which resembles many natural languages.

The reliability of workplace-based assessment in postgraduate medical education and training: a national evaluation in general practice in the United Kingdom.

PubMed

Murphy, Douglas J; Bruce, David A; Mercer, Stewart W; Eva, Kevin W

2009-05-01

To investigate the reliability and feasibility of six potential workplace-based assessment methods in general practice training: criterion audit, multi-source feedback from clinical and non-clinical colleagues, patient feedback (the CARE Measure), referral letters, significant event analysis, and video analysis of consultations. Performance of GP registrars (trainees) was evaluated with each tool to assess the reliabilities of the tools and feasibility, given raters and number of assessments needed. Participant experience of process determined by questionnaire. 171 GP registrars and their trainers, drawn from nine deaneries (representing all four countries in the UK), participated. The ability of each tool to differentiate between doctors (reliability) was assessed using generalisability theory. Decision studies were then conducted to determine the number of observations required to achieve an acceptably high reliability for "high-stakes assessment" using each instrument. Finally, descriptive statistics were used to summarise participants' ratings of their experience using these tools. Multi-source feedback from colleagues and patient feedback on consultations emerged as the two methods most likely to offer a reliable and feasible opinion of workplace performance. Reliability co-efficients of 0.8 were attainable with 41 CARE Measure patient questionnaires and six clinical and/or five non-clinical colleagues per doctor when assessed on two occasions. For the other four methods tested, 10 or more assessors were required per doctor in order to achieve a reliable assessment, making the feasibility of their use in high-stakes assessment extremely low. Participant feedback did not raise any major concerns regarding the acceptability, feasibility, or educational impact of the tools. The combination of patient and colleague views of doctors' performance, coupled with reliable competence measures, may offer a suitable evidence-base on which to monitor progress and completion of doctors' training in general practice.

The reliability of the Glasgow Coma Scale: a systematic review.

PubMed

Reith, Florence C M; Van den Brande, Ruben; Synnot, Anneliese; Gruen, Russell; Maas, Andrew I R

2016-01-01

The Glasgow Coma Scale (GCS) provides a structured method for assessment of the level of consciousness. Its derived sum score is applied in research and adopted in intensive care unit scoring systems. Controversy exists on the reliability of the GCS. The aim of this systematic review was to summarize evidence on the reliability of the GCS. A literature search was undertaken in MEDLINE, EMBASE and CINAHL. Observational studies that assessed the reliability of the GCS, expressed by a statistical measure, were included. Methodological quality was evaluated with the consensus-based standards for the selection of health measurement instruments checklist and its influence on results considered. Reliability estimates were synthesized narratively. We identified 52 relevant studies that showed significant heterogeneity in the type of reliability estimates used, patients studied, setting and characteristics of observers. Methodological quality was good (n = 7), fair (n = 18) or poor (n = 27). In good quality studies, kappa values were ≥0.6 in 85%, and all intraclass correlation coefficients indicated excellent reliability. Poor quality studies showed lower reliability estimates. Reliability for the GCS components was higher than for the sum score. Factors that may influence reliability include education and training, the level of consciousness and type of stimuli used. Only 13% of studies were of good quality and inconsistency in reported reliability estimates was found. Although the reliability was adequate in good quality studies, further improvement is desirable. From a methodological perspective, the quality of reliability studies needs to be improved. From a clinical perspective, a renewed focus on training/education and standardization of assessment is required.

Explosive Bolt Dual-Initiated from One Side

NASA Technical Reports Server (NTRS)

Snow, Eric

2011-01-01

An explosive bolt has been developed that has a one-sided dual initiation train all the way down to the pyro charge for high reliability, while still allowing the other side of the bolt to remain in place after actuation to act as a thermal seal in an extremely high-temperature environment. This lightweight separation device separates at a single fracture plane, and has as much redundancy/reliability as possible. The initiation train comes into the explosive bolt from one side.

Temporal Correlations and Neural Spike Train Entropy

NASA Astrophysics Data System (ADS)

Schultz, Simon R.; Panzeri, Stefano

2001-06-01

Sampling considerations limit the experimental conditions under which information theoretic analyses of neurophysiological data yield reliable results. We develop a procedure for computing the full temporal entropy and information of ensembles of neural spike trains, which performs reliably for limited samples of data. This approach also yields insight to the role of correlations between spikes in temporal coding mechanisms. The method, when applied to recordings from complex cells of the monkey primary visual cortex, results in lower rms error information estimates in comparison to a ``brute force'' approach.

TCOPPE school environmental audit tool: assessing safety and walkability of school environments.

PubMed

Lee, Chanam; Kim, Hyung Jin; Dowdy, Diane M; Hoelscher, Deanna M; Ory, Marcia G

2013-09-01

Several environmental audit instruments have been developed for assessing streets, parks and trails, but none for schools. This paper introduces a school audit tool that includes 3 subcomponents: 1) street audit, 2) school site audit, and 3) map audit. It presents the conceptual basis and the development process of this instrument, and the methods and results of the reliability assessments. Reliability tests were conducted by 2 trained auditors on 12 study schools (high-low income and urban-suburban-rural settings). Kappa statistics (categorical, factual items) and ICC (Likert-scale, perceptual items) were used to assess a) interrater, b) test-retest, and c) peak vs. off-peak hour reliability tests. For the interrater reliability test, the average Kappa was 0.839 and the ICC was 0.602. For the test-retest reliability, the average Kappa was 0.903 and the ICC was 0.774. The peak-off peak reliability was 0.801. Rural schools showed the most consistent results in the peak-off peak and test-retest assessments. For interrater tests, urban schools showed the highest ICC, and rural schools showed the highest Kappa. Most items achieved moderate to high levels of reliabilities in all study schools. With proper training, this audit can be used to assess school environments reliably for research, outreach, and policy-support purposes.

Influence of previous experience on resistance training on reliability of one-repetition maximum test.

PubMed

Ritti-Dias, Raphael Mendes; Avelar, Ademar; Salvador, Emanuel Péricles; Cyrino, Edilson Serpeloni

2011-05-01

The 1-repetition maximum test (1RM) has been widely used to assess maximal strength. However, to improve accuracy in assessing maximal strength, several sessions of the 1RM test are recommended. The aim of this study was to analyze the influence of previous resistance training experience on the reliability of 1RM test. Thirty men were assigned to the following 2 groups according to their previous resistance training experience: no previous resistance training experience (NOEXP) and more than 24 months of resistance training experience (EXP). All subjects performed the 1RM tests in bench press and squat in 4 sessions on distinct days. There was a significant session × group effect in bench press (F = 3.09; p < 0.03) and squat (F = 2.76; p < 0.05) showing that only the NOEXP increased maximal strength between the sessions. Significant increases (p < 0.05) in maximal strength occurred in the NOEXP between session 1 and the other sessions in bench press (session 1 vs. 2 = +3.8%; session 1 vs. 3 = +7.4%; session 1 vs. 4 = +10.1%), and squat (session 1 vs. 2 = +7.6%; session 1 vs. 3 = +10.1%; session 1 vs. 4 = +11.2%). Moreover, in bench press, maximal strength in sessions 3 and 4 were significantly higher than in session 2. The results of the present study suggest that the reliability of the 1RM test is influenced by the subject's previous experience in resistance training. Subjects without experience in resistance training require more practice and familiarization and show greater increases in maximal strength between sessions than subjects with previous experience in resistance training.

Assuring reliability program effectiveness.

NASA Technical Reports Server (NTRS)

Ball, L. W.

1973-01-01

An attempt is made to provide simple identification and description of techniques that have proved to be most useful either in developing a new product or in improving reliability of an established product. The first reliability task is obtaining and organizing parts failure rate data. Other tasks are parts screening, tabulation of general failure rates, preventive maintenance, prediction of new product reliability, and statistical demonstration of achieved reliability. Five principal tasks for improving reliability involve the physics of failure research, derating of internal stresses, control of external stresses, functional redundancy, and failure effects control. A final task is the training and motivation of reliability specialist engineers.

The cardiovascular in-training examination: development, implementation, results, and future directions.

PubMed

Kuvin, Jeffrey T; Soto, Amanda; Foster, Lauren; Dent, John; Kates, Andrew M; Polk, Donna M; Rosenzweig, Barry; Indik, Julia

2015-03-31

The American College of Cardiology (ACC), in collaboration with the National Board of Medical Examiners (NBME), developed the first standardized in-training examination (ITE) for cardiovascular disease fellows-in-training (FITs). In addition to testing knowledge, this examination uses the newly developed ACC Curricular Milestones to provide specific, competency-based feedback to program directors and FITs. The ACC ITE has been administered more than 5,000 times since 2011. This analysis sought to report the initial experience with the ITE, including feasibility and reliability of test development and implementation, as well as the ability of this process to provide useful feedback in key content areas. The annual ACC ITE has been available to cardiovascular disease fellowship programs in the United States since 2011. Questions for this Web-based, secure, multiple-choice examination were developed by a group of cardiovascular disease specialists and each question was analyzed by the NBME to ensure quality. Scores were equated and standardized to allow for comparability. Trainees and program directors were provided detailed feedback, including a list of the curricular competencies tested by those questions answered incorrectly. The ITE was administered 5,118 times. In 2013, the examination was taken by 1,969 fellows, representing 194 training programs. Among the 3 training years, there was consistency in the examination scores. Total test scores and scores within each of the content areas increased with each FIT year (there was a statistically significant difference in each cohort's average scale score across administration years). There was also significant improvement in examination scores across the fellowship years. The ACC ITE is a powerful tool available to all training programs to assess medical knowledge. This examination also delivers robust and timely feedback addressing individual knowledge gaps, and thus, may serve as a basis for improving training curricula. Copyright © 2015 American College of Cardiology Foundation. Published by Elsevier Inc. All rights reserved.

Comprehensive proficiency-based inanimate training for robotic surgery: reliability, feasibility, and educational benefit.

PubMed

Arain, Nabeel A; Dulan, Genevieve; Hogg, Deborah C; Rege, Robert V; Powers, Cathryn E; Tesfay, Seifu T; Hynan, Linda S; Scott, Daniel J

2012-10-01

We previously developed a comprehensive proficiency-based robotic training curriculum demonstrating construct, content, and face validity. This study aimed to assess reliability, feasibility, and educational benefit associated with curricular implementation. Over an 11-month period, 55 residents, fellows, and faculty (robotic novices) from general surgery, urology, and gynecology were enrolled in a 2-month curriculum: online didactics, half-day hands-on tutorial, and self-practice using nine inanimate exercises. Each trainee completed a questionnaire and performed a single proctored repetition of each task before (pretest) and after (post-test) training. Tasks were scored for time and errors using modified FLS metrics. For inter-rater reliability (IRR), three trainees were scored by two raters and analyzed using intraclass correlation coefficients (ICC). Data from eight experts were analyzed using ICC and Cronbach's α to determine test-retest reliability and internal consistency, respectively. Educational benefit was assessed by comparing baseline (pretest) and final (post-test) trainee performance; comparisons used Wilcoxon signed-rank test. Of the 55 trainees that pretested, 53 (96 %) completed all curricular components in 9-17 h and reached proficiency after completing an average of 72 ± 28 repetitions over 5 ± 1 h. Trainees indicated minimal prior robotic experience and "poor comfort" with robotic skills at baseline (1.8 ± 0.9) compared to final testing (3.1 ± 0.8, p < 0.001). IRR data for the composite score revealed an ICC of 0.96 (p < 0.001). Test-retest reliability was 0.91 (p < 0.001) and internal consistency was 0.81. Performance improved significantly after training for all nine tasks and according to composite scores (548 ± 176 vs. 914 ± 81, p < 0.001), demonstrating educational benefit. This curriculum is associated with high reliability measures, demonstrated feasibility for a large cohort of trainees, and yielded significant educational benefit. Further studies and adoption of this curriculum are encouraged.

INTRA-RATER RELIABILITY OF THE MULTIPLE SINGLE-LEG HOP-STABILIZATION TEST AND RELATIONSHIPS WITH AGE, LEG DOMINANCE AND TRAINING.

PubMed

Sawle, Leanne; Freeman, Jennifer; Marsden, Jonathan

2017-04-01

Balance is a complex construct, affected by multiple components such as strength and co-ordination. However, whilst assessing an athlete's dynamic balance is an important part of clinical examination, there is no gold standard measure. The multiple single-leg hop-stabilization test is a functional test which may offer a method of evaluating the dynamic attributes of balance, but it needs to show adequate intra-tester reliability. The purpose of this study was to assess the intra-rater reliability of a dynamic balance test, the multiple single-leg hop-stabilization test on the dominant and non-dominant legs. Intra-rater reliability study. Fifteen active participants were tested twice with a 10-minute break between tests. The outcome measure was the multiple single-leg hop-stabilization test score, based on a clinically assessed numerical scoring system. Results were analysed using an Intraclass Correlations Coefficient (ICC 2,1 ) and Bland-Altman plots. Regression analyses explored relationships between test scores, leg dominance, age and training (an alpha level of p = 0.05 was selected). ICCs for intra-rater reliability were 0.85 for the dominant and non-dominant legs (confidence intervals = 0.62-0.95 and 0.61-0.95 respectively). Bland-Altman plots showed scores within two standard deviations. A significant correlation was observed between the dominant and non-dominant leg on balance scores (R 2 =0.49, p<0.05), and better balance was associated with younger participants in their non-dominant leg (R 2 =0.28, p<0.05) and their dominant leg (R 2 =0.39, p<0.05), and a higher number of hours spent training for the non-dominant leg R 2 =0.37, p<0.05). The multiple single-leg hop-stabilisation test demonstrated strong intra-tester reliability with active participants. Younger participants who trained more, have better balance scores. This test may be a useful measure for evaluating the dynamic attributes of balance. 3.

The Shoulder Objective Practical Assessment Tool: Evaluation of a New Tool Assessing Residents Learning in Diagnostic Shoulder Arthroscopy.

PubMed

Talbot, Christopher L; Holt, Edward M; Gooding, Benjamin W T; Tennent, Thomas D; Foden, Philip

2015-08-01

To design and validate an objective practical assessment tool for diagnostic shoulder arthroscopy that would provide residents with a method to evaluate their progression in this field of surgery and to identify specific learning needs. We designed and evaluated the shoulder Objective Practical Assessment Tool (OPAT). The shoulder OPAT was designed by us, and scoring domains were created using a Delphi process. The shoulder OPAT was trialed by members of the British Elbow & Shoulder Society Education Committee for internal consistency and ease of use before being offered to other trainers and residents. Inter-rater reliability and intrarater reliability were calculated. One hundred forty orthopaedic residents, of varying seniority, within 5 training regions in the United Kingdom, were questioned regarding the tool. A pilot study of 6 residents was undertaken. Internal consistency was 0.77 (standardized Cronbach α). Inter-rater reliability was 0.60, and intrarater reliability was 0.82. The Spearman correlation coefficient (r) between the global summary score for the shoulder OPAT and the current assessment tool used in postgraduate training for orthopaedic residents undertaking diagnostic shoulder arthroscopy equaled 0.74. Of the residents, 82% agreed or strongly agreed when asked if the shoulder OPAT would be a useful tool in monitoring progression and 72% agreed or strongly agreed with the introduction of the shoulder OPAT within the orthopaedic domain. This study shows that the shoulder OPAT fulfills several aspects of reliability and validity when tested. Despite the inter-rater reliability being 0.60, we believe that the shoulder OPAT has the potential to play a role alongside the current assessment tool in the training of orthopaedic residents. The shoulder OPAT can be used to assess residents during shoulder arthroscopy and has the potential for use in medical education, as well as arthroscopic skills training in the operating theater. Copyright © 2015 Arthroscopy Association of North America. Published by Elsevier Inc. All rights reserved.

Ergonomics Calibration Training Utilizing Photography for Dental Hygiene Faculty Members.

PubMed

Partido, Brian B

2017-10-01

Dental and dental hygiene clinical faculty members often do not provide consistent instruction, especially since most procedures involve clinical judgment. Although instructional variations frequently translate into variations in student performance, the effect of inconsistent instruction is unknown, especially related to ergonomics. The aim of this study was to determine whether photography-assisted calibration training would improve interrater reliability among dental hygiene faculty members in ergonomics evaluation. The photography-assisted ergonomics calibration program incorporated features to improve accessibility and optimize the quality of the training. The study used a two-group repeated measures design with a convenience sample of 11 dental hygiene faculty members (eight full-time and three part-time) during the autumn 2016 term at one U.S. dental school. At weeks one and seven, all participants evaluated imaged postures of five dental students using a modified-dental operator posture assessment instrument. During weeks three and five, training group participants completed calibration training using independent and group review of imaged postures. All pre-training and post-training evaluations were evaluated for interrater reliability. Two-way random effects intraclass coefficient (ICC) values were calculated to measure the effects of the training on interrater reliability. The average measure of ICC of the training group improved from 0.694 with a 95% confidence interval (CI) of 0.001 to 0.965 (F(4,8)=3.465, p>0.05) to 0.766 with a 95% CI of 0.098 to 0.972 (F(4,8)=7.913, p<0.01). The average measure of ICC of the control group improved from 0.821 with a 95% CI of 0.480 to 0.978 (F(4,28)=7.702, p<0.01) to 0.846 with a 95% CI of 0.542 to 0.981 (F(4,28)=8.561, p<0.01). These results showed that the photography-assisted calibration training with the opportunity to reconcile different opinions resulted in improved agreement among these faculty members.

Tapering Practices of Strongman Athletes: Test-Retest Reliability Study

PubMed Central

Pritchard, Hayden J; Keogh, Justin WL

2017-01-01

Background Little is currently known about the tapering practices of strongman athletes. We have developed an Internet-based comprehensive self-report questionnaire examining the training and tapering practices of strongman athletes. Objective The objective of this study was to document the test-retest reliability of questions associated with the Internet-based comprehensive self-report questionnaire on the tapering practices of strongman athletes. The information will provide insight on the reliability and usefulness of the online questionnaire for use with strongman athletes. Methods Invitations to complete an Internet questionnaire were sent via Facebook Messenger to identified strongman athletes. The survey consisted of four main areas of inquiry, including demographics and background information, training practices, tapering, and tapering practices. Of the 454 athletes that completed the survey over the 8-week period, 130 athletes responded on Facebook Messenger indicating that they intended to complete, or had completed, the survey. These participants were asked if they could complete the online questionnaire a second time for a test-retest reliability analysis. Sixty-four athletes (mean age 33.3 years, standard deviation [SD] 7.7; mean height 178.2 cm, SD 11.0; mean body mass 103.7 kg, SD 24.8) accepted this invitation and completed the survey for the second time after a minimum 7-day period from the date of their first completion. Agreement between athlete responses was measured using intraclass correlation coefficients (ICCs) and kappa statistics. Confidence intervals (at 95%) were reported for all measures and significance was set at P<.05. Results Test-retest reliability for demographic and training practices items were significant (P<.001) and showed excellent (ICC range=.84 to .98) and fair to almost perfect agreement (κ range=.37-.85). Moderate to excellent agreements (ICC range=.56-.84; P<.01) were observed for all tapering practice measures except for the number of days athletes started their usual taper before a strongman competition (ICC=.30). When the number of days were categorized with additional analyses, moderate reliability was observed (κ=.43; P<.001). Fair to substantial agreement was observed for the majority of tapering practices measures (κrange=.38-.73; P<.001) except for how training frequency (κ=.26) and the percentage and type of resistance training performed, which changed in the taper (κ=.20). Good to excellent agreement (ICC=.62-.93; P<.05) was observed for items relating to strongman events and traditional exercises performed during the taper. Only the time at which the Farmer’s Walk was last performed before competition showed poor reliability (ICC=.27). Conclusions We have developed a low cost, self-reported, online retrospective questionnaire, which provided stable and reliable answers for most of the demographic, training, and tapering practice questions. The results of this study support the inferences drawn from the Tapering Practices of Strongman Athletes Study. PMID:29089292

Perceptions of Support Networks During the Graduate-Assistant Athletic Trainer Experience.

PubMed

Mazerolle, Stephanie M; Clines, Stephanie; Eason, Christianne M; Pitney, William A

2015-12-01

The graduate-assistant position can be a highly influential experience because it is often the first time novice athletic trainers (ATs) are practicing autonomously. To gain an understanding of how graduate-assistant ATs (GAATs) perceive professional socialization and mentorship during their assistantships. Semistructured phone interviews. Graduate-assistant ATs in various clinical settings. Twenty-five GAATs (20 women, 5 men) studying in 1 of 3 academic tracks (postprofessional athletic training = 8, athletic training-based curriculum = 11, non-athletic training-based curriculum = 6). Median age was 24 years. Phone interviews were digitally recorded and transcribed verbatim. Data analysis used principles of the general inductive approach. Credibility was maintained using peer review, field notes, and intercoder reliability. Participants identified peer support throughout their experiences, in both academic and clinical settings. The GAATs frequently relied on other GAATs for support due to shared experiences and understanding of workloads. Participants described difficulty receiving supervisor support from full-time staff ATs due to the supervisors' workload and time constraints, limiting their availability for mentoring. Guidance from academic support personnel occurred only in athletic training-centered academic programs. Communication emerged as helpful for incoming GAATs; the previous GAATs provided formal mentorship via job descriptions highlighting role responsibilities and expectations. Differences between assistantship types were noted only in terms of receiving balanced mentorship between the academic and clinical staffs, such that students studying in postprofessional athletic training programs perceived more balanced support. Our results confirm the literature regarding the GAAT's pursuit of continued formal mentoring. The GAATs perceived less support from full-time AT staff members due to limited availability. Therefore, GAATs leaned on their peers for support during the graduate experience.

Perceptions of Support Networks During the Graduate-Assistant Athletic Trainer Experience

PubMed Central

Mazerolle, Stephanie M.; Clines, Stephanie; Eason, Christianne M.; Pitney, William A.

2015-01-01

Context  The graduate-assistant position can be a highly influential experience because it is often the first time novice athletic trainers (ATs) are practicing autonomously. Objective  To gain an understanding of how graduate-assistant ATs (GAATs) perceive professional socialization and mentorship during their assistantships. Design  Semistructured phone interviews. Setting  Graduate-assistant ATs in various clinical settings. Patients or Other Participants  Twenty-five GAATs (20 women, 5 men) studying in 1 of 3 academic tracks (postprofessional athletic training = 8, athletic training-based curriculum = 11, non–athletic training-based curriculum = 6). Median age was 24 years. Data Collection and Analysis  Phone interviews were digitally recorded and transcribed verbatim. Data analysis used principles of the general inductive approach. Credibility was maintained using peer review, field notes, and intercoder reliability. Results  Participants identified peer support throughout their experiences, in both academic and clinical settings. The GAATs frequently relied on other GAATs for support due to shared experiences and understanding of workloads. Participants described difficulty receiving supervisor support from full-time staff ATs due to the supervisors' workload and time constraints, limiting their availability for mentoring. Guidance from academic support personnel occurred only in athletic training-centered academic programs. Communication emerged as helpful for incoming GAATs; the previous GAATs provided formal mentorship via job descriptions highlighting role responsibilities and expectations. Differences between assistantship types were noted only in terms of receiving balanced mentorship between the academic and clinical staffs, such that students studying in postprofessional athletic training programs perceived more balanced support. Conclusions  Our results confirm the literature regarding the GAAT's pursuit of continued formal mentoring. The GAATs perceived less support from full-time AT staff members due to limited availability. Therefore, GAATs leaned on their peers for support during the graduate experience. PMID:26565423

Convergence among Data Sources, Response Bias, and Reliability and Validity of a Structured Job Analysis Questionnaire.

ERIC Educational Resources Information Center

Smith, Jack E.; Hakel, Milton D.

1979-01-01

Examined are questions pertinent to the use of the Position Analysis Questionnaire: Who can use the PAQ reliably and validly? Must one rely on trained job analysts? Can people having no direct contact with the job use the PAQ reliably and validly? Do response biases influence PAQ responses? (Author/KC)

Confirmatory Factor Analysis of the System for Evaluation of Teaching Qualities (SETQ) in Graduate Medical Training.

PubMed

Boerebach, Benjamin C M; Lombarts, Kiki M J M H; Arah, Onyebuchi A

2016-03-01

The System for Evaluation of Teaching Qualities (SETQ) was developed as a formative system for the continuous evaluation and development of physicians' teaching performance in graduate medical training. It has been seven years since the introduction and initial exploratory psychometric analysis of the SETQ questionnaires. This study investigates the validity and reliability of the SETQ questionnaires across hospitals and medical specialties using confirmatory factor analyses (CFAs), reliability analysis, and generalizability analysis. The SETQ questionnaires were tested in a sample of 3,025 physicians and 2,848 trainees in 46 hospitals. The CFA revealed acceptable fit of the data to the previously identified five-factor model. The high internal consistency estimates suggest satisfactory reliability of the subscales. These results provide robust evidence for the validity and reliability of the SETQ questionnaires for evaluating physicians' teaching performance. © The Author(s) 2014.

Is the Scale for Measuring Motivational Interviewing Skills a valid and reliable instrument for measuring the primary care professionals motivational skills?: EVEM study protocol

PubMed Central

2012-01-01

Background Lifestyle is one of the main determinants of people’s health. It is essential to find the most effective prevention strategies to be used to encourage behavioral changes in their patients. Many theories are available that explain change or adherence to specific health behaviors in subjects. In this sense the named Motivational Interviewing has increasingly gained relevance. Few well-validated instruments are available for measuring doctors’ communication skills, and more specifically the Motivational Interviewing. Methods/Design The hypothesis of this study is that the Scale for Measuring Motivational Interviewing Skills (EVEM questionnaire) is a valid and reliable instrument for measuring the primary care professionals skills to get behavior change in patients. To test the hypothesis we have designed a prospective, observational, multi-center study to validate a measuring instrument. –Scope: Thirty-two primary care centers in Spain. -Sampling and Size: a) face and consensual validity: A group composed of 15 experts in Motivational Interviewing. b) Assessment of the psychometric properties of the scale; 50 physician- patient encounters will be videoed; a total of 162 interviews will be conducted with six standardized patients, and another 200 interviews will be conducted with 50 real patients (n=362). Four physicians will be specially trained to assess 30 interviews randomly selected to test the scale reproducibility. -Measurements for to test the hypothesis: a) Face validity: development of a draft questionnaire based on a theoretical model, by using Delphi-type methodology with experts. b) Scale psychometric properties: intraobservers will evaluate video recorded interviews: content-scalability validity (Exploratory Factor Analysis), internal consistency (Cronbach alpha), intra-/inter-observer reliability (Kappa index, intraclass correlation coefficient, Bland & Altman methodology), generalizability, construct validity and sensitivity to change (Pearson product–moment correlation coefficient). Discussion The verification of the hypothesis that EVEM is a valid and reliable tool for assessing motivational interviewing would be a major breakthrough in the current theoretical and practical knowledge, as it could be used to assess if the providers put into practice a patient centered communication style and can be used both for training or researching purposes. Trials registration Dislip-EM study NCT01282190 (ClinicalTrials.gov) PMID:23173902

The validity and reliability of script concordance test in otolaryngology residency training.

PubMed

Iravani, Kamyar; Amini, Mitra; Doostkam, Aida; Dehbozorgian, Mahnaz

2016-04-01

The script concordance test (SCT) is one the best tools used to evaluate clinical reasoning in ill-defined clinical situations. The aim of this study was to demonstrate SCT application in otolaryngology residency training. A 20 item otolaryngology SCT containing 60 questions was administered to 26 otolaryngology residents. The test was prepared by two otolaryngologists familiar to medical education. These questions have been validated by otolaryngology experts. The panel consisted of 9 academic staff in the field of otolaryngology. Pearson correlation test was used to assess the reliability of the test. The obtained mean scores were 68.4±5.8 (out of 100) for residents and 78.2±6.4(out of 100) for experts. There was a significant difference between the two scores (p<0.005). Cronbach's alpha value was 0.80. The SCT is a reliable tool to evaluate clinical reasoning in otolaryngology residents. It should be included in otolaryngology residency training.

Predicting wettability behavior of fluorosilica coated metal surface using optimum neural network

NASA Astrophysics Data System (ADS)

Taghipour-Gorjikolaie, Mehran; Valipour Motlagh, Naser

2018-02-01

The interaction between variables, which are effective on the surface wettability, is very complex to predict the contact angles and sliding angles of liquid drops. In this paper, in order to solve this complexity, artificial neural network was used to develop reliable models for predicting the angles of liquid drops. Experimental data are divided into training data and testing data. By using training data and feed forward structure for the neural network and using particle swarm optimization for training the neural network based models, the optimum models were developed. The obtained results showed that regression index for the proposed models for the contact angles and sliding angles are 0.9874 and 0.9920, respectively. As it can be seen, these values are close to unit and it means the reliable performance of the models. Also, it can be inferred from the results that the proposed model have more reliable performance than multi-layer perceptron and radial basis function based models.

Performing the unexplainable: Implicit task performance reveals individually reliable sequence learning without explicit knowledge

PubMed Central

Sanchez, Daniel J.; Gobel, Eric W.; Reber, Paul J.

2015-01-01

Memory-impaired patients express intact implicit perceptual–motor sequence learning, but it has been difficult to obtain a similarly clear dissociation in healthy participants. When explicit memory is intact, participants acquire some explicit knowledge and performance improvements from implicit learning may be subtle. Therefore, it is difficult to determine whether performance exceeds what could be expected on the basis of the concomitant explicit knowledge. Using a challenging new sequence-learning task, robust implicit learning was found in healthy participants with virtually no associated explicit knowledge. Participants trained on a repeating sequence that was selected randomly from a set of five. On a performance test of all five sequences, performance was best on the trained sequence, and two-thirds of the participants exhibited individually reliable improvement (by chi-square analysis). Participants could not reliably indicate which sequence had been trained by either recognition or recall. Only by expressing their knowledge via performance were participants able to indicate which sequence they had learned. PMID:21169570

Reliability of performance velocity for jump squats under feedback and nonfeedback conditions.

PubMed

Randell, Aaron D; Cronin, John B; Keogh, Justin Wl; Gill, Nicholas D; Pedersen, Murray C

2011-12-01

Randell, AD, Cronin, JB, Keogh, JWL, Gill, ND, and Pedersen, MC. Reliability of performance velocity for jump squats under feedback and nonfeedback conditions. J Strength Cond Res 25(12): 3514-3518, 2011-Advancements in the monitoring of kinematic and kinetic variables during resistance training have resulted in the ability to continuously monitor performance and provide feedback during training. If equipment and software can provide reliable instantaneous feedback related to the variable of interest during training, it is thought that this may result in goal-oriented movement tasks that increase the likelihood of transference to on-field performance or at the very least improve the mechanical variable of interest. The purpose of this study was to determine the reliability of performance velocity for jump squats under feedback and nonfeedback conditions over 3 consecutive training sessions. Twenty subjects were randomly allocated to a feedback or nonfeedback group, and each group performed a total of 3 "jump squat" training sessions with the velocity of each repetition measured using a linear position transducer. There was less change in mean velocities between sessions 1-2 and sessions 2-3 (0.07 and 0.02 vs. 0.13 and -0.04 m·s), less random variation (TE = 0.06 and 0.06 vs. 0.10 and 0.07 m·s) and greater consistency (intraclass correlation coefficient = 0.83 and 0.87 vs. 0.53 and 0.74) between sessions for the feedback condition as compared to the nonfeedback condition. It was concluded that there is approximately a 50-50 probability that the provision of feedback was beneficial to the performance in the squat jump over multiple sessions. It is suggested that this has the potential for increasing transference to on-field performance or at the very least improving the mechanical variable of interest.

Transfer in motion discrimination learning was no greater in double training than in single training.

PubMed

Huang, Jinfeng; Liang, Ju; Zhou, Yifeng; Liu, Zili

2017-06-01

We investigated the controversy regarding double training in motion discrimination learning. We collected data from 43 participants in a motion direction discrimination learning task with either double training (i.e., training plus exposure) or single training (i.e., no exposure). By pooling these data with those in the literature, we had data in double training from 28 participants and in single training from 36 participants. We found that, in double training, the transfer along the exposed direction was less than that along the trained direction, indicating incomplete transfer. Importantly, the transfer in double training was not reliably greater than that in single training.

The relationship between workload and training - An introduction

NASA Technical Reports Server (NTRS)

Hart, Sandra G.

1986-01-01

This paper reviews the relationships among workload, performance, and training. Its goal is to introduce the concepts of workload and training and to suggest how they may be related. It suggests some of the practical and theoretical benefits to be derived from their joint consideration. Training effectiveness can be improved by monitoring trainee workload and the reliability of workload predictions, and measures can be improved by identifying and controlling the training levels of experimental subjects.

Bootstrap study of genome-enabled prediction reliabilities using haplotype blocks across Nordic Red cattle breeds.

PubMed

Cuyabano, B C D; Su, G; Rosa, G J M; Lund, M S; Gianola, D

2015-10-01

This study compared the accuracy of genome-enabled prediction models using individual single nucleotide polymorphisms (SNP) or haplotype blocks as covariates when using either a single breed or a combined population of Nordic Red cattle. The main objective was to compare predictions of breeding values of complex traits using a combined training population with haplotype blocks, with predictions using a single breed as training population and individual SNP as predictors. To compare the prediction reliabilities, bootstrap samples were taken from the test data set. With the bootstrapped samples of prediction reliabilities, we built and graphed confidence ellipses to allow comparisons. Finally, measures of statistical distances were used to calculate the gain in predictive ability. Our analyses are innovative in the context of assessment of predictive models, allowing a better understanding of prediction reliabilities and providing a statistical basis to effectively calibrate whether one prediction scenario is indeed more accurate than another. An ANOVA indicated that use of haplotype blocks produced significant gains mainly when Bayesian mixture models were used but not when Bayesian BLUP was fitted to the data. Furthermore, when haplotype blocks were used to train prediction models in a combined Nordic Red cattle population, we obtained up to a statistically significant 5.5% average gain in prediction accuracy, over predictions using individual SNP and training the model with a single breed. Copyright © 2015 American Dairy Science Association. Published by Elsevier Inc. All rights reserved.

Combining satellite imagery and machine learning to predict poverty.

PubMed

Jean, Neal; Burke, Marshall; Xie, Michael; Davis, W Matthew; Lobell, David B; Ermon, Stefano

2016-08-19

Reliable data on economic livelihoods remain scarce in the developing world, hampering efforts to study these outcomes and to design policies that improve them. Here we demonstrate an accurate, inexpensive, and scalable method for estimating consumption expenditure and asset wealth from high-resolution satellite imagery. Using survey and satellite data from five African countries--Nigeria, Tanzania, Uganda, Malawi, and Rwanda--we show how a convolutional neural network can be trained to identify image features that can explain up to 75% of the variation in local-level economic outcomes. Our method, which requires only publicly available data, could transform efforts to track and target poverty in developing countries. It also demonstrates how powerful machine learning techniques can be applied in a setting with limited training data, suggesting broad potential application across many scientific domains. Copyright © 2016, American Association for the Advancement of Science.

Parasitology: United Kingdom National Quality Assessment Scheme.

PubMed Central

Hawthorne, M.; Chiodini, P. L.; Snell, J. J.; Moody, A. H.; Ramsay, A.

1992-01-01

AIMS: To assess the results from parasitology laboratories taking part in a quality assessment scheme between 1986 and 1991; and to compare performance with repeat specimens. METHODS: Quality assessment of blood parasitology, including tissue parasites (n = 444; 358 UK, 86 overseas), and faecal parasitology, including extra-intestinal parasites (n = 205; 141 UK, 64 overseas), was performed. RESULTS: Overall, the standard of performance was poor. A questionnaire distributed to participants showed that a wide range of methods was used, some of which were considered inadequate to achieve reliable results. Teaching material was distributed to participants from time to time in an attempt to improve standards. CONCLUSIONS: Since the closure of the IMLS fellowship course in 1972, fewer opportunities for specialised training in parasitology are available: more training is needed. Poor performance in the detection of malarial parasites is mainly attributable to incorrect speciation, misidentification, and lack of equipment such as an eyepiece graticule. PMID:1452791

Working Memory Training Does Not Improve Performance on Measures of Intelligence or Other Measures of “Far Transfer”

PubMed Central

Melby-Lervåg, Monica; Redick, Thomas S.; Hulme, Charles

2016-01-01

It has been claimed that working memory training programs produce diverse beneficial effects. This article presents a meta-analysis of working memory training studies (with a pretest-posttest design and a control group) that have examined transfer to other measures (nonverbal ability, verbal ability, word decoding, reading comprehension, or arithmetic; 87 publications with 145 experimental comparisons). Immediately following training there were reliable improvements on measures of intermediate transfer (verbal and visuospatial working memory). For measures of far transfer (nonverbal ability, verbal ability, word decoding, reading comprehension, arithmetic) there was no convincing evidence of any reliable improvements when working memory training was compared with a treated control condition. Furthermore, mediation analyses indicated that across studies, the degree of improvement on working memory measures was not related to the magnitude of far-transfer effects found. Finally, analysis of publication bias shows that there is no evidential value from the studies of working memory training using treated controls. The authors conclude that working memory training programs appear to produce short-term, specific training effects that do not generalize to measures of “real-world” cognitive skills. These results seriously question the practical and theoretical importance of current computerized working memory programs as methods of training working memory skills. PMID:27474138

Positive reinforcement training for a trunk wash in Nepal's working elephants: demonstrating alternatives to traditional elephant training techniques.

PubMed

Fagen, Ariel; Acharya, Narayan; Kaufman, Gretchen E

2014-01-01

Many trainers of animals in the zoo now rely on positive reinforcement training to teach animals to voluntarily participate in husbandry and veterinary procedures in an effort to improve behavioral reliability, captive management, and welfare. However, captive elephant handlers in Nepal still rely heavily on punishment- and aversion-based methods. The aim of this project was to determine the effectiveness of secondary positive reinforcement (SPR) in training free-contact elephants in Nepal to voluntarily participate in a trunk wash for the purpose of tuberculosis testing. Five female elephants, 4 juveniles and 1 adult, were enrolled in the project. Data were collected in the form of minutes of training, number of offers made for each training task, and success rate for each task in performance tests. Four out of 5 elephants, all juveniles, successfully learned the trunk wash in 35 sessions or fewer, with each session lasting a mean duration of 12 min. The elephants' performance improved from a mean success rate of 39.0% to 89.3% during the course of the training. This study proves that it is feasible to efficiently train juvenile, free-contact, traditionally trained elephants in Nepal to voluntarily and reliably participate in a trunk wash using only SPR techniques.

Positive Reinforcement Training for a Trunk Wash in Nepal's Working Elephants: Demonstrating Alternatives to Traditional Elephant Training Techniques

PubMed Central

Fagen, Ariel; Acharya, Narayan; Kaufman, Gretchen E.

2016-01-01

Many trainers of animals in the zoo now rely on positive reinforcement training to teach animals to voluntarily participate in husbandry and veterinary procedures in an effort to improve behavioral reliability, captive management, and welfare. However, captive elephant handlers in Nepal still rely heavily on punishment- and aversion-based methods. The aim of this project was to determine the effectiveness of secondary positive reinforcement (SPR) in training free-contact elephants in Nepal to voluntarily participate in a trunk wash for the purpose of tuberculosis testing. Five female elephants, 4 juveniles and 1 adult, were enrolled in the project. Data were collected in the form of minutes of training, number of offers made for each training task, and success rate for each task in performance tests. Four out of 5 elephants, all juveniles, successfully learned the trunk wash in 35 sessions or fewer, with each session lasting a mean duration of 12 min. The elephants’ performance improved from a mean success rate of 39.0% to 89.3% during the course of the training. This study proves that it is feasible to efficiently train juvenile, free-contact, traditionally trained elephants in Nepal to voluntarily and reliably participate in a trunk wash using only SPR techniques. PMID:24410366

Reliability and responsiveness of the Self-Efficacy in Assessing, Training and Spotting wheelchair skills (SEATS) outcome measure.

PubMed

Rushton, Paula W; Smith, Emma M; Miller, William C; Kirby, R Lee; Daoust, Geneviève

2018-01-31

The aim of this study was to evaluate the internal consistency, test-retest reliability and responsiveness of the Self-Efficacy in Assessing, Training and Spotting manual wheelchair skills (SEATS-M) and Self-Efficacy in Assessing, Training and Spotting power wheelchair skills (SEATS-P). A 2-week test-retest design was used with a convenience sample of occupational and physical therapists who worked at a provincial rehabilitation centre (inpatient and outpatient services). Sixteen participants completed the SEATS-M and 18 participants completed the SEATS-P. For the SEATS-M assessment, training, spotting and documentation sections, Cronbach's alpha coefficients ranged from 0.90 to 0.97, the 2-week intraclass correlation coefficients (ICC 1,1 ) ranged from 0.81 to 0.95, the standard error of measurements (SEM) ranged from 5.06 to 8.70 and the smallest real differences (SRD) ranged from 6.24 to 8.18. For the SEATS-P assessment, training, spotting and documentation sections, Cronbach's alpha coefficients ranged from 0.83 to 0.92, the ICCs ranged from 0.72 to 0.86, the SEMs ranged from 4.54 to 8.91 and the SRDs ranged from 5.90 to 8.27. There is preliminary evidence that both the SEATS-M and the SEATS-P have high internal consistency, good test-retest reliability and support for responsiveness. These tools can be used in evaluating clinician self-efficacy with assessing, training, spotting and documenting wheelchair skills included on the Wheelchair Skills Test. Implications for Rehabilitation There is preliminary evidence that the SEATS-M and SEATS-P are reliable and responsive outcome measures that can be used to evaluate the self-efficacy of clinicians to administer the Wheelchair Skills Program. Measurement of clinicians' self-efficacy in this area of practice may enable an enhanced understanding of the areas in which clinicians lack self-efficacy, thereby informing the development of improved knowledge translation interventions.

Blended Training on Scientific Software: A Study on How Scientific Data Are Generated

ERIC Educational Resources Information Center

Skordaki, Efrosyni-Maria; Bainbridge, Susan

2018-01-01

This paper presents the results of a research study on scientific software training in blended learning environments. The investigation focused on training approaches followed by scientific software users whose goal is the reliable application of such software. A key issue in current literature is the requirement for a theory-substantiated…

Optimization of genomic selection training populations with a genetic algorithm

USDA-ARS?s Scientific Manuscript database

In this article, we derive a computationally efficient statistic to measure the reliability of estimates of genetic breeding values for a fixed set of genotypes based on a given training set of genotypes and phenotypes. We adopt a genetic algorithm scheme to find a training set of certain size from ...

Development of the Internet-Enabled System for Exercise Telerehabilitation and Cardiovascular Training.

PubMed

Dedov, Vadim N; Dedova, Irina V

2015-07-01

Sustained exercise training could significantly improve patient rehabilitation and management of noncommunicable diseases in the community. This study aimed to develop a universal telecare system for delivery of exercise rehabilitation and cardiovascular training services at home. An innovative bilateral leg training device was equipped with an electronic system for the ongoing measurement of training activities with the device. A single-item parameter reflecting the intensity of training was monitored using several modern telecommunication technologies. According to the application protocol, eight volunteers first tried the device for 30-60 min to determine their personal training capacity. Then, they were provided with equipment to use at home for 4 weeks. Adherence to daily training was assessed by the number of training days per week, training intensity, and duration of training sessions. The system provided reliable recording of training activities with the device using (1) long-term data logging without an ongoing connection to the computer, (2) wireless monitoring and recording of training activities on a stand-alone computer, and (3) a secure cloud-based monitoring over the Internet connection using electronic devices, including smartphones. Overall analysis of recordings and phone feedbacks to participants took only approximately 5 h for the duration of study. This study, although of a pilot nature, described the comprehensive exercise telerehabilitation system integrating mobile training equipment with personalized training protocols and remote monitoring. A single-item electronic parameter of the system usage facilitated time-effective data management. Wireless connection allowed various locations of device application and several monitoring arrangements ranging from real-time monitoring to long-term recording of exercise activities. A cloud-based software platform enabled management of multiple users at distance. Implementation of this model may facilitate both accessibility and availability of personalized exercise telerehabilitation services. Further studies would validate it in the clinical and healthcare environment.

High Reliability and the Evaluation of ATC System Configuration by Communizing Resources

NASA Astrophysics Data System (ADS)

Yamamoto, Masanori

Automatic Train Control (ATC) in the railway signalling system is required high safety, high availability, reduction of unit, energy saving and cost reduction. This paper described the resources communization redundancy of the ATC system that shared the redundant units in preparation for common use units in order to accommodate with this issue by keeping safety and availability in the same level of conventional ATC. It was evaluated on N+2 redundant system which established 2 spares for the common use system N piece in transmission division. It was done the safety evaluation of the N+2 redundant system by way of hazard analysis of FTA method and safety issue was confirmed by FMEA. The new redundant system concludes that 19% of downsizing and 36% of the energy saving are surely possible.

Summative assessment of undergraduates' communication competence in challenging doctor-patient encounters. Evaluation of the Düsseldorf CoMeD-OSCE.

PubMed

Mortsiefer, Achim; Immecke, Janine; Rotthoff, Thomas; Karger, André; Schmelzer, Regine; Raski, Bianca; Schmitten, Jürgen In der; Altiner, Attila; Pentzek, Michael

2014-06-01

To evaluate the summative assessment (OSCE) of a communication training programme for dealing with challenging doctor-patient encounters in the 4th study year. Our OSCE consists of 4 stations (breaking bad news, guilt and shame, aggressive patients, shared decision making), using a 4-item global rating (GR) instrument. We calculated reliability coefficients for different levels, discriminability of single items and interrater reliability. Validity was estimated by gender differences and accordance between GR and a checklist. In a pooled sample of 456 students in 3 OSCEs over 3 terms, total reliability was α=0.64, reliability coefficients for single stations were >0.80, and discriminability in 3 of 4 stations was within the range of 0.4-0.7. Except for one station, interrater reliability was moderate to strong. Reliability on item level was poor and pointed to some problems with the use of the GR. The application of the GR on regular undergraduate medical education shows moderate reliability in need of improvement and some traits of validity. Ongoing development and evaluation is needed with particular regard to the training of the examiners. Our CoMeD-OSCE proved suitable for the summative assessment of communication skills in challenging doctor-patient encounters. Copyright © 2014 Elsevier Ireland Ltd. All rights reserved.

A reliability of the prototype trunk training system for sitting balance.

PubMed

Jeong, Juri; Park, Dae-Sung; Lee, Hyelim; Eun, Seondeok

2014-11-01

[Purpose] Cerebral palsy is a disorder that affects balance in the sitting position. Cerebral palsy patients need trunk muscle strengthening and balance training. In order to improve trunk control sensory-motor control training is carried out on an unstable surface. We have developed a Trunk Training System (TTS) that can provide visual feedback using a tilt sensor for balance training in the sitting position. Before using the TTS for training children with cerebral palsy experiments were conducted with healthy adult subjects and the TTS to gather basic data for its improvement. [Subjects] The subjects were 11 healthy men (n=3) and women (n=8). [Methods] Subjects trained at two levels (5°, 10°), in four different directions (anterior, posterior, left, right), three times each. TTS outcome indices (stability index, performance time) were measured. [Results] The stability index and performance time showed high correlation (-0.6

The Critical Thinking Analytic Rubric (CTAR): Investigating Intra-Rater and Inter-Rater Reliability of a Scoring Mechanism for Critical Thinking Performance Assessments

ERIC Educational Resources Information Center

Saxton, Emily; Belanger, Secret; Becker, William

2012-01-01

The purpose of this study was to investigate the intra-rater and inter-rater reliability of the Critical Thinking Analytic Rubric (CTAR). The CTAR is composed of 6 rubric categories: interpretation, analysis, evaluation, inference, explanation, and disposition. To investigate inter-rater reliability, two trained raters scored four sets of…

An innovative localisation algorithm for railway vehicles

NASA Astrophysics Data System (ADS)

Allotta, B.; D'Adamio, P.; Malvezzi, M.; Pugi, L.; Ridolfi, A.; Rindi, A.; Vettori, G.

2014-11-01

In modern railway automatic train protection and automatic train control systems, odometry is a safety relevant on-board subsystem which estimates the instantaneous speed and the travelled distance of the train; a high reliability of the odometry estimate is fundamental, since an error on the train position may lead to a potentially dangerous overestimation of the distance available for braking. To improve the odometry estimate accuracy, data fusion of different inputs coming from a redundant sensor layout may be used. The aim of this work has been developing an innovative localisation algorithm for railway vehicles able to enhance the performances, in terms of speed and position estimation accuracy, of the classical odometry algorithms, such as the Italian Sistema Controllo Marcia Treno (SCMT). The proposed strategy consists of a sensor fusion between the information coming from a tachometer and an Inertial Measurements Unit (IMU). The sensor outputs have been simulated through a 3D multibody model of a railway vehicle. The work has provided the development of a custom IMU, designed by ECM S.p.a, in order to meet their industrial and business requirements. The industrial requirements have to be compliant with the European Train Control System (ETCS) standards: the European Rail Traffic Management System (ERTMS), a project developed by the European Union to improve the interoperability among different countries, in particular as regards the train control and command systems, fixes some standard values for the odometric (ODO) performance, in terms of speed and travelled distance estimation. The reliability of the ODO estimation has to be taken into account basing on the allowed speed profiles. The results of the currently used ODO algorithms can be improved, especially in case of degraded adhesion conditions; it has been verified in the simulation environment that the results of the proposed localisation algorithm are always compliant with the ERTMS requirements. The estimation strategy has good performance also under degraded adhesion conditions and could be put on board of high-speed railway vehicles; it represents an accurate and reliable solution. The IMU board is tested via a dedicated Hardware in the Loop (HIL) test rig: it includes an industrial robot able to replicate the motion of the railway vehicle. Through the generated experimental outputs the performances of the innovative localisation algorithm have been evaluated: the HIL test rig permitted to test the proposed algorithm, avoiding expensive (in terms of time and cost) on-track tests, obtaining encouraging results. In fact, the preliminary results show a significant improvement of the position and speed estimation performances compared to those obtained with SCMT algorithms, currently in use on the Italian railway network.

Test-Retest Reliability of Measures Commonly Used to Measure Striatal Dysfunction across Multiple Testing Sessions: A Longitudinal Study.

PubMed

Palmer, Clare E; Langbehn, Douglas; Tabrizi, Sarah J; Papoutsi, Marina

2017-01-01

Cognitive impairment is common amongst many neurodegenerative movement disorders such as Huntington's disease (HD) and Parkinson's disease (PD) across multiple domains. There are many tasks available to assess different aspects of this dysfunction, however, it is imperative that these show high test-retest reliability if they are to be used to track disease progression or response to treatment in patient populations. Moreover, in order to ensure effects of practice across testing sessions are not misconstrued as clinical improvement in clinical trials, tasks which are particularly vulnerable to practice effects need to be highlighted. In this study we evaluated test-retest reliability in mean performance across three testing sessions of four tasks that are commonly used to measure cognitive dysfunction associated with striatal impairment: a combined Simon Stop-Signal Task; a modified emotion recognition task; a circle tracing task; and the trail making task. Practice effects were seen between sessions 1 and 2 across all tasks for the majority of dependent variables, particularly reaction time variables; some, but not all, diminished in the third session. Good test-retest reliability across all sessions was seen for the emotion recognition, circle tracing, and trail making test. The Simon interference effect and stop-signal reaction time (SSRT) from the combined-Simon-Stop-Signal task showed moderate test-retest reliability, however, the combined SSRT interference effect showed poor test-retest reliability. Our results emphasize the need to use control groups when tracking clinical progression or use pre-baseline training on tasks susceptible to practice effects.

A Persian-version of the stuttering severity instrument-version four (SSI-4): How the new additions to SSI-4 complement its stuttering severity score?

PubMed

Tahmasebi, Neda; Shafie, Bijan; Karimi, Hamid; Mazaheri, Masood

The fourth version of the Stuttering Severity Instrument (SSI-4) has been available since 2009. It has some modifications and new features which make it more appropriate at least for clinical practice, although further documentation is needed. The objective of the current research was to translate SSI-4 into Persian language and to discuss its relative and absolute reliability as well as its criterion validity for Persian adults who stutter (PWS). We also aimed to study how the new subjective self-reports of the SSI-4 complement the stuttering severity score obtained from the SSI-3 or the SSI-4. The cross-cultural guideline recommended by the International Quality of Life Assessment project was used to translate the SSI-4 into Persian language. Thirty five PWS from ages 17 to 42 were recruited and 10 speech and language pathologists assessed their stuttering severity using either the SSI-4 or stuttering severity ratings (SR) to test validity and reliability of the Persian translated version. A very high inter-judge relative reliability along with a poor absolute inter-judge reliability was found for the SSI-4 scores. The results were more promising for the intra-judge absolute reliability. Test-retest reliability of the complementary questions to the SSI-4 was also found acceptable. However, no strong relationship was found between the SSI-4 scores and its complementary questions. The Persian version of the SSI-4 can be used reliably by trained SLPs for research and clinical purposes, but not to document small changes in stuttering severity. We argue that the response of participants to the complementary self-report questions should also be considered in calculating their stuttering severity score. Copyright © 2018 Elsevier Inc. All rights reserved.

Metroliner Auxiliary Power Electrical System Reliability Study

DOT National Transportation Integrated Search

1971-06-01

The reliability of the electrical system of any vehicle is greatly affected by the way the system is configured. The propulsion and braking systems of a train must be unaffected by failures occurring in the nonessential power areas. With these criter...

A methodology for assessing the professional development needs of nurses and midwives in Indonesia: paper 1 of 3

PubMed Central

Hennessy, Deborah; Hicks, Carolyn; Hilan, Aflah; Kawonal, Yoanna

2006-01-01

Background Despite recent developments, health care provision in Indonesia remains suboptimal. Difficult terrain, economic crises, endemic diseases and high population numbers, coupled with limited availability of qualified health care professionals, all contribute to poor health status. In a country with a population of 220 million, there are currently an estimated 50 nurses and 26 midwives per 100 000 people. In line with government initiatives, this series of studies was undertaken to establish the training and development needs of nurses and midwives working within a variety of contexts in Indonesia, with the ultimate aim of enhancing care provision within these domains. Methods An established, psychometrically valid and reliable training needs instrument was modified for use within the Indonesian context. While this technique has had widespread international use in the developed world, its application for developing countries has not yet been established. The standard form consists of a biographical cover sheet and a core set of 30 items (all health-related tasks), which have to be rated along two seven-point scales. The first of these scales asks respondents to assess how important the task is to their job and the second scale is a self-assessment of respondents' current performance level of the task. By comparing the importance rating with the performance rating, an index of training need can be obtained (high importance and low performance indicating a significant training need). The modifications incorporated for use in this series of studies were a further 10 items, which were constructed following expert group and focus group discussions and a review of the relevant literature. Pilot trials with 109 respondents confirmed its feasibility and acceptability. The instrument was then administered to 524 nurses and 332 midwives across Indonesia. Results The data were subjected to a retrospective factor analysis, using a Varimax rotation and Cronbach's α to check the instrument's validity and reliability following modification. The results yielded six factors, which accounted for >53% of the variance, each of which had a Cronbach's α score of between 0.8644 and 0.7068. Conclusion The results suggest that the modified instrument remained valid and reliable for use in the Indonesian nursing and midwifery context. PMID:16623935

A research coding method for the basic patient-centered interview.

PubMed

Grayson-Sneed, Katelyn A; Smith, Sandi W; Smith, Robert C

2017-03-01

To develop a more reliable coding method of medical interviewing focused on data-gathering and emotion-handling. Two trained (30h) undergraduates rated videotaped interviews from 127 resident-simulated patient (SP) interactions. Trained on 45 videotapes, raters coded 25 of 127 study set tapes for patient-centeredness. Guetzkow's U, Cohen's Kappa, and percent of agreement were used to measure raters' reliability in unitizing and coding residents' skills for eliciting: agenda (3 yes/no items), physical story (2), personal story (6), emotional story (15), using indirect skills (4), and general patient-centeredness (3). 45 items were dichotomized from the earlier, Likert scale-based method and were reduced to 33 during training. Guetzkow's U ranged from 0.00 to 0.087. Kappa ranged from 0.86 to 1.00 for the 6 variables and 33 individual items. The overall kappa was 0.90, and percent of agreement was 97.5%. Percent of agreement by item ranged from 84 to 100%. A simple, highly reliable coding method, weighted (by no. of items) to highlight personal elements of an interview, was developed and is recommended as a criterion standard research coding method. An easily conducted, reliable coding procedure can be the basis for everyday questionnaires like patient satisfaction with patient-centeredness. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

Simulation-Based Training for Colonoscopy

PubMed Central

Preisler, Louise; Svendsen, Morten Bo Søndergaard; Nerup, Nikolaj; Svendsen, Lars Bo; Konge, Lars

2015-01-01

Abstract The aim of this study was to create simulation-based tests with credible pass/fail standards for 2 different fidelities of colonoscopy models. Only competent practitioners should perform colonoscopy. Reliable and valid simulation-based tests could be used to establish basic competency in colonoscopy before practicing on patients. Twenty-five physicians (10 consultants with endoscopic experience and 15 fellows with very little endoscopic experience) were tested on 2 different simulator models: a virtual-reality simulator and a physical model. Tests were repeated twice on each simulator model. Metrics with discriminatory ability were identified for both modalities and reliability was determined. The contrasting-groups method was used to create pass/fail standards and the consequences of these were explored. The consultants significantly performed faster and scored higher than the fellows on both the models (P < 0.001). Reliability analysis showed Cronbach α = 0.80 and 0.87 for the virtual-reality and the physical model, respectively. The established pass/fail standards failed one of the consultants (virtual-reality simulator) and allowed one fellow to pass (physical model). The 2 tested simulations-based modalities provided reliable and valid assessments of competence in colonoscopy and credible pass/fail standards were established for both the tests. We propose to use these standards in simulation-based training programs before proceeding to supervised training on patients. PMID:25634177

A succinct rating scale for radiology report quality

PubMed Central

Yang, Chengwu; Ouyang, Tao; Peterson, Christine M; Sarwani, Nabeel I; Tappouni, Rafel; Bruno, Michael

2014-01-01

Context: Poorly written radiology reports are common among residents and are a significant challenge for radiology education. While training may improve report quality, a professionally developed reliable and valid scale to measure report quality does not exist. Objectives: To develop a measurement tool for report quality, the quality of report scale, with rigorous validation through empirical data. Methods: A research team of an experienced psychometrician and six senior radiologists conducted qualitative and quantitative studies. Five items were identified for the quality of report scale, each measuring a distinct aspect of report quality. Two dedicated training sessions were designed and implemented to help residents generate high-quality reports. In a blinded fashion, the quality of report scale was applied to 804 randomly selected reports issued before (n = 403) and after (n = 401) training. Full-scale psychometrical assessments were implemented onto the quality of report scale’s item- and scale-scores from the reports. The quality of report scale scores were correlated with report professionalism and attendings’ preference and were compared pre-/post-training. Results: The quality of report scale showed sound psychometrical properties, with high validity and reliability. Reports with higher quality of report scale score were more professional and preferable by attendings. Training improved the quality of report scale score, empirically validating the quality of report scale further. Conclusion: While succinct and practitioner friendly, the quality of report scale is a reliable and valid measure of radiology report quality and has the potential to be easily adapted to other fields such as pathology, where similar training would be beneficial. PMID:26770756

Transferring Aviation Practices into Clinical Medicine for the Promotion of High Reliability.

PubMed

Powell-Dunford, Nicole; McPherson, Mark K; Pina, Joseph S; Gaydos, Steven J

2017-05-01

Aviation is a classic example of a high reliability organization (HRO)-an organization in which catastrophic events are expected to occur without control measures. As health care systems transition toward high reliability, aviation practices are increasingly transferred for clinical implementation. A PubMed search using the terms aviation, crew resource management, and patient safety was undertaken. Manuscripts authored by physician pilots and accident investigation regulations were analyzed. Subject matter experts involved in adoption of aviation practices into the medical field were interviewed. A PubMed search yielded 621 results with 22 relevant for inclusion. Improved clinical outcomes were noted in five research trials in which aviation practices were adopted, particularly with regard to checklist usage and crew resource-management training. Effectiveness of interventions was influenced by intensity of application, leadership involvement, and provision of staff training. The usefulness of incorporating mishap investigation techniques has not been established. Whereas aviation accident investigation is highly standardized, the investigation of medical error is characterized by variation. The adoption of aviation practices into clinical medicine facilitates an evolution toward high reliability. Evidence for the efficacy of the checklist and crew resource-management training is robust. Transference of aviation accident investigation practices is preliminary. A standardized, independent investigation process could facilitate the development of a safety culture commensurate with that achieved in the aviation industry.Powell-Dunford N, McPherson MK, Pina JS, Gaydos SJ. Transferring aviation practices into clinical medicine for the promotion of high reliability. Aerosp Med Hum Perform. 2017; 88(5):487-491.

Guidelines for a Training Course in Noise Survey Techniques.

ERIC Educational Resources Information Center

Shadley, John; And Others

The course is designed to train noise survey technicians during a 3-5 day period to make reliable measurements of 75 percent of the noise problems encountered in the community. The more complex noise problems remaining will continue to be handled by experienced specialists. These technicians will be trained to assist State and local governments in…

Traditional and nontraditional internships in government

NASA Technical Reports Server (NTRS)

Stohrer, Freda F.; Pinelli, Thomas E.

1980-01-01

Traditional and nontraditional methods for training technical writers-editors within the federal government are discussed. It is concluded that cooperative education that combines work experience with classroom instruction provides an excellent method for locating and training competent and reliable young professionals.

77 FR 34075 - Notice of Quarterly Report (January 1, 2012-March 31, 2012)

Federal Register 2010, 2011, 2012, 2013, 2014

2012-06-08

... trained in commercial cultivation, agriculture. services to Number of agriculture and agribusinesses... improved Latrines. Knowledge of good hygiene practices. Households with reliable water services... stakeholders reached by public outreach efforts. Personnel trained. [[Page 34089

18 CFR 40.3 - Availability of Reliability Standards.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 18 Conservation of Power and Water Resources 1 2010-04-01 2010-04-01 false Availability of Reliability Standards. 40.3 Section 40.3 Conservation of Power and Water Resources FEDERAL ENERGY REGULATORY... THE BULK-POWER SYSTEM § 40.3 Availability of Reliability Standards. The Electric Reliability...

Robot-aided assessment of lower extremity functions: a review.

PubMed

Maggioni, Serena; Melendez-Calderon, Alejandro; van Asseldonk, Edwin; Klamroth-Marganska, Verena; Lünenburger, Lars; Riener, Robert; van der Kooij, Herman

2016-08-02

The assessment of sensorimotor functions is extremely important to understand the health status of a patient and its change over time. Assessments are necessary to plan and adjust the therapy in order to maximize the chances of individual recovery. Nowadays, however, assessments are seldom used in clinical practice due to administrative constraints or to inadequate validity, reliability and responsiveness. In clinical trials, more sensitive and reliable measurement scales could unmask changes in physiological variables that would not be visible with existing clinical scores.In the last decades robotic devices have become available for neurorehabilitation training in clinical centers. Besides training, robotic devices can overcome some of the limitations in traditional clinical assessments by providing more objective, sensitive, reliable and time-efficient measurements. However, it is necessary to understand the clinical needs to be able to develop novel robot-aided assessment methods that can be integrated in clinical practice.This paper aims at providing researchers and developers in the field of robotic neurorehabilitation with a comprehensive review of assessment methods for the lower extremities. Among the ICF domains, we included those related to lower extremities sensorimotor functions and walking; for each chapter we present and discuss existing assessments used in routine clinical practice and contrast those to state-of-the-art instrumented and robot-aided technologies. Based on the shortcomings of current assessments, on the identified clinical needs and on the opportunities offered by robotic devices, we propose future directions for research in rehabilitation robotics. The review and recommendations provided in this paper aim to guide the design of the next generation of robot-aided functional assessments, their validation and their translation to clinical practice.

A valid and reliable method to measure jump-specific training and competition load in elite volleyball players.

PubMed

Skazalski, C; Whiteley, R; Hansen, C; Bahr, R

2018-05-01

Use of a commercially available wearable device to monitor jump load with elite volleyball players has become common practice. The purpose of this study was to evaluate the validity and reliability of this device, the Vert, to count jumps and measure jump height with professional volleyball players. Jump count accuracy was determined by comparing jumps recorded by the device to jumps observed through systematic video analysis of three practice sessions and two league matches performed by a men's professional volleyball team. Jumps performed by 14 players were each coded for time and jump type and individually matched to device recorded jumps. Jump height validity of the device was examined against reference standards as participants performed countermovement jumps on a force plate and volleyball-specific jumps with a Vertec. The Vert device accurately counted 99.3% of the 3637 jumps performed during practice and match play. The device showed excellent jump height interdevice reliability for two devices placed in the same pouch during volleyball jumps (r = .99, 95% CI 0.98-0.99). The device had a minimum detectable change (MDC) of 9.7 cm and overestimated jump height by an average of 5.5 cm (95% CI 4.5-6.5) across all volleyball jumps. The Vert device demonstrates excellent accuracy counting volleyball-specific jumps during training and competition. While the device is not recommended to measure maximal jumping ability when precision is needed, it provides an acceptable measure of on-court jump height that can be used to monitor athlete jump load. © 2018 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

[Training needs for health and nutrition personnel in developing nations].

PubMed

Delgado, H L; Valverde, V; Angel, L O

1983-01-01

This paper examines some of the factors related to the training of health and nutrition personnel in developing countries in general and in Latin America in particular. It addresses both professional and technical level staff responsible for the formulation of plans and programs, and auxiliary personnel responsible for the delivery of services. Using the systems analysis model, the need for training human resources in research techniques and decision-making on a scientific basis is taken as an example for a discussion of aspects of the diagnosis, planning, execution and evaluation of health and nutrition programs. Various ways are indicated of giving training in research techniques, using the personnel available and employing simple instructions and methods, currently under-used, such as gathering anthropometric data. The authors stress that if an effort were to be made in the countries, the risk factors influencing health and nutrition, particularly among the maternal-child group, might well be identified quickly and without great cost. The case of the height census in Costa Rica is reviewed as a practical example of programs that have originated in developing countries and that make maximum use of auxiliary personnel in obtaining reliable, valid information for decision-making purposes. This census included children in the first grade of the primary cycle at the national level. The contents of the tutorial training program of the Institute of Nutrition of Central America and Panama (INCAP) are also described.

The Surgeons' Leadership Inventory (SLI): a taxonomy and rating system for surgeons' intraoperative leadership skills.

PubMed

Henrickson Parker, Sarah; Flin, Rhona; McKinley, Aileen; Yule, Steven

2013-06-01

Surgeons must demonstrate leadership to optimize performance and maximize patient safety in the operating room, but no behavior rating tool is available to measure leadership. Ten focus groups with members of the operating room team discussed surgeons' intraoperative leadership. Surgeons' leadership behaviors were extracted and used to finalize the Surgeons' Leadership Inventory (SLI), which was checked by surgeons (n = 6) for accuracy and face validity. The SLI was used to code video recordings (n = 5) of operations to test reliability. Eight elements of surgeons' leadership were included in the SLI: (1) maintaining standards, (2) managing resources, (3) making decisions, (4) directing, (5) training, (6) supporting others, (7) communicating, and (8) coping with pressure. Interrater reliability to code videos of surgeons' behaviors while operating using this tool was acceptable (κ = .70). The SLI is empirically grounded in focus group data and both the leadership and surgical literature. The interrater reliability of the system was acceptable. The inventory could be used for rating surgeons' leadership in the operating room for research or as a basis for postoperative feedback on performance. Copyright © 2013 Elsevier Inc. All rights reserved.

Factors Influencing the Reliability of the Glasgow Coma Scale: A Systematic Review.

PubMed

Reith, Florence Cm; Synnot, Anneliese; van den Brande, Ruben; Gruen, Russell L; Maas, Andrew Ir

2017-06-01

The Glasgow Coma Scale (GCS) characterizes patients with diminished consciousness. In a recent systematic review, we found overall adequate reliability across different clinical settings, but reliability estimates varied considerably between studies, and methodological quality of studies was overall poor. Identifying and understanding factors that can affect its reliability is important, in order to promote high standards for clinical use of the GCS. The aim of this systematic review was to identify factors that influence reliability and to provide an evidence base for promoting consistent and reliable application of the GCS. A comprehensive literature search was undertaken in MEDLINE, EMBASE, and CINAHL from 1974 to July 2016. Studies assessing the reliability of the GCS in adults or describing any factor that influences reliability were included. Two reviewers independently screened citations, selected full texts, and undertook data extraction and critical appraisal. Methodological quality of studies was evaluated with the consensus-based standards for the selection of health measurement instruments checklist. Data were synthesized narratively and presented in tables. Forty-one studies were included for analysis. Factors identified that may influence reliability are education and training, the level of consciousness, and type of stimuli used. Conflicting results were found for experience of the observer, the pathology causing the reduced consciousness, and intubation/sedation. No clear influence was found for the professional background of observers. Reliability of the GCS is influenced by multiple factors and as such is context dependent. This review points to the potential for improvement from training and education and standardization of assessment methods, for which recommendations are presented. Copyright © 2017 by the Congress of Neurological Surgeons.

Reliability of the dynavision™ d2 for assessing reaction time performance.

PubMed

Wells, Adam J; Hoffman, Jay R; Beyer, Kyle S; Jajtner, Adam R; Gonzalez, Adam M; Townsend, Jeremy R; Mangine, Gerald T; Robinson, Edward H; McCormack, William P; Fragala, Maren S; Stout, Jeffrey R

2014-01-01

Recently, the Dynavision™ D2 Visuomotor Training Device (D2) has emerged as a tool in the assessment of reaction time (RT); however, information regarding the reliability of the D2 have been limited, and to date, reliability data have been limited to non- generalizable samples. Therefore, the purpose of this study was to establish intraclass correlation coefficients (ICC2,1) for the D2 that are generalizable across a population of recreationally active young adults. Forty-two recreationally active men and women (age: 23.41 ± 4.84 years; height: 1.72 ± 0.11 m; mass: 76.62 ± 18.26 Kg) completed 6 trials for three RT tasks of increasing complexity. Each trial was separated by at least 48-hours. A repeated measures ANOVA was used to detect differences in performance across the six trials. Intraclass correlation coefficients (ICC2,1) standard error of measurement (SEM), and minimal differences (MD) were used to determine the reliability of the D2 from the two sessions with the least significant difference score. Moderate to strong reliability was demonstrated for visual RT (ICC2,1: 0.84, SEM: 0.033), and reactive ability in both Mode A and Mode B tasks (Mode A hits: ICC2,1: 0.75, SEM: 5.44; Mode B hits: ICC2,1: 0.73, SEM: 8.57). Motor RT (ICC2,1: 0.63, SEM: 0.035s) showed fair reliability, while average RT per hit for Modes A and B showed moderate reliability (ICC2,1: 0.68, SEM: 0.43 s and ICC2,1: 0.72, SEM: 0.03 s respectively). It appears that one familiarization trial is necessary for the choice reaction time (CRT) task while three familiarization trials are necessary for reactive RT tasks. In conclusion, results indicate that the Dynavision™ D2 is a reliable device to assess neuromuscular reactivity given that an adequate practice is provided. The data presented are generalizable to a population of recreationally active young adults. Key PointsThe Dynavision™ D2 is a light-training reaction device, developed to train sensory motor integration through the visual system, offering the ability to assess visual and motor reaction to both central and peripheral stimuli, with a capacity to integrate increasing levels of cognitive challenge.The Dynavision™ D2 is a reliable instrument for assessing reaction time in recreationally active young adults.It is recommended that one familiarization trial is necessary for the choice reaction time task assessment to learn the test protocol, while three familiarization trials are needed for reactive ability in Mode A and Mode B before a subsequent reliable baseline score can be established.Significant training effects were observed for all reaction time tests and should be taken into account with continuous trials.

Identifying depression post-stroke in patients with aphasia: a systematic review of the reliability, validity and feasibility of available instruments.

PubMed

van Dijk, Mariska J; de Man-van Ginkel, Janneke M; Hafsteinsdóttir, Thóra B; Schuurmans, Marieke J

2016-08-01

To identify and critically appraise the evidence for instruments assessing depression in stroke patients with aphasia. The PubMed, CINAHL, Web of Science, Psych Info and Cochrane databases were searched from inception until May 2015. Of the 383 titles found in the search, 15 articles met the inclusion criteria and six instruments were identified: The Aphasic Depression Rating Scale, the Clinical Global Impression-Scale, the Stroke Aphasic Depression Questionnaire (four versions), the Signs of Depression Scale, the Visual Analogue Mood Scale (three versions) and the Visual Analogue Self Esteem Scale. Supporting evidence for reliability and validity was limited owing to methodological flaws in the studies influencing the ratings of methodological quality. Feasibility data were available for all instruments. Rating time of the instruments ranged from less than one minute to five minutes, two instruments required extensive training. A number of instruments to assess depressive symptoms in patients with aphasia are available. None of the instruments however, were found to be sufficiently investigated and most of the studies identified were of low methodological quality. Given the present evidence, the Stroke Aphasic Depression Questionnaire-10, the Stroke Aphasic Depression Questionnaire-H10 and the Signs of Depression Scale are the most feasible and can be recommended for clinical practice. © The Author(s) 2015.

Hand assessment in older adults with musculoskeletal hand problems: a reliability study.

PubMed

Myers, Helen L; Thomas, Elaine; Hay, Elaine M; Dziedzic, Krysia S

2011-01-07

Musculoskeletal hand pain is common in the general population. This study aims to investigate the inter- and intra-observer reliability of two trained observers conducting a simple clinical interview and physical examination for hand problems in older adults. The reliability of applying the American College of Rheumatology (ACR) criteria for hand osteoarthritis to community-dwelling older adults will also be investigated. Fifty-five participants aged 50 years and over with a current self-reported hand problem and registered with one general practice were recruited from a previous health questionnaire study. Participants underwent a standardised, structured clinical interview and physical examination by two independent trained observers and again by one of these observers a month later. Agreement beyond chance was summarised using Kappa statistics and intra-class correlation coefficients. Median values for inter- and intra-observer reliability for clinical interview questions were found to be "substantial" and "moderate" respectively [median agreement beyond chance (Kappa) was 0.75 (range: -0.03, 0.93) for inter-observer ratings and 0.57 (range: -0.02, 1.00) for intra-observer ratings]. Inter- and intra-observer reliability for physical examination items was variable, with good reliability observed for some items, such as grip and pinch strength, and poor reliability observed for others, notably assessment of altered sensation, pain on resisted movement and judgements based on observation and palpation of individual features at single joints, such as bony enlargement, nodes and swelling. Moderate agreement was observed both between and within observers when applying the ACR criteria for hand osteoarthritis. Standardised, structured clinical interview is reliable for taking a history in community-dwelling older adults with self reported hand problems. Agreement between and within observers for physical examination items is variable. Low Kappa values may have resulted, in part, from a low prevalence of clinical signs and symptoms in the study participants. The decision to use clinical interview and hand assessment variables in clinical practice or further research in primary care should include consideration of clinical applicability and training alongside reliability. Further investigation is required to determine the relationship between these clinical questions and assessments and the clinical course of hand pain and hand problems in community-dwelling older adults.

Validation of a Detailed Scoring Checklist for Use During Advanced Cardiac Life Support Certification

PubMed Central

McEvoy, Matthew D.; Smalley, Jeremy C.; Nietert, Paul J.; Field, Larry C.; Furse, Cory M.; Blenko, John W.; Cobb, Benjamin G.; Walters, Jenna L.; Pendarvis, Allen; Dalal, Nishita S.; Schaefer, John J.

2012-01-01

Introduction Defining valid, reliable, defensible, and generalizable standards for the evaluation of learner performance is a key issue in assessing both baseline competence and mastery in medical education. However, prior to setting these standards of performance, the reliability of the scores yielding from a grading tool must be assessed. Accordingly, the purpose of this study was to assess the reliability of scores generated from a set of grading checklists used by non-expert raters during simulations of American Heart Association (AHA) MegaCodes. Methods The reliability of scores generated from a detailed set of checklists, when used by four non-expert raters, was tested by grading team leader performance in eight MegaCode scenarios. Videos of the scenarios were reviewed and rated by trained faculty facilitators and by a group of non-expert raters. The videos were reviewed “continuously” and “with pauses.” Two content experts served as the reference standard for grading, and four non-expert raters were used to test the reliability of the checklists. Results Our results demonstrate that non-expert raters are able to produce reliable grades when using the checklists under consideration, demonstrating excellent intra-rater reliability and agreement with a reference standard. The results also demonstrate that non-expert raters can be trained in the proper use of the checklist in a short amount of time, with no discernible learning curve thereafter. Finally, our results show that a single trained rater can achieve reliable scores of team leader performance during AHA MegaCodes when using our checklist in continuous mode, as measures of agreement in total scoring were very strong (Lin’s Concordance Correlation Coefficient = 0.96; Intraclass Correlation Coefficient = 0.97). Discussion We have shown that our checklists can yield reliable scores, are appropriate for use by non-expert raters, and are able to be employed during continuous assessment of team leader performance during the review of a simulated MegaCode. This checklist may be more appropriate for use by Advanced Cardiac Life Support (ACLS) instructors during MegaCode assessments than current tools provided by the AHA. PMID:22863996

Reliability training

NASA Technical Reports Server (NTRS)

Lalli, Vincent R. (Editor); Malec, Henry A. (Editor); Dillard, Richard B.; Wong, Kam L.; Barber, Frank J.; Barina, Frank J.

1992-01-01

Discussed here is failure physics, the study of how products, hardware, software, and systems fail and what can be done about it. The intent is to impart useful information, to extend the limits of production capability, and to assist in achieving low cost reliable products. A review of reliability for the years 1940 to 2000 is given. Next, a review of mathematics is given as well as a description of what elements contribute to product failures. Basic reliability theory and the disciplines that allow us to control and eliminate failures are elucidated.

From a formal training program in musculoskeletal ultrasound (MSUS) to a high reproducibility for Doppler ultrasound in rheumatoid arthritis.

PubMed

Villota, Orlando; Diaz, Mario; Ceron, Carmen; Moller, Ingrid; Naredo, Esperanza; Saaibi, Diego Luis

2017-07-28

To assess the intra- and inter-observer reliability of ultrasound (US) in scoring B-mode, Doppler synovitis and combined B-mode and Doppler synovitis scores in different peripheral joints of rheumatoid arthritis (RA) patients. Four rheumatologists with a formal training in musculoskeletal US (MSKUS) particularly focus on definitions and scoring synovitis on B-mode and Doppler mode participated in a patient-based reliability exercise on 16 active RA patients. The four rheumatologists independently and consecutively performed a B-mode and power Doppler (PD) US assessment of 7 joints of each patient in two rounds in a blinded fashion. Each joint was semi quantitatively scored from 0 to 3 for B-mode synovitis (BS), Doppler synovitis (DS), and combined B-mode/Doppler synovitis (CS). Intraobserver reliability was assessed by Cohen's κ. Interobserver reliability was assessed by unweight Light's κ. The mean prevalence of synovitis on B-mode was 83% of joints; scores ranging from grade 1 in 18% of joints, to grade 3 in 33%. In 55% of joints synovial PD signal was detected and the distribution of scores range from 14% of joints for grade 3, to 26% for grade 2. After a total of 448 joints scanned with 896 adquired images our intraobserver and interobserver reliability was good to excellent for most of the joints. Formal, structured and continuous training in musculoskeletal ultrasound would bring a good to excellent reproducibility in rheumatological hands with a high reliability in real time acquisition BS, DS and CS modalities for scoring synovitis in patients with active rheumatoid arthritis. Copyright© Bentham Science Publishers; For any queries, please email at epub@benthamscience.org.

Developing a comprehensive training curriculum for integrated predictive maintenance

NASA Astrophysics Data System (ADS)

Wurzbach, Richard N.

2002-03-01

On-line equipment condition monitoring is a critical component of the world-class production and safety histories of many successful nuclear plant operators. From addressing availability and operability concerns of nuclear safety-related equipment to increasing profitability through support system reliability and reduced maintenance costs, Predictive Maintenance programs have increasingly become a vital contribution to the maintenance and operation decisions of nuclear facilities. In recent years, significant advancements have been made in the quality and portability of many of the instruments being used, and software improvements have been made as well. However, the single most influential component of the success of these programs is the impact of a trained and experienced team of personnel putting this technology to work. Changes in the nature of the power generation industry brought on by competition, mergers, and acquisitions, has taken the historically stable personnel environment of power generation and created a very dynamic situation. As a result, many facilities have seen a significant turnover in personnel in key positions, including predictive maintenance personnel. It has become the challenge for many nuclear operators to maintain the consistent contribution of quality data and information from predictive maintenance that has become important in the overall equipment decision process. These challenges can be met through the implementation of quality training to predictive maintenance personnel and regular updating and re-certification of key technology holders. The use of data management tools and services aid in the sharing of information across sites within an operating company, and with experts who can contribute value-added data management and analysis. The overall effectiveness of predictive maintenance programs can be improved through the incorporation of newly developed comprehensive technology training courses. These courses address the use of key technologies such as vibration analysis, infrared thermography, and oil analysis not as singular entities, but as a toolbox resource from which to address overall equipment and plant reliability in a structured program and decision environment.

Genomic selection in a commercial winter wheat population.

PubMed

He, Sang; Schulthess, Albert Wilhelm; Mirdita, Vilson; Zhao, Yusheng; Korzun, Viktor; Bothe, Reiner; Ebmeyer, Erhard; Reif, Jochen C; Jiang, Yong

2016-03-01

Genomic selection models can be trained using historical data and filtering genotypes based on phenotyping intensity and reliability criterion are able to increase the prediction ability. We implemented genomic selection based on a large commercial population incorporating 2325 European winter wheat lines. Our objectives were (1) to study whether modeling epistasis besides additive genetic effects results in enhancement on prediction ability of genomic selection, (2) to assess prediction ability when training population comprised historical or less-intensively phenotyped lines, and (3) to explore the prediction ability in subpopulations selected based on the reliability criterion. We found a 5 % increase in prediction ability when shifting from additive to additive plus epistatic effects models. In addition, only a marginal loss from 0.65 to 0.50 in accuracy was observed using the data collected from 1 year to predict genotypes of the following year, revealing that stable genomic selection models can be accurately calibrated to predict subsequent breeding stages. Moreover, prediction ability was maximized when the genotypes evaluated in a single location were excluded from the training set but subsequently decreased again when the phenotyping intensity was increased above two locations, suggesting that the update of the training population should be performed considering all the selected genotypes but excluding those evaluated in a single location. The genomic prediction ability was substantially higher in subpopulations selected based on the reliability criterion, indicating that phenotypic selection for highly reliable individuals could be directly replaced by applying genomic selection to them. We empirically conclude that there is a high potential to assist commercial wheat breeding programs employing genomic selection approaches.

The relationship between faculty performance assessment and results on the in-training examination for residents in an emergency medicine training program.

PubMed

Ryan, James G; Barlas, David; Pollack, Simcha

2013-12-01

Medical knowledge (MK) in residents is commonly assessed by the in-training examination (ITE) and faculty evaluations of resident performance. We assessed the reliability of clinical evaluations of residents by faculty and the relationship between faculty assessments of resident performance and ITE scores. We conducted a cross-sectional, observational study at an academic emergency department with a postgraduate year (PGY)-1 to PGY-3 emergency medicine residency program, comparing summative, quarterly, faculty evaluation data for MK and overall clinical competency (OC) with annual ITE scores, accounting for PGY level. We also assessed the reliability of faculty evaluations using a random effects, intraclass correlation analysis. We analyzed data for 59 emergency medicine residents during a 6-year period. Faculty evaluations of MK and OC were highly reliable (κ  =  0.99) and remained reliable after stratification by year of training (mean κ  =  0.68-0.84). Assessments of resident performance (MK and OC) and the ITE increased with PGY level. The MK and OC results had high correlations with PGY level, and ITE scores correlated moderately with PGY. The OC and MK results had a moderate correlation with ITE score. When residents were grouped by PGY level, there was no significant correlation between MK as assessed by the faculty and the ITE score. Resident clinical performance and ITE scores both increase with resident PGY level, but ITE scores do not predict resident clinical performance compared with peers at their PGY level.

The Relationship Between Faculty Performance Assessment and Results on the In-Training Examination for Residents in an Emergency Medicine Training Program

PubMed Central

Ryan, James G.; Barlas, David; Pollack, Simcha

2013-01-01

Background Medical knowledge (MK) in residents is commonly assessed by the in-training examination (ITE) and faculty evaluations of resident performance. Objective We assessed the reliability of clinical evaluations of residents by faculty and the relationship between faculty assessments of resident performance and ITE scores. Methods We conducted a cross-sectional, observational study at an academic emergency department with a postgraduate year (PGY)-1 to PGY-3 emergency medicine residency program, comparing summative, quarterly, faculty evaluation data for MK and overall clinical competency (OC) with annual ITE scores, accounting for PGY level. We also assessed the reliability of faculty evaluations using a random effects, intraclass correlation analysis. Results We analyzed data for 59 emergency medicine residents during a 6-year period. Faculty evaluations of MK and OC were highly reliable (κ  =  0.99) and remained reliable after stratification by year of training (mean κ  =  0.68–0.84). Assessments of resident performance (MK and OC) and the ITE increased with PGY level. The MK and OC results had high correlations with PGY level, and ITE scores correlated moderately with PGY. The OC and MK results had a moderate correlation with ITE score. When residents were grouped by PGY level, there was no significant correlation between MK as assessed by the faculty and the ITE score. Conclusions Resident clinical performance and ITE scores both increase with resident PGY level, but ITE scores do not predict resident clinical performance compared with peers at their PGY level. PMID:24455005

Redesign of the System for Evaluation of Teaching Qualities in Anesthesiology Residency Training (SETQ Smart).

PubMed

Lombarts, Kiki M J M H; Ferguson, Andrew; Hollmann, Markus W; Malling, Bente; Arah, Onyebuchi A

2016-11-01

Given the increasing international recognition of clinical teaching as a competency and regulation of residency training, evaluation of anesthesiology faculty teaching is needed. The System for Evaluating Teaching Qualities (SETQ) Smart questionnaires were developed for assessing teaching performance of faculty in residency training programs in different countries. This study investigated (1) the structure, (2) the psychometric qualities of the new tools, and (3) the number of residents' evaluations needed per anesthesiology faculty to use the instruments reliably. Two SETQ Smart questionnaires-for faculty self-evaluation and for resident evaluation of faculty-were developed. A multicenter survey was conducted among 399 anesthesiology faculty and 430 residents in six countries. Statistical analyses included exploratory factor analysis, reliability analysis using Cronbach α, item-total scale correlations, interscale correlations, comparison of composite scales to global ratings, and generalizability analysis to assess residents' evaluations needed per faculty. In total, 240 residents completed 1,622 evaluations of 247 faculty. The SETQ Smart questionnaires revealed six teaching qualities consisting of 25 items. Cronbach α's were very high (greater than 0.95) for the overall SETQ Smart questionnaires and high (greater than 0.80) for the separate teaching qualities. Interscale correlations were all within the acceptable range of moderate correlation. Overall, questionnaire and scale scores correlated moderately to highly with the global ratings. For reliable feedback to individual faculty, three to five resident evaluations are needed. The first internationally piloted questionnaires for evaluating individual anesthesiology faculty teaching performance can be reliably, validly, and feasibly used for formative purposes in residency training.

Working Memory Training Does Not Improve Performance on Measures of Intelligence or Other Measures of "Far Transfer": Evidence From a Meta-Analytic Review.

PubMed

Melby-Lervåg, Monica; Redick, Thomas S; Hulme, Charles

2016-07-01

It has been claimed that working memory training programs produce diverse beneficial effects. This article presents a meta-analysis of working memory training studies (with a pretest-posttest design and a control group) that have examined transfer to other measures (nonverbal ability, verbal ability, word decoding, reading comprehension, or arithmetic; 87 publications with 145 experimental comparisons). Immediately following training there were reliable improvements on measures of intermediate transfer (verbal and visuospatial working memory). For measures of far transfer (nonverbal ability, verbal ability, word decoding, reading comprehension, arithmetic) there was no convincing evidence of any reliable improvements when working memory training was compared with a treated control condition. Furthermore, mediation analyses indicated that across studies, the degree of improvement on working memory measures was not related to the magnitude of far-transfer effects found. Finally, analysis of publication bias shows that there is no evidential value from the studies of working memory training using treated controls. The authors conclude that working memory training programs appear to produce short-term, specific training effects that do not generalize to measures of "real-world" cognitive skills. These results seriously question the practical and theoretical importance of current computerized working memory programs as methods of training working memory skills. © The Author(s) 2016.

A data fusion approach for track monitoring from multiple in-service trains

NASA Astrophysics Data System (ADS)

Lederman, George; Chen, Siheng; Garrett, James H.; Kovačević, Jelena; Noh, Hae Young; Bielak, Jacobo

2017-10-01

We present a data fusion approach for enabling data-driven rail-infrastructure monitoring from multiple in-service trains. A number of researchers have proposed using vibration data collected from in-service trains as a low-cost method to monitor track geometry. The majority of this work has focused on developing novel features to extract information about the tracks from data produced by individual sensors on individual trains. We extend this work by presenting a technique to combine extracted features from multiple passes over the tracks from multiple sensors aboard multiple vehicles. There are a number of challenges in combining multiple data sources, like different relative position coordinates depending on the location of the sensor within the train. Furthermore, as the number of sensors increases, the likelihood that some will malfunction also increases. We use a two-step approach that first minimizes position offset errors through data alignment, then fuses the data with a novel adaptive Kalman filter that weights data according to its estimated reliability. We show the efficacy of this approach both through simulations and on a data-set collected from two instrumented trains operating over a one-year period. Combining data from numerous in-service trains allows for more continuous and more reliable data-driven monitoring than analyzing data from any one train alone; as the number of instrumented trains increases, the proposed fusion approach could facilitate track monitoring of entire rail-networks.

LipidFrag: Improving reliability of in silico fragmentation of lipids and application to the Caenorhabditis elegans lipidome

PubMed Central

Neumann, Steffen; Schmitt-Kopplin, Philippe

2017-01-01

Lipid identification is a major bottleneck in high-throughput lipidomics studies. However, tools for the analysis of lipid tandem MS spectra are rather limited. While the comparison against spectra in reference libraries is one of the preferred methods, these libraries are far from being complete. In order to improve identification rates, the in silico fragmentation tool MetFrag was combined with Lipid Maps and lipid-class specific classifiers which calculate probabilities for lipid class assignments. The resulting LipidFrag workflow was trained and evaluated on different commercially available lipid standard materials, measured with data dependent UPLC-Q-ToF-MS/MS acquisition. The automatic analysis was compared against manual MS/MS spectra interpretation. With the lipid class specific models, identification of the true positives was improved especially for cases where candidate lipids from different lipid classes had similar MetFrag scores by removing up to 56% of false positive results. This LipidFrag approach was then applied to MS/MS spectra of lipid extracts of the nematode Caenorhabditis elegans. Fragments explained by LipidFrag match known fragmentation pathways, e.g., neutral losses of lipid headgroups and fatty acid side chain fragments. Based on prediction models trained on standard lipid materials, high probabilities for correct annotations were achieved, which makes LipidFrag a good choice for automated lipid data analysis and reliability testing of lipid identifications. PMID:28278196

Organizing safety: conditions for successful information assurance programs.

PubMed

Collmann, Jeff; Coleman, Johnathan; Sostrom, Kristen; Wright, Willie

2004-01-01

Organizations must continuously seek safety. When considering computerized health information systems, "safety" includes protecting the integrity, confidentiality, and availability of information assets such as patient information, key components of the technical information system, and critical personnel. "High Reliability Theory" (HRT) argues that organizations with strong leadership support, continuous training, redundant safety mechanisms, and "cultures of high reliability" can deploy and safely manage complex, risky technologies such as nuclear weapons systems or computerized health information systems. In preparation for the Health Insurance Portability and Accountability Act (HIPAA) of 1996, the Office of the Assistant Secretary of Defense (Health Affairs), the Offices of the Surgeons General of the United States Army, Navy and Air Force, and the Telemedicine and Advanced Technology Research Center (TATRC), US Army Medical Research and Materiel Command sponsored organizational, doctrinal, and technical projects that individually and collectively promote conditions for a "culture of information assurance." These efforts include sponsoring the "P3 Working Group" (P3WG), an interdisciplinary, tri-service taskforce that reviewed all relevant Department of Defense (DoD), Miliary Health System (MHS), Army, Navy and Air Force policies for compliance with the HIPAA medical privacy and data security regulations; supporting development, training, and deployment of OCTAVE(sm), a self-directed information security risk assessment process; and sponsoring development of the Risk Information Management Resource (RIMR), a Web-enabled enterprise portal about health information assurance.

Is the rearfoot pattern the most frequently foot strike pattern among recreational shod distance runners?

PubMed

de Almeida, Matheus Oliveira; Saragiotto, Bruno Tirotti; Yamato, Tiê Parma; Lopes, Alexandre Dias

2015-02-01

To determine the distribution of the foot strike patterns among recreational shod runners and to compare the personal and training characteristics between runners with different foot strike patterns. Cross-sectional study. Areas of running practice in São Paulo, Brazil. 514 recreational shod runners older than 18 years and free of injury. Foot strike patterns were evaluated with a high-speed camera (250 Hz) and photocells to assess the running speed of participants. Personal and training characteristics were collected through a questionnaire. The inter-rater reliability of the visual foot strike pattern classification method was 96.7% and intra-rater reliability was 98.9%. 95.1% (n = 489) of the participants were rearfoot strikers, 4.1% (n = 21) were midfoot strikers, and four runners (0.8%) were forefoot strikers. There were no significant differences between strike patterns for personal and training characteristics. This is the first study to demonstrate that almost all recreational shod runners were rearfoot strikers. The visual method of evaluation seems to be a reliable and feasible option to classify foot strike pattern. Copyright © 2014 Elsevier Ltd. All rights reserved.

Ensembles of novelty detection classifiers for structural health monitoring using guided waves

NASA Astrophysics Data System (ADS)

Dib, Gerges; Karpenko, Oleksii; Koricho, Ermias; Khomenko, Anton; Haq, Mahmoodul; Udpa, Lalita

2018-01-01

Guided wave structural health monitoring uses sparse sensor networks embedded in sophisticated structures for defect detection and characterization. The biggest challenge of those sensor networks is developing robust techniques for reliable damage detection under changing environmental and operating conditions (EOC). To address this challenge, we develop a novelty classifier for damage detection based on one class support vector machines. We identify appropriate features for damage detection and introduce a feature aggregation method which quadratically increases the number of available training observations. We adopt a two-level voting scheme by using an ensemble of classifiers and predictions. Each classifier is trained on a different segment of the guided wave signal, and each classifier makes an ensemble of predictions based on a single observation. Using this approach, the classifier can be trained using a small number of baseline signals. We study the performance using Monte-Carlo simulations of an analytical model and data from impact damage experiments on a glass fiber composite plate. We also demonstrate the classifier performance using two types of baseline signals: fixed and rolling baseline training set. The former requires prior knowledge of baseline signals from all EOC, while the latter does not and leverages the fact that EOC vary slowly over time and can be modeled as a Gaussian process.

Fourteen years of progress testing in radiology residency training: experiences from The Netherlands.

PubMed

Rutgers, D R; van Raamt, F; van Lankeren, W; Ravesloot, C J; van der Gijp, A; Ten Cate, Th J; van Schaik, J P J

2018-05-01

To describe the development of the Dutch Radiology Progress Test (DRPT) for knowledge testing in radiology residency training in The Netherlands from its start in 2003 up to 2016. We reviewed all DRPTs conducted since 2003. We assessed key changes and events in the test throughout the years, as well as resident participation and dispensation for the DRPT, test reliability and discriminative power of test items. The DRPT has been conducted semi-annually since 2003, except for 2015 when one digital DRPT failed. Key changes in these years were improvements in test analysis and feedback, test digitalization (2013) and inclusion of test items on nuclear medicine (2016). From 2003 to 2016, resident dispensation rates increased (Pearson's correlation coefficient 0.74, P-value <0.01) to maximally 16 %. Cronbach´s alpha for test reliability varied between 0.83 and 0.93. The percentage of DRPT test items with negative item-rest-correlations, indicating relatively poor discriminative power, varied between 4 % and 11 %. Progress testing has proven feasible and sustainable in Dutch radiology residency training, keeping up with innovations in the radiological profession. Test reliability and discriminative power of test items have remained fair over the years, while resident dispensation rates have increased. • Progress testing allows for monitoring knowledge development from novice to senior trainee. • In postgraduate medical training, progress testing is used infrequently. • Progress testing is feasible and sustainable in radiology residency training.

Creating an infrastructure for training in the responsible conduct of research: the University of Pittsburgh's experience.

PubMed

Barnes, Barbara E; Friedman, Charles P; Rosenberg, Jerome L; Russell, Joanne; Beedle, Ari; Levine, Arthur S

2006-02-01

In response to public concerns about the consequences of research misconduct, academic institutions have become increasingly cognizant of the need to implement comprehensive, effective training in the responsible conduct of research (RCR) for faculty, staff, students, and external collaborators. The ability to meet this imperative is challenging as universities confront declining financial resources and increasing complexity of the research enterprise. The authors describe the University of Pittsburgh's design, implementation, and evaluation of a Web-based, institution-wide RCR training program called Research and Practice Fundamentals (RPF). This project, established in 2000, was embedded in the philosophy, organizational structure, and technology developed through the Integrated Advanced Information Management Systems grant from the National Library of Medicine. Utilizing a centralized, comprehensive approach, the RPF system provides an efficient mechanism for deploying content to a large, diverse cohort of learners and supports the needs of research administrators by providing access to information about who has successfully completed the training. During its first 3 years of operation, the RPF served over 17,000 users and issued more than 38,000 training certificates. The 18 modules that are currently available address issues required by regulatory mandates and other content areas important to the research community. RPF users report high levels of satisfaction with content and ease of using the system. Future efforts must explore methods to integrate non-RCR education and training into a centralized, cohesive structure. The University of Pittsburgh's experience with the RPF demonstrates the importance of developing an infrastructure for training that is comprehensive, scalable, reliable, centralized, affordable, and sustainable.

A Holistic Approach to Systems Development

NASA Technical Reports Server (NTRS)

Wong, Douglas T.

2008-01-01

Introduces a Holistic and Iterative Design Process. Continuous process but can be loosely divided into four stages. More effort spent early on in the design. Human-centered and Multidisciplinary. Emphasis on Life-Cycle Cost. Extensive use of modeling, simulation, mockups, human subjects, and proven technologies. Human-centered design doesn t mean the human factors discipline is the most important Disciplines should be involved in the design: Subsystem vendors, configuration management, operations research, manufacturing engineering, simulation/modeling, cost engineering, hardware engineering, software engineering, test and evaluation, human factors, electromagnetic compatibility, integrated logistics support, reliability/maintainability/availability, safety engineering, test equipment, training systems, design-to-cost, life cycle cost, application engineering etc. 9

Looking forward and back to relapse: implications for research and practice.

PubMed

Connors, G J; Longabaugh, R; Miller, W R

1996-12-01

In this commentary, the three principal investigators of the Relapse Replication and Extension Project (RREP) reflect on clinical and research implications of study findings from the three collaborating sites. A primary purpose of RREP was to study the reliability and validity of a taxonomy of relapse antecedents originally proposed by Marlatt two decades ago. Under the best of research conditions, with extensive training and practice, it was difficult to achieve reliability of coding with the original three-level system, although with only two levels of classification more reasonable albeit variable reliability was found. Modifications may improve the taxonomy's reliability, but RREP data indicate that a more appropriate strategy is to measure possible antecedents of relapse by continuous scales such as those provided by Annis, Heather and Litman. There is reasonably consistent evidence for two common antecedents of relapse: negative emotional states, and positive emotional states in a social context. Antecedents of relapse show only modest consistency within individuals from one occasion to the next. The causes to which clients attribute relapses may exert a significant effect on future drinking episodes. Stable and internal attributions, such as are commonly associated with a dispositional disease model, may serve to perpetuate relapse. From the RREP studies, the availability of coping skills appears to be a potent protective factor, and ineffective coping a consistent predictor of relapse. Implications for clinical research and practice are considered.

The Development and Validation of a Spanish Elicited Imitation Test of Oral Language Proficiency for the Missionary Training Center

ERIC Educational Resources Information Center

Thompson, Carrie A.

2013-01-01

The Missionary Training Center (MTC), affiliated with the Church of Jesus Christ of Latter-day Saints, needs a reliable and cost effective way to measure the oral language proficiency of missionaries learning Spanish. The MTC needed to measure incoming missionaries' Spanish language proficiency for training and classroom assignment as well as to…

The Effectiveness of a Training Program Based on Practice of Careers in Vocational Interests Development

ERIC Educational Resources Information Center

Mahasneh, Omar. M.; Farajat, Amani. M.

2015-01-01

The present research was conducted to identify the effectiveness of a training program based on practice of careers in vocational interests development, to answer questions about the study and test its hypothesis the training program had been prepared and the adoption of a measure of vocational interests, as validity and reliability of each of…

Reliability and Validity of a Submaximal Warm-up Test for Monitoring Training Status in Professional Soccer Players.

PubMed

Rabbani, Alireza; Kargarfard, Mehdi; Twist, Craig

2018-02-01

Rabbani, A, Kargarfard, M, and Twist, C. Reliability and validity of a submaximal warm-up test for monitoring training status in professional soccer players. J Strength Cond Res 32(2): 326-333, 2018-Two studies were conducted to assess the reliability and validity of a submaximal warm-up test (SWT) in professional soccer players. For the reliability study, 12 male players performed an SWT over 3 trials, with 1 week between trials. For the validity study, 14 players of the same team performed an SWT and a 30-15 intermittent fitness test (30-15IFT) 7 days apart. Week-to-week reliability in selected heart rate (HR) responses (exercise heart rate [HRex], heart rate recovery [HRR] expressed as the number of beats recovered within 1 minute [HRR60s], and HRR expressed as the mean HR during 1 minute [HRpost1]) was determined using the intraclass correlation coefficient (ICC) and typical error of measurement expressed as coefficient of variation (CV). The relationships between HR measures derived from the SWT and the maximal speed reached at the 30-15IFT (VIFT) were used to assess validity. The range for ICC and CV values was 0.83-0.95 and 1.4-7.0% in all HR measures, respectively, with the HRex as the most reliable HR measure of the SWT. Inverse large (r = -0.50 and 90% confidence limits [CLs] [-0.78 to -0.06]) and very large (r = -0.76 and CL, -0.90 to -0.45) relationships were observed between HRex and HRpost1 with VIFT in relative (expressed as the % of maximal HR) measures, respectively. The SWT is a reliable and valid submaximal test to monitor high-intensity intermittent running fitness in professional soccer players. In addition, the test's short duration (5 minutes) and simplicity mean that it can be used regularly to assess training status in high-level soccer players.

Rigorous Training of Dogs Leads to High Accuracy in Human Scent Matching-To-Sample Performance

PubMed Central

Marchal, Sophie; Bregeras, Olivier; Puaux, Didier; Gervais, Rémi; Ferry, Barbara

2016-01-01

Human scent identification is based on a matching-to-sample task in which trained dogs are required to compare a scent sample collected from an object found at a crime scene to that of a suspect. Based on dogs’ greater olfactory ability to detect and process odours, this method has been used in forensic investigations to identify the odour of a suspect at a crime scene. The excellent reliability and reproducibility of the method largely depend on rigor in dog training. The present study describes the various steps of training that lead to high sensitivity scores, with dogs matching samples with 90% efficiency when the complexity of the scents presented during the task in the sample is similar to that presented in the in lineups, and specificity reaching a ceiling, with no false alarms in human scent matching-to-sample tasks. This high level of accuracy ensures reliable results in judicial human scent identification tests. Also, our data should convince law enforcement authorities to use these results as official forensic evidence when dogs are trained appropriately. PMID:26863620

Evoking prescribed spike times in stochastic neurons

NASA Astrophysics Data System (ADS)

Doose, Jens; Lindner, Benjamin

2017-09-01

Single cell stimulation in vivo is a powerful tool to investigate the properties of single neurons and their functionality in neural networks. We present a method to determine a cell-specific stimulus that reliably evokes a prescribed spike train with high temporal precision of action potentials. We test the performance of this stimulus in simulations for two different stochastic neuron models. For a broad range of parameters and a neuron firing with intermediate firing rates (20-40 Hz) the reliability in evoking the prescribed spike train is close to its theoretical maximum that is mainly determined by the level of intrinsic noise.

76 FR 4912 - Proposed Information Collection Activity; Comment Request

Federal Register 2010, 2011, 2012, 2013, 2014

2011-01-27

... reliable evaluation design to produce accurate evidence of the effect of HPOG on individuals and health job training programs systems. The goals of the HPOG evaluation are to establish a performance management... grantee organizations (higher education Institutions, workforce investment boards, private training...

Magpies can use local cues to retrieve their food caches.

PubMed

Feenders, Gesa; Smulders, Tom V

2011-03-01

Much importance has been placed on the use of spatial cues by food-hoarding birds in the retrieval of their caches. In this study, we investigate whether food-hoarding birds can be trained to use local cues ("beacons") in their cache retrieval. We test magpies (Pica pica) in an active hoarding-retrieval paradigm, where local cues are always reliable, while spatial cues are not. Our results show that the birds use the local cues to retrieve their caches, even when occasionally contradicting spatial information is available. The design of our study does not allow us to test rigorously whether the birds prefer using local over spatial cues, nor to investigate the process through which they learn to use local cues. We furthermore provide evidence that magpies develop landmark preferences, which improve their retrieval accuracy. Our findings support the hypothesis that birds are flexible in their use of memory information, using a combination of the most reliable or salient information to retrieve their caches. © Springer-Verlag 2010

Integrating Machine Learning into a Crowdsourced Model for Earthquake-Induced Damage Assessment

NASA Technical Reports Server (NTRS)

Rebbapragada, Umaa; Oommen, Thomas

2011-01-01

On January 12th, 2010, a catastrophic 7.0M earthquake devastated the country of Haiti. In the aftermath of an earthquake, it is important to rapidly assess damaged areas in order to mobilize the appropriate resources. The Haiti damage assessment effort introduced a promising model that uses crowdsourcing to map damaged areas in freely available remotely-sensed data. This paper proposes the application of machine learning methods to improve this model. Specifically, we apply work on learning from multiple, imperfect experts to the assessment of volunteer reliability, and propose the use of image segmentation to automate the detection of damaged areas. We wrap both tasks in an active learning framework in order to shift volunteer effort from mapping a full catalog of images to the generation of high-quality training data. We hypothesize that the integration of machine learning into this model improves its reliability, maintains the speed of damage assessment, and allows the model to scale to higher data volumes.

An empirical look at the Defense Mechanism Test (DMT): reliability and construct validity.

PubMed

Ekehammar, Bo; Zuber, Irena; Konstenius, Marja-Liisa

2005-07-01

Although the Defense Mechanism Test (DMT) has been in use for almost half a century, there are still quite contradictory views about whether it is a reliable instrument, and if so, what it really measures. Thus, based on data from 39 female students, we first examined DMT inter-coder reliability by analyzing the agreement among trained judges in their coding of the same DMT protocols. Second, we constructed a "parallel" photographic picture that retained all structural characteristic of the original and analyzed DMT parallel-test reliability. Third, we examined the construct validity of the DMT by (a) employing three self-report defense-mechanism inventories and analyzing the intercorrelations between DMT defense scores and corresponding defenses in these instruments, (b) studying the relationships between DMT responses and scores on trait and state anxiety, and (c) relating DMT-defense scores to measures of self-esteem. The main results showed that the DMT can be coded with high reliability by trained coders, that the parallel-test reliability is unsatisfactory compared to traditional psychometric standards, that there is a certain generalizability in the number of perceptual distortions that people display from one picture to another, and that the construct validation provided meager empirical evidence for the conclusion that the DMT measures what it purports to measure, that is, psychological defense mechanisms.

Interrater Reliability of the Supports Intensity Scale (SIS)

ERIC Educational Resources Information Center

Thompson, James R.; Tasse, Marc J.; McLaughlin, Colleen A.

2008-01-01

The interrater reliability of the Supports Intensity Scale (SIS) was investigated under the condition that interviewers had to have been trained and/or experienced in its administration and scoring. Both corrected and noncorrected Pearson's product-moment coefficients were generated to assess interinterviewer, interrespondent, and mixed interrater…

Field studies of safety security rescue technologies through training and response activities

NASA Astrophysics Data System (ADS)

Murphy, Robin R.; Stover, Sam

2006-05-01

This paper describes the field-oriented philosophy of the Institute for Safety Security Rescue Technology (iSSRT) and summarizes the activities and lessons learned during calendar year 2005 of its two centers: the Center for Robot-Assisted Search and Rescue and the NSF Safety Security Rescue industry/university cooperative research center. In 2005, iSSRT participated in four responses (La Conchita, CA, Mudslides, Hurricane Dennis, Hurricane Katrina, Hurricane Wilma) and conducted three field experiments (NJTF-1, Camp Hurricane, Richmond, MO). The lessons learned covered mobility, operator control units, wireless communications, and general reliability. The work has collectively identified six emerging issues for future work. Based on these studies, a 10-hour, 1 continuing education unit credit course on rescue robotics has been created and is available. Rescue robots and sensors are available for loan upon request.

W5″ Test: A simple method for measuring mean power output in the bench press exercise.

PubMed

Tous-Fajardo, Julio; Moras, Gerard; Rodríguez-Jiménez, Sergio; Gonzalo-Skok, Oliver; Busquets, Albert; Mujika, Iñigo

2016-11-01

The aims of the present study were to assess the validity and reliability of a novel simple test [Five Seconds Power Test (W5″ Test)] for estimating the mean power output during the bench press exercise at different loads, and its sensitivity to detect training-induced changes. Thirty trained young men completed as many repetitions as possible in a time of ≈5 s at 25%, 45%, 65% and 85% of one-repetition maximum (1RM) in two test sessions separated by four days. The number of repetitions, linear displacement of the bar and time needed to complete the test were recorded by two independent testers, and a linear encoder was used as the criterion measure. For each load, the mean power output was calculated in the W5″ Test as mechanical work per time unit and compared with that obtained from the linear encoder. Subsequently, 20 additional subjects (10 training group vs. 10 control group) were assessed before and after completing a seven-week training programme designed to improve maximal power. Results showed that both assessment methods correlated highly in estimating mean power output at different loads (r range: 0.86-0.94; p < .01) and detecting training-induced changes (R(2): 0.78). Good to excellent intra-tester (intraclass correlation coefficient (ICC) range: 0.81-0.97) and excellent inter-tester (ICC range: 0.96-0.99; coefficient of variation range: 2.4-4.1%) reliability was found for all loads. The W5″ Test was shown to be a valid, reliable and sensitive method for measuring mean power output during the bench press exercise in subjects who have previous resistance training experience.

Development, validity and reliability testing of the East Midlands Evaluation Tool (EMET) for measuring impacts on trainees' confidence and competence following end of life care training.

PubMed

Whittaker, B; Parry, R; Bird, L; Watson, S; Faull, C

2017-02-02

To develop, test and validate a versatile questionnaire, the East Midlands Evaluation Tool (EMET), for measuring effects of end of life care training events on trainees' self-reported confidence and competence. A paper-based questionnaire was designed on the basis of the English Department of Health's core competences for end of life care, with sections for completion pretraining, immediately post-training and also for longer term follow-up. Preliminary versions were field tested at 55 training events delivered by 13 organisations to 1793 trainees working in diverse health and social care backgrounds. Iterative rounds of development aimed to maximise relevance to events and trainees. Internal consistency was assessed by calculating interitem correlations on questionnaire responses during field testing. Content validity was assessed via qualitative content analysis of (1) responses to questionnaires completed by field tester trainers and (2) field notes from a workshop with a separate cohort of experienced trainers. Test-retest reliability was assessed via repeat administration to a cohort of student nurses. The EMET comprises 27 items with Likert-scaled responses supplemented with questions seeking free-text responses. It measures changes in self-assessed confidence and competence on 5 subscales: communication skills; assessment and care planning; symptom management; advance care planning; overarching values and knowledge. Test-retest reliability was found to be good, as was internal consistency: the questions successfully assess different aspects of the same underlying concept. The EMET provides a time-efficient, reliable and flexible means of evaluating effects of training on self-reported confidence and competence in the key elements of end of life care. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.

Assessing communication skills of clinical call handlers working at an out-of-hours centre: development of the RICE rating scale.

PubMed

Derkx, Hay P; Rethans, Jan-Joost E; Knottnerus, J André; Ram, Paul M

2007-05-01

Out-of-hours centres provide telephone support to patients with medical problems. In most of these centres specially-trained nurses handle incoming telephone calls. They assess patients' needs, the degree of urgency, and determine the level of care required. Assessment of the medical problem and the quality of 'care-by-phone' depend on the medical and communication skills of the call handlers. To develop a valid, reliable, and practical rating scale to evaluate the communication skills of call handlers working at an out-of-hours centre and to improve quality of communication. Qualitative study with focus groups followed by validation of the rating scale and measurement of reliability (internal consistency). Out-of-hours centres in the Netherlands. A focus group developed the rating scale. Experts with experience in training and evaluating communication skills of medical students and GPs commented on the scale to ensure content validity. The reliability of the rating scale was tested in a pilot in which ten specially-trained assessors scored six telephone calls each. The scale, known as the RICE rating scale, has 17 items divided over four different phases of the telephone consultation: Reason for calling; Information gathering; Conclusion; and Evaluation (RICE). Content validity of the scale was assessed by two experts. Reliability of the scale tested in the pilot was 0.73 (Cronbach's alpha). Establishing a rating scale to assess the communication skills of call handlers which meets common scientific demands, such as content validity and reliability, proved successful. This instrument can be used to give feedback to call handlers.

Test-Retest Reliability of a Novel Isokinetic Squat Device With Strength-Trained Athletes.

PubMed

Bridgeman, Lee A; McGuigan, Michael R; Gill, Nicholas D; Dulson, Deborah K

2016-11-01

Bridgeman, LA, McGuigan, MR, Gill, ND, and Dulson, DK. Test-retest reliability of a novel isokinetic squat device with strength-trained athletes. J Strength Cond Res 30(11): 3261-3265, 2016-The aim of this study was to investigate the test-retest reliability of a novel multijoint isokinetic squat device. The subjects in this study were 10 strength-trained athletes. Each subject completed 3 maximal testing sessions to assess peak concentric and eccentric force (N) over a 3-week period using the Exerbotics squat device. Mean differences between eccentric and concentric force across the trials were calculated. Intraclass correlation coefficients (ICCs) and coefficients of variation (CVs) for the variables of interest were calculated using an excel reliability spreadsheet. Between trials 1 and 2 an 11.0 and 2.3% increase in mean concentric and eccentric forces, respectively, was reported. Between trials 2 and 3 a 1.35% increase in the mean concentric force production and a 1.4% increase in eccentric force production was reported. The mean concentric peak force CV and ICC across the 3 trials was 10% (7.6-15.4) and 0.95 (0.87-0.98) respectively. However, the mean eccentric peak force CV and ICC across the trials was 7.2% (5.5-11.1) and 0.90 (0.76-0.97), respectively. Based on these findings it is suggested that the Exerbotics squat device shows good test-retest reliability. Therefore practitioners and investigators may consider its use to monitor changes in concentric and eccentric peak force.

Can Physicians Identify Inappropriate Nuclear Stress Tests? An Examination of Inter-rater Reliability for the 2009 Appropriate Use Criteria for Radionuclide Imaging

PubMed Central

Ye, Siqin; Rabbani, LeRoy E.; Kelly, Christopher R.; Kelly, Maureen R.; Lewis, Matthew; Paz, Yehuda; Peck, Clara L.; Rao, Shaline; Bokhari, Sabahat; Weiner, Shepard D.; Einstein, Andrew J.

2014-01-01

Background We sought to determine inter-rater reliability of the 2009 Appropriate Use Criteria (AUC) for radionuclide imaging (RNI) and whether physicians at various levels of training can effectively identify nuclear stress tests with inappropriate indications. Methods and Results Four hundred patients were randomly selected from a consecutive cohort of patients undergoing nuclear stress testing at an academic medical center. Raters with different levels of training (including cardiology attending physicians, cardiology fellows, internal medicine hospitalists, and internal medicine interns) classified individual nuclear stress tests using the 2009 AUC. Consensus classification by two cardiologists was considered the operational gold standard, and sensitivity and specificity of individual raters for identifying inappropriate tests was calculated. Inter-rater reliability of the AUC was assessed using Cohen’s kappa statistics for pairs of different raters. The mean age of patients was 61.5 years; 214 (54%) were female. The cardiologists rated 256 (64%) of 400 NSTs as appropriate, 68 (18%) as uncertain, 55 (14%) as inappropriate; 21 (5%) tests were unable to be classified. Inter-rater reliability for non-cardiologist raters was modest (unweighted Cohen’s kappa, 0.51, 95% confidence interval, 0.45 to 0.55). Sensitivity of individual raters for identifying inappropriate tests ranged from 47% to 82%, while specificity ranged from 85% to 97%. Conclusions Inter-rater reliability for the 2009 AUC for RNI is modest, and there is considerable variation in the ability of raters at different levels of training to identify inappropriate tests. PMID:25563660

Wearable Lactate Threshold Predicting Device is Valid and Reliable in Runners.

PubMed

Borges, Nattai R; Driller, Matthew W

2016-08-01

Borges, NR and Driller, MW. Wearable lactate threshold predicting device is valid and reliable in runners. J Strength Cond Res 30(8): 2212-2218, 2016-A commercially available device claiming to be the world's first wearable lactate threshold predicting device (WLT), using near-infrared LED technology, has entered the market. The aim of this study was to determine the levels of agreement between the WLT-derived lactate threshold workload and traditional methods of lactate threshold (LT) calculation and the interdevice and intradevice reliability of the WLT. Fourteen (7 male, 7 female; mean ± SD; age: 18-45 years, height: 169 ± 9 cm, mass: 67 ± 13 kg, V[Combining Dot Above]O2max: 53 ± 9 ml·kg·min) subjects ranging from recreationally active to highly trained athletes completed an incremental exercise test to exhaustion on a treadmill. Blood lactate samples were taken at the end of each 3-minute stage during the test to determine lactate threshold using 5 traditional methods from blood lactate analysis which were then compared against the WLT predicted value. In a subset of the population (n = 12), repeat trials were performed to determine both inter-reliability and intrareliability of the WLT device. Intraclass correlation coefficient (ICC) found high to very high agreement between the WLT and traditional methods (ICC > 0.80), with TEMs and mean differences ranging between 3.9-10.2% and 1.3-9.4%. Both interdevice and intradevice reliability resulted in highly reproducible and comparable results (CV < 1.2%, TEM <0.2 km·h, ICC > 0.97). This study suggests that the WLT is a practical, reliable, and noninvasive tool for use in predicting LT in runners.

Reliability of the ECHOWS Tool for Assessment of Patient Interviewing Skills.

PubMed

Boissonnault, Jill S; Evans, Kerrie; Tuttle, Neil; Hetzel, Scott J; Boissonnault, William G

2016-04-01

History taking is an important component of patient/client management. Assessment of student history-taking competency can be achieved via a standardized tool. The ECHOWS tool has been shown to be valid with modest intrarater reliability in a previous study but did not demonstrate sufficient power to definitively prove its stability. The purposes of this study were: (1) to assess the reliability of the ECHOWS tool for student assessment of patient interviewing skills and (2) to determine whether the tool discerns between novice and experienced skill levels. A reliability and construct validity assessment was conducted. Three faculty members from the United States and Australia scored videotaped histories from standardized patients taken by students and experienced clinicians from each of these countries. The tapes were scored twice, 3 to 6 weeks apart. Reliability was assessed using interclass correlation coefficients (ICCs) and repeated measures. Analysis of variance models assessed the ability of the tool to discern between novice and experienced skill levels. The ECHOWS tool showed excellent intrarater reliability (ICC [3,1]=.74-.89) and good interrater reliability (ICC [2,1]=.55) as a whole. The summary of performance (S) section showed poor interrater reliability (ICC [2,1]=.27). There was no statistical difference in performance on the tool between novice and experienced clinicians. A possible ceiling effect may occur when standardized patients are not coached to provide complex and obtuse responses to interviewer questions. Variation in familiarity with the ECHOWS tool and in use of the online training may have influenced scoring of the S section. The ECHOWS tool demonstrates excellent intrarater reliability and moderate interrater reliability. Sufficient training with the tool prior to student assessment is recommended. The S section must evolve in order to provide a more discerning measure of interviewing skills. © 2016 American Physical Therapy Association.

Fair and Just Culture, Team Behavior, and Leadership Engagement: The Tools to Achieve High Reliability

PubMed Central

Frankel, Allan S; Leonard, Michael W; Denham, Charles R

2006-01-01

Background Disparate health care provider attitudes about autonomy, teamwork, and administrative operations have added to the complexity of health care delivery and are a central factor in medicine's unacceptably high rate of errors. Other industries have improved their reliability by applying innovative concepts to interpersonal relationships and administrative hierarchical structures (Chandler 1962). In the last 10 years the science of patient safety has become more sophisticated, with practical concepts identified and tested to improve the safety and reliability of care. Objective Three initiatives stand out as worthy regarding interpersonal relationships and the application of provider concerns to shape operational change: The development and implementation of Fair and Just Culture principles, the broad use of Teamwork Training and Communication, and tools like WalkRounds that promote the alignment of leadership and frontline provider perspectives through effective use of adverse event data and provider comments. Methods Fair and Just Culture, Teamwork Training, and WalkRounds are described, and implementation examples provided. The argument is made that they must be systematically and consistently implemented in an integrated fashion. Conclusions There are excellent examples of institutions applying Just Culture principles, Teamwork Training, and Leadership WalkRounds—but to date, they have not been comprehensively instituted in health care organizations in a cohesive and interdependent manner. To achieve reliability, organizations need to begin thinking about the relationship between these efforts and linking them conceptually. PMID:16898986

Nerve ultrasound reliability of upper limbs: Effects of examiner training.

PubMed

Garcia-Santibanez, Rocio; Dietz, Alexander R; Bucelli, Robert C; Zaidman, Craig M

2018-02-01

Duration of training to reliably measure nerve cross-sectional area with ultrasound is unknown. A retrospective review was performed of ultrasound data, acquired and recorded by 2 examiners-an expert and either a trainee with 2 months (novice) or a trainee with 12 months (experienced) of experience. Data on median, ulnar, and radial nerves were reviewed for 42 patients. Interrater reliability was good and varied most with nerve site but little with experience. Coefficient of variation (CoV) range was 9.33%-22.5%. Intraclass correlation coefficient (ICC) was good to excellent (0.65-95) except ulnar nerve-wrist/forearm and radial nerve-humerus (ICC = 0.39-0.59). Interrater differences did not vary with nerve size or body mass index. Expert-novice and expert-experienced interrater differences and CoV were similar. The ulnar nerve-wrist expert-novice interrater difference decreased with time (r s  = -0.68, P = 0.001). A trainee with at least 2 months of experience can reliably measure upper limb nerves. Reliability varies by nerve and location and slightly improves with time. Muscle Nerve 57: 189-192, 2018. © 2017 Wiley Periodicals, Inc.

76 FR 10624 - Agency Information Collection Activities; Submission for OMB Review; Comment Request; Employment...

Federal Register 2010, 2011, 2012, 2013, 2014

2011-02-25

...) sponsored information collection request (ICR) titled, ``Employment and Training Data Validation Requirement... report accurate and reliable program and financial information. Data validation requires States and... Employment Program. The Employment and Training Data Validation Requirement is an information collection is...

Impact of pandemic flu training on ability of medical personnel to recognize an index case of avian influenza.

PubMed

Adini, Bruria; Goldberg, Avishay; Cohen, Robert; Bar-Dayan, Yaron

2012-04-01

This study investigated the relationship between training programmes for pandemic flu and level of knowledge of health-care professionals with performance in an avian flu exercise. Training programmes of all general hospitals in Israel for managing a pandemic influenza were evaluated. Spearman's ρ correlation was used to analyse the relationship between training scores and level of knowledge of medical personnel with performance in an avian flu exercise. Hospital preparedness levels were evaluated at two time points and Wilcoxon signed-rank test was used to determine if overall preparedness scores improved over time. Evaluation of training programmes for pandemic influenza showed high to very high scores in most hospitals (mean 85, SD 22). Significant correlations between training and performance in the exercise were noted for: implementation of training programmes 0.91, P = 0.000; designating personnel for training 0.87, P = 0.000; content of training 0.61, P = 0.001; and training materials 0.36, P = 0.05. Overall reliability of the evaluation scores was 0.82 and reliability for two of the sub-scales was: implementation of the programme 0.78; and designating personnel for training 0.37. No significant correlation was found between level of knowledge and performance in the exercise. Training programmes for hospital personnel for pandemic flu have a significant role in improving performance in case of pandemic flu. The key component of the training programme appears to be the implementation of the programme. Use of knowledge tests should be further investigated, as they do not appear to correlate with the level of emergency preparedness for pandemic influenza.

The need to approximate the use-case in clinical machine learning

PubMed Central

Saeb, Sohrab; Jayaraman, Arun; Mohr, David C.; Kording, Konrad P.

2017-01-01

Abstract The availability of smartphone and wearable sensor technology is leading to a rapid accumulation of human subject data, and machine learning is emerging as a technique to map those data into clinical predictions. As machine learning algorithms are increasingly used to support clinical decision making, it is vital to reliably quantify their prediction accuracy. Cross-validation (CV) is the standard approach where the accuracy of such algorithms is evaluated on part of the data the algorithm has not seen during training. However, for this procedure to be meaningful, the relationship between the training and the validation set should mimic the relationship between the training set and the dataset expected for the clinical use. Here we compared two popular CV methods: record-wise and subject-wise. While the subject-wise method mirrors the clinically relevant use-case scenario of diagnosis in newly recruited subjects, the record-wise strategy has no such interpretation. Using both a publicly available dataset and a simulation, we found that record-wise CV often massively overestimates the prediction accuracy of the algorithms. We also conducted a systematic review of the relevant literature, and found that this overly optimistic method was used by almost half of the retrieved studies that used accelerometers, wearable sensors, or smartphones to predict clinical outcomes. As we move towards an era of machine learning-based diagnosis and treatment, using proper methods to evaluate their accuracy is crucial, as inaccurate results can mislead both clinicians and data scientists. PMID:28327985

A unified approach to validation, reliability, and education study design for surgical technical skills training.

PubMed

Sweet, Robert M; Hananel, David; Lawrenz, Frances

2010-02-01

To present modern educational psychology theory and apply these concepts to validity and reliability of surgical skills training and assessment. In a series of cross-disciplinary meetings, we applied a unified approach of behavioral science principles and theory to medical technical skills education given the recent advances in the theories in the field of behavioral psychology and statistics. While validation of the individual simulation tools is important, it is only one piece of a multimodal curriculum that in and of itself deserves examination and study. We propose concurrent validation throughout the design of simulation-based curriculum rather than once it is complete. We embrace the concept that validity and curriculum development are interdependent, ongoing processes that are never truly complete. Individual predictive, construct, content, and face validity aspects should not be considered separately but as interdependent and complementary toward an end application. Such an approach could help guide our acceptance and appropriate application of these exciting new training and assessment tools for technical skills training in medicine.

Perceptual learning increases the strength of the earliest signals in visual cortex.

PubMed

Bao, Min; Yang, Lin; Rios, Cristina; He, Bin; Engel, Stephen A

2010-11-10

Training improves performance on most visual tasks. Such perceptual learning can modify how information is read out from, and represented in, later visual areas, but effects on early visual cortex are controversial. In particular, it remains unknown whether learning can reshape neural response properties in early visual areas independent from feedback arising in later cortical areas. Here, we tested whether learning can modify feedforward signals in early visual cortex as measured by the human electroencephalogram. Fourteen subjects were trained for >24 d to detect a diagonal grating pattern in one quadrant of the visual field. Training improved performance, reducing the contrast needed for reliable detection, and also reliably increased the amplitude of the earliest component of the visual evoked potential, the C1. Control orientations and locations showed smaller effects of training. Because the C1 arises rapidly and has a source in early visual cortex, our results suggest that learning can increase early visual area response through local receptive field changes without feedback from later areas.

Prior Individual Training and Self-Organized Queuing during Group Emergency Escape of Mice from Water Pool

PubMed Central

Saloma, Caesar; Perez, Gay Jane; Gavile, Catherine Ann; Ick-Joson, Jacqueline Judith; Palmes-Saloma, Cynthia

2015-01-01

We study the impact of prior individual training during group emergency evacuation using mice that escape from an enclosed water pool to a dry platform via any of two possible exits. Experimenting with mice avoids serious ethical and legal issues that arise when dealing with unwitting human participants while minimizing concerns regarding the reliability of results obtained from simulated experiments using ‘actors’. First, mice were trained separately and their individual escape times measured over several trials. Mice learned quickly to swim towards an exit–they achieved their fastest escape times within the first four trials. The trained mice were then placed together in the pool and allowed to escape. No two mice were permitted in the pool beforehand and only one could pass through an exit opening at any given time. At first trial, groups of trained mice escaped seven and five times faster than their corresponding control groups of untrained mice at pool occupancy rate ρ of 11.9% and 4%, respectively. Faster evacuation happened because trained mice: (a) had better recognition of the available pool space and took shorter escape routes to an exit, (b) were less likely to form arches that blocked an exit opening, and (c) utilized the two exits efficiently without preference. Trained groups achieved continuous egress without an apparent leader-coordinator (self-organized queuing)—a collective behavior not experienced during individual training. Queuing was unobserved in untrained groups where mice were prone to wall seeking, aimless swimming and/or blind copying that produced circuitous escape routes, biased exit use and clogging. The experiments also reveal that faster and less costly group training at ρ = 4%, yielded an average individual escape time that is comparable with individualized training. However, group training in a more crowded pool (ρ = 11.9%) produced a longer average individual escape time. PMID:25693170

77 FR 47670 - Amended Certification Regarding Eligibility To Apply for Worker Adjustment Assistance

Federal Register 2010, 2011, 2012, 2013, 2014

2012-08-09

... DEPARTMENT OF LABOR Employment and Training Administration [TA-W-75,151; TA-W-75,151A] Amended... Reliability Center, A Subsidiary of Navistar International Corporation, Truck Division, Including All On-Site... Reliability Center, a Subsidiary of Navistar International Corporation, Truck Division, 3033 Wayne Trace, Fort...

76 FR 60536 - Notice of Quarterly Report (April 1, 2011-June 30, 2011)

Federal Register 2010, 2011, 2012, 2013, 2014

2011-09-29

... Number of farmers of cultivation, trained in commercial services to agriculture. agriculture and Number.... Knowledge of good hygiene practices. Households with reliable water services. Enterprises with reliable... Antananarivo. Average time for Land Services Offices to issue a duplicate copy of a title. Average cost to a...

Assessing the Assessment: Rubrics Training for Pre-Service and New In-Service Teachers

ERIC Educational Resources Information Center

Lovorn, Michael G.; Rezaei, Ali Reza

2011-01-01

Recent studies report that the use of rubrics may not improve the reliability of assessment if raters are not well trained on how to design and employ them effectively. The intent of this two-phase study was to test if training pre-service and new in-service teachers in the construction, use, and evaluation of rubrics would improve the reliability…

Surgeons' attitude toward a competency-based training and assessment program: results of a multicenter survey.

PubMed

Hopmans, Cornelis J; den Hoed, Pieter T; Wallenburg, Iris; van der Laan, Lijkckle; van der Harst, Erwin; van der Elst, Maarten; Mannaerts, Guido H H; Dawson, Imro; van Lanschot, Jan J B; Ijzermans, Jan N M

2013-01-01

Currently, most surgical training programs are focused on the development and evaluation of professional competencies. Also in the Netherlands, competency-based training and assessment programs were introduced to restructure postgraduate medical training. The current surgical residency program is based on the Canadian Medical Education Directives for Specialists (CanMEDS) competencies and uses assessment tools to evaluate residents' competence progression. In this study, we examined the attitude of surgical residents and attending surgeons toward a competency-based training and assessment program used to restructure general surgical training in the Netherlands in 2009. In 2011, all residents (n = 51) and attending surgeons (n = 108) in 1 training region, consisting of 7 hospitals, were surveyed. Participants were asked to rate the importance of the CanMEDS competencies and the suitability of the adopted assessment tools. Items were rated on a 5-point Likert scale and considered relevant when at least 80% of the respondents rated an item with a score of 4 or 5 (indicating a positive attitude). Reliability was evaluated by calculating the Cronbach's α, and the Mann-Whitney test was applied to assess differences between groups. The response rate was 88% (n = 140). The CanMEDS framework demonstrated good reliability (Cronbach's α = 0.87). However, the importance of the competencies 'Manager' (78%) and 'Health Advocate' (70%) was undervalued. The assessment tools failed to achieve an acceptable reliability (Cronbach's α = 0.55), and individual tools were predominantly considered unsuitable for assessment. Exceptions were the tools 'in-training evaluation report' (91%) and 'objective structured assessment of technical skill' (82%). No significant differences were found between the residents and the attending surgeons. This study has demonstrated that, 2 years after the reform of the general surgical residency program, residents and attending surgeons in a large training region in the Netherlands do not acknowledge the importance of all CanMEDS competencies and consider the assessment tools generally unsuitable for competence evaluation. Copyright © 2013 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Reliability and validity of the adapted Resistance Training Skills Battery for Children.

PubMed

Furzer, Bonnie J; Bebich-Philip, Marc D; Wright, Kemi E; Reid, Siobhan L; Thornton, Ashleigh L

2017-12-29

Resistance training (RT) is emerging as a training modality to improve motor function and facilitate physical activity participation in children across the motor proficiency spectrum. Although RT competency assessments have been established and validated among adolescent cohorts, the extent to which these methods are suitable for assessing children's RT skills is unknown. This project aimed to assess the psychometric properties of the adapted Resistance Training Skills Battery for Children (RTSBc), in children with varying motor proficiency. Repeated measures design with 40 participants (M age=8.2±1.7years) displaying varying levels of motor proficiency. Participants performed the adapted RTSBc on two occasions, receiving a score for their execution of each component, in addition to an overall RT skill quotient child (RTSQc). Cronbach's alpha, intra-class correlation (ICC), Bland-Altman analysis, and typical error were used to assess test-retest reliability. To examine construct validity, exploratory factor analysis was performed alongside computing correlations between participants' muscle strength, motor proficiency, age, lean muscle mass, and RTSQc. The RTSBc displayed an acceptable level of internal consistency (alpha=0.86) and test-retest reliability (ICC range=0.86-0.99). Exploratory factor analysis supported internal test structure, with all six RT skills loading strongly on a single factor (range 0.56-0.89). Analyses of structural validity revealed positive correlations for RTSQc in relation to motor proficiency (r=0.52, p<0.001) and strength scores (r=0.61, p<0.001). Analyses revealed support for the construct validity and test-retest reliability of the RTSBc, providing preliminary evidence that the RTSBc is appropriate for use in the assessment of children's RT competency. Copyright © 2018 Sports Medicine Australia. Published by Elsevier Ltd. All rights reserved.

[Multicenter validation of an evaluation tool for clinical training activities (SVAT) of the nursing students].

PubMed

Finotto, Sergio; Gradellini, Cinzia; Bandini, Stefania; Burrai, Francesco; Lucchi Casadei, Sandra; Villani, Carolina; Vincenzi, Simone; Mecugni, Daniela

2017-01-01

To evaluate the psychometric characteristics of the Scheda di Valutazione delle Attività di Tirocinio (SVAT). The degree courses in Nursing of the University of Modena and Reggio Emilia, site of Reggio Emilia, the University of Bologna Formative Section BO1, Imola and training center of Cesena, the University of Ferrara training centers of Ferrara and Codigoro were all enrolled in the research. For the content validation the reactive Delphi method was chosen. The panel of experts expressed a qualitative-intuitive judgment on the adequacy of language and on the stimulus material (SVAT). For internal consistency Cronbach's alpha was calculated the. The test-retest method was used for the reliability of stability. all indicators of the SVAT have achieved a degree of consensus not less than 80% demonstrating its content validity. The face validity is demonstrated by an average score equal to or greater than 7 obtained by all indicators. The reliability of internal consistency of the SVAT was appraised by Cronbach's alpha that was 0.987 for the entire instrument. The reliability of the stability has been calculated through the correlation's coefficient expressed by Pearson's r that was 0.983 (p = 1.3E-198). in Italy there is no a "gold standard" tool to evaluate the clinical performance of nursing students during and at the end of their clinical training. The SVAT proves to be a valuable and reliable tool it furthermore could stimulate the discussion and the debate among educators and nurses, so that also in our country, it may be possible develop and refine tools that support the evaluation of clinical skills of nursing students.

Reverse lactate threshold: a novel single-session approach to reliable high-resolution estimation of the anaerobic threshold.

PubMed

Dotan, Raffy

2012-06-01

The multisession maximal lactate steady-state (MLSS) test is the gold standard for anaerobic threshold (AnT) estimation. However, it is highly impractical, requires high fitness level, and suffers additional shortcomings. Existing single-session AnT-estimating tests are of compromised validity, reliability, and resolution. The presented reverse lactate threshold test (RLT) is a single-session, AnT-estimating test, aimed at avoiding the pitfalls of existing tests. It is based on the novel concept of identifying blood lactate's maximal appearance-disappearance equilibrium by approaching the AnT from higher, rather than from lower exercise intensities. Rowing, cycling, and running case data (4 recreational and competitive athletes, male and female, aged 17-39 y) are presented. Subjects performed the RLT test and, on a separate session, a single 30-min MLSS-type verification test at the RLT-determined intensity. The RLT and its MLSS verification exhibited exceptional agreement at 0.5% discrepancy or better. The RLT's training sensitivity was demonstrated by a case of 2.5-mo training regimen following which the RLT's 15-W improvement was fully MLSS-verified. The RLT's test-retest reliability was examined in 10 trained and untrained subjects. Test 2 differed from test 1 by only 0.3% with an intraclass correlation of 0.997. The data suggest RLT to accurately and reliably estimate AnT (as represented by MLSS verification) with high resolution and in distinctly different sports and to be sensitive to training adaptations. Compared with MLSS, the single-session RLT is highly practical and its lower fitness requirements make it applicable to athletes and untrained individuals alike. Further research is needed to establish RLT's validity and accuracy in larger samples.

Predicting space telerobotic operator training performance from human spatial ability assessment

NASA Astrophysics Data System (ADS)

Liu, Andrew M.; Oman, Charles M.; Galvan, Raquel; Natapoff, Alan

2013-11-01

Our goal was to determine whether existing tests of spatial ability can predict an astronaut's qualification test performance after robotic training. Because training astronauts to be qualified robotics operators is so long and expensive, NASA is interested in tools that can predict robotics performance before training begins. Currently, the Astronaut Office does not have a validated tool to predict robotics ability as part of its astronaut selection or training process. Commonly used tests of human spatial ability may provide such a tool to predict robotics ability. We tested the spatial ability of 50 active astronauts who had completed at least one robotics training course, then used logistic regression models to analyze the correlation between spatial ability test scores and the astronauts' performance in their evaluation test at the end of the training course. The fit of the logistic function to our data is statistically significant for several spatial tests. However, the prediction performance of the logistic model depends on the criterion threshold assumed. To clarify the critical selection issues, we show how the probability of correct classification vs. misclassification varies as a function of the mental rotation test criterion level. Since the costs of misclassification are low, the logistic models of spatial ability and robotic performance are reliable enough only to be used to customize regular and remedial training. We suggest several changes in tracking performance throughout robotics training that could improve the range and reliability of predictive models.

Autism detection in early childhood (ADEC): reliability and validity data for a Level 2 screening tool for autistic disorder.

PubMed

Nah, Yong-Hwee; Young, Robyn L; Brewer, Neil; Berlingeri, Genna

2014-03-01

The Autism Detection in Early Childhood (ADEC; Young, 2007) was developed as a Level 2 clinician-administered autistic disorder (AD) screening tool that was time-efficient, suitable for children under 3 years, easy to administer, and suitable for persons with minimal training and experience with AD. A best estimate clinical Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev.; DSM-IV-TR; American Psychiatric Association, 2000) diagnosis of AD was made for 70 children using all available information and assessment results, except for the ADEC data. A screening study compared these children on the ADEC with 57 children with other developmental disorders and 64 typically developing children. Results indicated high internal consistency (α = .91). Interrater reliability and test-retest reliability of the ADEC were also adequate. ADEC scores reliably discriminated different diagnostic groups after controlling for nonverbal IQ and Vineland Adaptive Behavior Composite scores. Construct validity (using exploratory factor analysis) and concurrent validity using performance on the Autism Diagnostic Observation Schedule (Lord et al., 2000), the Autism Diagnostic Interview-Revised (Le Couteur, Lord, & Rutter, 2003), and DSM-IV-TR criteria were also demonstrated. Signal detection analysis identified the optimal ADEC cutoff score, with the ADEC identifying all children who had an AD (N = 70, sensitivity = 1.0) but overincluding children with other disabilities (N = 13, specificity ranging from .74 to .90). Together, the reliability and validity data indicate that the ADEC has potential to be established as a suitable and efficient screening tool for infants with AD. 2014 APA

Objective structured clinical interview training using a virtual human patient.

PubMed

Parsons, Thomas D; Kenny, Patrick; Ntuen, Celestine A; Pataki, Caroly S; Pato, Michele T; Rizzo, Albert A; St-George, Cheryl; Sugar, Jeffery

2008-01-01

Effective interview skills are a core competency for psychiatry residents and developing psychotherapists. Although schools commonly make use of standardized patients to teach interview skills, the diversity of the scenarios standardized patients can characterize is limited by availability of human actors. Further, there is the economic concern related to the time and money needed to train standardized patients. Perhaps most damaging is the "standardization" of standardized patients -- will they in fact consistently proffer psychometrically reliable and valid interactions with the training clinicians. Virtual Human Agent (VHA) technology has evolved to a point where researchers may begin developing mental health applications that make use of virtual reality patients. The work presented here is a preliminary attempt at what we believe to be a large application area. Herein we describe an ongoing study of our virtual patients (VP). We present an approach that allows novice mental health clinicians to conduct an interview with a virtual character that emulates an adolescent male with conduct disorder. This study illustrates the ways in which a variety of core research components developed at the University of Southern California facilitates the rapid development of mental health applications.

AN-CASE NET-CENTRIC modeling and simulation

NASA Astrophysics Data System (ADS)

Baskinger, Patricia J.; Chruscicki, Mary Carol; Turck, Kurt

2009-05-01

The objective of mission training exercises is to immerse the trainees into an environment that enables them to train like they would fight. The integration of modeling and simulation environments that can seamlessly leverage Live systems, and Virtual or Constructive models (LVC) as they are available offers a flexible and cost effective solution to extending the "war-gaming" environment to a realistic mission experience while evolving the development of the net-centric enterprise. From concept to full production, the impact of new capabilities on the infrastructure and concept of operations, can be assessed in the context of the enterprise, while also exposing them to the warfighter. Training is extended to tomorrow's tools, processes, and Tactics, Techniques and Procedures (TTPs). This paper addresses the challenges of a net-centric modeling and simulation environment that is capable of representing a net-centric enterprise. An overview of the Air Force Research Laboratory's (AFRL) Airborne Networking Component Architecture Simulation Environment (AN-CASE) is provide as well as a discussion on how it is being used to assess technologies for the purpose of experimenting with new infrastructure mechanisms that enhance the scalability and reliability of the distributed mission operations environment.

Surface-enhanced Raman scattering detection of ammonium nitrate samples fabricated using drop-on-demand inkjet technology.

PubMed

Farrell, Mikella E; Holthoff, Ellen L; Pellegrino, Paul M

2014-01-01

The United States Army and the first responder community are increasingly focusing efforts on energetic materials detection and identification. Main hazards encountered in theater include homemade explosives and improvised explosive devices, in part fabricated from simple components like ammonium nitrate (AN). In order to accurately detect and identify these unknowns (energetic or benign), fielded detection systems must be accurately trained using well-understood universal testing substrates. These training substrates must contain target species at known concentrations and recognized polymorphic phases. Ammonium nitrate is an explosive precursor material that demonstrates several different polymorphic phases dependent upon how the material is deposited onto testing substrates. In this paper, known concentrations of AN were uniformly deposited onto commercially available surface-enhanced Raman scattering (SERS) substrates using a drop-on-demand inkjet printing system. The phase changes observed after the deposition of AN under several solvent conditions are investigated. Characteristics of the collected SERS spectra of AN are discussed, and it is demonstrated that an understanding of the exact nature of the AN samples deposited will result in an increased ability to accurately and reliably "train" hazard detection systems.

PubMed Central

Scarponi, L.; Pedrali, S.; Pizzorni, N.; Pinotti, C.; Foieni, F.; Zuccotti, G.; Schindler, A.

2017-01-01

SUMMARY The large majority of the available dysphagia screening tools has been developed for the stroke population. Only few screening tools are suitable for heterogeneous groups of patients admitted to a subacute care unit. The Royal Brisbane and Women's Hospital (RBWH) dysphagia screening tool is a nurse-administered, evidence-based swallow screening tool for generic acute hospital use that demonstrates excellent sensitivity and specificity. No Italian version of this tool is available to date. The aim of this study was to determine the reliability and screening accuracy of the Italian version of the RBWH (I-RBWH) dysphagia screening tool. A total of 105 patients consecutively admitted to a subacute care unit were enrolled. Using the I-RBWH tool, each patient was evaluated twice by trained nurses and once by a speech and language pathologist (SLP) blind to nurses' scores. The SLP also performed standardised clinical assessment of swallowing using the Mann assessment of swallowing ability (MASA). During the first and the second administration of the I-RBWH by nurses, 28 and 27 patients, respectively, were considered at risk of dysphagia, and 27 were considered at risk after SLP assessment. Intra- and inter-rater reliability was satisfactory. Comparison between nurse I-RBWH scores and MASA examination demonstrated a sensitivity and specificity of the I-RBWH dysphagia screening tool up to 93% and 96%, respectively; the positive and negative predictive values were 90% and 97%, respectively. Thus, the current findings support the reliability and accuracy of the I-RBWH tool for dysphagia screening of patients in subacute settings. Its application in clinical practice is recommended. PMID:28374867

Evidence-Based Diagnosis and Treatment of the Painful Sacroiliac Joint

PubMed Central

Laslett, Mark

2008-01-01

Sacroiliac joint (SIJ) pain refers to the pain arising from the SIJ joint structures. SIJ dysfunction generally refers to aberrant position or movement of SIJ structures that may or may not result in pain. This paper aims to clarify the difference between these clinical concepts and present current available evidence regarding diagnosis and treatment of SIJ disorders. Tests for SIJ dysfunction generally have poor inter-examiner reliability. A reference standard for SIJ dysfunction is not readily available, so validity of the tests for this disorder is unknown. Tests that stress the SIJ in order to provoke familiar pain have acceptable inter-examiner reliability and have clinically useful validity against an acceptable reference standard. It is unknown if provocation tests can reliably identify extra-articular SIJ sources of pain. Three or more positive pain provocation SIJ tests have sensitivity and specificity of 91% and 78%, respectively. Specificity of three or more positive tests increases to 87% in patients whose symptoms cannot be made to move towards the spinal midline, i.e., centralize. In chronic back pain populations, patients who have three or more positive provocation SIJ tests and whose symptoms cannot be made to centralize have a probability of having SIJ pain of 77%, and in pregnant populations with back pain, a probability of 89%. This combination of test findings could be used in research to evaluate the efficacy of specific treatments for SIJ pain. Treatments most likely to be effective are specific lumbopelvic stabilization training and injections of corticosteroid into the intra-articular space. PMID:19119403

Objective structured assessment of nontechnical skills: Reliability of a global rating scale for the in-training assessment in the operating room.

PubMed

Dedy, Nicolas J; Szasz, Peter; Louridas, Marisa; Bonrath, Esther M; Husslein, Heinrich; Grantcharov, Teodor P

2015-06-01

Nontechnical skills are critical for patient safety in the operating room (OR). As a result, regulatory bodies for accreditation and certification have mandated the integration of these competencies into postgraduate education. A generally accepted approach to the in-training assessment of nontechnical skills, however, is lacking. The goal of the present study was to develop an evidence-based and reliable tool for the in-training assessment of residents' nontechnical performance in the OR. The Objective Structured Assessment of Nontechnical Skills tool was designed as a 5-point global rating scale with descriptive anchors for each item, based on existing evidence-based frameworks of nontechnical skills, as well as resident training requirements. The tool was piloted on scripted videos and refined in an iterative process. The final version was used to rate residents' performance in recorded OR crisis simulations and during live observations in the OR. A total of 37 simulations and 10 live procedures were rated. Interrater agreement was good for total mean scores, both in simulation and in the real OR, with intraclass correlation coefficients >0.90 in all settings for average and single measures. Internal consistency of the scale was high (Cronbach's alpha = 0.80). The Objective Structured Assessment of Nontechnical Skills global rating scale was developed as an evidence-based tool for the in-training assessment of residents' nontechnical performance in the OR. Unique descriptive anchors allow for a criterion-referenced assessment of performance. Good reliability was demonstrated in different settings, supporting applications in research and education. Copyright © 2015 Elsevier Inc. All rights reserved.

Development of Measurability and Importance Scales for the NATA Athletic Training Educational Competencies.

ERIC Educational Resources Information Center

Wolfe, Edward W.; Nogle, Sally

2002-01-01

Developed and validated an instrument designed to measure the perceived measurability and importance of the National Athletic Trainers' Association Athletic Training Educational Competencies. Data from 931 athletic trainers and sport medicine physicians support 6 constructs, each of which demonstrates high reliability. (SLD)

Clinical pastoral education as professional training: some entrance, curriculum and assessment implications.

PubMed

Little, N Keith

2010-01-01

Clinical Pastoral Education is professional training for pastoral care. This paper compares CPE against the professional training model. While limiting the discussion to Christian pastoral care, the professional education model suggests a clarification of the trainee's theological and other entry requirements for a basic unit, a more thoughtful provision of information during CPE training, a careful attention to group membership and an appropriate integration with the theological curriculum. It also suggests more specific competency standards and more reliable, valid and objective assessment methods.

TuMore: generation of synthetic brain tumor MRI data for deep learning based segmentation approaches

NASA Astrophysics Data System (ADS)

Lindner, Lydia; Pfarrkirchner, Birgit; Gsaxner, Christina; Schmalstieg, Dieter; Egger, Jan

2018-03-01

Accurate segmentation and measurement of brain tumors plays an important role in clinical practice and research, as it is critical for treatment planning and monitoring of tumor growth. However, brain tumor segmentation is one of the most challenging tasks in medical image analysis. Since manual segmentations are subjective, time consuming and neither accurate nor reliable, there exists a need for objective, robust and fast automated segmentation methods that provide competitive performance. Therefore, deep learning based approaches are gaining interest in the field of medical image segmentation. When the training data set is large enough, deep learning approaches can be extremely effective, but in domains like medicine, only limited data is available in the majority of cases. Due to this reason, we propose a method that allows to create a large dataset of brain MRI (Magnetic Resonance Imaging) images containing synthetic brain tumors - glioblastomas more specifically - and the corresponding ground truth, that can be subsequently used to train deep neural networks.

Toward automated face detection in thermal and polarimetric thermal imagery

NASA Astrophysics Data System (ADS)

Gordon, Christopher; Acosta, Mark; Short, Nathan; Hu, Shuowen; Chan, Alex L.

2016-05-01

Visible spectrum face detection algorithms perform pretty reliably under controlled lighting conditions. However, variations in illumination and application of cosmetics can distort the features used by common face detectors, thereby degrade their detection performance. Thermal and polarimetric thermal facial imaging are relatively invariant to illumination and robust to the application of makeup, due to their measurement of emitted radiation instead of reflected light signals. The objective of this work is to evaluate a government off-the-shelf wavelet based naïve-Bayes face detection algorithm and a commercial off-the-shelf Viola-Jones cascade face detection algorithm on face imagery acquired in different spectral bands. New classifiers were trained using the Viola-Jones cascade object detection framework with preprocessed facial imagery. Preprocessing using Difference of Gaussians (DoG) filtering reduces the modality gap between facial signatures across the different spectral bands, thus enabling more correlated histogram of oriented gradients (HOG) features to be extracted from the preprocessed thermal and visible face images. Since the availability of training data is much more limited in the thermal spectrum than in the visible spectrum, it is not feasible to train a robust multi-modal face detector using thermal imagery alone. A large training dataset was constituted with DoG filtered visible and thermal imagery, which was subsequently used to generate a custom trained Viola-Jones detector. A 40% increase in face detection rate was achieved on a testing dataset, as compared to the performance of a pre-trained/baseline face detector. Insights gained in this research are valuable in the development of more robust multi-modal face detectors.

An imputed genotype resource for the laboratory mouse

PubMed Central

Szatkiewicz, Jin P.; Beane, Glen L.; Ding, Yueming; Hutchins, Lucie; de Villena, Fernando Pardo-Manuel; Churchill, Gary A.

2009-01-01

We have created a high-density SNP resource encompassing 7.87 million polymorphic loci across 49 inbred mouse strains of the laboratory mouse by combining data available from public databases and training a hidden Markov model to impute missing genotypes in the combined data. The strong linkage disequilibrium found in dense sets of SNP markers in the laboratory mouse provides the basis for accurate imputation. Using genotypes from eight independent SNP resources, we empirically validated the quality of the imputed genotypes and demonstrate that they are highly reliable for most inbred strains. The imputed SNP resource will be useful for studies of natural variation and complex traits. It will facilitate association study designs by providing high density SNP genotypes for large numbers of mouse strains. We anticipate that this resource will continue to evolve as new genotype data become available for laboratory mouse strains. The data are available for bulk download or query at http://cgd.jax.org/. PMID:18301946

RNABindRPlus: a predictor that combines machine learning and sequence homology-based methods to improve the reliability of predicted RNA-binding residues in proteins.

PubMed

Walia, Rasna R; Xue, Li C; Wilkins, Katherine; El-Manzalawy, Yasser; Dobbs, Drena; Honavar, Vasant

2014-01-01

Protein-RNA interactions are central to essential cellular processes such as protein synthesis and regulation of gene expression and play roles in human infectious and genetic diseases. Reliable identification of protein-RNA interfaces is critical for understanding the structural bases and functional implications of such interactions and for developing effective approaches to rational drug design. Sequence-based computational methods offer a viable, cost-effective way to identify putative RNA-binding residues in RNA-binding proteins. Here we report two novel approaches: (i) HomPRIP, a sequence homology-based method for predicting RNA-binding sites in proteins; (ii) RNABindRPlus, a new method that combines predictions from HomPRIP with those from an optimized Support Vector Machine (SVM) classifier trained on a benchmark dataset of 198 RNA-binding proteins. Although highly reliable, HomPRIP cannot make predictions for the unaligned parts of query proteins and its coverage is limited by the availability of close sequence homologs of the query protein with experimentally determined RNA-binding sites. RNABindRPlus overcomes these limitations. We compared the performance of HomPRIP and RNABindRPlus with that of several state-of-the-art predictors on two test sets, RB44 and RB111. On a subset of proteins for which homologs with experimentally determined interfaces could be reliably identified, HomPRIP outperformed all other methods achieving an MCC of 0.63 on RB44 and 0.83 on RB111. RNABindRPlus was able to predict RNA-binding residues of all proteins in both test sets, achieving an MCC of 0.55 and 0.37, respectively, and outperforming all other methods, including those that make use of structure-derived features of proteins. More importantly, RNABindRPlus outperforms all other methods for any choice of tradeoff between precision and recall. An important advantage of both HomPRIP and RNABindRPlus is that they rely on readily available sequence and sequence-derived features of RNA-binding proteins. A webserver implementation of both methods is freely available at http://einstein.cs.iastate.edu/RNABindRPlus/.

Combining two open source tools for neural computation (BioPatRec and Netlab) improves movement classification for prosthetic control.

PubMed

Prahm, Cosima; Eckstein, Korbinian; Ortiz-Catalan, Max; Dorffner, Georg; Kaniusas, Eugenijus; Aszmann, Oskar C

2016-08-31

Controlling a myoelectric prosthesis for upper limbs is increasingly challenging for the user as more electrodes and joints become available. Motion classification based on pattern recognition with a multi-electrode array allows multiple joints to be controlled simultaneously. Previous pattern recognition studies are difficult to compare, because individual research groups use their own data sets. To resolve this shortcoming and to facilitate comparisons, open access data sets were analysed using components of BioPatRec and Netlab pattern recognition models. Performances of the artificial neural networks, linear models, and training program components were compared. Evaluation took place within the BioPatRec environment, a Matlab-based open source platform that provides feature extraction, processing and motion classification algorithms for prosthetic control. The algorithms were applied to myoelectric signals for individual and simultaneous classification of movements, with the aim of finding the best performing algorithm and network model. Evaluation criteria included classification accuracy and training time. Results in both the linear and the artificial neural network models demonstrated that Netlab's implementation using scaled conjugate training algorithm reached significantly higher accuracies than BioPatRec. It is concluded that the best movement classification performance would be achieved through integrating Netlab training algorithms in the BioPatRec environment so that future prosthesis training can be shortened and control made more reliable. Netlab was therefore included into the newest release of BioPatRec (v4.0).

The interrater and intrarater reliability of the Philpott-Javer staging system based on level of training.

PubMed

Parhar, Harman S; Thamboo, Andrew; Habib, Al-Rahim; Chang, Brent; Gan, Eng Cern; Javer, Amin R

2014-04-01

The Philpott-Javer postoperative endoscopic mucosal staging system for allergic fungal rhinosinusitis has previously demonstrated acceptable interrater reliability among rhinologists. There are, however, numerous learners involved in patient care at tertiary centers. This study aims to analyze the interrater and intrarater reliability of this system among learners in otolaryngology at different stages in training. A prospective analysis of retrospectively collected endoscopic photographs. A tertiary care teaching hospital (January 2013). Fifty patients undergoing routine follow-up. Three photographs from each of 50 patients undergoing routine postsurgical nasoendoscopy were reviewed. Images were played twice, 1 week apart, in 2 differently randomized cycles and scored according to Philpott-Javer criteria by a rhinologist, a rhinology fellow, a senior otolaryngology resident, a junior otolaryngology resident, and a medical student. Interobserver reliability was assessed using the intraclass correlation coefficient, while intrarater reliability was assessed by Shrout-Fleiss κ values. Agreement between each learner and the rhinologist was also assessed using κ values. The interclass correlation among the 5 raters was 0.7600 (95% confidence interval, 0.6917-0.8161) for the Philpott-Javer scoring system, suggesting substantial reliability. Intrarater data showed substantial to almost-perfect reliability (κ values between 0.668 and 0.815) among all raters using this system. There was also moderate to substantial agreement between the learners and the rhinologist (κ values between 0.534 and 0.710). Results suggest that the Philpott-Javer staging system has acceptable intrarater and interrater reliability among learners of differing levels of clinical experience and is suitable for evaluating progress following surgery.

Interrater and Intrarater Reliability of the Tuck Jump Assessment by Health Professionals of Varied Educational Backgrounds

PubMed Central

Dudley, Lisa A.; Smith, Craig A.; Olson, Brandon K.; Chimera, Nicole J.

2013-01-01

Objective. The Tuck Jump Assessment (TJA), a clinical plyometric assessment, identifies 10 jumping and landing technique flaws. The study objective was to investigate TJA interrater and intrarater reliability with raters of different educational and clinical backgrounds. Methods. 40 participants were video recorded performing the TJA using published protocol and instructions. Five raters of varied educational and clinical backgrounds scored the TJA. Each score of the 10 technique flaws was summed for the total TJA score. Approximately one month later, 3 raters scored the videos again. Intraclass correlation coefficients determined interrater (5 and 3 raters for first and second session, resp.) and intrarater (3 raters) reliability. Results. Interrater reliability with 5 raters was poor (ICC = 0.47; 95% confidence intervals (CI) 0.33–0.62). Interrater reliability between 3 raters who completed 2 scoring sessions improved from 0.52 (95% CI 0.35–0.68) for session one to 0.69 (95% CI 0.55–0.81) for session two. Intrarater reliability was poor to moderate, ranging from 0.44 (95% CI 0.22–0.68) to 0.72 (95% CI 0.55–0.84). Conclusion. Published protocol and training of raters were insufficient to allow consistent TJA scoring. There may be a learned effect with the TJA since interrater reliability improved with repetition. TJA instructions and training should be modified and enhanced before clinical implementation. PMID:26464881

Within-trial contrast: when is a failure to replicate not a type I error?

PubMed

Zentall, Thomas R; Singer, Rebecca A

2007-05-01

Vasconcelos, Urcuioli, and Lionello-DeNolf (2007) report the results of five experiments that fail to replicate the results of our within-trial contrast study (Clement, Feltus, Kaiser, & Zentall, 2000) and suggest that our results may represent a Type I Error. We believe that this conclusion is not warranted because (a) there is considerable evidence in support of the effect and (b) the amount of training that they gave to their pigeons prior to test may not have been sufficient to observe the effect reliably. We suggest that when sufficient training is provided, reliable contrast can be found.

How Reliable is the Acetabular Cup Position Assessment from Routine Radiographs?

PubMed Central

Carvajal Alba, Jaime A.; Vincent, Heather K.; Sodhi, Jagdeep S.; Latta, Loren L.; Parvataneni, Hari K.

2017-01-01

Abstract Background: Cup position is crucial for optimal outcomes in total hip arthroplasty. Radiographic assessment of component position is routinely performed in the early postoperative period. Aims: The aims of this study were to determine in a controlled environment if routine radiographic methods accurately and reliably assess the acetabular cup position and to assess if there is a statistical difference related to the rater’s level of training. Methods: A pelvic model was mounted in a spatial frame. An acetabular cup was fixed in different degrees of version and inclination. Standardized radiographs were obtained. Ten observers including five fellowship-trained orthopaedic surgeons and five orthopaedic residents performed a blind assessment of cup position. Inclination was assessed from anteroposterior radiographs of the pelvis and version from cross-table lateral radiographs of the hip. Results: The radiographic methods used showed to be imprecise specially when the cup was positioned at the extremes of version and inclination. An excellent inter-observer reliability (Intra-class coefficient > 0,9) was evidenced. There were no differences related to the level of training of the raters. Conclusions: These widely used radiographic methods should be interpreted cautiously and computed tomography should be utilized in cases when further intervention is contemplated. PMID:28852355

Current status of validation for robotic surgery simulators - a systematic review.

PubMed

Abboudi, Hamid; Khan, Mohammed S; Aboumarzouk, Omar; Guru, Khurshid A; Challacombe, Ben; Dasgupta, Prokar; Ahmed, Kamran

2013-02-01

To analyse studies validating the effectiveness of robotic surgery simulators. The MEDLINE(®), EMBASE(®) and PsycINFO(®) databases were systematically searched until September 2011. References from retrieved articles were reviewed to broaden the search. The simulator name, training tasks, participant level, training duration and evaluation scoring were extracted from each study. We also extracted data on feasibility, validity, cost-effectiveness, reliability and educational impact. We identified 19 studies investigating simulation options in robotic surgery. There are five different robotic surgery simulation platforms available on the market. In all, 11 studies sought opinion and compared performance between two different groups; 'expert' and 'novice'. Experts ranged in experience from 21-2200 robotic cases. The novice groups consisted of participants with no prior experience on a robotic platform and were often medical students or junior doctors. The Mimic dV-Trainer(®), ProMIS(®), SimSurgery Educational Platform(®) (SEP) and Intuitive systems have shown face, content and construct validity. The Robotic Surgical SimulatorTM system has only been face and content validated. All of the simulators except SEP have shown educational impact. Feasibility and cost-effectiveness of simulation systems was not evaluated in any trial. Virtual reality simulators were shown to be effective training tools for junior trainees. Simulation training holds the greatest potential to be used as an adjunct to traditional training methods to equip the next generation of robotic surgeons with the skills required to operate safely. However, current simulation models have only been validated in small studies. There is no evidence to suggest one type of simulator provides more effective training than any other. More research is needed to validate simulated environments further and investigate the effectiveness of animal and cadaveric training in robotic surgery. © 2012 BJU International.

Managing cognitive difficulties after traumatic brain injury: a review of online resources for families.

PubMed

Poulin, Valérie; Dawson, Deirdre R; Bottari, Carolina; Verreault, Cynthia; Turcotte, Samantha; Jean, Alexandra

2018-03-22

To identify and critically appraise the content, readability, reliability and usability of websites providing information for managing cognitive difficulties in everyday life for the families of adults with moderate to severe traumatic brain injury. Systematic searches on the Internet for relevant websites were conducted using five search engines, and through consultation of the lists of resources published on websites of traumatic brain injury organizations. Two team members assessed eligibility of the websites. To be included, they had to provide information related to management of cognitive difficulties following moderate to severe traumatic brain injury, to be in English or French and available free of charge. Two reviewers evaluated each website according to: (1) its readability using Flesch-Kincaid Grade Level; (2) the quality of its content using a checklist of eight recommendations for managing memory, attention and executive function problems; (3) its usability (e.g., clear design) and reliability (e.g., currency of information) using the Minervation Validation Instrument for Health Care Web Sites. Of the 38 websites included, 10 provide specific tips for families that cover several domains of cognitive function, including memory, attention and executive function. The most frequent recommendations focused on the use of environmental supports for memory problems (n = 33 websites). The readability of information is below the recommended grade 7 for only nine of the websites. All sites show acceptable usability, but their quality is variable in terms of reliability of the information. This review provides useful information for selecting online resources to educate families about the management of cognitive difficulties following moderate to severe traumatic brain injury, as a complement to information and training provided by the rehabilitation team. Implications for rehabilitation This review describes standardized criteria for the evaluation of the content, readability, reliability and usability of websites for family education post-TBI. Given the variability in the content, the readability and the reliability of websites providing information for families about the management of cognitive difficulties post-TBI, careful attention to the selection of appropriate resources is required. Findings from this review may facilitate clinicians' identification of relevant websites to educate families about the management of cognitive difficulties post-TBI, as a complement to other information and training from the rehabilitation team.

Southeastern Virtual Institute for Health Equity and Wellness (SEVIEW), Phase 2

DTIC Science & Technology

2016-09-01

locations through “ Train the Trainer” and equip them with information about treatment resources to amplify the number of personnel who can independently... training goal was to equip participants with enough current, foundational knowledge about substance abuse that they could reliably inform others in their...Upon completion of STEER training , the 68 participants reported feeling more equipped to share the knowledge they gained with others in their

Defense Planning for National Security: Navigation Aids for the Mystery Tour

DTIC Science & Technology

2014-03-01

to provide in the necessary quantity. 2. Training: superior training regimes are not en- tirely reliable as keys to victory. Although rigorous... eccentric misbehavior, the question of how much tolerance is permissible should not be ignored. The reason is not simply academic pedant- ry. A single

77 FR 50097 - Notice of Commission Staff Attendance

Federal Register 2010, 2011, 2012, 2013, 2014

2012-08-20

....m.-4 p.m., Local Time Markets and Reliability Committee August 23, 2012, 9 a.m.-3:30 p.m., Local... Combined Markets and Reliability Committee/Members Committee September 27, 2012, 9 a.m.-5 p.m., Local Time... Riverfront, Wilmington, DE The PJM Conference & Training Center, Norristown, PA The above-referenced meetings...

Reliability of Grading High School Work in English

ERIC Educational Resources Information Center

Brimi, Hunter M.

2011-01-01

This research replicates the work of Starch and Elliot (1912) by examining the reliability of the grading by English teachers in a single school district. Ninety high school teachers graded the same student paper following professional development sessions in which they were trained to use NWREL's "6+1 Traits of Writing." These participants had…

Composite Reliability of a Workplace-Based Assessment Toolbox for Postgraduate Medical Education

ERIC Educational Resources Information Center

Moonen-van Loon, J. M. W.; Overeem, K.; Donkers, H. H. L. M.; van der Vleuten, C. P. M.; Driessen, E. W.

2013-01-01

In recent years, postgraduate assessment programmes around the world have embraced workplace-based assessment (WBA) and its related tools. Despite their widespread use, results of studies on the validity and reliability of these tools have been variable. Although in many countries decisions about residents' continuation of training and…

77 FR 41452 - Comment Request for Information Collection on Employment and Training (ET) Handbook 361...

Federal Register 2010, 2011, 2012, 2013, 2014

2012-07-13

... the completeness and reliability of the performance data included in it [that] * * * describes any material inadequacies in the completeness and reliability of the data.'' (OMB Circular A-11, Section 230.2... validity (accuracy) of the counts of transactions or measurements of status as follows. In the validation...

Further Investigation of the SI Scale of the MMPI: Reliabilities, Correlates, and Subscale Utility.

ERIC Educational Resources Information Center

Williams, Carolyn L.

1983-01-01

Administered the SI scale of the Minnesota Multiphasic Personality Inventory (MMPI) and several measures typically used in social skills and assertiveness training research to college students (N=218). Results demonstrated acceptable reliability and support for the utility of the subscales of the SI scale of the MMPI. (LLL)

The Reliability, Validity, and Usefulness of the Objective Structured Clinical Examination (OSCE) in Dental Education

ERIC Educational Resources Information Center

Graham, Roseanna

2010-01-01

This study evaluated the reliability, validity, and educational usefulness of a comprehensive, multidisciplinary Objective Structured Clinical Examination (OSCE) in dental education. The OSCE was administered to dental students at the Columbia University College of Dental Medicine (CDM) before they entered clinical training. Participants in this…

Improving patient safety: patient-focused, high-reliability team training.

PubMed

McKeon, Leslie M; Cunningham, Patricia D; Oswaks, Jill S Detty

2009-01-01

Healthcare systems are recognizing "human factor" flaws that result in adverse outcomes. Nurses work around system failures, although increasing healthcare complexity makes this harder to do without risk of error. Aviation and military organizations achieve ultrasafe outcomes through high-reliability practice. We describe how reliability principles were used to teach nurses to improve patient safety at the front line of care. Outcomes include safety-oriented, teamwork communication competency; reflections on safety culture and clinical leadership are discussed.

HiRel: Hybrid Automated Reliability Predictor (HARP) integrated reliability tool system, (version 7.0). Volume 3: HARP Graphics Oriented (GO) input user's guide

NASA Technical Reports Server (NTRS)

Bavuso, Salvatore J.; Rothmann, Elizabeth; Mittal, Nitin; Koppen, Sandra Howell

1994-01-01

The Hybrid Automated Reliability Predictor (HARP) integrated Reliability (HiRel) tool system for reliability/availability prediction offers a toolbox of integrated reliability/availability programs that can be used to customize the user's application in a workstation or nonworkstation environment. HiRel consists of interactive graphical input/output programs and four reliability/availability modeling engines that provide analytical and simulative solutions to a wide host of highly reliable fault-tolerant system architectures and is also applicable to electronic systems in general. The tool system was designed at the outset to be compatible with most computing platforms and operating systems, and some programs have been beta tested within the aerospace community for over 8 years. This document is a user's guide for the HiRel graphical preprocessor Graphics Oriented (GO) program. GO is a graphical user interface for the HARP engine that enables the drawing of reliability/availability models on a monitor. A mouse is used to select fault tree gates or Markov graphical symbols from a menu for drawing.

Reliability and Accuracy of Cross-sectional Radiographic Assessment of Severe Knee Osteoarthritis: Role of Training and Experience.

PubMed

Klara, Kristina; Collins, Jamie E; Gurary, Ellen; Elman, Scott A; Stenquist, Derek S; Losina, Elena; Katz, Jeffrey N

2016-07-01

To dêtermine the reliability of radiographic assessment of knee osteoarthritis (OA) by nonclinician readers compared to an experienced radiologist. The radiologist trained 3 nonclinicians to evaluate radiographic characteristics of knee OA. The radiologist and nonclinicians read preoperative films of 36 patients prior to total knee replacement. Intrareader and interreader reliability were measured using the weighted κ statistic and intraclass correlation coefficient (ICC). Scores κ < 0.20 indicated slight agreement, 0.21-0.40 fair, 0.41-0.60 moderate, 0.61-0.80 substantial, and 0.81-1.0 almost perfect agreement. Intrareader reliability among nonclinicians (κ) ranged from 0.40 to 1.0 for individual radiographic features and 0.72 to 1.0 for Kellgren-Lawrence (KL) grade. ICC ranged from 0.89 to 0.98 for the Osteoarthritis Research Society International (OARSI) summary score. Interreader agreement among nonclinicians ranged from κ of 0.45 to 0.94 for individual features, and 0.66 to 0.97 for KL grade. ICC ranged from 0.87 to 0.96 for the OARSI Summary Score. Interreader reliability between nonclinicians and the radiologist ranged from κ of 0.56 to 0.85 for KL grade. ICC ranged from 0.79 to 0.88 for the OARSI Summary Score. Intrareader and interreader agreement was variable for individual radiograph features but substantial for summary KL grade and OARSI Summary Score. Investigators face tradeoffs between cost and reader experience. These data suggest that in settings where costs are constrained, trained nonclinicians may be suitable readers of radiographic knee OA, particularly if a summary score (KL grade or OARSI Score) is used to determine radiographic severity.

Reliability of sprint test indices in well-trained cyclists.

PubMed

Coleman, D A; Wiles, J D; Nunn, M; Smith, M F

2005-06-01

The study aim was to assess reliability of repeated laboratory sprint tests in well-trained endurance cyclists. Eleven male cyclists (mean +/- standard deviation: 27 +/- 6 yr, 1.79 +/- 0.04 m, 70.1 +/- 3.3 kg) performed a maximal 30-second sprint test on four separate occasions using their own bicycle fitted with an SRM powermeter on a Kingcycle air-braked ergometer. Peak power output (W (peak)), mean power (W (mean)) and an index of fatigue (FI) were calculated. Three minutes post sprint, capillarised blood lactate measurements were taken and analysed. No significant differences (p > 0.05) were found between trials for W (peak), W (mean), FI and blood lactate concentration. Repeatability of W (peak), W (mean), and fatigue index improved across trials 2 and 3 when compared to trials 1 and 2. The highest CV for these variables was recorded between trials 3 and 4. The CV for W (peak) was 4.5 +/- 1.6 %, W (mean) 2.4 +/- 1.2 %, and FI 17.2 +/- 7.1 %. Intraclass reliability coefficients were 0.93 (95 % CI 0.84 - 0.98), 0.94 (95 % CI 0.86 - 0.98) and 0.89 (95 % CI 0.69 - 0.95) respectively. Blood lactate concentration ranged between 5.35 and 14.52 mmol.l(-1), with a mean CV of 12.1 +/- 4.2 %. The CV for trials 2 and 3 revealed the highest CV for blood lactate concentration (15.1 %). The lowest CV for this variable (10.2 %) was recorded between trials 3 and 4. The intraclass reliability coefficient for blood lactate concentration was 0.79 (95 % CI 0.58 - 0.93). The results of this study indicate that there is no improvement in the reliability of sprint test indices when assessing well-trained, experienced cyclists, riding on their own cycle equipment.

The Community Child Health Network Life Stress Interview: a brief chronic stress measure for community health research.

PubMed

Tanner Stapleton, Lynlee R; Dunkel Schetter, Christine; Dooley, Larissa N; Guardino, Christine M; Huynh, Jan; Paek, Cynthia; Clark-Kauffman, Elizabeth; Schafer, Peter; Woolard, Richard; Lanzi, Robin Gaines

2016-07-01

Chronic stress is implicated in many theories as a contributor to a wide range of physical and mental health problems. The current study describes the development of a chronic stress measure that was based on the UCLA Life Stress Interview (LSI) and adapted in collaboration with community partners for use in a large community health study of low-income, ethnically diverse parents of infants in the USA (Community Child Health Network [CCHN]). We describe the instrument, its purpose and adaptations, implementation, and results of a reliability study in a subsample of the larger study cohort. Interviews with 272 mothers were included in the present study. Chronic stress was assessed using the CCHN LSI, an instrument designed for administration by trained community interviewers to assess four domains of chronic stress, each rated by interviewers. Significant correlations ranging from small to moderate in size between chronic stress scores on this measure, other measures of stress, biomarkers of allostatic load, and mental health provide initial evidence of construct and concurrent validity. Reliability data for interviewer ratings are also provided. This relatively brief interview (15 minutes) is available for use and may be a valuable tool for researchers seeking to measure chronic stress reliably and validly in future studies with time constraints.

EEG-neurofeedback for optimising performance. II: creativity, the performing arts and ecological validity.

PubMed

Gruzelier, John H

2014-07-01

As a continuation of a review of evidence of the validity of cognitive/affective gains following neurofeedback in healthy participants, including correlations in support of the gains being mediated by feedback learning (Gruzelier, 2014a), the focus here is on the impact on creativity, especially in the performing arts including music, dance and acting. The majority of research involves alpha/theta (A/T), sensory-motor rhythm (SMR) and heart rate variability (HRV) protocols. There is evidence of reliable benefits from A/T training with advanced musicians especially for creative performance, and reliable benefits from both A/T and SMR training for novice music performance in adults and in a school study with children with impact on creativity, communication/presentation and technique. Making the SMR ratio training context ecologically relevant for actors enhanced creativity in stage performance, with added benefits from the more immersive training context. A/T and HRV training have benefitted dancers. The neurofeedback evidence adds to the rapidly accumulating validation of neurofeedback, while performing arts studies offer an opportunity for ecological validity in creativity research for both creative process and product. Copyright © 2013 Elsevier Ltd. All rights reserved.

Hardware Evaluation of the Horizontal Exercise Fixture with Weight Stack

NASA Technical Reports Server (NTRS)

Newby, Nate; Leach, Mark; Fincke, Renita; Sharp, Carwyn

2009-01-01

HEF with weight stack seems to be a very sturdy and reliable exercise device that should function well in a bed rest training setting. A few improvements should be made to both the hardware and software to improve usage efficiency, but largely, this evaluation has demonstrated HEF's robustness. The hardware offers loading to muscles, bones, and joints, potentially sufficient to mitigate the loss of muscle mass and bone mineral density during long-duration bed rest campaigns. With some minor modifications, the HEF with weight stack equipment provides the best currently available means of performing squat, heel raise, prone row, bench press, and hip flexion/extension exercise in a supine orientation.

As accessible as a book on a library shelf: the imperative of routine simulation in modern health care.

PubMed

Gordon, James A

2012-01-01

Technology-enhanced patient simulation has emerged as an important new modality for teaching and learning in medicine. In particular, immersive simulation platforms that replicate the clinical environment promise to revolutionize medical education by enabling an enhanced level of safety, standardization, and efficiency across health-care training. Such an experiential approach seems unique in reliably catalyzing a level of emotional engagement that fosters immediate and indelible learning and allows for increasingly reliable levels of performance evaluation-all in a completely risk-free environment. As such, medical simulation is poised to emerge as a critical component of training and certification throughout health care, promising to fundamentally enhance quality and safety across disciplines. To encourage routine simulation-based practice as part of its core quality and safety mission, Massachusetts General Hospital now incorporates simulation resources within its historic medical library (est. 1847), located at the center of the campus. In this new model, learners go to the library not only to read about a patient's illness, but also to take care of their "patient." Such an approach redefines and advances the central role of the library on the campus and ensures that simulation-based practice is centrally available as part of everyday hospital operations. This article describes the reasons for identifying simulation as an institutional priority leading up to the Massachusetts General Hospital Bicentennial Celebration (1811-2011) and for creating a simulation-based learning laboratory within a hospital library.

INFLUENCES OF RESPONSE RATE AND DISTRIBUTION ON THE CALCULATION OF INTEROBSERVER RELIABILITY SCORES

PubMed Central

Rolider, Natalie U.; Iwata, Brian A.; Bullock, Christopher E.

2012-01-01

We examined the effects of several variations in response rate on the calculation of total, interval, exact-agreement, and proportional reliability indices. Trained observers recorded computer-generated data that appeared on a computer screen. In Study 1, target responses occurred at low, moderate, and high rates during separate sessions so that reliability results based on the four calculations could be compared across a range of values. Total reliability was uniformly high, interval reliability was spuriously high for high-rate responding, proportional reliability was somewhat lower for high-rate responding, and exact-agreement reliability was the lowest of the measures, especially for high-rate responding. In Study 2, we examined the separate effects of response rate per se, bursting, and end-of-interval responding. Response rate and bursting had little effect on reliability scores; however, the distribution of some responses at the end of intervals decreased interval reliability somewhat, proportional reliability noticeably, and exact-agreement reliability markedly. PMID:23322930

Cleft audit protocol for speech (CAPS-A): a comprehensive training package for speech analysis.

PubMed

Sell, D; John, A; Harding-Bell, A; Sweeney, T; Hegarty, F; Freeman, J

2009-01-01

The previous literature has largely focused on speech analysis systems and ignored process issues, such as the nature of adequate speech samples, data acquisition, recording and playback. Although there has been recognition of the need for training on tools used in speech analysis associated with cleft palate, little attention has been paid to this issue. To design, execute, and evaluate a training programme for speech and language therapists on the systematic and reliable use of the Cleft Audit Protocol for Speech-Augmented (CAPS-A), addressing issues of standardized speech samples, data acquisition, recording, playback, and listening guidelines. Thirty-six specialist speech and language therapists undertook the training programme over four days. This consisted of two days' training on the CAPS-A tool followed by a third day, making independent ratings and transcriptions on ten new cases which had been previously recorded during routine audit data collection. This task was repeated on day 4, a minimum of one month later. Ratings were made using the CAPS-A record form with the CAPS-A definition table. An analysis was made of the speech and language therapists' CAPS-A ratings at occasion 1 and occasion 2 and the intra- and inter-rater reliability calculated. Trained therapists showed consistency in individual judgements on specific sections of the tool. Intraclass correlation coefficients were calculated for each section with good agreement on eight of 13 sections. There were only fair levels of agreement on anterior oral cleft speech characteristics, non-cleft errors/immaturities and voice. This was explained, at least in part, by their low prevalence which affects the calculation of the intraclass correlation coefficient statistic. Speech and language therapists benefited from training on the CAPS-A, focusing on specific aspects of speech using definitions of parameters and scalar points, in order to apply the tool systematically and reliably. Ratings are enhanced by ensuring a high degree of attention to the nature of the data, standardizing the speech sample, data acquisition, the listening process together with the use of high-quality recording and playback equipment. In addition, a method is proposed for maintaining listening skills following training as part of an individual's continuing education.

Rhythm and Melody Tasks for School-Aged Children With and Without Musical Training: Age-Equivalent Scores and Reliability.

PubMed

Ireland, Kierla; Parker, Averil; Foster, Nicholas; Penhune, Virginia

2018-01-01

Measuring musical abilities in childhood can be challenging. When music training and maturation occur simultaneously, it is difficult to separate the effects of specific experience from age-based changes in cognitive and motor abilities. The goal of this study was to develop age-equivalent scores for two measures of musical ability that could be reliably used with school-aged children (7-13) with and without musical training. The children's Rhythm Synchronization Task (c-RST) and the children's Melody Discrimination Task (c-MDT) were adapted from adult tasks developed and used in our laboratories. The c-RST is a motor task in which children listen and then try to synchronize their taps with the notes of a woodblock rhythm while it plays twice in a row. The c-MDT is a perceptual task in which the child listens to two melodies and decides if the second was the same or different. We administered these tasks to 213 children in music camps (musicians, n = 130) and science camps (non-musicians, n = 83). We also measured children's paced tapping, non-paced tapping, and phonemic discrimination as baseline motor and auditory abilities We estimated internal-consistency reliability for both tasks, and compared children's performance to results from studies with adults. As expected, musically trained children outperformed those without music lessons, scores decreased as difficulty increased, and older children performed the best. Using non-musicians as a reference group, we generated a set of age-based z-scores, and used them to predict task performance with additional years of training. Years of lessons significantly predicted performance on both tasks, over and above the effect of age. We also assessed the relation between musician's scores on music tasks, baseline tasks, auditory working memory, and non-verbal reasoning. Unexpectedly, musician children outperformed non-musicians in two of three baseline tasks. The c-RST and c-MDT fill an important need for researchers interested in evaluating the impact of musical training in longitudinal studies, those interested in comparing the efficacy of different training methods, and for those assessing the impact of training on non-musical cognitive abilities such as language processing.

Rhythm and Melody Tasks for School-Aged Children With and Without Musical Training: Age-Equivalent Scores and Reliability

PubMed Central

Ireland, Kierla; Parker, Averil; Foster, Nicholas; Penhune, Virginia

2018-01-01

Measuring musical abilities in childhood can be challenging. When music training and maturation occur simultaneously, it is difficult to separate the effects of specific experience from age-based changes in cognitive and motor abilities. The goal of this study was to develop age-equivalent scores for two measures of musical ability that could be reliably used with school-aged children (7–13) with and without musical training. The children's Rhythm Synchronization Task (c-RST) and the children's Melody Discrimination Task (c-MDT) were adapted from adult tasks developed and used in our laboratories. The c-RST is a motor task in which children listen and then try to synchronize their taps with the notes of a woodblock rhythm while it plays twice in a row. The c-MDT is a perceptual task in which the child listens to two melodies and decides if the second was the same or different. We administered these tasks to 213 children in music camps (musicians, n = 130) and science camps (non-musicians, n = 83). We also measured children's paced tapping, non-paced tapping, and phonemic discrimination as baseline motor and auditory abilities We estimated internal-consistency reliability for both tasks, and compared children's performance to results from studies with adults. As expected, musically trained children outperformed those without music lessons, scores decreased as difficulty increased, and older children performed the best. Using non-musicians as a reference group, we generated a set of age-based z-scores, and used them to predict task performance with additional years of training. Years of lessons significantly predicted performance on both tasks, over and above the effect of age. We also assessed the relation between musician's scores on music tasks, baseline tasks, auditory working memory, and non-verbal reasoning. Unexpectedly, musician children outperformed non-musicians in two of three baseline tasks. The c-RST and c-MDT fill an important need for researchers interested in evaluating the impact of musical training in longitudinal studies, those interested in comparing the efficacy of different training methods, and for those assessing the impact of training on non-musical cognitive abilities such as language processing. PMID:29674984

Enhancing computer self-efficacy and attitudes in multi-ethnic older adults: a randomised controlled study

PubMed Central

Laganà, Luciana; Oliver, Taylor; Ainsworth, Andrew; Edwards, Marc

2014-01-01

Several studies have documented the health-related benefits of older adults' use of computer technology, but before they can be realised, older individuals must be positively inclined and confident in their ability to engage in computer-based environments. To facilitate the assessment of computer technology attitudes, one aim of the longitudinal study reported in this paper was to test and refine a new 22-item measure of computer technology attitudes designed specifically for older adults, as none such were available.1 Another aim was to replicate, on a much larger scale, the successful findings of a preliminary study that tested a computer technology training programme for older adults (Laganà 2008). Ninety-six older men and women, mainly from non-European-American backgrounds, were randomly assigned to the waitlist/control or the experimental group. The same six-week one-on-one training was administered to the control subjects at the completion of their post-test. The revised (17-item) version of the Older Adults' Computer Technology Attitudes Scale (OACTAS) showed strong reliability: the results of a factor analysis were robust, and two analyses of covariance demonstrated that the training programme induced significant changes in attitudes and self-efficacy. Such results encourage the recruitment of older persons into training programmes aimed at increasing computer technology attitudes and self-efficacy. PMID:25512679

Applied Virtual Reality Research and Applications at NASA/Marshall Space Flight Center

NASA Technical Reports Server (NTRS)

Hale, Joseph P.

1995-01-01

A Virtual Reality (VR) applications program has been under development at NASA/Marshall Space Flight Center (MSFC) since 1989. The objectives of the MSFC VR Applications Program are to develop, assess, validate, and utilize VR in hardware development, operations development and support, mission operations training and science training. Before this technology can be utilized with confidence in these applications, it must be validated for each particular class of application. That is, the precision and reliability with which it maps onto real settings and scenarios, representative of a class, must be calculated and assessed. The approach of the MSFC VR Applications Program is to develop and validate appropriate virtual environments and associated object kinematic and behavior attributes for specific classes of applications. These application-specific environments and associated simulations will be validated, where possible, through empirical comparisons with existing, accepted tools and methodologies. These validated VR analytical tools will then be available for use in the design and development of space systems and operations and in training and mission support systems. Specific validation studies for selected classes of applications have been completed or are currently underway. These include macro-ergonomic "control-room class" design analysis, Spacelab stowage reconfiguration training, a full-body micro-gravity functional reach simulator, and a gross anatomy teaching simulator. This paper describes the MSFC VR Applications Program and the validation studies.

Ensembles of novelty detection classifiers for structural health monitoring using guided waves

DOE Office of Scientific and Technical Information (OSTI.GOV)

Dib, Gerges; Karpenko, Oleksii; Koricho, Ermias

Guided wave structural health monitoring uses sparse sensor networks embedded in sophisticated structures for defect detection and characterization. The biggest challenge of those sensor networks is developing robust techniques for reliable damage detection under changing environmental and operating conditions. To address this challenge, we develop a novelty classifier for damage detection based on one class support vector machines. We identify appropriate features for damage detection and introduce a feature aggregation method which quadratically increases the number of available training observations.We adopt a two-level voting scheme by using an ensemble of classifiers and predictions. Each classifier is trained on a differentmore » segment of the guided wave signal, and each classifier makes an ensemble of predictions based on a single observation. Using this approach, the classifier can be trained using a small number of baseline signals. We study the performance using monte-carlo simulations of an analytical model and data from impact damage experiments on a glass fiber composite plate.We also demonstrate the classifier performance using two types of baseline signals: fixed and rolling baseline training set. The former requires prior knowledge of baseline signals from all environmental and operating conditions, while the latter does not and leverages the fact that environmental and operating conditions vary slowly over time and can be modeled as a Gaussian process.« less

Strength testing and training of rowers: a review.

PubMed

Lawton, Trent W; Cronin, John B; McGuigan, Michael R

2011-05-01

In the quest to maximize average propulsive stroke impulses over 2000-m racing, testing and training of various strength parameters have been incorporated into the physical conditioning plans of rowers. Thus, the purpose of this review was 2-fold: to identify strength tests that were reliable and valid correlates (predictors) of rowing performance; and, to establish the benefits gained when strength training was integrated into the physical preparation plans of rowers. The reliability of maximal strength and power tests involving leg extension (e.g. leg pressing) and arm pulling (e.g. prone bench pull) was high (intra-class correlations 0.82-0.99), revealing that elite rowers were significantly stronger than their less competitive peers. The greater strength of elite rowers was in part attributed to the correlation between strength and greater lean body mass (r = 0.57-0.63). Dynamic lower body strength tests that determined the maximal external load for a one-repetition maximum (1RM) leg press (kg), isokinetic leg extension peak force (N) or leg press peak power (W) proved to be moderately to strongly associated with 2000-m ergometer times (r = -0.54 to -0.68; p < 0.05). Repetition tests that assess muscular or strength endurance by quantifying the number of repetitions accrued at a fixed percentage of the strength maximum (e.g. 50-70% 1RM leg press) or set absolute load (e.g. 40 kg prone bench pulls) were less reliable and more time consuming when compared with briefer maximal strength tests. Only leg press repetition tests were correlated with 2000-m ergometer times (e.g. r = -0.67; p < 0.05). However, these tests differentiate training experience and muscle morphology, in that those individuals with greater training experience and/or proportions of slow twitch fibres performed more repetitions. Muscle balance ratios derived from strength data (e.g. hamstring-quadriceps ratio <45% or knee extensor-elbow flexor ratio around 4.2 ± 0.22 to 1) appeared useful in the pathological assessment of low back pain or rib injury history associated with rowing. While strength partially explained variances in 2000-m ergometer performance, concurrent endurance training may be counterproductive to strength development over the shorter term (i.e. <12 weeks). Therefore, prioritization of strength training within the sequence of training units should be considered, particularly over the non-competition phase (e.g. 2-6 sets × 4-12 repetitions, three sessions a week). Maximal strength was sustained when infrequent (e.g. one or two sessions a week) but intense (e.g. 73-79% of maximum) strength training units were scheduled; however, it was unclear whether training adaptations should emphasize maximal strength, endurance or power in order to enhance performance during the competition phase. Additionally, specific on-water strength training practices such as towing ropes had not been reported. Further research should examine the on-water benefits associated with various strength training protocols, in the context of the training phase, weight division, experience and level of rower, if limitations to the reliability and precision of performance data (e.g. 2000-m time or rank) can be controlled. In conclusion, while positive ergometer time-trial benefits of clinical and practical significance were reported with strength training, a lack of statistical significance was noted, primarily due to an absence of quality long-term controlled experimental research designs.

[Development of Autogenic Training Clinical Effectiveness Scale (ATCES)].

PubMed

Ikezuki, Makoto; Miyauchi, Yuko; Yamaguchi, Hajime; Koshikawa, Fusako

2002-02-01

The purpose of the present study was to develop a scale measuring clinical effectiveness of autogenic training. In Study 1, 167 undergraduates completed a survey of items concerning physical and mental states, which were thought to vary in the course of autogenic training. With item and factor analyses, 20 items were selected, and the resulting scale (ATCES) had high discrimination and clear factor structure. In Study 2, reliability and concurrent and clinical validity of the scale were examined with three groups of respondents: 85 mentally healthy, 31 control, 13 clinical persons. The scale showed a high test-retest correlation (r = .83) and alpha coefficient (alpha = .86). ATCES had a Pearson correlation coefficient of r = .56 with General Health Questionnaire (GHQ-12), and r = .73 with trait anxiety (STAI-T). And ATCES successfully discriminated the mentally healthy and clinical groups in terms of clinical effectiveness. These results demonstrated high reliability and sufficient concurrent and clinical validity of the new scale.

Reliability of Wearable Inertial Measurement Units to Measure Physical Activity in Team Handball.

PubMed

Luteberget, Live S; Holme, Benjamin R; Spencer, Matt

2018-04-01

To assess the reliability and sensitivity of commercially available inertial measurement units to measure physical activity in team handball. Twenty-two handball players were instrumented with 2 inertial measurement units (OptimEye S5; Catapult Sports, Melbourne, Australia) taped together. They participated in either a laboratory assessment (n = 10) consisting of 7 team handball-specific tasks or field assessment (n = 12) conducted in 12 training sessions. Variables, including PlayerLoad™ and inertial movement analysis (IMA) magnitude and counts, were extracted from the manufacturers' software. IMA counts were divided into intensity bands of low (1.5-2.5 m·s -1 ), medium (2.5-3.5 m·s -1 ), high (>3.5 m·s -1 ), medium/high (>2.5 m·s -1 ), and total (>1.5 m·s -1 ). Reliability between devices and sensitivity was established using coefficient of variation (CV) and smallest worthwhile difference (SWD). Laboratory assessment: IMA magnitude showed a good reliability (CV = 3.1%) in well-controlled tasks. CV increased (4.4-6.7%) in more-complex tasks. Field assessment: Total IMA counts (CV = 1.8% and SWD = 2.5%), PlayerLoad (CV = 0.9% and SWD = 2.1%), and their associated variables (CV = 0.4-1.7%) showed a good reliability, well below the SWD. However, the CV of IMA increased when categorized into intensity bands (2.9-5.6%). The reliability of IMA counts was good when data were displayed as total, high, or medium/high counts. A good reliability for PlayerLoad and associated variables was evident. The CV of the previously mentioned variables was well below the SWD, suggesting that OptimEye's inertial measurement unit and its software are sensitive for use in team handball.

Classifying with confidence from incomplete information.

DOE PAGES

Parrish, Nathan; Anderson, Hyrum S.; Gupta, Maya R.; ...

2013-12-01

For this paper, we consider the problem of classifying a test sample given incomplete information. This problem arises naturally when data about a test sample is collected over time, or when costs must be incurred to compute the classification features. For example, in a distributed sensor network only a fraction of the sensors may have reported measurements at a certain time, and additional time, power, and bandwidth is needed to collect the complete data to classify. A practical goal is to assign a class label as soon as enough data is available to make a good decision. We formalize thismore » goal through the notion of reliability—the probability that a label assigned given incomplete data would be the same as the label assigned given the complete data, and we propose a method to classify incomplete data only if some reliability threshold is met. Our approach models the complete data as a random variable whose distribution is dependent on the current incomplete data and the (complete) training data. The method differs from standard imputation strategies in that our focus is on determining the reliability of the classification decision, rather than just the class label. We show that the method provides useful reliability estimates of the correctness of the imputed class labels on a set of experiments on time-series data sets, where the goal is to classify the time-series as early as possible while still guaranteeing that the reliability threshold is met.« less

Basic psychometric properties of the transfer assessment instrument (version 3.0).

PubMed

Tsai, Chung-Ying; Rice, Laura A; Hoelmer, Claire; Boninger, Michael L; Koontz, Alicia M

2013-12-01

To refine the Transfer Assessment Instrument (TAI 2.0), develop a training program for the TAI, and analyze the basic psychometric properties of the TAI 3.0, including reliability, standard error of measurement (SEM), minimal detectable change (MDC), and construct validity. Repeated measures. A winter sports clinic for disabled veterans. Wheelchair users (N=41) who perform sitting-pivot or standing-pivot transfers. Not applicable. TAI version 3.0, intraclass correlation coefficients, SEMs, and MDCs for reliable measurement of raters' responses. Spearman correlation coefficient, 1-way analysis of variance, and independent t tests to evaluate construct validity. TAI 3.0 had acceptable to high levels of reliability (range, .74-.88). The SEMs for part 1, part 2, and final scores ranged from .45 to .75. The MDC was 1.5 points on the 10-point scale for the final score. There were weak correlations (ρ range, -.13 to .25; P>.11) between TAI final scores and subjects' characteristics (eg, sex, body mass index, age, type of disability, length of wheelchair use, grip and elbow strength, sitting balance). With comprehensive training, the refined TAI 3.0 yields high reliability among raters of different clinical backgrounds and experience. TAI 3.0 was unbiased toward certain physical characteristics that may influence transfer. TAI fills a void in the field by providing a quantitative measurement of transfers and a tool that can be used to detect problems and guide transfer training. Copyright © 2013 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.

An instrument to characterize the environment for residents' evidence-based medicine learning and practice.

PubMed

Mi, Misa; Moseley, James L; Green, Michael L

2012-02-01

Many residency programs offer training in evidence-based medicine (EBM). However, these curricula often fail to achieve optimal learning outcomes, perhaps because they neglect various contextual factors in the learning environment. We developed and validated an instrument to characterize the environment for EBM learning and practice in residency programs. An EBM Environment Scale was developed following scale development principles. A survey was administered to residents across six programs in primary care specialties at four medical centers. Internal consistency reliability was analyzed with Cronbach's coefficient alpha. Validity was assessed by comparing predetermined subscales with the survey's internal structure as assessed via factor analysis. Scores were also compared for subgroups based on residency program affiliation and residency characteristics. Out of 262 eligible residents, 124 completed the survey (response rate 47%). The overall mean score was 3.89 (standard deviation=0.56). The initial reliability analysis of the 48-item scale had a high reliability coefficient (Cronbach α=.94). Factor analysis and further item analysis resulted in a shorter 36-item scale with a satisfactory reliability coefficient (Cronbach α=.86). Scores were higher for residents with prior EBM training in medical school (4.14 versus 3.62) and in residency (4.25 versus 3.69). If further testing confirms its properties, the EBM Environment Scale may be used to understand the influence of the learning environment on the effectiveness of EBM training. Additionally, it may detect changes in the EBM learning environment in response to programmatic or institutional interventions.

Feasibility of using microbiology diagnostic tests of moderate or high complexity at the point - of - care in a delivery suite.

PubMed

Gray, J W; Milner, P J; Edwards, E H; Daniels, J P; Khan, K S

2012-07-01

Point-of-care testing (POCT) is one of the fastest growing sectors of laboratory diagnostics. Most tests in routine use are haematology or biochemistry tests that are of low complexity. Microbiology POCT has been constrained by a lack of tests that are both accurate and of low complexity. We describe our experience of the practical issues around using more complex POCT for detection of Group B streptococci (GBS) in swabs from labouring women. We evaluated two tests for their feasibility in POCT: an optical immune assay (Biostar OIA Strep B, Inverness Medical, Princetown, NJ) and a PCR (IDI-Strep B, Cepheid, Sunnyvale, CA), which have been categorised as being of moderate and high complexity, respectively. A total of 12 unqualified midwifery assistants (MA) were trained to undertake testing on the delivery suite. A systematic approach to the introduction and management of POC testing was used. Modelling showed that the probability of test results being available within a clinically useful timescale was high. However, in the clinical setting, we found it impossible to maintain reliable availability of trained testers. Implementation of more complex POC testing is technically feasible, but it is expensive, and may be difficult to achieve in a busy delivery suite.

Simulation Technology for Skills Training and Competency Assessment in Medical Education

PubMed Central

Obeso, Vivian T.; Issenberg, S. Barry

2007-01-01

Medical education during the past decade has witnessed a significant increase in the use of simulation technology for teaching and assessment. Contributing factors include: changes in health care delivery and academic environments that limit patient availability as educational opportunities; worldwide attention focused on the problem of medical errors and the need to improve patient safety; and the paradigm shift to outcomes-based education with its requirements for assessment and demonstration of competence. The use of simulators addresses many of these issues: they can be readily available at any time and can reproduce a wide variety of clinical conditions on demand. In lieu of the customary (and arguably unethical) system, whereby novices carry out the practice required to master various techniques—including invasive procedures—on real patients, simulation-based education allows trainees to hone their skills in a risk-free environment. Evaluators can also use simulators for reliable assessments of competence in multiple domains. For those readers less familiar with medical simulators, this article aims to provide a brief overview of these educational innovations and their uses; for decision makers in medical education, we hope to broaden awareness of the significant potential of these new technologies for improving physician training and assessment, with a resultant positive impact on patient safety and health care outcomes. PMID:18095044

Cross-sectional-derived determinants of satisfaction with physician-scientist training among Canadian MD/PhD graduates.

PubMed

Twa, David D W; Skinnider, Michael A; Squair, Jordan W; Lukac, Christine D

2017-01-01

Although MD/PhD programs require considerable commitment on behalf of students and learning institutions, they serve as an integral means of training future physician-scientists; individuals who engage in translational medicine. As attrition from these programs has longstanding effects on the community of translational medicine and comes at substantial cost to MD/PhD programs, we aimed to identify determinants that were associated with satisfaction among MD/PhD graduates, a feature that might inform on limiting program attrition. Anonymized data from a national survey of 139 Canadian MD/PhD alumni was analyzed. Factor analysis was conducted to evaluate the reliability of three questions that measured satisfaction and logistic regression was used to assess the association of outcomes with 17 independent determinants. Eighty-one percent of graduates were satisfied with MD/PhD training. Factor analysis confirmed the reliability of the questions measuring satisfaction. Determinants of self-reported satisfaction with physician-scientist training included co-authorship of more than six manuscripts during MD/PhD training. Additionally, protected research time at the place of current appointment was strongly associated with agreement that MD/PhD training had helped career progression. Demographic variables were not associated with any satisfaction indicator. Taken together, the majority of Canadian MD/PhD graduates are satisfied with their physician-scientist training. Project collaboration leading to co-authorships and protected research time were strongly associated with training satisfaction among graduates. If the value of collaboration can be realized among current and future physician-scientist trainees who are dissatisfied with their training, this might ultimately reduce program attrition.

HiRel: Hybrid Automated Reliability Predictor (HARP) integrated reliability tool system, (version 7.0). Volume 1: HARP introduction and user's guide

NASA Technical Reports Server (NTRS)

Bavuso, Salvatore J.; Rothmann, Elizabeth; Dugan, Joanne Bechta; Trivedi, Kishor S.; Mittal, Nitin; Boyd, Mark A.; Geist, Robert M.; Smotherman, Mark D.

1994-01-01

The Hybrid Automated Reliability Predictor (HARP) integrated Reliability (HiRel) tool system for reliability/availability prediction offers a toolbox of integrated reliability/availability programs that can be used to customize the user's application in a workstation or nonworkstation environment. HiRel consists of interactive graphical input/output programs and four reliability/availability modeling engines that provide analytical and simulative solutions to a wide host of reliable fault-tolerant system architectures and is also applicable to electronic systems in general. The tool system was designed to be compatible with most computing platforms and operating systems, and some programs have been beta tested, within the aerospace community for over 8 years. Volume 1 provides an introduction to the HARP program. Comprehensive information on HARP mathematical models can be found in the references.

Training Guide for Observation and Interviewing in Marine Corps Task Analysis. Training Manual 3

DTIC Science & Technology

1975-08-01

McGraw- Hill Book Co., 1954. This is a comprehensive text that treats theory and sta- tistical operations in psychological testing . Chapter 14 deals...with reliability and validity of measures. -44- Reliabilitv and Validity (cont’d.) Cronbach, L., ESSENTIALS OF PSYCHOLOGICAL TESTING , 2nd ed., New...Commandant of the Marine Corps (Code RD) And Monitored By Personnel and Training Research Programs Psychological Sciences Division Office of Naval

Reliability studies of diagnostic methods in Indian traditional Ayurveda medicine: An overview

PubMed Central

Kurande, Vrinda Hitendra; Waagepetersen, Rasmus; Toft, Egon; Prasad, Ramjee

2013-01-01

Recently, a need to develop supportive new scientific evidence for contemporary Ayurveda has emerged. One of the research objectives is an assessment of the reliability of diagnoses and treatment. Reliability is a quantitative measure of consistency. It is a crucial issue in classification (such as prakriti classification), method development (pulse diagnosis), quality assurance for diagnosis and treatment and in the conduct of clinical studies. Several reliability studies are conducted in western medicine. The investigation of the reliability of traditional Chinese, Japanese and Sasang medicine diagnoses is in the formative stage. However, reliability studies in Ayurveda are in the preliminary stage. In this paper, examples are provided to illustrate relevant concepts of reliability studies of diagnostic methods and their implication in practice, education, and training. An introduction to reliability estimates and different study designs and statistical analysis is given for future studies in Ayurveda. PMID:23930037

Chemical and canine analysis as complimentary techniques for the identification of active odors of the invasive fungus, Raffaelea lauricola.

PubMed

Simon, Alison G; Mills, DeEtta K; Furton, Kenneth G

2017-06-01

Raffaelea lauricola, a fungus causing a vascular wilt (laurel wilt) in Lauraceae trees, was introduced into the United States in the early 2000s. It has devastated forests in the Southeast and has now moved into the commercial avocado groves in southern Florida. Trained detection canines are currently one of the few successful methods for early detection of pre-symptomatic diseased trees. In order to achieve the universal and frequent training required to have successful detection canines, it is desirable to create accessible, safe, and long-lasting training aids. However, identification of odorants and compounds is limited by several factors, including both the availability of chemicals and the need to present chemicals individually and in combination to detection canines. A method for the separation and identification of volatile organic compounds (VOCs) from environmental substances for the creation of such a canine training aid is presented here. Headspace solid phase microextraction-gas chromatography-mass spectrometry (HS-SPME-GC-MS) was used to identify the odors present in avocado trees infected with the R. lauricola phytopathogen. Twenty-eight compounds were detected using this method, with nine present in greater than 80% of samples. The majority of these compounds were not commercially available as standard reference materials, and a canine trial was designed to identify the active odors without the need of pure chemical compounds. To facilitate the creation of a canine training aid, the VOCs above R. lauricola were separated by venting a 0.53mm ID solgel-wax gas chromatography column to the atmosphere. Ten minute fractions of the odor profile were collected on cotton gauze in glass vials and presented to the detection canines in a series of field trials. The canines alerted to the VOCs from the vials that correspond to a portion of the chromatogram containing the most volatile species from R. lauricola. This innovative fractionation and collection method can be used to develop reliable and cost effective canine training aids. Copyright © 2017 Elsevier B.V. All rights reserved.

Development of the Human Factors Skills for Healthcare Instrument: a valid and reliable tool for assessing interprofessional learning across healthcare practice settings.

PubMed

Reedy, Gabriel B; Lavelle, Mary; Simpson, Thomas; Anderson, Janet E

2017-10-01

A central feature of clinical simulation training is human factors skills, providing staff with the social and cognitive skills to cope with demanding clinical situations. Although these skills are critical to safe patient care, assessing their learning is challenging. This study aimed to develop, pilot and evaluate a valid and reliable structured instrument to assess human factors skills, which can be used pre- and post-simulation training, and is relevant across a range of healthcare professions. Through consultation with a multi-professional expert group, we developed and piloted a 39-item survey with 272 healthcare professionals attending training courses across two large simulation centres in London, one specialising in acute care and one in mental health, both serving healthcare professionals working across acute and community settings. Following psychometric evaluation, the final 12-item instrument was evaluated with a second sample of 711 trainees. Exploratory factor analysis revealed a 12-item, one-factor solution with good internal consistency (α=0.92). The instrument had discriminant validity, with newly qualified trainees scoring significantly lower than experienced trainees ( t (98)=4.88, p<0.001) and was sensitive to change following training in acute and mental health settings, across professional groups (p<0.001). Confirmatory factor analysis revealed an adequate model fit (RMSEA=0.066). The Human Factors Skills for Healthcare Instrument provides a reliable and valid method of assessing trainees' human factors skills self-efficacy across acute and mental health settings. This instrument has the potential to improve the assessment and evaluation of human factors skills learning in both uniprofessional and interprofessional clinical simulation training.

An improved cellular automata model for train operation simulation with dynamic acceleration

NASA Astrophysics Data System (ADS)

Li, Wen-Jun; Nie, Lei

2018-03-01

Urban rail transit plays an important role in the urban public traffic because of its advantages of fast speed, large transport capacity, high safety, reliability and low pollution. This study proposes an improved cellular automaton (CA) model by considering the dynamic characteristic of the train acceleration to analyze the energy consumption and train running time. Constructing an effective model for calculating energy consumption to aid train operation improvement is the basis for studying and analyzing energy-saving measures for urban rail transit system operation.

Disease severity assessment in epidemiological studies: accuracy and reliability of visual estimates of Septoria leaf blotch (SLB) in winter wheat

USDA-ARS?s Scientific Manuscript database

The accuracy and reliability of visual assessments of SLB severity by raters (i.e. one plant pathologist with extensive experience and three other raters trained prior to field observations using standard area diagrams and DISTRAIN) was determined by comparison with assumed actual values obtained by...

Influences of Response Rate and Distribution on the Calculation of Interobserver Reliability Scores

ERIC Educational Resources Information Center

Rolider, Natalie U.; Iwata, Brian A.; Bullock, Christopher E.

2012-01-01

We examined the effects of several variations in response rate on the calculation of total, interval, exact-agreement, and proportional reliability indices. Trained observers recorded computer-generated data that appeared on a computer screen. In Study 1, target responses occurred at low, moderate, and high rates during separate sessions so that…

An Application of Generalizability Theory on Writing Assessment: Effects of Marking Components Weighting

ERIC Educational Resources Information Center

Lam, Ling Chi Tenny

2010-01-01

In writing assessment, there are quite a number of factors influencing the marking stability and the reliability of the assessment such as the attitude towards marking and consistency of markers, the physical environment, the design of the items, and marking rubrics. Even the methods to train markers have effects on the reliability of the…

Transferable Competences of Young People with a High Dropout Risk in Vocational Training in Germany

ERIC Educational Resources Information Center

Frey, Andreas; Balzer, Lars; Ruppert, Jean-Jacques

2014-01-01

This paper examines whether the subjective beliefs on their competences of 409 trainees in machinery, sales, and logistics constitute a reliable and valid way to measure transferable competences. The analysis of results attributes satisfactory to good reliability values to the assessment procedure. Furthermore, it could be shown that young people…

76 FR 11514 - Comment Request for information Collection for Employment and Training (ET) Handbook 361...

Federal Register 2010, 2011, 2012, 2013, 2014

2011-03-02

... reliability of the performance data included in it [that] * * * describes any material inadequacies in the completeness and reliability of the data.'' (OMB Circular A-11, Section 230.2(f)). The Administrations' agenda... to operate between the mid-1970s and 2000. The WV program checked the validity of 29 report elements...

Second-Order Conditioning of Human Causal Learning

ERIC Educational Resources Information Center

Jara, Elvia; Vila, Javier; Maldonado, Antonio

2006-01-01

This article provides the first demonstration of a reliable second-order conditioning (SOC) effect in human causal learning tasks. It demonstrates the human ability to infer relationships between a cause and an effect that were never paired together during training. Experiments 1a and 1b showed a clear and reliable SOC effect, while Experiments 2a…

A Preliminary Examination of the In-Training Evaluation Report

ERIC Educational Resources Information Center

Skakun, Ernest N.; And Others

1975-01-01

The In-Training Evaluation Report (ITER), in use by the Royal College of Physicians and Surgeons of Canada for examining the competencies of candidates eligible for the certifying examination, was tested for validity and reliability. This analysis suggests revisions but declares the ITEA a useful instrument to aid in candidate assessment. (JT)

The Coordinators of Psychiatric Education (COPE) Residency In-Training Exam: A Preliminary Psychometric Assessment

ERIC Educational Resources Information Center

Montgomery, Gregory P. J.; Crockford, David N.; Hecker, Kent

2010-01-01

Objective: The Coordinators of Psychiatric Education (COPE) Residency In-Training Exam is a formative exam for Canadian psychiatric residents that was reconstructed using assessment best practices. An assessment of psychometric properties was subsequently performed on the exam to ensure preliminary validity and reliability. Methods: An exam…

Observer Use of Standardized Observation Protocols in Consequential Observation Systems

ERIC Educational Resources Information Center

Bell, Courtney A.; Yi, Qi; Jones, Nathan D.; Lewis, Jennifer M.; McLeod, Monica; Liu, Shuangshuang

2014-01-01

Evidence from a handful of large-scale studies suggests that although observers can be trained to score reliably using observation protocols, there are concerns related to initial training and calibration activities designed to keep observers scoring accurately over time (e.g., Bell, et al, 2012; BMGF, 2012). Studies offer little insight into how…

The Development of Chorus Motivation Scale (CMS) for Prospective Music Teacher

ERIC Educational Resources Information Center

Ozgul, Ilhan; Yigit, Nalan

2017-01-01

The purpose of this study was to develop a Chorus Motivation Scale (CMS) that is tested in terms of reliability and construct validity by determining the student perceptions of effective motivation strategies in Chorus training in Turkish Music Teacher Training Model. In order to develop a Chorus Motivation Scale, Questionnaire-Effective…

The Effects of Instructor Differences Upon Student Progress in Naval Aviation Training. Medical Research Progress No. 4.

ERIC Educational Resources Information Center

Waag, Wayne L.; Shannon, Richard H.

The investigation attempted to determine: whether instructor differences could be measured quantitatively; if such differences affected the grades which they assigned; if such differences affected the student's progress through the flight training program. Using an unstructured rating form, it was found that reliable instructor differences could…

The Application of Strength of Association Statistics to the Item Analysis of an In-Training Examination in Diagnostic Radiology.

ERIC Educational Resources Information Center

Diamond, James J.; McCormick, Janet

1986-01-01

Using item responses from an in-training examination in diagnostic radiology, the application of a strength of association statistic to the general problem of item analysis is illustrated. Criteria for item selection, general issues of reliability, and error of measurement are discussed. (Author/LMO)

Optimal design of the absolute positioning sensor for a high-speed maglev train and research on its fault diagnosis.

PubMed

Zhang, Dapeng; Long, Zhiqiang; Xue, Song; Zhang, Junge

2012-01-01

This paper studies an absolute positioning sensor for a high-speed maglev train and its fault diagnosis method. The absolute positioning sensor is an important sensor for the high-speed maglev train to accomplish its synchronous traction. It is used to calibrate the error of the relative positioning sensor which is used to provide the magnetic phase signal. On the basis of the analysis for the principle of the absolute positioning sensor, the paper describes the design of the sending and receiving coils and realizes the hardware and the software for the sensor. In order to enhance the reliability of the sensor, a support vector machine is used to recognize the fault characters, and the signal flow method is used to locate the faulty parts. The diagnosis information not only can be sent to an upper center control computer to evaluate the reliability of the sensors, but also can realize on-line diagnosis for debugging and the quick detection when the maglev train is off-line. The absolute positioning sensor we study has been used in the actual project.

Do in-training evaluation reports deserve their bad reputations? A study of the reliability and predictive ability of ITER scores and narrative comments.

PubMed

Ginsburg, Shiphra; Eva, Kevin; Regehr, Glenn

2013-10-01

Although scores on in-training evaluation reports (ITERs) are often criticized for poor reliability and validity, ITER comments may yield valuable information. The authors assessed across-rotation reliability of ITER scores in one internal medicine program, ability of ITER scores and comments to predict postgraduate year three (PGY3) performance, and reliability and incremental predictive validity of attendings' analysis of written comments. Numeric and narrative data from the first two years of ITERs for one cohort of residents at the University of Toronto Faculty of Medicine (2009-2011) were assessed for reliability and predictive validity of third-year performance. Twenty-four faculty attendings rank-ordered comments (without scores) such that each resident was ranked by three faculty. Mean ITER scores and comment rankings were submitted to regression analyses; dependent variables were PGY3 ITER scores and program directors' rankings. Reliabilities of ITER scores across nine rotations for 63 residents were 0.53 for both postgraduate year one (PGY1) and postgraduate year two (PGY2). Interrater reliabilities across three attendings' rankings were 0.83 for PGY1 and 0.79 for PGY2. There were strong correlations between ITER scores and comments within each year (0.72 and 0.70). Regressions revealed that PGY1 and PGY2 ITER scores collectively explained 25% of variance in PGY3 scores and 46% of variance in PGY3 rankings. Comment rankings did not improve predictions. ITER scores across multiple rotations showed decent reliability and predictive validity. Comment ranks did not add to the predictive ability, but correlation analyses suggest that trainee performance can be measured through these comments.

An intervention to improve the reliability of manuscript reviews for the Journal of the American Academy of Child and Adolescent Psychiatry.

PubMed

Strayhorn, J; McDermott, J F; Tanguay, P

1993-06-01

The effects of methods used to improve the interrater reliability of reviewers' ratings of manuscripts submitted to the Journal of the American Academy of Child and Adolescent Psychiatry were studied. Reviewers' ratings of consecutive manuscripts submitted over approximately 1 year were first analyzed; 296 pairs of ratings were studied. Intraclass correlations and confidence intervals for the correlations were computed for the two main ratings by which reviewers quantified the quality of the article: a 1-10 overall quality rating and a recommendation for acceptance or rejection with four possibilities along that continuum. Modifications were then introduced, including a multi-item rating scale and two training manuals to accompany it. Over the next year, 272 more articles were rated, and reliabilities were computed for the new scale and for the scales previously used. The intraclass correlation of the most reliable rating before the intervention was 0.27; the reliability of the new rating procedure was 0.43. The difference between these two was significant. The reliability for the new rating scale was in the fair to good range, and it became even better when the ratings of the two reviewers were averaged and the reliability stepped up by the Spearman-Brown formula. The new rating scale had excellent internal consistency and correlated highly with other quality ratings. The data confirm that the reliability of ratings of scientific articles may be improved by increasing the number of rating scale points, eliciting ratings of separate, concrete items rather than a global judgment, using training manuals, and averaging the scores of multiple reviewers.

The Reliability of Individualized Load-Velocity Profiles.

PubMed

Banyard, Harry G; Nosaka, K; Vernon, Alex D; Haff, G Gregory

2017-11-15

This study examined the reliability of peak velocity (PV), mean propulsive velocity (MPV), and mean velocity (MV) in the development of load-velocity profiles (LVP) in the full depth free-weight back squat performed with maximal concentric effort. Eighteen resistance-trained men performed a baseline one-repetition maximum (1RM) back squat trial and three subsequent 1RM trials used for reliability analyses, with 48-hours interval between trials. 1RM trials comprised lifts from six relative loads including 20, 40, 60, 80, 90, and 100% 1RM. Individualized LVPs for PV, MPV, or MV were derived from loads that were highly reliable based on the following criteria: intra-class correlation coefficient (ICC) >0.70, coefficient of variation (CV) ≤10%, and Cohen's d effect size (ES) <0.60. PV was highly reliable at all six loads. Importantly, MPV and MV were highly reliable at 20, 40, 60, 80 and 90% but not 100% 1RM (MPV: ICC=0.66, CV=18.0%, ES=0.10, standard error of the estimate [SEM]=0.04m·s -1 ; MV: ICC=0.55, CV=19.4%, ES=0.08, SEM=0.04m·s -1 ). When considering the reliable ranges, almost perfect correlations were observed for LVPs derived from PV 20-100% (r=0.91-0.93), MPV 20-90% (r=0.92-0.94) and MV 20-90% (r=0.94-0.95). Furthermore, the LVPs were not significantly different (p>0.05) between trials, movement velocities, or between linear regression versus second order polynomial fits. PV 20-100% , MPV 20-90% , and MV 20-90% are reliable and can be utilized to develop LVPs using linear regression. Conceptually, LVPs can be used to monitor changes in movement velocity and employed as a method for adjusting sessional training loads according to daily readiness.

Tailoring a Human Reliability Analysis to Your Industry Needs

NASA Technical Reports Server (NTRS)

DeMott, D. L.

2016-01-01

Companies at risk of accidents caused by human error that result in catastrophic consequences include: airline industry mishaps, medical malpractice, medication mistakes, aerospace failures, major oil spills, transportation mishaps, power production failures and manufacturing facility incidents. Human Reliability Assessment (HRA) is used to analyze the inherent risk of human behavior or actions introducing errors into the operation of a system or process. These assessments can be used to identify where errors are most likely to arise and the potential risks involved if they do occur. Using the basic concepts of HRA, an evolving group of methodologies are used to meet various industry needs. Determining which methodology or combination of techniques will provide a quality human reliability assessment is a key element to developing effective strategies for understanding and dealing with risks caused by human errors. There are a number of concerns and difficulties in "tailoring" a Human Reliability Assessment (HRA) for different industries. Although a variety of HRA methodologies are available to analyze human error events, determining the most appropriate tools to provide the most useful results can depend on industry specific cultures and requirements. Methodology selection may be based on a variety of factors that include: 1) how people act and react in different industries, 2) expectations based on industry standards, 3) factors that influence how the human errors could occur such as tasks, tools, environment, workplace, support, training and procedure, 4) type and availability of data, 5) how the industry views risk & reliability, and 6) types of emergencies, contingencies and routine tasks. Other considerations for methodology selection should be based on what information is needed from the assessment. If the principal concern is determination of the primary risk factors contributing to the potential human error, a more detailed analysis method may be employed versus a requirement to provide a numerical value as part of a probabilistic risk assessment. Industries involved with humans operating large equipment or transport systems (ex. railroads or airlines) would have more need to address the man machine interface than medical workers administering medications. Human error occurs in every industry; in most cases the consequences are relatively benign and occasionally beneficial. In cases where the results can have disastrous consequences, the use of Human Reliability techniques to identify and classify the risk of human errors allows a company more opportunities to mitigate or eliminate these types of risks and prevent costly tragedies.

Nutrition Knowledge and Training Needs in the School Environment

NASA Astrophysics Data System (ADS)

Jones, Anna Marie

The nutrition environment in schools can influence the risk for childhood overweight and obesity, which in turn can have life-long implications for risk of chronic disease. This dissertation aimed to examine the nutrition environment in primary public schools in California with regards to the amount of nutrition education provided in the classroom, the nutrition knowledge of teachers, and the training needs of school nutrition personnel. In order to determine nutrition knowledge of teachers, a valid and reliable questionnaire was developed to assess knowledge. The systematic process involved cognitive interviews, a mail-based pretest that utilized a random sample of addresses in California, and validity and reliability testing in a sample of university students. Results indicated that the questionnaire had adequate construct validity, internal consistency reliability, and test-retest reliability. Following the validation of the knowledge questionnaire, it was used in a study of public school teachers in California to determine the relationship between demographic and classroom characteristics and nutrition knowledge, in addition to barriers to nutrition education and resources used to plan nutrition lessons. Nutrition knowledge was not found to be associated with teaching nutrition in the classroom, however it was associated with gender, identifying as Hispanic or Latino, and grade level grouping taught. The most common barriers to nutrition education were time, and unrelated subject matter. The most commonly used resources to plan nutrition lessons were Dairy Council of California educational materials. The school nutrition program was the second area of the school nutrition environment to be examined, and the primary focus was to determine the perceived training needs of California school nutrition personnel. Respondents indicated a need for training in topics related to: program management; the Healthy, Hunger-Free Kids Act of 2010; nutrition, health and wellness; planning, preparing, and serving meals; and communication and marketing. Those employed in residential child care institutions expressed a strong need for training specific to this type of program. Overall, the results of this dissertation contribute to the body of knowledge about nutrition in the school environment and raise interesting questions to be examined in future studies.

Resilient Systemics to Telehealth Support for Clinical Psychiatry and Psychology.

PubMed

Fiorini, Rodolfo A; De Giacomo, Piero; L'Abate, Luciano

2015-01-01

Reliably expanding our clinical practice and lowering our overhead with telepsychiatry, telepsychology, distance counseling and online therapy, requires resilient and antifragile system and tools. When utilized appropriately these technologies may provide greater access to needed services to include more reliable treatment, consultation, supervision, and training. The wise and proper use of technology is fundamental to create and boost outstanding social results. We present, as an example, the main steps to achieve application resilience and antifragility at system level, for diagnostic and therapeutic telepractice and telehealth support, devoted to psychiatry and psychology application. This article presents a number of innovations that can take psychotherapy treatment, supervision, training, and research forward, towards increased effectiveness application.

Study of complete interconnect reliability for a GaAs MMIC power amplifier

NASA Astrophysics Data System (ADS)

Lin, Qian; Wu, Haifeng; Chen, Shan-ji; Jia, Guoqing; Jiang, Wei; Chen, Chao

2018-05-01

By combining the finite element analysis (FEA) and artificial neural network (ANN) technique, the complete prediction of interconnect reliability for a monolithic microwave integrated circuit (MMIC) power amplifier (PA) at the both of direct current (DC) and alternating current (AC) operation conditions is achieved effectively in this article. As a example, a MMIC PA is modelled to study the electromigration failure of interconnect. This is the first time to study the interconnect reliability for an MMIC PA at the conditions of DC and AC operation simultaneously. By training the data from FEA, a high accuracy ANN model for PA reliability is constructed. Then, basing on the reliability database which is obtained from the ANN model, it can give important guidance for improving the reliability design for IC.

The need to approximate the use-case in clinical machine learning.

PubMed

Saeb, Sohrab; Lonini, Luca; Jayaraman, Arun; Mohr, David C; Kording, Konrad P

2017-05-01

The availability of smartphone and wearable sensor technology is leading to a rapid accumulation of human subject data, and machine learning is emerging as a technique to map those data into clinical predictions. As machine learning algorithms are increasingly used to support clinical decision making, it is vital to reliably quantify their prediction accuracy. Cross-validation (CV) is the standard approach where the accuracy of such algorithms is evaluated on part of the data the algorithm has not seen during training. However, for this procedure to be meaningful, the relationship between the training and the validation set should mimic the relationship between the training set and the dataset expected for the clinical use. Here we compared two popular CV methods: record-wise and subject-wise. While the subject-wise method mirrors the clinically relevant use-case scenario of diagnosis in newly recruited subjects, the record-wise strategy has no such interpretation. Using both a publicly available dataset and a simulation, we found that record-wise CV often massively overestimates the prediction accuracy of the algorithms. We also conducted a systematic review of the relevant literature, and found that this overly optimistic method was used by almost half of the retrieved studies that used accelerometers, wearable sensors, or smartphones to predict clinical outcomes. As we move towards an era of machine learning-based diagnosis and treatment, using proper methods to evaluate their accuracy is crucial, as inaccurate results can mislead both clinicians and data scientists. © The Author 2017. Published by Oxford University Press.

Preliminary testing of the reliability and feasibility of SAGE: a system to measure and score engagement with and use of research in health policies and programs.

PubMed

Makkar, Steve R; Williamson, Anna; D'Este, Catherine; Redman, Sally

2017-12-19

Few measures of research use in health policymaking are available, and the reliability of such measures has yet to be evaluated. A new measure called the Staff Assessment of Engagement with Evidence (SAGE) incorporates an interview that explores policymakers' research use within discrete policy documents and a scoring tool that quantifies the extent of policymakers' research use based on the interview transcript and analysis of the policy document itself. We aimed to conduct a preliminary investigation of the usability, sensitivity, and reliability of the scoring tool in measuring research use by policymakers. Nine experts in health policy research and two independent coders were recruited. Each expert used the scoring tool to rate a random selection of 20 interview transcripts, and each independent coder rated 60 transcripts. The distribution of scores among experts was examined, and then, interrater reliability was tested within and between the experts and independent coders. Average- and single-measure reliability coefficients were computed for each SAGE subscales. Experts' scores ranged from the limited to extensive scoring bracket for all subscales. Experts as a group also exhibited at least a fair level of interrater agreement across all subscales. Single-measure reliability was at least fair except for three subscales: Relevance Appraisal, Conceptual Use, and Instrumental Use. Average- and single-measure reliability among independent coders was good to excellent for all subscales. Finally, reliability between experts and independent coders was fair to excellent for all subscales. Among experts, the scoring tool was comprehensible, usable, and sensitive to discriminate between documents with varying degrees of research use. Secondly, the scoring tool yielded scores with good reliability among the independent coders. There was greater variability among experts, although as a group, the tool was fairly reliable. The alignment between experts' and independent coders' ratings indicates that the independent coders were scoring in a manner comparable to health policy research experts. If the present findings are replicated in a larger sample, end users (e.g. policy agency staff) could potentially be trained to use SAGE to reliably score research use within their agencies, which would provide a cost-effective and time-efficient approach to utilising this measure in practice.

Buying a Laser - Tips and Pearls

PubMed Central

Aurangabadkar, Sanjeev J; Mysore, Venkataram; Ahmed, E Suhail

2014-01-01

Lasers and aesthetic procedures have transformed dermatology practice. They have aided in the treatment of hitherto untreatable conditions and allowed better financial remuneration to the physician. The availability of a variety of laser devices of different makes, specifications and pricing has lead to confusion and dilemma in the mind of the buying physician. There are presently no guidelines available for buying a laser. Since purchase of a laser involves large investments, careful consideration to laser specifications, training, costing, warranty, availability of spares, and reliability of service are important prerequisites. This article describes various factors that are needed to be considered and also attempts to lay down criteria to be assessed while buying a laser system that will be useful to physicians before investing in a laser machine. Practice points Meticulous planning of the type of machine, specifications, financial aspects, maintenance and warranties is important.It is wise to sign a contract or agreement between the buyer and seller before purchase of a laser which covers key aspects of installation, after sales service and maintenance of the machine.Adequate training is essential; understanding laser physics and laser-tissue interaction goes a long way in getting the best out of the machine.The credibility of the dealer and company should be ascertained in order to be assured of after-sales service.Buying used machines, sharing of equipment to offset high initial investments is a good option but even more care is required to ensure proper functioning and maintenance. PMID:25136218

A machine learning pipeline for automated registration and classification of 3D lidar data

NASA Astrophysics Data System (ADS)

Rajagopal, Abhejit; Chellappan, Karthik; Chandrasekaran, Shivkumar; Brown, Andrew P.

2017-05-01

Despite the large availability of geospatial data, registration and exploitation of these datasets remains a persis- tent challenge in geoinformatics. Popular signal processing and machine learning algorithms, such as non-linear SVMs and neural networks, rely on well-formatted input models as well as reliable output labels, which are not always immediately available. In this paper we outline a pipeline for gathering, registering, and classifying initially unlabeled wide-area geospatial data. As an illustrative example, we demonstrate the training and test- ing of a convolutional neural network to recognize 3D models in the OGRIP 2007 LiDAR dataset using fuzzy labels derived from OpenStreetMap as well as other datasets available on OpenTopography.org. When auxiliary label information is required, various text and natural language processing filters are used to extract and cluster keywords useful for identifying potential target classes. A subset of these keywords are subsequently used to form multi-class labels, with no assumption of independence. Finally, we employ class-dependent geometry extraction routines to identify candidates from both training and testing datasets. Our regression networks are able to identify the presence of 6 structural classes, including roads, walls, and buildings, in volumes as big as 8000 m3 in as little as 1.2 seconds on a commodity 4-core Intel CPU. The presented framework is neither dataset nor sensor-modality limited due to the registration process, and is capable of multi-sensor data-fusion.

Buying a laser - tips and pearls.

PubMed

Aurangabadkar, Sanjeev J; Mysore, Venkataram; Ahmed, E Suhail

2014-04-01

Lasers and aesthetic procedures have transformed dermatology practice. They have aided in the treatment of hitherto untreatable conditions and allowed better financial remuneration to the physician. The availability of a variety of laser devices of different makes, specifications and pricing has lead to confusion and dilemma in the mind of the buying physician. There are presently no guidelines available for buying a laser. Since purchase of a laser involves large investments, careful consideration to laser specifications, training, costing, warranty, availability of spares, and reliability of service are important prerequisites. This article describes various factors that are needed to be considered and also attempts to lay down criteria to be assessed while buying a laser system that will be useful to physicians before investing in a laser machine. Meticulous planning of the type of machine, specifications, financial aspects, maintenance and warranties is important.It is wise to sign a contract or agreement between the buyer and seller before purchase of a laser which covers key aspects of installation, after sales service and maintenance of the machine.Adequate training is essential; understanding laser physics and laser-tissue interaction goes a long way in getting the best out of the machine.The credibility of the dealer and company should be ascertained in order to be assured of after-sales service.Buying used machines, sharing of equipment to offset high initial investments is a good option but even more care is required to ensure proper functioning and maintenance.

Evaluation of machine learning algorithms for prediction of regions of high Reynolds averaged Navier Stokes uncertainty

DOE PAGES

Ling, Julia; Templeton, Jeremy Alan

2015-08-04

Reynolds Averaged Navier Stokes (RANS) models are widely used in industry to predict fluid flows, despite their acknowledged deficiencies. Not only do RANS models often produce inaccurate flow predictions, but there are very limited diagnostics available to assess RANS accuracy for a given flow configuration. If experimental or higher fidelity simulation results are not available for RANS validation, there is no reliable method to evaluate RANS accuracy. This paper explores the potential of utilizing machine learning algorithms to identify regions of high RANS uncertainty. Three different machine learning algorithms were evaluated: support vector machines, Adaboost decision trees, and random forests.more » The algorithms were trained on a database of canonical flow configurations for which validated direct numerical simulation or large eddy simulation results were available, and were used to classify RANS results on a point-by-point basis as having either high or low uncertainty, based on the breakdown of specific RANS modeling assumptions. Classifiers were developed for three different basic RANS eddy viscosity model assumptions: the isotropy of the eddy viscosity, the linearity of the Boussinesq hypothesis, and the non-negativity of the eddy viscosity. It is shown that these classifiers are able to generalize to flows substantially different from those on which they were trained. As a result, feature selection techniques, model evaluation, and extrapolation detection are discussed in the context of turbulence modeling applications.« less

The Imperial Paediatric Emergency Training Toolkit (IPETT) for use in paediatric emergency training: development and evaluation of feasibility and validity.

PubMed

Lambden, Simon; DeMunter, Claudine; Dowson, Anne; Cooper, Mehrengise; Gautama, Sanjay; Sevdalis, Nick

2013-06-01

To develop and test the feasibility, reliability, and validity of a practical toolkit for the assessment and feedback of skills required to manage paediatric emergencies in critical care settings. The Imperial Paediatric Emergency Training Toolkit (IPETT) was developed based on current evidence-base and expert input. IPETT assesses both technical and non-technical skills. The technical component covers skills in the areas of clinical assessment, airway and breathing, cardiovascular, and drugs. The non-technical component is based on the validated NOTECHS tool and covers communication and interaction, cooperation and team skills, leadership and managerial skills, and decision-making. The reliability (internal consistency), content validity (inter-correlations between different skills) and concurrent validity (correlations between global technical and non-technical scores) of IPETT were prospectively evaluated in 45 simulated paediatric crises carried out in a PICU with anaesthetic and paediatric trainees (N=52). Non-parametric analyses were carried out. Significance was set at P<0.05. Cronbach alpha reliability coefficients were overall acceptable for the technical (alpha range=0.638-0.810) and good for the non-technical (alpha range=0.701-0.899) component of IPETT. The median inter-skill correlation was rho=0.564 and rho=0.549 for the technical and non-technical components, respectively. These indicate good content validity, as the skills were inter-related but not redundant. We also demonstrate a correlation between the global technical and non-technical scores (rho=0.471) - all Ps<0.05 during the assessments. IPETT offers a psychometrically viable and feasible to use tool in the context of paediatric emergencies training. This study shows that assessment of technical and non-technical skills in combination may offer a more clinically relevant model for training in paediatric emergencies. Further validation should aim to demonstrate skill retention over time and skill transfer from simulation-based training to real emergencies. Copyright © 2013. Published by Elsevier Ireland Ltd.

Internal validation of the RapidHIT® ID system.

PubMed

Wiley, Rachel; Sage, Kelly; LaRue, Bobby; Budowle, Bruce

2017-11-01

Traditionally, forensic DNA analysis has required highly skilled forensic geneticists in a dedicated laboratory to generate short tandem repeat (STR) profiles. STR profiles are routinely used either to associate or exclude potential donors of forensic biological evidence. The typing of forensic reference samples has become more demanding, especially with the requirement in some jurisdictions to DNA profile arrestees. The Rapid DNA (RDNA) platform, the RapidHIT ® ID (IntegenX ® , Pleasanton, CA), is a fully automated system capable of processing reference samples in approximately 90min with minimal human intervention. Thus, the RapidHIT ID instrument can be deployed to non-laboratory environments (e.g., booking stations) and run by trained atypical personnel such as law enforcement. In order to implement the RapidHIT ID platform, validation studies are needed to define the performance and limitations of the system. Internal validation studies were undertaken with four early-production RapidHIT ID units. Reliable and concordant STR profiles were obtained from reference buccal swabs. Throughout the study, no contamination was observed. The overall first-pass success rate with an "expert-like system" was 72%, which is comparable to another current RDNA platform commercially available. The system's second-pass success rate (involving manual interpretation on first-pass inconclusive results) increased to 90%. Inhibitors (i.e., coffee, smoking tobacco, and chewing tobacco) did not appear to affect typing by the instrument system; however, substrate (i.e., swab type) did impact typing success. Additionally, one desirable feature not available with other Rapid systems is that in the event of a system failed run, a swab can be recovered and subsequently re-analyzed in a new sample cartridge. Therefore, rarely should additional sampling or swab consumption be necessary. The RapidHIT ID system is a robust and reliable tool capable of generating complete STR profiles within the forensic DNA typing laboratory or with proper training in decentralized environments by non-laboratory personnel. Copyright © 2017 Elsevier B.V. All rights reserved.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Jens Korsgaard

The main conclusions from the work carried out under this contract are: An ordinary seafarer can learn by training on a simulator, to moor large tanker vessels to the Hydrostatic Mooring, safely and quickly, in all weather conditions up to storms generating waves with a significant wave height of 8 m. Complete conceptual design of the Hydrostatic Mooring buoy was carried out which proved that the buoy could be constructed entirely from commercially available standard components and materials. The design is robust, and damage resistant. The mooring tests had a 100% success rate from the point of view of themore » buoy being securely attached and moored to the vessel following every mooring attempt. The tests had an 80% success rate from the point of view of the buoy being adequately centered such that petroleum transfer equipment on the vessel could be attached to the corresponding equipment on the buoy. The results given in Table 3-2 of the mooring tests show a consistently improving performance from test to test by the Captain that performed the mooring operations. This is not surprising, in view of the fact that the Captain had only three days of training on the simulator prior to conducting the tests, that the maneuvering required is non-standard, and the test program itself lasted four days. One conclusion of the test performance is that the Captain was not fully trained at the initiation of the test. It may therefore be concluded that a thoroughly trained navigator would probably be able to make the mooring such that the fluid transfer equipment can be connected with reliability in excess of 90%. Considering that the typical standard buoy has enough power aboard to make eight mooring attempts, this implies that the probability that the mooring attempt should fail because of the inability to connect the fluid transfer equipment is of the order of 10{sup {minus}8}. It may therefore be concluded that the mooring operation between a Hydrostatic Mooring and a large tanker vessel can be carried out with near absolute reliability in all sea states up to a sea state where the significant wave height is 8 m.« less

Limited generalization with varied, as compared to specific, practice in short-term motor learning.

PubMed

Willey, Chéla R; Liu, Zili

2018-01-01

The schema theory of learning predicts that varied training in motor learning should give rise to better transfer than specific training. For example, throwing beanbags during practice to targets 5 and 9ft away should better generalize to targets 7 and 11ft away, as compared to only throwing to a target 7ft away. In this study, we tested this prediction in a throwing task, when the pretest, practice, and posttest were all completed within an hour. Participants in the varied group practiced throwing at 5 and 9ft targets, while participants in the specific group practiced throwing at 7ft only. All participants reliably reduced errors from pretest to posttest. The varied group never outperformed the specific group at the 7ft target (the trained target for the specific group). They did not reliably outperform the specific group at 11ft, either. The numerically better performance at 11ft by the varied group was due, as it turned out in a subsequent experiment, to the fact that 11ft was closer to 9ft (one of the two training targets for the varied group) than to 7ft (the training target for the specific group). We conclude that varied training played a very limited role in short-term motor learning. Copyright © 2017 Elsevier B.V. All rights reserved.

Single-shot T2 mapping using overlapping-echo detachment planar imaging and a deep convolutional neural network.

PubMed

Cai, Congbo; Wang, Chao; Zeng, Yiqing; Cai, Shuhui; Liang, Dong; Wu, Yawen; Chen, Zhong; Ding, Xinghao; Zhong, Jianhui

2018-04-24

An end-to-end deep convolutional neural network (CNN) based on deep residual network (ResNet) was proposed to efficiently reconstruct reliable T 2 mapping from single-shot overlapping-echo detachment (OLED) planar imaging. The training dataset was obtained from simulations that were carried out on SPROM (Simulation with PRoduct Operator Matrix) software developed by our group. The relationship between the original OLED image containing two echo signals and the corresponding T 2 mapping was learned by ResNet training. After the ResNet was trained, it was applied to reconstruct the T 2 mapping from simulation and in vivo human brain data. Although the ResNet was trained entirely on simulated data, the trained network was generalized well to real human brain data. The results from simulation and in vivo human brain experiments show that the proposed method significantly outperforms the echo-detachment-based method. Reliable T 2 mapping with higher accuracy is achieved within 30 ms after the network has been trained, while the echo-detachment-based OLED reconstruction method took approximately 2 min. The proposed method will facilitate real-time dynamic and quantitative MR imaging via OLED sequence, and deep convolutional neural network has the potential to reconstruct maps from complex MRI sequences efficiently. © 2018 International Society for Magnetic Resonance in Medicine.

Autogenic-feedback training - A treatment for motion and space sickness

NASA Technical Reports Server (NTRS)

Cowings, Patricia S.

1990-01-01

A training method for preventing the occurrence of motion sickness in humans, called autogenic-feedback training (AFT), is described. AFT is based on a combination of biofeedback and autogenic therapy which involves training physiological self-regulation as an alternative to pharmacological management. AFT was used to reliably increase tolerance to motion-sickness-inducing tests in both men and women ranging in age from 18 to 54 years. The effectiveness of AFT is found to be significantly higher than that of protective adaptation training. Data obtained show that there is no apparent effect from AFT on measures of vestibular perception and no side effects.

Measuring competence in endoscopic sinus surgery.

PubMed

Syme-Grant, J; White, P S; McAleer, J P G

2008-02-01

Competence based education is currently being introduced into higher surgical training in the UK. Valid and reliable performance assessment tools are essential to ensure competencies are achieved. No such tools have yet been reported in the UK literature. We sought to develop and pilot test an Endoscopic Sinus Surgery Competence Assessment Tool (ESSCAT). The ESSCAT was designed for in-theatre assessment of higher surgical trainees in the UK. The ESSCAT rating matrix was developed through task analysis of ESS procedures. All otolaryngology consultants and specialist registrars in Scotland were given the opportunity to contribute to its refinement. Two cycles of in-theatre testing were used to ensure utility and gather quantitative data on validity and reliability. Videos of trainees performing surgery were used in establishing inter-rater reliability. National consultation, the consensus derived minimum standard of performance, Cronbach's alpha = 0.89 and demonstration of trainee learning (p = 0.027) during the in vivo application of the ESSCAT suggest a high level of validity. Inter-rater reliability was moderate for competence decisions (Cohen's Kappa = 0.5) and good for total scores (Intra-Class Correlation Co-efficient = 0.63). Intra-rater reliability was good for both competence decisions (Kappa = 0.67) and total scores (Kendall's Tau-b = 0.73). The ESSCAT generates a valid and reliable assessment of trainees' in-theatre performance of endoscopic sinus surgery. In conjunction with ongoing evaluation of the instrument we recommend the use of the ESSCAT in higher specialist training in otolaryngology in the UK.

Validity evidence and reliability of a simulated patient feedback instrument.

PubMed

Schlegel, Claudia; Woermann, Ulrich; Rethans, Jan-Joost; van der Vleuten, Cees

2012-01-27

In the training of healthcare professionals, one of the advantages of communication training with simulated patients (SPs) is the SP's ability to provide direct feedback to students after a simulated clinical encounter. The quality of SP feedback must be monitored, especially because it is well known that feedback can have a profound effect on student performance. Due to the current lack of valid and reliable instruments to assess the quality of SP feedback, our study examined the validity and reliability of one potential instrument, the 'modified Quality of Simulated Patient Feedback Form' (mQSF). Content validity of the mQSF was assessed by inviting experts in the area of simulated clinical encounters to rate the importance of the mQSF items. Moreover, generalizability theory was used to examine the reliability of the mQSF. Our data came from videotapes of clinical encounters between six simulated patients and six students and the ensuing feedback from the SPs to the students. Ten faculty members judged the SP feedback according to the items on the mQSF. Three weeks later, this procedure was repeated with the same faculty members and recordings. All but two items of the mQSF received importance ratings of > 2.5 on a four-point rating scale. A generalizability coefficient of 0.77 was established with two judges observing one encounter. The findings for content validity and reliability with two judges suggest that the mQSF is a valid and reliable instrument to assess the quality of feedback provided by simulated patients.

Training set expansion: an approach to improving the reconstruction of biological networks from limited and uneven reliable interactions

PubMed Central

Yip, Kevin Y.; Gerstein, Mark

2009-01-01

Motivation: An important problem in systems biology is reconstructing complete networks of interactions between biological objects by extrapolating from a few known interactions as examples. While there are many computational techniques proposed for this network reconstruction task, their accuracy is consistently limited by the small number of high-confidence examples, and the uneven distribution of these examples across the potential interaction space, with some objects having many known interactions and others few. Results: To address this issue, we propose two computational methods based on the concept of training set expansion. They work particularly effectively in conjunction with kernel approaches, which are a popular class of approaches for fusing together many disparate types of features. Both our methods are based on semi-supervised learning and involve augmenting the limited number of gold-standard training instances with carefully chosen and highly confident auxiliary examples. The first method, prediction propagation, propagates highly confident predictions of one local model to another as the auxiliary examples, thus learning from information-rich regions of the training network to help predict the information-poor regions. The second method, kernel initialization, takes the most similar and most dissimilar objects of each object in a global kernel as the auxiliary examples. Using several sets of experimentally verified protein–protein interactions from yeast, we show that training set expansion gives a measurable performance gain over a number of representative, state-of-the-art network reconstruction methods, and it can correctly identify some interactions that are ranked low by other methods due to the lack of training examples of the involved proteins. Contact: mark.gerstein@yale.edu Availability: The datasets and additional materials can be found at http://networks.gersteinlab.org/tse. PMID:19015141

Cross-education of muscular strength following unilateral resistance training: a meta-analysis.

PubMed

Manca, A; Dragone, D; Dvir, Z; Deriu, Franca

2017-11-01

Cross-education (CE) of strength is a well-known phenomenon whereby exercise of one limb can induce strength gains in the contralateral untrained limb. The only available meta-analyses on CE, which date back to a decade ago, estimated a modest 7.8% increase in contralateral strength following unilateral training. However, in recent years new evidences have outlined larger contralateral gains, which deserve to be systematically evaluated. Therefore, the aim of this meta-analysis was to appraise current data on CE and determine its overall magnitude of effect. Five databases were searched from inception to December 2016. All randomized controlled trials focusing on unilateral resistance training were carefully checked by two reviewers who also assessed the eligibility of the identified trials and extracted data independently. The risk of bias was assessed using the Cochrane Risk-of-Bias tool. Thirty-one studies entered the meta-analysis. Data from 785 subjects were pooled and subgroup analyses by body region (upper/lower limb) and type of training (isometric/concentric/eccentric/isotonic-dynamic) were performed. The pooled estimate of CE was a significant 11.9% contralateral increase (95% CI 9.1-14.8; p < 0.00001; upper limb: + 9.4%, p < 0.00001; lower limb: + 16.4%, p < 0.00001). Significant CE effects were induced by isometric (8.2%; p = 0.0003), concentric (11.3%; p < 0.00001), eccentric (17.7%; p = 0.003) and isotonic-dynamic training (15.9%; p < 0.00001), although a high risk of bias was detected across the studies. Unilateral resistance training induces significant contraction type-dependent gains in the contralateral untrained limb. Methodological issues in the included studies are outlined to provide guidance for a reliable quantification of CE in future studies.

Explicit knowledge of stimulus-outcome contingencies and stimulus control of selective attention and instrumental action in human smoking behaviour.

PubMed

Hogarth, Lee; Dickinson, Anthony; Duka, Theodora

2005-02-01

External stimuli (S+) that reliably signal that addictive drugs are available command the focus of selective attention and control instrumental action that procures the drug. According to incentive salience theory, as the contingency between the S+ and the drug is learned the magnitude of attentional orienting towards the S+ increases. By contrast, alternative theories propose that processing of the S+ becomes more efficient with training such that the measured attentional orienting response elicited by the S+ decreases. The aim of the present study was to prompt half of participants to acquire explicit knowledge of the stimulus-reinforcer contingencies arranged in training, to examine the impact of this manipulation on the magnitude of attentional orienting towards the S+. Smokers (n=32) completed an instrumental discrimination training procedure in which a set of stimuli were established as differential predictors that an instrumental response would yield tobacco-smoke reinforcement. During training, attention for the stimuli and performance of the instrumental tobacco-seeking response were measured in parallel. One group (n=16) was prompted to develop explicit knowledge of the discriminative contingencies in training whereas another group (n=16) underwent discrimination training without prompting. The prompted group reported accurate knowledge of the contingencies and showed no attentional orienting response towards the S+. By contrast, the unprompted group reported inaccurate knowledge of the contingencies and showed an attentional orienting response towards the S+. The S+ appeared to control the instrumental tobacco-seeking response in both groups equally. The results suggest that attention for drug paired S+ is associated with the process of learning about the relationship between those cues and the drug.

A research coding method to evaluate medical clinicians conduct of behavioral health care in patients with unexplained symptoms.

PubMed

Grayson-Sneed, Katelyn A; Smith, Robert C

2018-04-01

Develop a reliable coding method of a Behavioral Health Treatment Model for patients with Medically Unexplained Symptoms (BHTM-MUS). Two undergraduates trained for 30h coded videotaped interviews from 161 resident-simulated patient (SP) interactions. Trained on 45 videotapes, coders coded 33 (20%) of 161 study set tapes for the BHTM-MUS. Guetzkow's U, Cohen's Kappa, and percent of agreement were used to measure coders' reliability in unitizing and coding residents' skills for eliciting: education and informing (4 yes/no items), motivating (2), treatment statements (5), commitment and goals (2), negotiates plan (8), non-emotion patient-centered skills (4), and patient-centered emotional skills (8). 60 items were dichotomized a priori from analysis of the BHTM-MUS and were reduced to 33 during training. Guetzkow's U ranged from .00 to .082. Kappa ranged from 0.76 to 0.97 for the 7 variables and 33 individual items. The overall kappa was 0.87, and percent of agreement was 95.7%. Percent of agreement by item ranged from 85 to 100%. A highly reliable coding method is recommended to evaluate medical clinicians' behavioral care skills in patients with unexplained symptoms. A way to rate behavioral care in patients with unexplained symptoms. Copyright © 2017 Elsevier B.V. All rights reserved.

AO Distal Radius Fracture Classification: Global Perspective on Observer Agreement.

PubMed

Jayakumar, Prakash; Teunis, Teun; Giménez, Beatriz Bravo; Verstreken, Frederik; Di Mascio, Livio; Jupiter, Jesse B

2017-02-01

Background  The primary objective of this study was to test interobserver reliability when classifying fractures by consensus by AO types and groups among a large international group of surgeons. Secondarily, we assessed the difference in inter- and intraobserver agreement of the AO classification in relation to geographical location, level of training, and subspecialty. Methods  A randomized set of radiographic and computed tomographic images from a consecutive series of 96 distal radius fractures (DRFs), treated between October 2010 and April 2013, was classified using an electronic web-based portal by an invited group of participants on two occasions. Results  Interobserver reliability was substantial when classifying AO type A fractures but fair and moderate for type B and C fractures, respectively. No difference was observed by location, except for an apparent difference between participants from India and Australia classifying type B fractures. No statistically significant associations were observed comparing interobserver agreement by level of training and no differences were shown comparing subspecialties. Intra-rater reproducibility was "substantial" for fracture types and "fair" for fracture groups with no difference accounting for location, training level, or specialty. Conclusion  Improved definition of reliability and reproducibility of this classification may be achieved using large international groups of raters, empowering decision making on which system to utilize. Level of Evidence  Level III.

AO Distal Radius Fracture Classification: Global Perspective on Observer Agreement

PubMed Central

Jayakumar, Prakash; Teunis, Teun; Giménez, Beatriz Bravo; Verstreken, Frederik; Di Mascio, Livio; Jupiter, Jesse B.

2016-01-01

Background The primary objective of this study was to test interobserver reliability when classifying fractures by consensus by AO types and groups among a large international group of surgeons. Secondarily, we assessed the difference in inter- and intraobserver agreement of the AO classification in relation to geographical location, level of training, and subspecialty. Methods A randomized set of radiographic and computed tomographic images from a consecutive series of 96 distal radius fractures (DRFs), treated between October 2010 and April 2013, was classified using an electronic web-based portal by an invited group of participants on two occasions. Results Interobserver reliability was substantial when classifying AO type A fractures but fair and moderate for type B and C fractures, respectively. No difference was observed by location, except for an apparent difference between participants from India and Australia classifying type B fractures. No statistically significant associations were observed comparing interobserver agreement by level of training and no differences were shown comparing subspecialties. Intra-rater reproducibility was “substantial” for fracture types and “fair” for fracture groups with no difference accounting for location, training level, or specialty. Conclusion Improved definition of reliability and reproducibility of this classification may be achieved using large international groups of raters, empowering decision making on which system to utilize. Level of Evidence Level III PMID:28119795

ETARA PC version 3.3 user's guide: Reliability, availability, maintainability simulation model

NASA Technical Reports Server (NTRS)

Hoffman, David J.; Viterna, Larry A.

1991-01-01

A user's manual describing an interactive, menu-driven, personal computer based Monte Carlo reliability, availability, and maintainability simulation program called event time availability reliability (ETARA) is discussed. Given a reliability block diagram representation of a system, ETARA simulates the behavior of the system over a specified period of time using Monte Carlo methods to generate block failure and repair intervals as a function of exponential and/or Weibull distributions. Availability parameters such as equivalent availability, state availability (percentage of time as a particular output state capability), continuous state duration and number of state occurrences can be calculated. Initial spares allotment and spares replenishment on a resupply cycle can be simulated. The number of block failures are tabulated both individually and by block type, as well as total downtime, repair time, and time waiting for spares. Also, maintenance man-hours per year and system reliability, with or without repair, at or above a particular output capability can be calculated over a cumulative period of time or at specific points in time.

Reliability of Maximal Strength Testing in Novice Weightlifters

NASA Technical Reports Server (NTRS)

Loehr, James A.; Lee, Stuart M. C.; Feiveson, Alan H.; Ploutz-Snyder, Lori L.

2009-01-01

The one repetition maximum (1RM) is a criterion measure of muscle strength. However, the reliability of 1RM testing in novice subjects has received little attention. Understanding this information is crucial to accurately interpret changes in muscle strength. To evaluate the test-retest reliability of a squat (SQ), heel raise (HR), and deadlift (DL) 1RM in novice subjects. Twenty healthy males (31 plus or minus 5 y, 179.1 plus or minus 6.1 cm, 81.4 plus or minus 10.6 kg) with no weight training experience in the previous six months participated in four 1RM testing sessions, with each session separated by 5-7 days. SQ and HR 1RM were conducted using a smith machine; DL 1RM was assessed using free weights. Session 1 was considered a familiarization and was not included in the statistical analyses. Repeated measures analysis of variance with Tukey fs post-hoc tests were used to detect between-session differences in 1RM (p.0.05). Test-retest reliability was evaluated by intraclass correlation coefficients (ICC). During Session 2, the SQ and DL 1RM (SQ: 90.2 }4.3, DL: 75.9 }3.3 kg) were less than Session 3 (SQ: 95.3 }4.1, DL: 81.5 plus or minus 3.5 kg) and Session 4 (SQ: 96.6 }4.0, DL: 82.4 }3.9 kg), but there were no differences between Session 3 and Session 4. HR 1RM measured during Session 2 (150.1 }3.7 kg) and Session 3 (152.5 }3.9 kg) were not different from one another, but both were less than Session 4 (157.5 }3.8 kg). The reliability (ICC) of 1RM measures for Sessions 2-4 were 0.88, 0.83, and 0.87, for SQ, HR, and DL, respectively. When considering only Sessions 3 and 4, the reliability was 0.93, 0.91, and 0.86 for SQ, HR, and DL, respectively. One familiarization session and 2 test sessions (for SQ and DL) were required to obtain excellent reliability (ICC greater than or equal to 0.90) in 1RM values with novice subjects. We were unable to attain this level of reliability following 3 HR testing sessions therefore additional sessions may be required to obtain an ICC of greater than or equal to 0.90. Future resistive exercise studies should consider the reliability of specific measures to ensure that changes in strength with training are attributable to training and not learning effects associated with 1RM testing.

Absolute and relative reliability of acute effects of aerobic exercise on executive function in seniors.

PubMed

Donath, Lars; Ludyga, Sebastian; Hammes, Daniel; Rossmeissl, Anja; Andergassen, Nadin; Zahner, Lukas; Faude, Oliver

2017-10-25

Aging is accompanied by a decline of executive function. Aerobic exercise training induces moderate improvements of cognitive domains (i.e., attention, processing, executive function, memory) in seniors. Most conclusive data are obtained from studies with dementia or cognitive impairment. Confident detection of exercise training effects requires adequate between-day reliability and low day-to-day variability obtained from acute studies, respectively. These absolute and relative reliability measures have not yet been examined for a single aerobic training session in seniors. Twenty-two healthy and physically active seniors (age: 69 ± 3 y, BMI: 24.8 ± 2.2, VO 2peak : 32 ± 6 mL/kg/bodyweight) were enrolled in this randomized controlled cross-over study. A repeated between-day comparison [i.e., day 1 (habituation) vs. day 2 & day 2 vs. day 3] of executive function testing (Eriksen-Flanker-Test, Stroop-Color-Test, Digit-Span, Five-Point-Test) before and after aerobic cycling exercise at 70% of the heart rate reserve [0.7 × (HR max - HR rest )] was conducted. Reliability measures were calculated for pre, post and change scores. Large between-day differences between day 1 and 2 were found for reaction times (Flanker- and Stroop Color testing) and completed figures (Five-Point test) at pre and post testing (0.002 < p < 0.05, 0.16 < ɳ p 2  < 0.38). These differences notably declined when comparing day 2 and 3. Absolute between days variability (CoV) dropped from 10 to 5% when comparing day 2 vs. day 3 instead of day 1 vs. day 2. Also ICC ranges increased from day 1 vs. day 2 (0.65 < ICC < 0.87) to day 2 vs. day 3 (0.40 < ICC < 0.93). Interestingly, reliability measures for pre-post change scores were low (0.02 < ICC < 0.71). These data did not improve when comparing day 2 with day 3. During inhibition tests, reaction times showed excellent reliability values compared to the poor to fair reliability of accuracy. Notable habituation to the whole testing procedure should be considered as it increased the reliability of different executive function tests. Change scores of executive function after acute aerobic exercise cannot be detected reliably. Large intra- and inter-individual of responses to acute aerobic exercise in seniors can be presumed.

Evaluation of odometry algorithm performances using a railway vehicle dynamic model

NASA Astrophysics Data System (ADS)

Allotta, B.; Pugi, L.; Ridolfi, A.; Malvezzi, M.; Vettori, G.; Rindi, A.

2012-05-01

In modern railway Automatic Train Protection and Automatic Train Control systems, odometry is a safety relevant on-board subsystem which estimates the instantaneous speed and the travelled distance of the train; a high reliability of the odometry estimate is fundamental, since an error on the train position may lead to a potentially dangerous overestimation of the distance available for braking. To improve the odometry estimate accuracy, data fusion of different inputs coming from a redundant sensor layout may be used. Simplified two-dimensional models of railway vehicles have been usually used for Hardware in the Loop test rig testing of conventional odometry algorithms and of on-board safety relevant subsystems (like the Wheel Slide Protection braking system) in which the train speed is estimated from the measures of the wheel angular speed. Two-dimensional models are not suitable to develop solutions like the inertial type localisation algorithms (using 3D accelerometers and 3D gyroscopes) and the introduction of Global Positioning System (or similar) or the magnetometer. In order to test these algorithms correctly and increase odometry performances, a three-dimensional multibody model of a railway vehicle has been developed, using Matlab-Simulink™, including an efficient contact model which can simulate degraded adhesion conditions (the development and prototyping of odometry algorithms involve the simulation of realistic environmental conditions). In this paper, the authors show how a 3D railway vehicle model, able to simulate the complex interactions arising between different on-board subsystems, can be useful to evaluate the odometry algorithm and safety relevant to on-board subsystem performances.

How Differences among Data Collectors Are Reflected in the Reliability and Validity of Data Collected by Likert-Type Scales?

ERIC Educational Resources Information Center

Köksal, Mustafa Serdar; Ertekin, Pelin; Çolakoglu, Özgür Murat

2014-01-01

The purpose of this study is to investigate association of data collectors' differences with the differences in reliability and validity of scores regarding affective variables (motivation toward science learning and science attitude) that are measured by Likert-type scales. Four researchers trained in data collection and seven science teachers…

The Reliability of a 5km Run Test on a Motorized Treadmill

ERIC Educational Resources Information Center

Driller, Matthew; Brophy-Williams, Ned; Walker, Anthony

2017-01-01

The purpose of the present study was to determine the reliability of a 5km run test on a motorized treadmill. Over three consecutive weeks, 12 well-trained runners completed three 5km time trials on a treadmill following a standardized warm-up. Runners were partially-blinded to their running speed and distance covered. Total time to complete the…

An Attitude Scale on Individual Instrument and Individual Instrument Course: Validity-Reliability Research

ERIC Educational Resources Information Center

Kuçukosmanoglu, Hayrettin Onur

2015-01-01

The main purpose of this study is to develop a scale to determine students' attitude levels on individual instruments and individual instrument courses in instrument training, which is an important dimension of music education, and to conduct a validity-reliability research of the scale that has been developed. The scale consists of 16 items. The…

Enhanced Communication Network Solution for Positive Train Control Implementation

NASA Technical Reports Server (NTRS)

Fatehi, M. T.; Simon, J.; Chang, W.; Chow, E. T.; Burleigh, S. C.

2011-01-01

The commuter and freight railroad industry is required to implement Positive Train Control (PTC) by 2015 (2012 for Metrolink), a challenging network communications problem. This paper will discuss present technologies developed by the National Aeronautics and Space Administration (NASA) to overcome comparable communication challenges encountered in deep space mission operations. PTC will be based on a new cellular wireless packet Internet Protocol (IP) network. However, ensuring reliability in such a network is difficult due to the "dead zones" and transient disruptions we commonly experience when we lose calls in commercial cellular networks. These disruptions make it difficult to meet PTC s stringent reliability (99.999%) and safety requirements, deployment deadlines, and budget. This paper proposes innovative solutions based on space-proven technologies that would help meet these challenges: (1) Delay Tolerant Networking (DTN) technology, designed for use in resource-constrained, embedded systems and currently in use on the International Space Station, enables reliable communication over networks in which timely data acknowledgments might not be possible due to transient link outages. (2) Policy-Based Management (PBM) provides dynamic management capabilities, allowing vital data to be exchanged selectively (with priority) by utilizing alternative communication resources. The resulting network may help railroads implement PTC faster, cheaper, and more reliably.

Reliability, Validity, and Responsiveness of the QuickDASH in Patients With Upper Limb Amputation.

PubMed

Resnik, Linda; Borgia, Matthew

2015-09-01

To examine the internal consistency, test-retest reliability, validity, and responsiveness of the shortened version of the Disabilities of the Arm, Shoulder and Hand (QuickDASH) questionnaire in persons with upper limb amputation. Cross-sectional and longitudinal. Three sites participating in the U.S. Department of Veterans Affairs Home Study of the DEKA Arm. A convenience sample of upper limb amputees (N=44). Training with a multifunction upper limb prosthesis. Multiple outcome measures including the QuickDASH were administered twice within 1 week, and for a subset of 20 persons, after completion of in-laboratory training with the DEKA Arm. Scale alphas and intraclass correlation coefficient type 3,1 (ICC3,1) were used to examine reliability. Minimum detectable change (MDC) scores were calculated. Analyses of variance, comparing QuickDASH scores by the amount of prosthetic use and amputation level, were used for known-group validity analyses with alpha set at .05. Pairwise correlations between QuickDASH and other measures were used to examine concurrent validity. Responsiveness was measured by effect size (ES) and standardized response mean (SRM). QuickDASH alpha was .83, and ICC was .87 (95% confidence interval, .77-.93). MDC at the 95% confidence level (MDC95%) was 17.4. Full- or part-time prosthesis users had better QuickDASH scores compared with nonprosthesis users (P=.021), as did those with more distal amputations at both baseline (P=.042) and with the DEKA Arm (P=.024). The QuickDASH was correlated with concurrent measures of activity limitation as expected. The ES and SRM after training with the DEKA Arm were 0.6. This study provides evidence of reliability and validity of the QuickDASH in persons with upper limb amputation. Results provide preliminary evidence of responsiveness to prosthetic device type/training. Further research with a larger sample is needed to confirm results. Copyright © 2015 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.

Enhancement of the bulbocavernosus reflex during intraoperative neurophysiological monitoring through the use of double train stimulation: a pilot study.

PubMed

Skinner, Stanley; Chiri, Chala A; Wroblewski, Jill; Transfeldt, Ensor E

2007-02-01

Electrophysiological bulbocavernosus reflex (BCR) testing, during surgeries in which the constituent neural components are at risk, might supplement other low sacral (S2-4) stimulation/recording techniques. However, intraoperative BCR is not always reliably implemented. We proposed to analyze BCR signals in five surgical patients monitored with the novel application of double train stimulation (DTS) to determine if the potential could be enhanced. We prospectively planned a regime of DTS BCR with a series of intertrain delays in five monitored patients at risk for low sacral neural injury. Patients were maintained with propofol, opiate infusion, and low inhalant anesthesia without muscle relaxant. Cutaneous sensory nerves of the penis (or clitoris) were stimulated using two consecutive pulse trains (DTS). Intertrain delays were 75, 100, 125, 150, 175, 200, and 250 ms. For BCR recording, uncoated paired wires were inserted into the external anal sphincter (EAS) bilaterally. For each trial, waveform amplitude, duration, and turn count measures for the first (single train) and second (double train) response were recorded. Percent increase/decrease of the second train response compared to the first train response was calculated. There was at least a 30% increase in measures of amplitude, turn count, and duration of the second train response in 22/28, 22/28, and 14/28 of the total trials respectively. There was an insufficient number of independent observations to determine statistical significance. Intraoperative BCR is currently obtained with some difficulty using pulse train stimulation. Our preliminary evidence has identified BCR waveform enhancement using DTS and suggests that the reliability of intraoperative BCR acquisition may be further improved by the addition of this technique. Our data are insufficient to define the best intertrain interval.

Development and Testing of a Direct Observation Code Training Protocol for Elementary Aged Students with Attention Deficit/Hyperactivity Disorder

ERIC Educational Resources Information Center

Steiner, Naomi J.; Sidhu, Tahnee; Rene, Kirsten; Tomasetti, Kathryn; Frenette, Elizabeth; Brennan, Robert T.

2013-01-01

Observational measures can add objective data to both research and clinical evaluations of children's behavior in the classroom. However, they pose challenges for training and attaining high levels of interrater reliability between observers. The Behavioral Observation of Students in Schools (BOSS) is a commonly used school-based observation…

A Methodology for Optimizing the Training and Utilization of Physical Therapy Personnel.

ERIC Educational Resources Information Center

Dumas, Neil S.; Muthard, John E.

A method for analyzing the work in a department of physical therapy was devised and applied in a teaching hospital. Physical therapists, trained as observer-investigators, helped refine the coding system and were able to reliably record job behavior in the physical therapy department. The nature of the therapist's and aide's job was described and…

Development methodology of the novel Endoscopic stone treatment step 1 (EST s1) training/assessment curriculum.

PubMed

Veneziano, Domenico; Ahmed, Kamran; Van Cleynenbreugel, Ben S E P; Gözen, Ali Serdar; Palou, Joan; Sarica, Kemal; Liatsikos, Evangelos N; Sanguedolce, Francesco; Honeck, Patrick; Alvarez-Maestro, Mario; Papatsoris, Athanasios; Kallidonis, Panagiotis; Greco, Francesco; Breda, Alberto; Somani, Bhaskar

2017-07-10

Background Simulation based technical-skill assessment is a core topic of debate, especially in high-risk environments. After the introduction of the E-BLUS exam for basic laparoscopy, no more technical training/assessment urological protocols have been developed in Europe. Objective We describe the methodology used in the development of the novel Endoscopic Stone Treatment step 1 (EST s1) assessment curriculum. Materials and Methods The "full life cycle curriculum development" template was followed for curriculum development. A CTA was run to define the most important steps and details of RIRS, in accordance with EAU Urolithiasis guidelines. Training tasks were created between April 2015 and September 2015. Tasks and metrics were further analyzed by a consensus meeting with the EULIS board in February 2016. A review, aimed to study available simulators and their accordance with task requirements, was subsequently run in London on March 2016. After initial feedback and further tests, content validity of this protocol was achieved during EUREP 2016. Results The EST s1 curriculum development, took 23 months. 72 participants tested the 5 preliminary tasks during EUREP 2015, with sessions of 45 minutes each. Likert-scale questionnaires were filled-out to score the quality of training. The protocol was modified accordingly and 25 participants tested the 4 tasks during the hands-on training sessions of the ESUT 2016 congress. 134 participants finally participated in the validation study in EUREP 2016. During the same event 10 experts confirmed content validity by filling-out a Likert-scale questionnaire. Conclusion We described a reliable and replicable methodology that can be followed to develop training/assessment protocols for surgical procedures. The expert consensus meetings, strict adherence to guidelines and updated literature search towards an Endourology curriculum allowed correct training and assessment protocol development. It is the first step towards standardized simulation training in Endourology with a potential for worldwide adoption.

Positive-unlabeled learning for disease gene identification

PubMed Central

Yang, Peng; Li, Xiao-Li; Mei, Jian-Ping; Kwoh, Chee-Keong; Ng, See-Kiong

2012-01-01

Background: Identifying disease genes from human genome is an important but challenging task in biomedical research. Machine learning methods can be applied to discover new disease genes based on the known ones. Existing machine learning methods typically use the known disease genes as the positive training set P and the unknown genes as the negative training set N (non-disease gene set does not exist) to build classifiers to identify new disease genes from the unknown genes. However, such kind of classifiers is actually built from a noisy negative set N as there can be unknown disease genes in N itself. As a result, the classifiers do not perform as well as they could be. Result: Instead of treating the unknown genes as negative examples in N, we treat them as an unlabeled set U. We design a novel positive-unlabeled (PU) learning algorithm PUDI (PU learning for disease gene identification) to build a classifier using P and U. We first partition U into four sets, namely, reliable negative set RN, likely positive set LP, likely negative set LN and weak negative set WN. The weighted support vector machines are then used to build a multi-level classifier based on the four training sets and positive training set P to identify disease genes. Our experimental results demonstrate that our proposed PUDI algorithm outperformed the existing methods significantly. Conclusion: The proposed PUDI algorithm is able to identify disease genes more accurately by treating the unknown data more appropriately as unlabeled set U instead of negative set N. Given that many machine learning problems in biomedical research do involve positive and unlabeled data instead of negative data, it is possible that the machine learning methods for these problems can be further improved by adopting PU learning methods, as we have done here for disease gene identification. Availability and implementation: The executable program and data are available at http://www1.i2r.a-star.edu.sg/∼xlli/PUDI/PUDI.html. Contact: xlli@i2r.a-star.edu.sg or yang0293@e.ntu.edu.sg Supplementary information: Supplementary Data are available at Bioinformatics online. PMID:22923290

Optical biosensor system for the quick and reliable detection of virus infections: VIROSENS

NASA Astrophysics Data System (ADS)

Proll, Günther; Hartjes, Anja; Sinclair, Alexander; Markovic, Goran; Pröll, Florian; Patel, Pranav; Niedrig, Matthias

2014-10-01

Viral infections are of special threat because they can induce severe courses of disease but only few medical treatments are available. Because of socio-economic and climate changes, increased worldwide mobility and population growth, the risk of newly occurring and quickly spreading viral pathogens has increased. A diagnosis of these diseases at an early stage is essential for a quick risk assessment and a proper health management as well as patient's treatment in an optimal way. Currently, the diagnosis of such diseases is based on time consuming and costly detection methods that can only be performed by specially trained personnel in laboratories at specific security levels. Aim of the project VIROSENS is the development of a biosensor platform that can specifically detect virus particles as well as virus-specific antibodies out of biological matrices like blood, serum, plasma and other body fluids. For this purpose, a disposable cartridge for such antibody- and virus-arrays is designed and developed within the project. The optical detection of viruses is performed with a portable device that will be benchmarked and evaluated concerning currently used standard detection methods in terms of its analytical performance. Within this project, a novel combination of serological tests and direct detection of virus particles will be developed, which will provide faster and more reliable results than presently available and used test systems.

The “e-Generation”: The Technological Usage and Experiences of Medical Students from a Developing Country

PubMed Central

Galappatthy, Priyasdarshani; Wathurapatha, Wasundara S.; Wijayabandara, Maheshi D. M. S.; Warapitiya, Dinuka S.; Weerasuriya, Krishantha

2017-01-01

Background The medical community is increasingly using Portable Electronic Devices (PEDs). We evaluated usage of PEDs and medical apps among medical students from Sri Lanka. Methods This descriptive cross-sectional study was conducted at Faculty of Medicine, University of Colombo. Medical students from 2nd to 5th year were invited for the study. A self-administered questionnaire was used to collect details of PEDs availability, accessibility, and usage, perceived advantages/barriers of PEDs, and availability, accessibility, and usage of medical apps. Results Sample size was 505 (response rate, 61.8%). Mean age was 23.2 ± 1.3 years and majority were females (60.4%, n = 305). Majority (87.5%, n = 442) of students owned a PED. Nonaffordability was the most common reason for having not owning a PED (46%). Nonaffordability and lack of knowledge were key determinants of the usage of PEDs and medical “apps.” Doubts about reliability and lack of knowledge regarding reliable electronic sources of information were other significant barriers. Conclusions Our results show that a significant majority of students owned a PED, a higher percentage than what is reported elsewhere. Considering barriers identified, it is important for institutions to promote usage of PEDs and medical apps by providing financial support, training, and knowledge to build confidence in technology. PMID:29138632

Coupling Matched Molecular Pairs with Machine Learning for Virtual Compound Optimization.

PubMed

Turk, Samo; Merget, Benjamin; Rippmann, Friedrich; Fulle, Simone

2017-12-26

Matched molecular pair (MMP) analyses are widely used in compound optimization projects to gain insights into structure-activity relationships (SAR). The analysis is traditionally done via statistical methods but can also be employed together with machine learning (ML) approaches to extrapolate to novel compounds. The here introduced MMP/ML method combines a fragment-based MMP implementation with different machine learning methods to obtain automated SAR decomposition and prediction. To test the prediction capabilities and model transferability, two different compound optimization scenarios were designed: (1) "new fragments" which occurs when exploring new fragments for a defined compound series and (2) "new static core and transformations" which resembles for instance the identification of a new compound series. Very good results were achieved by all employed machine learning methods especially for the new fragments case, but overall deep neural network models performed best, allowing reliable predictions also for the new static core and transformations scenario, where comprehensive SAR knowledge of the compound series is missing. Furthermore, we show that models trained on all available data have a higher generalizability compared to models trained on focused series and can extend beyond chemical space covered in the training data. Thus, coupling MMP with deep neural networks provides a promising approach to make high quality predictions on various data sets and in different compound optimization scenarios.

Consensus training: an effective tool to minimize variations in periodontal diagnosis and treatment planning among dental faculty and students.

PubMed

John, Vanchit; Lee, Seung-Jun; Prakasam, Sivaraman; Eckert, George J; Maupome, Gerardo

2013-08-01

Considerable disagreements and variations exist in diagnosis and treatment planning of periodontal disease. Achieving high interrater periodontal diagnosis can prove challenging. The objectives of this study were to measure variations in periodontal diagnosis and treatment planning among predoctoral periodontics faculty members after consensus training and to compare such variation with those identified in third- and fourth-year dental students. Nine electronically stored case vignettes and survey instruments were made available to eighteen faculty members and twenty dental students under standardized conditions. A chi-square test was used to compare responses between groups, and multirater kappa tests were used to evaluate interrater agreement/reliability. Of the nine cases, only one differed between groups significantly in terms of treatment. Also, third-year students differed from fourth-year students on the diagnosis of aggressive periodontitis versus chronic periodontitis. Most respondents were able to distinguish clearly among diagnoses of chronic periodontitis, aggressive periodontitis, and gingivitis. This study established a baseline assessment of the current status of consensus after training. We will reassess variations after addressing the specific challenges identified. Programs designed and implemented to help decrease the variation in periodontal diagnosis and treatment planning among faculty members may ultimately translate into better agreement and better standardization of dental instruction.

Virtual reality simulation training in Otolaryngology.

PubMed

Arora, Asit; Lau, Loretta Y M; Awad, Zaid; Darzi, Ara; Singh, Arvind; Tolley, Neil

2014-01-01

To conduct a systematic review of the validity data for the virtual reality surgical simulator platforms available in Otolaryngology. Ovid and Embase databases searched July 13, 2013. Four hundred and nine abstracts were independently reviewed by 2 authors. Thirty-six articles which fulfilled the search criteria were retrieved and viewed in full text. These articles were assessed for quantitative data on at least one aspect of face, content, construct or predictive validity. Papers were stratified by simulator, sub-specialty and further classified by the validation method used. There were 21 articles reporting applications for temporal bone surgery (n = 12), endoscopic sinus surgery (n = 6) and myringotomy (n = 3). Four different simulator platforms were validated for temporal bone surgery and two for each of the other surgical applications. Face/content validation represented the most frequent study type (9/21). Construct validation studies performed on temporal bone and endoscopic sinus surgery simulators showed that performance measures reliably discriminated between different experience levels. Simulation training improved cadaver temporal bone dissection skills and operating room performance in sinus surgery. Several simulator platforms particularly in temporal bone surgery and endoscopic sinus surgery are worthy of incorporation into training programmes. Standardised metrics are necessary to guide curriculum development in Otolaryngology. Copyright © 2013 Surgical Associates Ltd. Published by Elsevier Ltd. All rights reserved.

Interexaminer reliability in clinical measurement of L*C*h* values of anterior teeth using a spectrophotometer.

PubMed

Hassel, Alexander J; Grossmann, Anne-christiane; Schmitter, Marc; Balke, Zibandeh; Buzello, Anja M

2007-01-01

The objective of this study was to investigate interexaminer reliability in the clinical measurement of the L*C*h* (lightness/value, chroma, hue) values of anterior teeth using a spectrophotometer (Vita Easyshade). The basic color of the maxillary right central incisors and canines of 23 subjects was spectrophotometrically determined by 4 clinicians and an experienced user (development manager) of the spectrophotometer. Also, to analyze the effect of different training with the instrument on interexaminer reliability, 2 of the clinicians were instructed in the use of the spectrophotometer by the experienced examiner, whereas the others instructed themselves by studying the operating manual. Agreement between all examiners was acceptable to excellent (intraclass coefficient > 0.4). The mean value of the measured differences for the central incisors of all subjects for L* values was 5 (for C* = 3.8, h* = 2.7 degrees) and for canines, the mean L* was 4.5 (C* = 3, h* = 1.6 degrees). Results from comparison of the 2 different training methods were inconsistent. Agreement with the experienced examiner ranged from not acceptable (C* values for incisors of self-instructed examiners) to excellent. The distribution of the measurements of 1 subject could lead to deviations in color, probably with clinical impact. For canines, the measurements were at least equally reproducible (in some cases significantly more reproducible) compared to central incisors. Because of the small number of examiners and the inconsistent results, it was not possible to reach a definite conclusion about the effect of different training methods on interexaminer reliability.

The impact of leadership and team behavior on standard of care delivered during human patient simulation: a pilot study for undergraduate medical students.

PubMed

Carlson, Jim; Min, Elana; Bridges, Diane

2009-01-01

Methodology to train team behavior during simulation has received increased attention, but standard performance measures are lacking, especially at the undergraduate level. Our purposes were to develop a reliable team behavior measurement tool and explore the relationship between team behavior and the delivery of an appropriate standard of care specific to the simulated case. Authors developed a unique team measurement tool based on previous work. Trainees participated in a simulated event involving the presentation of acute dyspnea. Performance was rated by separate raters using the team behavior measurement tool. Interrater reliability was assessed. The relationship between team behavior and the standard of care delivered was explored. The instrument proved to be reliable for this case and group of raters. Team behaviors had a positive relationship with the standard of medical care delivered specific to the simulated case. The methods used provide a possible method for training and assessing team performance during simulation.

Reliability and Availability Evaluation Program Manual.

DTIC Science & Technology

1982-11-01

research and development. The manual’s purpose was to provide a practical method for making reliability measurements, measurements directly related to... Research , Development, Test and Evaluation. RMA Reliability, Maintainability and Availability. R&R Repair and Refurbishment, Repair and Replacement, etc...length. phenomena such as mechanical wear and A number of researchers in the reliability chemical deterioration. Maintenance should field 14-pages 402

Design of fuel cell powered data centers for sufficient reliability and availability

NASA Astrophysics Data System (ADS)

Ritchie, Alexa J.; Brouwer, Jacob

2018-04-01

It is challenging to design a sufficiently reliable fuel cell electrical system for use in data centers, which require 99.9999% uptime. Such a system could lower emissions and increase data center efficiency, but the reliability and availability of such a system must be analyzed and understood. Currently, extensive backup equipment is used to ensure electricity availability. The proposed design alternative uses multiple fuel cell systems each supporting a small number of servers to eliminate backup power equipment provided the fuel cell design has sufficient reliability and availability. Potential system designs are explored for the entire data center and for individual fuel cells. Reliability block diagram analysis of the fuel cell systems was accomplished to understand the reliability of the systems without repair or redundant technologies. From this analysis, it was apparent that redundant components would be necessary. A program was written in MATLAB to show that the desired system reliability could be achieved by a combination of parallel components, regardless of the number of additional components needed. Having shown that the desired reliability was achievable through some combination of components, a dynamic programming analysis was undertaken to assess the ideal allocation of parallel components.

Improving Acoustic Models by Watching Television

NASA Technical Reports Server (NTRS)

Witbrock, Michael J.; Hauptmann, Alexander G.

1998-01-01

Obtaining sufficient labelled training data is a persistent difficulty for speech recognition research. Although well transcribed data is expensive to produce, there is a constant stream of challenging speech data and poor transcription broadcast as closed-captioned television. We describe a reliable unsupervised method for identifying accurately transcribed sections of these broadcasts, and show how these segments can be used to train a recognition system. Starting from acoustic models trained on the Wall Street Journal database, a single iteration of our training method reduced the word error rate on an independent broadcast television news test set from 62.2% to 59.5%.

NEON's Mobile Deployment Platform: A Resource for Community Research

NASA Astrophysics Data System (ADS)

Sanclements, M.

2015-12-01

Here we provide an update on construction and validation of the NEON Mobile Deployment Platforms (MDPs) as well as a description of the infrastructure and sensors available to researchers in the future. The MDPs will provide the means to observe stochastic or spatially important events, gradients, or quantities that cannot be reliably observed using fixed location sampling (e.g. fires and floods). Due to the transient temporal and spatial nature of such events, the MDPs will be designed to accommodate rapid deployment for time periods up to ~ 1 year. Broadly, the MDPs will be comprised of infrastructure and instrumentation capable of functioning individually or in conjunction with one another to support observations of ecological change, as well as education, training and outreach.

eSensor®: A Microarray Technology Based on Electrochemical Detection of Nucleic Acids and Its Application to Cystic Fibrosis Carrier Screening

NASA Astrophysics Data System (ADS)

Reed, Michael R.; Coty, William A.

We have developed a test for identification of carriers for cystic fibrosis using the eSensor® DNA detection technology. Oligonucleotide probes are deposited within self-assembled monolayers on gold electrodes arrayed upon printed circuit boards. These probes allow sequence-specific capture of amplicons containing a panel of mutation sites associated with cystic fibrosis. DNA targets are detected and mutations genotyped using a “sandwich” assay methodology employing electrochemical detection of ferrocene-labeled oligonucleotides for discrimination of carrier and non-carrier alleles. Performance of the cystic fibrosis application demonstrates sufficient accuracy and reliability for clinical diagnostic use, and the procedure can be performed by trained medical technologists available in the hospital laboratory.

Learning reliable manipulation strategies without initial physical models

NASA Technical Reports Server (NTRS)

Christiansen, Alan D.; Mason, Matthew T.; Mitchell, Tom M.

1990-01-01

A description is given of a robot, possessing limited sensory and effectory capabilities but no initial model of the effects of its actions on the world, that acquires such a model through exploration, practice, and observation. By acquiring an increasingly correct model of its actions, it generates increasingly successful plans to achieve its goals. In an apparently nondeterministic world, achieving reliability requires the identification of reliable actions and a preference for using such actions. Furthermore, by selecting its training actions carefully, the robot can significantly improve its learning rate.

Reliability of the Cardiff Test of basic life support and automated external defibrillation version 3.1.

PubMed

Whitfield, Richard H; Newcombe, Robert G; Woollard, Malcolm

2003-12-01

The introduction of the European Resuscitation Guidelines (2000) for cardiopulmonary resuscitation (CPR) and automated external defibrillation (AED) prompted the development of an up-to-date and reliable method of assessing the quality of performance of CPR in combination with the use of an AED. The Cardiff Test of basic life support (BLS) and AED version 3.1 was developed to meet this need and uses standardised checklists to retrospectively evaluate performance from analyses of video recordings and data drawn from a laptop computer attached to a training manikin. This paper reports the inter- and intra-observer reliability of this test. Data used to assess reliability were obtained from an investigation of CPR and AED skill acquisition in a lay responder AED training programme. Six observers were recruited to evaluate performance in 33 data sets, repeating their evaluation after a minimum interval of 3 weeks. More than 70% of the 42 variables considered in this study had a kappa score of 0.70 or above for inter-observer reliability or were drawn from computer data and therefore not subject to evaluator variability. 85% of the 42 variables had kappa scores for intra-observer reliability of 0.70 or above or were drawn from computer data. The standard deviations for inter- and intra-observer measures of time to first shock were 11.6 and 7.7 s, respectively. The inter- and intra-observer reliability for the majority of the variables in the Cardiff Test of BLS and AED version 3.1 is satisfactory. However, reliability is less acceptable with respect to shaking when checking for responsiveness, initial check/clearing of the airway, checks for signs of circulation, time to first shock and performance of interventions in the correct sequence. Further research is required to determine if modifications to the method of assessing these variables can increase reliability.

3D liver volume reconstructed for palpation training.

PubMed

Tibamoso, Gerardo; Perez-Gutierrez, Byron; Uribe-Quevedo, Alvaro

2013-01-01

Virtual Reality systems for medical procedures such as the palpation of different organs, requires fast, robust, accurate and reliable computational methods for providing realism during interaction with the 3D biological models. This paper presents the segmentation, reconstruction and palpation simulation of a healthy liver volume as a tool for training. The chosen method considers the mechanical characteristics and liver properties for correctly simulating palpation interactions, which results appropriate as a complementary tool for training medical students in familiarizing with the liver anatomy.

Predictive validity of a selection centre testing non-technical skills for recruitment to training in anaesthesia.

PubMed

Gale, T C E; Roberts, M J; Sice, P J; Langton, J A; Patterson, F C; Carr, A S; Anderson, I R; Lam, W H; Davies, P R F

2010-11-01

Assessment centres are an accepted method of recruitment in industry and are gaining popularity within medicine. We describe the development and validation of a selection centre for recruitment to speciality training in anaesthesia based on an assessment centre model incorporating the rating of candidate's non-technical skills. Expert consensus identified non-technical skills suitable for assessment at the point of selection. Four stations-structured interview, portfolio review, presentation, and simulation-were developed, the latter two being realistic scenarios of work-related tasks. Evaluation of the selection centre focused on applicant and assessor feedback ratings, inter-rater agreement, and internal consistency reliability coefficients. Predictive validity was sought via correlations of selection centre scores with subsequent workplace-based ratings of appointed trainees. Two hundred and twenty-four candidates were assessed over two consecutive annual recruitment rounds; 68 were appointed and followed up during training. Candidates and assessors demonstrated strong approval of the selection centre with more than 70% of ratings 'good' or 'excellent'. Mean inter-rater agreement coefficients ranged from 0.62 to 0.77 and internal consistency reliability of the selection centre score was high (Cronbach's α=0.88-0.91). The overall selection centre score was a good predictor of workplace performance during the first year of appointment. An assessment centre model based on the rating of non-technical skills can produce a reliable and valid selection tool for recruitment to speciality training in anaesthesia. Early results on predictive validity are encouraging and justify further development and evaluation.

A comparison of seminar and computer based training on the accuracy and reliability of raters using the Children's Global Assessment Scale (CGAS).

PubMed

Lundh, Anna; Kowalski, Jan; Sundberg, Carl Johan; Landén, Mikael

2012-11-01

The aim of this study was to compare two methods to conduct CGAS rater training. A total of 648 raters were randomized to training (CD or seminar), and rated five cases before and 12 months after training. The ICC at baseline/end of study was 0.71/0.78 (seminar), 0.76/0.78 (CD), and 0.67/0.79 (comparison). There were no differences in training effect in terms of agreement with expert ratings, which speaks in favor of using the less resource-demanding CD. However, the effect was modest in both groups, and untrained comparison group improved of the same order of magnitude, which proposes more extensive training.

Train integrity detection risk analysis based on PRISM

NASA Astrophysics Data System (ADS)

Wen, Yuan

2018-04-01

GNSS based Train Integrity Monitoring System (TIMS) is an effective and low-cost detection scheme for train integrity detection. However, as an external auxiliary system of CTCS, GNSS may be influenced by external environments, such as uncertainty of wireless communication channels, which may lead to the failure of communication and positioning. In order to guarantee the reliability and safety of train operation, a risk analysis method of train integrity detection based on PRISM is proposed in this article. First, we analyze the risk factors (in GNSS communication process and the on-board communication process) and model them. Then, we evaluate the performance of the model in PRISM based on the field data. Finally, we discuss how these risk factors influence the train integrity detection process.

28 CFR 345.81 - Pre-industrial training.

Code of Federal Regulations, 2010 CFR

2010-07-01

... training is available and the worker has not completed both the skill training and orientation phases of... pre-industrial training is not available, new FPI assignees will receive on-the-job training in pre-industrial pay status for a period of at least 30 days before being promoted into available fourth grade jobs. ...

Inter-rater reliability and generalizability of patient note scores using a scoring rubric based on the USMLE Step-2 CS format.

PubMed

Park, Yoon Soo; Hyderi, Abbas; Bordage, Georges; Xing, Kuan; Yudkowsky, Rachel

2016-10-01

Recent changes to the patient note (PN) format of the United States Medical Licensing Examination have challenged medical schools to improve the instruction and assessment of students taking the Step-2 clinical skills examination. The purpose of this study was to gather validity evidence regarding response process and internal structure, focusing on inter-rater reliability and generalizability, to determine whether a locally-developed PN scoring rubric and scoring guidelines could yield reproducible PN scores. A randomly selected subsample of historical data (post-encounter PN from 55 of 177 medical students) was rescored by six trained faculty raters in November-December 2014. Inter-rater reliability (% exact agreement and kappa) was calculated for five standardized patient cases administered in a local graduation competency examination. Generalizability studies were conducted to examine the overall reliability. Qualitative data were collected through surveys and a rater-debriefing meeting. The overall inter-rater reliability (weighted kappa) was .79 (Documentation = .63, Differential Diagnosis = .90, Justification = .48, and Workup = .54). The majority of score variance was due to case specificity (13 %) and case-task specificity (31 %), indicating differences in student performance by case and by case-task interactions. Variance associated with raters and its interactions were modest (<5 %). Raters felt that justification was the most difficult task to score and that having case and level-specific scoring guidelines during training was most helpful for calibration. The overall inter-rater reliability indicates high level of confidence in the consistency of note scores. Designs for scoring notes may optimize reliability by balancing the number of raters and cases.

Reliability of lower limb alignment measures using an established landmark-based method with a customized computer software program

PubMed Central

Sled, Elizabeth A.; Sheehy, Lisa M.; Felson, David T.; Costigan, Patrick A.; Lam, Miu; Cooke, T. Derek V.

2010-01-01

The objective of the study was to evaluate the reliability of frontal plane lower limb alignment measures using a landmark-based method by (1) comparing inter- and intra-reader reliability between measurements of alignment obtained manually with those using a computer program, and (2) determining inter- and intra-reader reliability of computer-assisted alignment measures from full-limb radiographs. An established method for measuring alignment was used, involving selection of 10 femoral and tibial bone landmarks. 1) To compare manual and computer methods, we used digital images and matching paper copies of five alignment patterns simulating healthy and malaligned limbs drawn using AutoCAD. Seven readers were trained in each system. Paper copies were measured manually and repeat measurements were performed daily for 3 days, followed by a similar routine with the digital images using the computer. 2) To examine the reliability of computer-assisted measures from full-limb radiographs, 100 images (200 limbs) were selected as a random sample from 1,500 full-limb digital radiographs which were part of the Multicenter Osteoarthritis (MOST) Study. Three trained readers used the software program to measure alignment twice from the batch of 100 images, with two or more weeks between batch handling. Manual and computer measures of alignment showed excellent agreement (intraclass correlations [ICCs] 0.977 – 0.999 for computer analysis; 0.820 – 0.995 for manual measures). The computer program applied to full-limb radiographs produced alignment measurements with high inter- and intra-reader reliability (ICCs 0.839 – 0.998). In conclusion, alignment measures using a bone landmark-based approach and a computer program were highly reliable between multiple readers. PMID:19882339

Reliable and valid assessment of competence in endoscopic ultrasonography and fine-needle aspiration for mediastinal staging of non-small cell lung cancer.

PubMed

Konge, L; Vilmann, P; Clementsen, P; Annema, J T; Ringsted, C

2012-10-01

Fine-needle aspiration (FNA) guided by endoscopic ultrasonography (EUS) is important in mediastinal staging of non-small cell lung cancer (NSCLC). Training standards and implementation strategies of this technique are currently under discussion. The aim of this study was to explore the reliability and validity of a newly developed EUS Assessment Tool (EUSAT) designed to measure competence in EUS - FNA for mediastinal staging of NSCLC. A total of 30 patients with proven or suspected NSCLC underwent EUS - FNA for mediastinal staging by three trainees and three experienced physicians. Their performances were assessed prospectively by three experts in EUS under direct observation and again 2 months later in a blinded fashion using digital video-recordings. Based on the assessments, intra-rater reliability, inter-rater reliability, and construct validity were explored. The intra-rater reliability was good (Cronbach's α = 0.80), but comparison of results based on direct observations and blinded video-recordings indicated a significant bias favoring consultants (P = 0.022). Inter-rater reliability was very good (Cronbach's α = 0.93). However, one rater assessing five procedures or two raters each assessing four procedures were necessary to secure a generalizability coefficient of 0.80. The assessment tool demonstrated construct validity by discriminating between trainees and experienced physicians (P = 0.034). Competency in mediastinal staging of NSCLC using EUS and EUS - FNA can be assessed in a reliable and valid way using the EUSAT assessment tool. Measuring and defining competency and training requirements could improve EUS quality and benefit patient care. © Georg Thieme Verlag KG Stuttgart · New York.

Evaluation of a Simpler Tool to Assess Nontechnical Skills During Simulated Critical Events.

PubMed

Watkins, Scott C; Roberts, David A; Boulet, John R; McEvoy, Matthew D; Weinger, Matthew B

2017-04-01

Management of critical events requires teams to employ nontechnical skills (NTS), such as teamwork, communication, decision making, and vigilance. We sought to estimate the reliability and provide evidence for the validity of the ratings gathered using a new tool for assessing the NTS of anesthesia providers, the behaviorally anchored rating scale (BARS), and compare its scores with those of an established NTS tool, the Anaesthetists' Nontechnical Skills (ANTS) scale. Six previously trained raters (4 novices and 2 experts) reviewed and scored 18 recorded simulated pediatric crisis management scenarios using a modified ANTS and a BARS tool. Pearson correlation coefficients were calculated separately for the novice and expert raters, by scenario, and overall. The intrarater reliability of the ANTS total score was 0.73 (expert, 0.57; novice, 0.84); for the BARS tool, it was 0.80 (expert, 0.79; novice, 0.81). The average interrater reliability of BARS scores (0.58) was better than ANTS scores (0.37), and the interrater reliabilities of scores from novices (0.69 BARS and 0.52 ANTS) were better than those obtained from experts (0.47 BARS and 0.21 ANTS) for both scoring instruments. The Pearson correlation between the ANTS and BARS total scores was 0.74. Overall, reliability estimates were better for the BARS scores than the ANTS scores. For both measures, the intrarater and interrater reliability was better for novices compared with domain experts, suggesting that properly trained novices can reliably assess the NTS of anesthesia providers managing a simulated critical event. There was substantial correlation between the 2 scoring instruments, suggesting that the tools measured similar constructs. The BARS tool can be an alternative to the ANTS scale for the formative assessment of NTS of anesthesia providers.

Developing and testing a street audit tool using Google Street View to measure environmental supportiveness for physical activity.

PubMed

Griew, Pippa; Hillsdon, Melvyn; Foster, Charlie; Coombes, Emma; Jones, Andy; Wilkinson, Paul

2013-08-23

Walking for physical activity is associated with substantial health benefits for adults. Increasingly research has focused on associations between walking behaviours and neighbourhood environments including street characteristics such as pavement availability and aesthetics. Nevertheless, objective assessment of street-level data is challenging. This research investigates the reliability of a new street characteristic audit tool designed for use with Google Street View, and assesses levels of agreement between computer-based and on-site auditing. The Forty Area STudy street VIEW (FASTVIEW) tool, a Google Street View based audit tool, was developed incorporating nine categories of street characteristics. Using the tool, desk-based audits were conducted by trained researchers across one large UK town during 2011. Both inter and intra-rater reliability were assessed. On-site street audits were also completed to test the criterion validity of the method. All reliability scores were assessed by percentage agreement and the kappa statistic. Within-rater agreement was high for each category of street characteristic (range: 66.7%-90.0%) and good to high between raters (range: 51.3%-89.1%). A high level of agreement was found between the Google Street View audits and those conducted in-person across the nine categories examined (range: 75.0%-96.7%). The audit tool was found to provide a reliable and valid measure of street characteristics. The use of Google Street View to capture street characteristic data is recommended as an efficient method that could substantially increase potential for large-scale objective data collection.

Adapting the helpful responses questionnaire to assess communication skills involved in delivering contingency management: preliminary psychometrics.

PubMed

Hartzler, Bryan

2015-08-01

A paper/pencil instrument, adapted from Miller and colleagues' (1991) Helpful Responses Questionnaire (HRQ), was developed to assess clinician skill with core communicative aspects involved in delivering contingency management (CM). The instrument presents a single vignette consisting of six points of client dialogue to which respondents write 'what they would say next.' In the context of an implementation/effectiveness hybrid trial, 19 staff clinicians at an opiate treatment program completed serial training outcome assessments before, following, and three months after CM training. Assessments included this adaptation of the HRQ, a multiple-choice CM knowledge test, and a recorded standardized patient encounter scored for CM skillfulness. Study results reveal promising psychometric properties for the instrument, including strong scoring reliability, internal consistency, concurrent and predictive validity, test-retest reliability and sensitivity to training effects. These preliminary findings suggest the instrument is a viable, practical method to assess clinician skill in communicative aspects of CM delivery. Copyright © 2015 Elsevier Inc. All rights reserved.

Reliability of the OSCE for Physical and Occupational Therapists

PubMed Central

Sakurai, Hiroaki; Kanada, Yoshikiyo; Sugiura, Yoshito; Motoya, Ikuo; Wada, Yosuke; Yamada, Masayuki; Tomita, Masao; Tanabe, Shigeo; Teranishi, Toshio; Tsujimura, Toru; Sawa, Syunji; Okanishi, Tetsuo

2014-01-01

[Purpose] To examine agreement rates between faculty members and clinical supervisors as OSCE examiners. [Subjects] The study subjects were involved physical and occupational therapists working in clinical environments for 1 to 5 years after graduating from training schools as OSCE examinees, and a physical or occupational therapy faculty member and a clinical supervisor as examiners. Another clinical supervisor acted as a simulated patient. [Methods] The agreement rate between the examiners for each OSCE item was calculated based on Cohen’s kappa coefficient to confirm inter-rater reliability. [Results] The agreement rates for the behavioral aspects of the items were higher in the second than in the first examination. Similar increases were also observed in the agreement rates for the technical aspects until the initiation of each activity; however, the rates decreased during the middle to terminal stages of continuous movements. [Conclusion] The results may reflect the recent implementation of measures for the integration of therapist education in training schools and clinical training facilities. PMID:25202170

Abstract numerical discrimination learning in rats.

PubMed

Taniuchi, Tohru; Sugihara, Junko; Wakashima, Mariko; Kamijo, Makiko

2016-06-01

In this study, we examined rats' discrimination learning of the numerical ordering positions of objects. In Experiments 1 and 2, five out of seven rats successfully learned to respond to the third of six identical objects in a row and showed reliable transfer of this discrimination to novel stimuli after being trained with three different training stimuli. In Experiment 3, the three rats from Experiment 2 continued to be trained to respond to the third object in an object array, which included an odd object that needed to be excluded when identifying the target third object. All three rats acquired this selective-counting task of specific stimuli, and two rats showed reliable transfer of this selective-counting performance to test sets of novel stimuli. In Experiment 4, the three rats from Experiment 3 quickly learned to respond to the third stimulus in object rows consisting of either six identical or six different objects. These results offer strong evidence for abstract numerical discrimination learning in rats.

HMMER Cut-off Threshold Tool (HMMERCTTER): Supervised classification of superfamily protein sequences with a reliable cut-off threshold.

PubMed

Pagnuco, Inti Anabela; Revuelta, María Victoria; Bondino, Hernán Gabriel; Brun, Marcel; Ten Have, Arjen

2018-01-01

Protein superfamilies can be divided into subfamilies of proteins with different functional characteristics. Their sequences can be classified hierarchically, which is part of sequence function assignation. Typically, there are no clear subfamily hallmarks that would allow pattern-based function assignation by which this task is mostly achieved based on the similarity principle. This is hampered by the lack of a score cut-off that is both sensitive and specific. HMMER Cut-off Threshold Tool (HMMERCTTER) adds a reliable cut-off threshold to the popular HMMER. Using a high quality superfamily phylogeny, it clusters a set of training sequences such that the cluster-specific HMMER profiles show cluster or subfamily member detection with 100% precision and recall (P&R), thereby generating a specific threshold as inclusion cut-off. Profiles and thresholds are then used as classifiers to screen a target dataset. Iterative inclusion of novel sequences to groups and the corresponding HMMER profiles results in high sensitivity while specificity is maintained by imposing 100% P&R self detection. In three presented case studies of protein superfamilies, classification of large datasets with 100% precision was achieved with over 95% recall. Limits and caveats are presented and explained. HMMERCTTER is a promising protein superfamily sequence classifier provided high quality training datasets are used. It provides a decision support system that aids in the difficult task of sequence function assignation in the twilight zone of sequence similarity. All relevant data and source codes are available from the Github repository at the following URL: https://github.com/BBCMdP/HMMERCTTER.

HMMER Cut-off Threshold Tool (HMMERCTTER): Supervised classification of superfamily protein sequences with a reliable cut-off threshold

PubMed Central

Pagnuco, Inti Anabela; Revuelta, María Victoria; Bondino, Hernán Gabriel; Brun, Marcel

2018-01-01

Background Protein superfamilies can be divided into subfamilies of proteins with different functional characteristics. Their sequences can be classified hierarchically, which is part of sequence function assignation. Typically, there are no clear subfamily hallmarks that would allow pattern-based function assignation by which this task is mostly achieved based on the similarity principle. This is hampered by the lack of a score cut-off that is both sensitive and specific. Results HMMER Cut-off Threshold Tool (HMMERCTTER) adds a reliable cut-off threshold to the popular HMMER. Using a high quality superfamily phylogeny, it clusters a set of training sequences such that the cluster-specific HMMER profiles show cluster or subfamily member detection with 100% precision and recall (P&R), thereby generating a specific threshold as inclusion cut-off. Profiles and thresholds are then used as classifiers to screen a target dataset. Iterative inclusion of novel sequences to groups and the corresponding HMMER profiles results in high sensitivity while specificity is maintained by imposing 100% P&R self detection. In three presented case studies of protein superfamilies, classification of large datasets with 100% precision was achieved with over 95% recall. Limits and caveats are presented and explained. Conclusions HMMERCTTER is a promising protein superfamily sequence classifier provided high quality training datasets are used. It provides a decision support system that aids in the difficult task of sequence function assignation in the twilight zone of sequence similarity. All relevant data and source codes are available from the Github repository at the following URL: https://github.com/BBCMdP/HMMERCTTER. PMID:29579071

Assessment of Nutrient Status in Athletes and the Need for Supplementation.

PubMed

Larson-Meyer, D Enette; Woolf, Kathleen; Burke, Louise

2018-03-01

Nutrition assessment is a necessary first step in advising athletes on dietary strategies that include dietary supplementation, and in evaluating the effectiveness of supplementation regimens. Although dietary assessment is the cornerstone component of the nutrition assessment process, it should be performed within the context of a complete assessment that includes collection/evaluation of anthropometric, biochemical, clinical, and environmental data. Collection of dietary intake data can be challenging, with the potential for significant error of validity and reliability, which include inherent errors of the collection methodology, coding of data by dietitians, estimation of nutrient composition using nutrient food tables and/or dietary software programs, and expression of data relative to reference standards including eating guidance systems, macronutrient guidelines for athletes, and recommended dietary allowances. Limitations in methodologies used to complete anthropometric assessment and biochemical analysis also exist, as reference norms for the athlete are not well established and practical and reliable biomarkers are not available for all nutrients. A clinical assessment collected from history information and the nutrition-focused physical exam may help identify overt nutrient deficiencies but may be unremarkable in the well-trained athlete. Assessment of potential food-drug interactions and environmental components further helps make appropriate dietary and supplement recommendations. Overall, the assessment process can help the athlete understand that supplement intake cannot make up for poor food choices and an inadequate diet, while a healthy diet helps ensure maximal benefit from supplementation. Establishment of reference norms specifically for well-trained athletes for the nutrition assessment process is a future research priority.

Spur, helical, and spiral bevel transmission life modeling

NASA Technical Reports Server (NTRS)

Savage, Michael; Rubadeux, Kelly L.; Coe, Harold H.; Coy, John J.

1994-01-01

A computer program, TLIFE, which estimates the life, dynamic capacity, and reliability of aircraft transmissions, is presented. The program enables comparisons of transmission service life at the design stage for optimization. A variety of transmissions may be analyzed including: spur, helical, and spiral bevel reductions as well as series combinations of these reductions. The basic spur and helical reductions include: single mesh, compound, and parallel path plus revert star and planetary gear trains. A variety of straddle and overhung bearing configurations on the gear shafts are possible as is the use of a ring gear for the output. The spiral bevel reductions include single and dual input drives with arbitrary shaft angles. The program is written in FORTRAN 77 and has been executed both in the personal computer DOS environment and on UNIX workstations. The analysis may be performed in either the SI metric or the English inch system of units. The reliability and life analysis is based on the two-parameter Weibull distribution lives of the component gears and bearings. The program output file describes the overall transmission and each constituent transmission, its components, and their locations, capacities, and loads. Primary output is the dynamic capacity and 90-percent reliability and mean lives of the unit transmissions and the overall system which can be used to estimate service overhaul frequency requirements. Two examples are presented to illustrate the information available for single element and series transmissions.

Enhancing the Communication of Climate Change Science

NASA Astrophysics Data System (ADS)

Somerville, R. C.; Hassol, S. J.

2011-12-01

Climate scientists have an important role to play in the critical task of informing the public, media and policymakers. Scientists can help in publicizing and illuminating climate science. However, this task requires combining climate science expertise with advanced communication skills. For example, it is entirely possible to convey scientific information accurately without using jargon or technical concepts unfamiliar to non-scientists. However, making this translation into everyday language is a job that few scientists have been trained to do. In this talk, we give examples from our recent experience working with scientists to enhance their ability to communicate well. Our work includes providing training, technical assistance, and communications tools to climate scientists and universities, government agencies, and research centers. Our experience ranges from preparing Congressional testimony to writing major climate science reports to appearing on television. We have also aided journalists in gathering reliable scientific information and identifying trustworthy experts. Additionally, we are involved in developing resources freely available online at climatecommunication.org. These include a feature on the links between climate change and extreme weather, a climate science primer, and graphics and video explaining key developments in climate change science.

Utility and Reliability of an App for the System for Observing Play and Recreation in Communities (iSOPARC®)

ERIC Educational Resources Information Center

Santos, Maria P. M.; Rech, Cassiano R.; Alberico, Claudia O.; Fermino, Rogério C.; Rios, Ana P.; David, João; Reis, Rodrigo S.; Sarmiento, Olga L.; McKenzie, Thomas L.; Mota, Jorge

2016-01-01

The app for the System for Observing Play and Recreation in Communities (iSOPARC®) was developed to enhance System for Observing Play and Recreation in Communities data collection and management. The study aim was to examine the usability and inter-rater reliability of iSOPARC®. Trained observers collected data in 16 park areas in two Latin…

Assessing Reliability and Validity of the "GroPromo" Audit Tool for Evaluation of Grocery Store Marketing and Promotional Environments

ERIC Educational Resources Information Center

Kerr, Jacqueline; Sallis, James F.; Bromby, Erica; Glanz, Karen

2012-01-01

Objective: To evaluate reliability and validity of a new tool for assessing the placement and promotional environment in grocery stores. Methods: Trained observers used the "GroPromo" instrument in 40 stores to code the placement of 7 products in 9 locations within a store, along with other promotional characteristics. To test construct validity,…

Reliability: A Comparison of Absence Measures.

DTIC Science & Technology

1985-09-01

28 8. Sick Leave Frequency Index ... . .. . . . . . . . 29 J.v -. ~~~~i j.- .i...- - AFIT/GSM/LSB/85S-19 Abstract Absenteeism is an important and...RELIABILITY: A COMPARISON OF ABSENCE MEASURES I. Literature Review Introduction Absenteeism may be defined as undesired work absence. This withdrawal...behavior is an important organizational problem because of probable reduced labor productivity and increased training costs. * Absenteeism is almost

Review on the Traction System Sensor Technology of a Rail Transit Train.

PubMed

Feng, Jianghua; Xu, Junfeng; Liao, Wu; Liu, Yong

2017-06-11

The development of high-speed intelligent rail transit has increased the number of sensors applied on trains. These play an important role in train state control and monitoring. These sensors generally work in a severe environment, so the key problem for sensor data acquisition is to ensure data accuracy and reliability. In this paper, we follow the sequence of sensor signal flow, present sensor signal sensing technology, sensor data acquisition, and processing technology, as well as sensor fault diagnosis technology based on the voltage, current, speed, and temperature sensors which are commonly used in train traction systems. Finally, intelligent sensors and future research directions of rail transit train sensors are discussed.

Review on the Traction System Sensor Technology of a Rail Transit Train

PubMed Central

Feng, Jianghua; Xu, Junfeng; Liao, Wu; Liu, Yong

2017-01-01

The development of high-speed intelligent rail transit has increased the number of sensors applied on trains. These play an important role in train state control and monitoring. These sensors generally work in a severe environment, so the key problem for sensor data acquisition is to ensure data accuracy and reliability. In this paper, we follow the sequence of sensor signal flow, present sensor signal sensing technology, sensor data acquisition, and processing technology, as well as sensor fault diagnosis technology based on the voltage, current, speed, and temperature sensors which are commonly used in train traction systems. Finally, intelligent sensors and future research directions of rail transit train sensors are discussed. PMID:28604615

Nutrition Risk in Home-Bound Older Adults: Using Dietician-Trained and Supervised Nutrition Volunteers for Screening and Intervention

ERIC Educational Resources Information Center

Laforest, Sophie; Goldin, Benita; Nour, Kareen; Roy, Marie-Andree; Payette, Helene

2007-01-01

Nutrition screening and early intervention in home-bound older adults are key to preventing unfavourable health outcomes and functional decline. This pilot study's objectives were (a) to test the reliability of the Elderly Nutrition Screening Tool (ENS [C]) when administered by dietician-trained and supervised nutrition volunteers, and (b) to…

Fetal Alcohol Syndrome: A Training Manual To Aid in Vocational Rehabilitation and Other Non-Medical Services.

ERIC Educational Resources Information Center

LaDue, Robin A.; Schacht, Robert M.; Tanner-Halverson, Patricia; McGowan, Mark

This training manual provides vocational rehabilitation and school counselors with background information and practical tools related to fetal alcohol syndrome (FAS), with particular reference to the needs of Native Americans. The most recent reliable data (1990) for American Indians and Alaska Natives show a rate of FAS over 10 times the national…

Plant and Industry Experience. MAS-122. Waste Isolation Division (WID). Management and Supervisor Training (MAST) Program.

ERIC Educational Resources Information Center

Westinghouse Electric Corp., Carlsbad, NM.

This learning module, which is part of a management and supervisor training program for managers and supervisors employed at the Department of Energy's Waste Isolation Division, is designed to prepare trainees to use plant and industry experience to improve plant safety and reliability. The following topics are covered in the module's individual…

Transforming the Public Provision of Training: Reorganisation or Privatisation? Long-Term Changes in Sweden and the Netherlands. CEDEFOP Discussion Dossier.

ERIC Educational Resources Information Center

Brandsma, Jittie; Noonan, Richard; Westphalen, Sven-Age

The public sector is becoming less concerned with who is providing a given service but more concerned about the quality, reliability, accessibility, and price of the service. In vocational education and training (VET), one consequence of this transformation is that the various public stakeholders involved with funding, purchasing, and providing…

Technical Training Requirements of Middle Management in the Greek Textile and Clothing Industries.

ERIC Educational Resources Information Center

Fotinopoulou, K.; Manolopoulos, N.

A case study of 16 companies in the Greek textile and clothing industry elicited the training needs of the industry's middle managers. The study concentrated on large and medium-sized work units, using a lengthy questionnaire. The study found that middle managers increasingly need to solve problems and ensure the reliability of new equipment and…

Consistency and reliability of judgements by assessors of case based discussions in general practice specialty training programmes in the United Kingdom.

PubMed

Bodgener, Susan; Denney, Meiling; Howard, John

2017-01-01

Case based discussions (CbDs) are a mandatory workplace assessment used throughout general practitioner (GP) specialty training; they contribute to the annual review of competence progression (ARCP) for each trainee. This study examined the judgements arising from CbDs made by different groups of assessors and whether or not these assessments supported ARCP decisions. The trainees selected were at the end of their first year of GP training and had been identified during their ARCPs to need extra training time. CbDs were specifically chosen as they are completed by both hospital and GP supervisors, enabling comparison between these two groups. The results raise concern with regard to the consistency of judgements made by different groups of assessors, with significant variance between assessors of different status and seniority. Further work needs to be done on whether the CbD in its current format is fit for purpose as one of the mandatory WPBAs for GP trainees, particularly during their hospital placements. There is a need to increase the inter-rater reliability of CbDs to ensure a consistent contribution to subsequent decisions about a trainee's overall progress.

Optimal Design of the Absolute Positioning Sensor for a High-Speed Maglev Train and Research on Its Fault Diagnosis

PubMed Central

Zhang, Dapeng; Long, Zhiqiang; Xue, Song; Zhang, Junge

2012-01-01

This paper studies an absolute positioning sensor for a high-speed maglev train and its fault diagnosis method. The absolute positioning sensor is an important sensor for the high-speed maglev train to accomplish its synchronous traction. It is used to calibrate the error of the relative positioning sensor which is used to provide the magnetic phase signal. On the basis of the analysis for the principle of the absolute positioning sensor, the paper describes the design of the sending and receiving coils and realizes the hardware and the software for the sensor. In order to enhance the reliability of the sensor, a support vector machine is used to recognize the fault characters, and the signal flow method is used to locate the faulty parts. The diagnosis information not only can be sent to an upper center control computer to evaluate the reliability of the sensors, but also can realize on-line diagnosis for debugging and the quick detection when the maglev train is off-line. The absolute positioning sensor we study has been used in the actual project. PMID:23112619

Transfer of infrared thermography predictive maintenance technologies to Soviet-designed nuclear power plants: experience at Chernobyl

NASA Astrophysics Data System (ADS)

Pugh, Ray; Huff, Roy

1999-03-01

The importance of infrared (IR) technology and analysis in today's world of predictive maintenance and reliability- centered maintenance cannot be understated. The use of infrared is especially important in facilities that are required to maintain a high degree of equipment reliability because of plant or public safety concerns. As with all maintenance tools, particularly those used in predictive maintenance approaches, training plays a key role in their effectiveness and the benefit gained from their use. This paper details an effort to transfer IR technology to Soviet- designed nuclear power plants in Russia, Ukraine, and Lithuania. Delivery of this technology and post-delivery training activities have been completed recently at the Chornobyl nuclear power plant in Ukraine. Many interesting challenges were encountered during this effort. Hardware procurement and delivery of IR technology to a sensitive country were complicated by United States regulations. Freight and shipping infrastructure and host-country customs policies complicated hardware transport. Training activities were complicated by special hardware, software and training material translation needs, limited communication opportunities, and site logistical concerns. These challenges and others encountered while supplying the Chornobyl plant with state-of-the-art IR technology are described in this paper.

A semi-supervised learning approach for RNA secondary structure prediction.

PubMed

Yonemoto, Haruka; Asai, Kiyoshi; Hamada, Michiaki

2015-08-01

RNA secondary structure prediction is a key technology in RNA bioinformatics. Most algorithms for RNA secondary structure prediction use probabilistic models, in which the model parameters are trained with reliable RNA secondary structures. Because of the difficulty of determining RNA secondary structures by experimental procedures, such as NMR or X-ray crystal structural analyses, there are still many RNA sequences that could be useful for training whose secondary structures have not been experimentally determined. In this paper, we introduce a novel semi-supervised learning approach for training parameters in a probabilistic model of RNA secondary structures in which we employ not only RNA sequences with annotated secondary structures but also ones with unknown secondary structures. Our model is based on a hybrid of generative (stochastic context-free grammars) and discriminative models (conditional random fields) that has been successfully applied to natural language processing. Computational experiments indicate that the accuracy of secondary structure prediction is improved by incorporating RNA sequences with unknown secondary structures into training. To our knowledge, this is the first study of a semi-supervised learning approach for RNA secondary structure prediction. This technique will be useful when the number of reliable structures is limited. Copyright © 2015 Elsevier Ltd. All rights reserved.

Design and Evaluation of a Training Protocol for a Photographic Method of Visual Estimation of Fruit and Vegetable Intake among Kindergarten Through Second-Grade Students.

PubMed

Masis, Natalie; McCaffrey, Jennifer; Johnson, Susan L; Chapman-Novakofski, Karen

2017-04-01

To design a replicable training protocol for visual estimation of fruit and vegetable (FV) intake of kindergarten through second-grade students through digital photography of lunch trays that results in reliable data for FV served and consumed. Protocol development through literature and researcher input was followed by 3 laboratory-based trainings of 3 trainees. Lunchroom data collection sessions were done at 2 elementary schools for kindergarten through second-graders. Intraclass correlation coefficients (ICCs) were used. By training 3, ICC was substantial for amount of FV served and consumed (0.86 and 0.95, respectively; P < .05). The ICC was moderate for percentage of fruits consumed (0.67; P = .06). In-school estimates for ICCs were all significant for amounts served at school 1 and percentage of FV consumed at both schools. The protocol resulted in reliable estimation of combined FV served and consumed using digital photography. The ability to estimate FV intake accurately will benefit intervention development and evaluation. Copyright © 2017 Society for Nutrition Education and Behavior. Published by Elsevier Inc. All rights reserved.

Electric Utility Transmission and Distribution Line Engineering Program

DOE Office of Scientific and Technical Information (OSTI.GOV)

Peter McKenny

2010-08-31

Economic development in the United States depends on a reliable and affordable power supply. The nation will need well educated engineers to design a modern, safe, secure, and reliable power grid for our future needs. An anticipated shortage of qualified engineers has caused considerable concern in many professional circles, and various steps are being taken nationwide to alleviate the potential shortage and ensure the North American power system's reliability, and our world-wide economic competitiveness. To help provide a well-educated and trained workforce which can sustain and modernize the nation's power grid, Gonzaga University's School of Engineering and Applied Science hasmore » established a five-course (15-credit hour) Certificate Program in Transmission and Distribution (T&D) Engineering. The program has been specifically designed to provide working utility engineering professionals with on-line access to advanced engineering courses which cover modern design practice with an industry-focused theoretical foundation. A total of twelve courses have been developed to-date and students may select any five in their area of interest for the T&D Certificate. As each course is developed and taught by a team of experienced engineers (from public and private utilities, consultants, and industry suppliers), students are provided a unique opportunity to interact directly with different industry experts over the eight weeks of each course. Course material incorporates advanced aspects of civil, electrical, and mechanical engineering disciplines that apply to power system design and are appropriate for graduate engineers. As such, target students for the certificate program include: (1) recent graduates with a Bachelor of Science Degree in an engineering field (civil, mechanical, electrical, etc.); (2) senior engineers moving from other fields to the utility industry (i.e. paper industry to utility engineering or project management positions); and (3) regular working professionals wishing to update their skills or increase their knowledge of utility engineering design practices and procedures. By providing graduate educational opportunities for the above groups, the T&D Program will help serve a strong industry need for training the next generation of engineers in the cost-effective design, construction, operation, and maintenance of modern electrical transmission and distribution systems. In addition to developing the on-line engineering courses described above, the T&D Program also focused significant efforts towards enhancing the training opportunities available to power system operators in the northwest. These efforts have included working with outside vendors to provide NERC-approved training courses in Gonzaga University's (GU) system operator training facility, support for an accurate system model which can be used in regional blackstart exercises, and the identification of a retired system operator who could provide actual regional training courses. The GU system operator training facility is also being used to recruit young workers, veterans, and various under-represented groups to the utility industry. Over the past three years students from Columbia Gorge Community College, Spokane Falls Community College, Walla Walla Community College, Central Washington University, Eastern Washington University, Gonzaga University, and various local high schools have attended short (one-day) system operator training courses free of charge. These collaboration efforts has been extremely well received by both students and industry, and meet T&D Program objectives of strengthening the power industry workforce while bridging the knowledge base across power worker categories, and recruiting new workers to replace a predominantly retirement age workforce. In the past three years the T&D Program has provided over 170 utility engineers with access to advanced engineering courses, been involved in training more than 300 power system operators, and provided well over 500 college and high school students with an experience in running a power system simulator and an exposure to various utility-related professions and craft trades.« less

Improving accuracy of genomic prediction in Brangus cattle by adding animals with imputed low-density SNP genotypes.

PubMed

Lopes, F B; Wu, X-L; Li, H; Xu, J; Perkins, T; Genho, J; Ferretti, R; Tait, R G; Bauck, S; Rosa, G J M

2018-02-01

Reliable genomic prediction of breeding values for quantitative traits requires the availability of sufficient number of animals with genotypes and phenotypes in the training set. As of 31 October 2016, there were 3,797 Brangus animals with genotypes and phenotypes. These Brangus animals were genotyped using different commercial SNP chips. Of them, the largest group consisted of 1,535 animals genotyped by the GGP-LDV4 SNP chip. The remaining 2,262 genotypes were imputed to the SNP content of the GGP-LDV4 chip, so that the number of animals available for training the genomic prediction models was more than doubled. The present study showed that the pooling of animals with both original or imputed 40K SNP genotypes substantially increased genomic prediction accuracies on the ten traits. By supplementing imputed genotypes, the relative gains in genomic prediction accuracies on estimated breeding values (EBV) were from 12.60% to 31.27%, and the relative gain in genomic prediction accuracies on de-regressed EBV was slightly small (i.e. 0.87%-18.75%). The present study also compared the performance of five genomic prediction models and two cross-validation methods. The five genomic models predicted EBV and de-regressed EBV of the ten traits similarly well. Of the two cross-validation methods, leave-one-out cross-validation maximized the number of animals at the stage of training for genomic prediction. Genomic prediction accuracy (GPA) on the ten quantitative traits was validated in 1,106 newly genotyped Brangus animals based on the SNP effects estimated in the previous set of 3,797 Brangus animals, and they were slightly lower than GPA in the original data. The present study was the first to leverage currently available genotype and phenotype resources in order to harness genomic prediction in Brangus beef cattle. © 2018 Blackwell Verlag GmbH.

Session-RPE for quantifying load of different youth taekwondo training sessions.

PubMed

Lupo, Corrado; Capranica, Laura; Cortis, Cristina; Guidotti, Flavia; Bianco, Antonino; Tessitore, Antonio

2017-03-01

The session rating of perceived exertion (session-RPE) proved to be a valuable method to quantify the internal training load (ITL) in taekwondo. However, no study validated this method in youth taekwondo athletes performing different training sessions. Thus this study aimed at evaluating the reliability of the session-RPE to monitor the ITL of prepubescent taekwondo athletes during pre-competitive (PC) and competitive (C) training sessions. Five female (age: 12.0±0.7 y; height: 1.54±0.08 m; body mass: 48.8±7.3 kg) and four male (age: 12.0±0.8 yrs; height: 1.55±0.07 m; body mass: 47.3±5.3 kg) taekwondo athletes were monitored during 100 individual sessions (PC: N.=33; C: N.=67). The Edwards' HR method was used as reference measure of ITL; the CR-10 RPE scale was administered at 1- and 30-minutes from the end of each session. No difference for gender emerged. The ITLs of C (Edwards: 228±40 arbitrary units, AU) resulted higher than that of PC (192±26 AU; P=0.04). Although all training typologies and data collections achieved significant correlations between Edwards' and session-RPE methods, a large relationship (r =0.71, P<0.001) emerged only for PC sessions evaluated at 30 minutes of the recovery phases. Findings support coaches of prepubescent taekwondo athletes to successfully use session-RPE to monitor the ITL of different training typologies. However, PC training evaluated at 30 minutes of the recovery phase represents the best condition for a highly reliable ITL perception.

Test-retest and interobserver reliability of quantitative sensory testing according to the protocol of the German Research Network on Neuropathic Pain (DFNS): a multi-centre study.

PubMed

Geber, Christian; Klein, Thomas; Azad, Shahnaz; Birklein, Frank; Gierthmühlen, Janne; Huge, Volker; Lauchart, Meike; Nitzsche, Dorothee; Stengel, Maike; Valet, Michael; Baron, Ralf; Maier, Christoph; Tölle, Thomas; Treede, Rolf-Detlef

2011-03-01

Quantitative sensory testing (QST) is an instrument to assess positive and negative sensory signs, helping to identify mechanisms underlying pathologic pain conditions. In this study, we evaluated the test-retest reliability (TR-R) and the interobserver reliability (IO-R) of QST in patients with sensory disturbances of different etiologies. In 4 centres, 60 patients (37 male and 23 female, 56.4±1.9years) with lesions or diseases of the somatosensory system were included. QST comprised 13 parameters including detection and pain thresholds for thermal and mechanical stimuli. QST was performed in the clinically most affected test area and a less or unaffected control area in a morning and an afternoon session on 2 consecutive days by examiner pairs (4 QSTs/patient). For both, TR-R and IO-R, there were high correlations (r=0.80-0.93) at the affected test area, except for wind-up ratio (TR-R: r=0.67; IO-R: r=0.56) and paradoxical heat sensations (TR-R: r=0.35; IO-R: r=0.44). Mean IO-R (r=0.83, 31% unexplained variance) was slightly lower than TR-R (r=0.86, 26% unexplained variance, P<.05); the difference in variance amounted to 5%. There were no differences between study centres. In a subgroup with an unaffected control area (n=43), reliabilities were significantly better in the test area (TR-R: r=0.86; IO-R: r=0.83) than in the control area (TR-R: r=0.79; IO-R: r=0.71, each P<.01), suggesting that disease-related systematic variance enhances reliability of QST. We conclude that standardized QST performed by trained examiners is a valuable diagnostic instrument with good test-retest and interobserver reliability within 2days. With standardized training, observer bias is much lower than random variance. Quantitative sensory testing performed by trained examiners is a valuable diagnostic instrument with good interobserver and test-retest reliability for use in patients with sensory disturbances of different etiologies to help identify mechanisms of neuropathic and non-neuropathic pain. Copyright © 2010 International Association for the Study of Pain. Published by Elsevier B.V. All rights reserved.

Reliability of length measurements collected by community nurses and health volunteers in rural growth monitoring and promotion services.

PubMed

Laar, Matilda E; Marquis, Grace S; Lartey, Anna; Gray-Donald, Katherine

2018-02-17

Length measurements are important in growth, monitoring and promotion (GMP) for the surveillance of a child's weight-for-length and length-for-age. These two indices provide an indication of a child's risk of becoming wasted or stunted, and are more informative about a child's growth than the widely used weight-for-age index (underweight). Although the introduction of length measurements in GMP is recommended by the World Health Organization, concerns about the reliability of length measurements collected in rural outreach settings have been expressed by stakeholders. Our aim was to describe the reliability and challenges associated with community health personnel measuring length for rural outreach GMP activities. Two reliability studies (A and B), using 10 children less than 24 months each, were conducted in the GMP services of a rural district in Ghana. Fifteen nurses and 15 health volunteers (HV) with no prior experience in length measurements were trained. Intra- and inter-observer technical error of measurement (TEM), average bias from expert anthropometrist, and coefficient of reliability (R) of length measurements were assessed and compared across sessions. Observations and interviews were used to understand the ability and experiences of health personnel with measuring length at outreach GMP. Inter-observer TEM was larger than intra-observer TEM for both nurses and HV at both sessions and was unacceptably (compared to error standards) high in both groups at both time points. Average biases from expert's measurements were within acceptable limits, however, both groups tended to underestimate length measurements. The R for lengths collected by nurses (92.3%) was higher at session B compared to that of HV (87.5%). Length measurements taken by nurses and HV, and those taken by an experienced anthropometrist at GMP sessions were of moderate agreement (kappa = 0.53, p < 0.0001). The reliability of length measurements improved after two refresher trainings for nurses but not for HV. In addition, length measurements taken during GMP sessions may be susceptible to errors due to overburdened health personnel and crowded GMP clinics. There is need for both pre- and in-service training of nurses and HV on length measurements and procedures to improve reliability of length measurements.

HiRel - Reliability/availability integrated workstation tool

NASA Technical Reports Server (NTRS)

Bavuso, Salvatore J.; Dugan, Joanne B.

1992-01-01

The HiRel software tool is described and demonstrated by application to the mission avionics subsystem of the Advanced System Integration Demonstrations (ASID) system that utilizes the PAVE PILLAR approach. HiRel marks another accomplishment toward the goal of producing a totally integrated computer-aided design (CAD) workstation design capability. Since a reliability engineer generally represents a reliability model graphically before it can be solved, the use of a graphical input description language increases productivity and decreases the incidence of error. The graphical postprocessor module HARPO makes it possible for reliability engineers to quickly analyze huge amounts of reliability/availability data to observe trends due to exploratory design changes. The addition of several powerful HARP modeling engines provides the user with a reliability/availability modeling capability for a wide range of system applications all integrated under a common interactive graphical input-output capability.

Teamwork as an Essential Component of High-Reliability Organizations

PubMed Central

Baker, David P; Day, Rachel; Salas, Eduardo

2006-01-01

Organizations are increasingly becoming dynamic and unstable. This evolution has given rise to greater reliance on teams and increased complexity in terms of team composition, skills required, and degree of risk involved. High-reliability organizations (HROs) are those that exist in such hazardous environments where the consequences of errors are high, but the occurrence of error is extremely low. In this article, we argue that teamwork is an essential component of achieving high reliability particularly in health care organizations. We describe the fundamental characteristics of teams, review strategies in team training, demonstrate the criticality of teamwork in HROs and finally, identify specific challenges the health care community must address to improve teamwork and enhance reliability. PMID:16898980

Reliability and validity enhancement: a treatment package for increasing fidelity of self-report.

PubMed

Bornstein, P H; Hamilton, S B; Miller, R K; Quevillon, R P; Spitzform, M

1977-07-01

This study investigated the effects of reliability and validity "enhancers" on fidelity of self-report data in an analogue therapy situation. Under the guise of a Concentration Skills Training Program, 57 Ss were assigned randomly to one of the following conditions: (a) Reliability Enhancement; (b) Truth Talk; (c) No Comment Control. Results indicated significant differences among groups (p less than .05). In addition, tests of multiple comparisons revealed that Reliability Enhancement was significantly different from Truth Talk in occurrences of unreliability (p less than .05). These findings are discussed in light of the increased reliance on self-report data in behavioral intervention, and recommendations are made for future research.