Relationships between Contextual and Task Performance and Interrater Agreement: Are There Any?

PubMed Central

Díaz-Cabrera, Dolores; Hernández-Fernaud, Estefanía; Rosales-Sánchez, Christian

2015-01-01

Work performance is one of the most important dependent variables in Work and Organizational Psychology. The main objective of this paper was to explore the relationships between citizenship performance and task performance measures obtained from different appraisers and their consistency through a seldom-used methodology, intraclass correlation coefficients. Participants were 135 public employees, the total staff in a local government department. Jobs were clustered into job families through a work analysis based on standard questionnaires. A task description technique was used to develop a performance appraisal questionnaire for each job family, with three versions: self-, supervisor-, and peer-evaluation, in addition to a measure of citizenship performance. Only when the self-appraisal bias is controlled, significant correlations appeared between task performance rates. However, intraclass correlations analyses show that only self- (contextual and task) performance measures are consistent, while interrater agreement disappears. These results provide some interesting clues about the procedure of appraisal instrument development, the role of appraisers, and the importance of choosing adequate consistency analysis methods. PMID:26473956

Evaluation of Mother-Child Agreement and Factorial Structures of the SCARED Questionnaire in an Italian Clinical Sample

PubMed Central

Scaini, Simona; Ogliari, Anna; De Carolis, Ludovica; Bellodi, Laura; Di Serio, Clelia; Brombin, Chiara

2017-01-01

Background: A great part of the literature has confirmed the importance of both child and parents reports as source of factual information, especially for childhood emotional syndromes. In our study we aimed at: (i) calculating mother-child agreement and (ii) evaluating factorial structure of the Screen for Child Anxiety Related Emotional Disorders (SCARED) questionnaire in an Italian clinical sample. The novelty of this contribution is two-fold: first, from a clinical point of view, we investigated the parent-child agreement level and examined separately the factorial structures of both parent and child versions of the SCARED for the first time in an Italian clinical sample. Second, unlike previous studies, we used statistical approaches specifically suited to account for the ordinal nature of the collected variables. Method: In a clinical sample of 171 children and adolescents aged 8–18 and their mothers we evaluated inter-rater agreement using weighted kappa indices to assess agreement for each item belonging to a certain SCARED subscale. Exploratory factor analysis for ordinal data was then performed on the polychoric correlation matrix calculated on SCARED items. Differences in the numbers of symptoms reported by children and parents were evaluated as well. Results and Conclusions: Our results reveal moderate to strong mother-child agreement. A significant age effect is present. Two different factorial solutions emerged for parent and child SCARED versions (a 5 factor structure for parents and a 6 factor solution in the child version, including a new factor “Worry about Parents”). This study confirmed the importance of evaluating both child and parent reports in assessment protocols for anxiety disorders. Our findings could help clinicians to determine which information, and from which rater, must be accounted for in evaluating treatment decisions. Moreover, we find that patients characteristics, such as gender and age, should be taken into account when

Evaluation of the activity of rheumatoid arthritis in clinical practice. Agreement between self-rated clinimetric evaluation and clinical evaluation with activity indexes: DAS28, CDAI and SDAI.

PubMed

Horta-Baas, Gabriel; Pérez Bolde-Hernández, Arturo; Hernández-Cabrera, María Fernanda; Vergara-Sánchez, Imelda; Romero-Figueroa, María Del Socorro

2017-10-11

To achieve control of rheumatoid arthritis (RA) it is necessary to be able to evaluate its activity. The American College of Rheumatology (ACR) recommends for this purpose indexes of activity that can be performed by the patient (PAS-II and RAPID-3) and IA including medical evaluation with laboratory studies (DAS28 and SDAI) or without them (CDAI). The objective was to analyze the concordance between self-rated clinimetric evaluation and clinimetric evaluation performed by the physician. Analytical cross-sectional study in 126 patients with RA. The agreement was evaluated through the weighted κ coefficient and the Krippendorff's α coefficient. The PAS-II and RAPID-3 significantly correlated with all variables included in the core set of measures recommended by the ACR/EULAR. The agreement between PAS-II and CDAI-SDAI was good (κ: 0.6, α: 0.61-0.62), and moderate with DAS28-ESR (κ: 0.53, α: 0.56). The concordance between RAPID-3 and CDAI-SDAI was moderate (κ: 0.55-0.57, α: 0.50-0.51), and moderate with DAS28-ESR (κ: 0.55, α: 0.53). When categorizing the activity in remission/low activity vs. moderate/severe activity, the agreement was greater with the PAS-II (0.59 vs. 0.34; P=.012). The good concordance between PAS-II and SDAI supports their use in clinical practice, especially if biomarkers of inflammation or the possibility of joint count are not available. However, in order to recommend its routine application in clinical practice, it is necessary to perform longitudinal studies that assess its responsiveness. Copyright © 2017 Elsevier España, S.L.U. All rights reserved.

A comparative agreement evaluation of two subaxial cervical spine injury classification systems: the AOSpine and the Allen and Ferguson schemes.

PubMed

Urrutia, Julio; Zamora, Tomas; Campos, Mauricio; Yurac, Ratko; Palma, Joaquin; Mobarec, Sebastian; Prada, Carlos

2016-07-01

We performed an agreement study using two subaxial cervical spine classification systems: the AOSpine and the Allen and Ferguson (A&F) classifications. We sought to determine which scheme allows better agreement by different evaluators and by the same evaluator on different occasions. Complete imaging studies of 65 patients with subaxial cervical spine injuries were classified by six evaluators (three spine sub-specialists and three senior orthopaedic surgery residents) using the AOSpine subaxial cervical spine classification system and the A&F scheme. The cases were displayed in a random sequence after a 6-week interval for repeat evaluation. The Kappa coefficient (κ) was used to determine inter- and intra-observer agreement. Inter-observer: considering the main AO injury types, the agreement was substantial for the AOSpine classification [κ = 0.61 (0.57-0.64)]; using AO sub-types, the agreement was moderate [κ = 0.57 (0.54-0.60)]. For the A&F classification, the agreement [κ = 0.46 (0.42-0.49)] was significantly lower than using the AOSpine scheme. Intra-observer: the agreement was substantial considering injury types [κ = 0.68 (0.62-0.74)] and considering sub-types [κ = 0.62 (0.57-0.66)]. Using the A&F classification, the agreement was also substantial [κ = 0.66 (0.61-0.71)]. No significant differences were observed between spine surgeons and orthopaedic residents in the overall inter- and intra-observer agreement, or in the inter- and intra-observer agreement of specific type of injuries. The AOSpine classification (using the four main injury types or at the sub-types level) allows a significantly better agreement than the A&F classification. The A&F scheme does not allow reliable communication between medical professionals.

A mixed-methods evaluation of the Educational Supervision Agreement for Wales

PubMed Central

Bullock, Alison; Groves, Caroline; Saayman, Anton Gerhard

2017-01-01

Objectives In a bid to promote high-quality postgraduate education and training and support the General Medical Council’s (GMC) implementation plan for trainer recognition, the Wales Deanery developed the Educational Supervision Agreement (EdSA). This is a three-way agreement between Educational Supervisors, Local Education Providers and the Wales Deanery which clarifies roles, responsibilities and expectations for all. This paper reports on the formative evaluation of the EdSA after 1 year. Design Evaluation of pan-Wales EdSA roll-out (2013–2015) employed a mixed-methods approach: questionnaires (n=191), interviews (n=11) with educational supervisors and discussion with key stakeholders (GMC, All-Wales Trainer Recognition Group, Clinical Directors). Numerical data were analysed in SPSS V.20; open comments underwent thematic content analysis. Participants The study involved Educational Supervisors working in different specialties across Wales, UK. Results At the point of data collection, survey respondents represented 14% of signed agreements. Respondents believed the Agreement professionalises the Educational Supervisor role (85%, n=159 agreed), increases the accountability of Educational Supervisors (87%; n=160) and health boards (72%, n=131), provides leverage to negotiate supporting professional activities’ (SPA) time (76%, n=142) and continuing professional development (CPD) activities (71%, n=131). Factor analysis identified three principal factors: professionalisation of the educational supervisor role, supporting practice through training and feedback and implementation of the Agreement. Conclusions Our evidence suggests that respondents believed the Agreement would professionalise and support their Educational Supervisor role. Respondents showed enthusiasm for the Agreement and its role in maintaining high standards of training. PMID:28600372

A prospective study assessing agreement and reliability of a geriatric evaluation.

PubMed

Locatelli, Isabella; Monod, Stéfanie; Cornuz, Jacques; Büla, Christophe J; Senn, Nicolas

2017-07-19

The present study takes place within a geriatric program, aiming at improving the diagnosis and management of geriatric syndromes in primary care. Within this program it was of prime importance to be able to rely on a robust and reproducible geriatric consultation to use as a gold standard for evaluating a primary care brief assessment tool. The specific objective of the present study was thus assessing the agreement and reliability of a comprehensive geriatric consultation. The study was conducted at the outpatient clinic of the Service of Geriatric Medicine, University of Lausanne, Switzerland. All community-dwelling older persons aged 70 years and above were eligible. Patients were excluded if they hadn't a primary care physician, they were unable to speak French, or they were already assessed by a geriatrician within the last 12 months. A set of 9 geriatricians evaluated 20 patients. Each patient was assessed twice within a 2-month delay. Geriatric consultations were based on a structured evaluation process, leading to rating the following geriatric conditions: functional, cognitive, visual, and hearing impairment, mood disorders, risk of fall, osteoporosis, malnutrition, and urinary incontinence. Reliability and agreement estimates on each of these items were obtained using a three-way Intraclass Correlation and a three-way Observed Disagreement index. The latter allowed a decomposition of overall disagreement into disagreements due to each source of error variability (visit, rater and random). Agreement ranged between 0.62 and 0.85. For most domains, geriatrician-related error variability explained an important proportion of disagreement. Reliability ranged between 0 and 0.8. It was poor/moderate for visual impairment, malnutrition and risk of fall, and good/excellent for functional/cognitive/hearing impairment, osteoporosis, incontinence and mood disorders. Six out of nine items of the geriatric consultation described in this study (functional

Morphological Deficits of Children with SLI: Evaluation of Number Marking and Agreement.

ERIC Educational Resources Information Center

Rice, Mabel L.; Oetting, Janna B.

1993-01-01

Grammatical deficits (e.g., missing feature, surface account, and missing agreement) reported for children with specific language impairment (SLI) were evaluated in spontaneous language transcripts from 108 preschool children. Results indicated that children with SLI do control number marking but find number agreement across clausal boundaries…

[Agreement between guidelines for pharmaco-economic research and never-before-published health-economics evaluations].

PubMed

Postma, M J; Kwik, J J; Rutten, W J M J; de Jong-van den Berg, L T W; Brouwers, J R B J

2002-06-08

To investigate whether the health-economics research published in Dutch journals is in agreement with the guidelines for pharmaco-economic research as published in 1999 by the Dutch 'College voor zorgverzekeringen' [Healthcare Insurance Board]. Descriptive. A literature search was performed for health-economic evaluations published during 1990-1999 in Dutch journals such as Nederlands Tijdschrift voor Geneeskunde (NTvG), Pharmaceutisch Weekblad and Tijdschrift voor Gezondheidswetenschappen. The resulting 19 studies, primarily published in the NTvG, were compared with the pharmaco-economic guidelines, summarised in 9 criteria. The social perspective was taken in 5 studies (26%), non-related medical costs were excluded in 19 (100%), relevant subgroup analyses were performed in 16 (84%), effectiveness was explicitly differentiated from efficacy in 15 (79%), incremental analysis was performed in 18 (95%), an adequate period of analysis was chosen in 16 (84%), costs, benefits and health gains were discounted in 13 (68%), sensitivity analysis was included in the investigation in 10 (53%), and estimated cost prices were used in 9 (47%). The methodology in recently published Dutch health-economics research was partly in agreement with the later official guidelines for pharmaco-economic studies in the Netherlands.

Rater Agreement Indexes for Performance Assessment.

ERIC Educational Resources Information Center

Burry-Stock, Judith A.; And Others

1996-01-01

It is argued that interrater agreement is a psychometric property which is theoretically different from classic reliability. Formulas are presented to illustrate a set of algebraically equivalent rater agreement indices that are intended to provide educational and psychological researchers with a practical way to establish a measure of rater…

Observers' Agreement on Measurements in Fiberoptic Endoscopic Evaluation of Swallowing.

PubMed

Pilz, Walmari; Vanbelle, Sophie; Kremer, Bernd; van Hooren, Michel R; van Becelaere, Tine; Roodenburg, Nel; Baijens, Laura W J

2016-04-01

This study analyzed the effect that dysphagia etiology, different observers, and bolus consistency might have on the level of agreement for measurements in FEES images reached by independent versus consensus panel rating. Sixty patients were included and divided into two groups according to dysphagia etiology: neurological or head and neck oncological. All patients underwent standardized FEES examination using thin and thick liquid consistencies. Two observers scored the same exams, first independently and then in a consensus panel. Four ordinal FEES variables were analyzed. Statistical analysis was performed using a linear weighted kappa coefficient and Bayesian multilevel model. Intra- and interobserver agreement on FEES measurements ranged from 0.76 to 0.93 and from 0.61 to 0.88, respectively. Dysphagia etiology did not influence observers' agreement level. However, bolus consistency resulted in decreased interobserver agreement for all measured FEES variables during thin liquid swallows. When rating on the consensus panel, the observers deviated considerably from the scores they had previously given on the independent rating task. Observer agreement on measurements in FEES exams was influenced by bolus consistency, not by dysphagia etiology. Therefore, observer agreement on FEES measurements should be analyzed by taking bolus consistency into account, as it might affect the interpretation of the outcome. Identifying factors that might influence agreement levels could lead to better understanding of the rating process and assist in developing a more precise measurement scale that would ensure higher levels of observer agreement for measurements in FEES exams.

Urban Partnership Agreement and Congestion Reduction Demonstration : National Evaluation Framework

DOT National Transportation Integrated Search

2008-11-21

This report provides an analytical framework for evaluating six deployments under the United States Department of Transportation (U.S. DOT) Urban Partnership Agreement (UPA) and Congestion Reduction Demonstration (CRD) Programs. The six UPA/CRD sites...

Minnesota urban partnership agreement national evaluation : exogenous factors test plan.

DOT National Transportation Integrated Search

2009-11-17

This report presents the exogenous factors test plan for the national evaluation of the Minnesota Urban Partnership Agreement (UPA) under the United States Department of Transportation (U.S. DOT) UPA Program. The Minnesota UPA projects focus on reduc...

Seattle/Lake Washington corridor urban partnership agreement : national evaluation report.

DOT National Transportation Integrated Search

2014-12-01

This document presents the final report on the national evaluation of the Seattle/Lake Washington Corridor (LWC) Urban Partnership Agreement (UPA) under the United States Department of Transportation (U.S. DOT) UPA Program. The Seattle UPA projects f...

New agreement measures based on survival processes

PubMed Central

Guo, Ying; Li, Ruosha; Peng, Limin; Manatunga, Amita K.

2013-01-01

Summary The need to assess agreement arises in many scenarios in biomedical sciences when measurements were taken by different methods on the same subjects. When the endpoints are survival outcomes, the study of agreement becomes more challenging given the special characteristics of time-to-event data. In this paper, we propose a new framework for assessing agreement based on survival processes that can be viewed as a natural representation of time-to-event outcomes. Our new agreement measure is formulated as the chance-corrected concordance between survival processes. It provides a new perspective for studying the relationship between correlated survival outcomes and offers an appealing interpretation as the agreement between survival times on the absolute distance scale. We provide a multivariate extension of the proposed agreement measure for multiple methods. Furthermore, the new framework enables a natural extension to evaluate time-dependent agreement structure. We develop nonparametric estimation of the proposed new agreement measures. Our estimators are shown to be strongly consistent and asymptotically normal. We evaluate the performance of the proposed estimators through simulation studies and then illustrate the methods using a prostate cancer data example. PMID:23844617

Analytical performance, agreement and user-friendliness of six point-of-care testing urine analysers for urinary tract infection in general practice.

PubMed

Schot, Marjolein J C; van Delft, Sanne; Kooijman-Buiting, Antoinette M J; de Wit, Niek J; Hopstaken, Rogier M

2015-05-18

Various point-of-care testing (POCT) urine analysers are commercially available for routine urine analysis in general practice. The present study compares analytical performance, agreement and user-friendliness of six different POCT urine analysers for diagnosing urinary tract infection in general practice. All testing procedures were performed at a diagnostic centre for primary care in the Netherlands. Urine samples were collected at four general practices. Analytical performance and agreement of the POCT analysers regarding nitrite, leucocytes and erythrocytes, with the laboratory reference standard, was the primary outcome measure, and analysed by calculating sensitivity, specificity, positive and negative predictive value, and Cohen's κ coefficient for agreement. Secondary outcome measures were the user-friendliness of the POCT analysers, in addition to other characteristics of the analysers. The following six POCT analysers were evaluated: Uryxxon Relax (Macherey Nagel), Urisys 1100 (Roche), Clinitek Status (Siemens), Aution 11 (Menarini), Aution Micro (Menarini) and Urilyzer (Analyticon). Analytical performance was good for all analysers. Compared with laboratory reference standards, overall agreement was good, but differed per parameter and per analyser. Concerning the nitrite test, the most important test for clinical practice, all but one showed perfect agreement with the laboratory standard. For leucocytes and erythrocytes specificity was high, but sensitivity was considerably lower. Agreement for leucocytes varied between good to very good, and for the erythrocyte test between fair and good. First-time users indicated that the analysers were easy to use. They expected higher productivity and accuracy when using these analysers in daily practice. The overall performance and user-friendliness of all six commercially available POCT urine analysers was sufficient to justify routine use in suspected urinary tract infections in general practice. Published by

Analytical performance, agreement and user-friendliness of six point-of-care testing urine analysers for urinary tract infection in general practice

PubMed Central

Schot, Marjolein J C; van Delft, Sanne; Kooijman-Buiting, Antoinette M J; de Wit, Niek J; Hopstaken, Rogier M

2015-01-01

Objective Various point-of-care testing (POCT) urine analysers are commercially available for routine urine analysis in general practice. The present study compares analytical performance, agreement and user-friendliness of six different POCT urine analysers for diagnosing urinary tract infection in general practice. Setting All testing procedures were performed at a diagnostic centre for primary care in the Netherlands. Urine samples were collected at four general practices. Primary and secondary outcome measures Analytical performance and agreement of the POCT analysers regarding nitrite, leucocytes and erythrocytes, with the laboratory reference standard, was the primary outcome measure, and analysed by calculating sensitivity, specificity, positive and negative predictive value, and Cohen's κ coefficient for agreement. Secondary outcome measures were the user-friendliness of the POCT analysers, in addition to other characteristics of the analysers. Results The following six POCT analysers were evaluated: Uryxxon Relax (Macherey Nagel), Urisys 1100 (Roche), Clinitek Status (Siemens), Aution 11 (Menarini), Aution Micro (Menarini) and Urilyzer (Analyticon). Analytical performance was good for all analysers. Compared with laboratory reference standards, overall agreement was good, but differed per parameter and per analyser. Concerning the nitrite test, the most important test for clinical practice, all but one showed perfect agreement with the laboratory standard. For leucocytes and erythrocytes specificity was high, but sensitivity was considerably lower. Agreement for leucocytes varied between good to very good, and for the erythrocyte test between fair and good. First-time users indicated that the analysers were easy to use. They expected higher productivity and accuracy when using these analysers in daily practice. Conclusions The overall performance and user-friendliness of all six commercially available POCT urine analysers was sufficient to justify routine

Benefits and costs of programmatic agreements

DOT National Transportation Integrated Search

2015-02-01

The performing organization, on behalf of the FHWA Office of Planning, Environment, and Realty, conducted a benefit-cost assessment of programmatic agreements and approaches. The assessment consisted of a case study approach that evaluated three agre...

25 CFR 170.621 - What if a tribe fails to substantially perform work under a contract or agreement?

Code of Federal Regulations, 2014 CFR

2014-04-01

... 25 Indians 1 2014-04-01 2014-04-01 false What if a tribe fails to substantially perform work under... Contracts and Agreements Under Isdeaa § 170.621 What if a tribe fails to substantially perform work under a contract or agreement? If a tribe fails to substantially perform work under a contract or agreement: (a...

25 CFR 170.621 - What if a tribe fails to substantially perform work under a contract or agreement?

Code of Federal Regulations, 2013 CFR

2013-04-01

... 25 Indians 1 2013-04-01 2013-04-01 false What if a tribe fails to substantially perform work under... Contracts and Agreements Under Isdeaa § 170.621 What if a tribe fails to substantially perform work under a contract or agreement? If a tribe fails to substantially perform work under a contract or agreement: (a...

50 CFR 260.72 - Fees for inspection service performed under cooperative agreement.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 50 Wildlife and Fisheries 7 2010-10-01 2010-10-01 false Fees for inspection service performed... CERTIFICATION Inspection and Certification of Establishments and Fishery Products for Human Consumption Fees and Charges § 260.72 Fees for inspection service performed under cooperative agreement. The fees to be charged...

Agreement studies in radiology research.

PubMed

Farzin, B; Gentric, J-C; Pham, M; Tremblay-Paquet, S; Brosseau, L; Roy, C; Jamali, S; Chagnon, M; Darsaut, T E; Guilbert, F; Naggara, O; Raymond, J

2017-03-01

The goal of this study was to estimate the frequency and the quality of agreement studies published in diagnostic imaging journals. All studies published between January 2011 and December 2012 in four radiology journals were reviewed. Four trained readers evaluated agreement studies using a 24-item form that included the 15 items of the Guidelines for Reporting Reliability and Agreement Studies criteria. Of 2229 source titles, 280 studies (13%) reported agreement. The mean number of patients per study was 81±99 (SD) (range, 0-180). Justification for sample size was found in 9 studies (3%). The number of raters was≤2 in 226 studies (81%). No intra-observer study was performed in 212 (76%) articles. Confidence intervals and interpretation of statistical estimates were provided in 98 (35%) and 147 (53%) of the studies, respectively. In 168 studies (60%), the agreement study was not mentioned in the discussion section. In 8 studies (3%), reporting of the agreement study was judged to be adequate. Twenty studies (7%) were dedicated to agreement. Agreement studies are preliminary and not adequately reported. Studies dedicated to agreement are infrequent. They are research opportunities that should be promoted. Copyright © 2016 Éditions françaises de radiologie. Published by Elsevier Masson SAS. All rights reserved.

Minnesota urban partnership agreement national evaluation : transit system data test plan.

DOT National Transportation Integrated Search

2009-11-17

This report presents the test plan for collecting and analyzing transit system data for the Minnesota Urban Partnership Agreement (UPA) National Evaluation under the United States Department of Transportation (U.S. DOT) UPA Program. The Minnesota UPA...

Successful associateship agreements.

PubMed

Crafton, B C

1997-08-01

When evaluating potential associateship agreements, dentists need to recognize and understand how status, noncompete clauses, scheduling and compensation affect the strength of an associateship agreement. Dentists should not enter an associateship agreement without fully understanding the agreement and its obligations or without the help of an accountant and an attorney.

Agreement of glaucoma specialists and experienced optometrists in gonioscopy and optic disc evaluation

PubMed Central

Kumar, Addepalli U.; Jonnadula, Ganesh B.; Garudadri, Chandrasekhar; Rao, Harsha L.; Senthil, Sirisha; Papas, Eric B.; Sankaridurg, Padmaja; Khanna, Rohit C.

2013-01-01

Purpose To compare the diagnostic performance of glaucoma specialists and experienced optometrists in gonioscopy and optic disc assessment. Methods This study was done to validate the diagnostic performance of two experienced optometrists for using their skills of detecting glaucoma using gonioscopy and optic disc assessment in a major epidemiological study, the L V Prasad Eye Institute Glaucoma Epidemiology and Molecular Genetics Study (LVPEI-GLEAMS). Gonioscopic findings for 150 eyes were categorized as 0, 1 and 2 for open angle, primary angle closure suspect (PACS) and primary angle closure (PAC) respectively. Optic disc findings for 200 eyes were categorized as 0, 1 and 2 for normal, suspects and glaucomatous respectively. Weighted kappa (κ) and diagnostic accuracy parameters were calculated. Two optometrists (#1 and #2) participated in the study. Results Agreement between glaucoma specialists and optometrist for interpretation of gonioscopy to discriminate PACS and PAC from open angles and for interpretation of optic disc to discriminate glaucomatous and suspicious discs from normal, the kappa (κ) was 0.92 and 0.84 and 0.90 and 0.89 for optometrists #1 and #2 respectively. Sensitivities and specificities were above 90% for gonioscopy. Optic disc evaluation had specificities greater than 95% to discriminate normal from glaucomatous discs while the sensitivities were 83% and 93% for optometrists #1 and #2 respectively. Conclusion Agreement between optometrists and glaucoma specialists, in diagnostic performance of gonioscopy and optic assessment was excellent with high sensitivity and specificity. Hence, we conclude that the experienced optometrists can detect glaucoma accurately in the LVPEI-GLEAMS.

Self-informant Agreement for Personality and Evaluative Person Descriptors: Comparing Methods for Creating Informant Measures.

PubMed

Simms, Leonard J; Zelazny, Kerry; Yam, Wern How; Gros, Daniel F

2010-05-01

Little attention typically is paid to the way self-report measures are translated for use in self-informant agreement studies. We studied two possible methods for creating informant measures: (a) the traditional method in which self-report items were translated from the first- to the third-person and (b) an alternative meta-perceptual method in which informants were directed to rate their perception of the targets' self-perception. We hypothesized that the latter method would yield stronger self-informant agreement for evaluative personality dimensions measured by indirect item markers. We studied these methods in a sample of 303 undergraduate friendship dyads. Results revealed mean-level differences between methods, similar self-informant agreement across methods, stronger agreement for Big Five dimensions than for evaluative dimensions, and incremental validity for meta-perceptual informant rating methods. Limited power reduced the interpretability of several sparse acquaintanceship effects. We conclude that traditional informant methods are appropriate for most personality traits, but meta-perceptual methods may be more appropriate when personality questionnaire items reflect indirect indicators of the trait being measured, which is particularly likely for evaluative traits.

The Effect of Initial Public Offering (IPO) Firm Legitimacy on Cooperative Agreements and Performance

DTIC Science & Technology

2000-04-04

4(1): 15-27. Keats , B.W. 1990. Diversification and business economic performance revisted: Issues of measurementand causality. Journal of... Economics 50 III. DEVELOPMENT OF- HYPOTHESES 53 Introduction 53 The Link Between Legitimacy and Cooperative Agreements 53 The Link Between...for cooperative agreements is the young , growth oriented firm (Doz and Hamel, 1998). These firms find that a disproportionate number and type of

Glaucoma progression detection: agreement, sensitivity, and specificity of expert visual field evaluation, event analysis, and trend analysis.

PubMed

Antón, Alfonso; Pazos, Marta; Martín, Belén; Navero, José Manuel; Ayala, Miriam Eleonora; Castany, Marta; Martínez, Patricia; Bardavío, Javier

2013-01-01

To assess sensitivity, specificity, and agreement among automated event analysis, automated trend analysis, and expert evaluation to detect glaucoma progression. This was a prospective study that included 37 eyes with a follow-up of 36 months. All had glaucomatous disks and fields and performed reliable visual fields every 6 months. Each series of fields was assessed with 3 different methods: subjective assessment by 2 independent teams of glaucoma experts, glaucoma/guided progression analysis (GPA) event analysis, and GPA (visual field index-based) trend analysis. Kappa agreement coefficient between methods and sensitivity and specificity for each method using expert opinion as gold standard were calculated. The incidence of glaucoma progression was 16% to 18% in 3 years but only 3 cases showed progression with all 3 methods. Kappa agreement coefficient was high (k=0.82) between subjective expert assessment and GPA event analysis, and only moderate between these two and GPA trend analysis (k=0.57). Sensitivity and specificity for GPA event and GPA trend analysis were 71% and 96%, and 57% and 93%, respectively. The 3 methods detected similar numbers of progressing cases. The GPA event analysis and expert subjective assessment showed high agreement between them and moderate agreement with GPA trend analysis. In a period of 3 years, both methods of GPA analysis offered high specificity, event analysis showed 83% sensitivity, and trend analysis had a 66% sensitivity.

SPSS Macros for Assessing the Reliability and Agreement of Student Evaluations of Teaching

ERIC Educational Resources Information Center

Morley, Donald D.

2009-01-01

This article reports and demonstrates two SPSS macros for calculating Krippendorff's alpha and intraclass reliability coefficients in repetitive situations where numerous coefficients are needed. Specifically, the reported SPSS macros were used to evaluate the interrater agreement and reliability of student evaluations of teaching in thousands of…

Verification of a computer-aided replica technique for evaluating prosthesis adaptation using statistical agreement analysis.

PubMed

Mai, Hang-Nga; Lee, Kyeong Eun; Lee, Kyu-Bok; Jeong, Seung-Mi; Lee, Seok-Jae; Lee, Cheong-Hee; An, Seo-Young; Lee, Du-Hyeong

2017-10-01

The purpose of this study was to evaluate the reliability of computer-aided replica technique (CART) by calculating its agreement with the replica technique (RT), using statistical agreement analysis. A prepared metal die and a metal crown were fabricated. The gap between the restoration and abutment was replicated using silicone indicator paste (n = 25). Gap measurements differed in the control (RT) and experimental (CART) groups. In the RT group, the silicone replica was manually sectioned, and the marginal and occlusal gaps were measured using a microscope. In the CART group, the gap was digitized using optical scanning and image superimposition, and the gaps were measured using a software program. The agreement between the measurement techniques was evaluated by using the 95% Bland-Altman limits of agreement and concordance correlation coefficients (CCC). The least acceptable CCC was 0.90. The RT and CART groups showed linear association, with a strong positive correlation in gap measurements, but without significant differences. The 95% limits of agreement between the paired gap measurements were 3.84% and 7.08% of the mean. The lower 95% confidence limits of CCC were 0.9676 and 0.9188 for the marginal and occlusal gap measurements, respectively, and the values were greater than the allowed limit. The CART is a reliable digital approach for evaluating the fit accuracy of fixed dental prostheses.

Self-informant Agreement for Personality and Evaluative Person Descriptors: Comparing Methods for Creating Informant Measures

PubMed Central

Simms, Leonard J.; Zelazny, Kerry; Yam, Wern How; Gros, Daniel F.

2011-01-01

Little attention typically is paid to the way self-report measures are translated for use in self-informant agreement studies. We studied two possible methods for creating informant measures: (a) the traditional method in which self-report items were translated from the first- to the third-person and (b) an alternative meta-perceptual method in which informants were directed to rate their perception of the targets’ self-perception. We hypothesized that the latter method would yield stronger self-informant agreement for evaluative personality dimensions measured by indirect item markers. We studied these methods in a sample of 303 undergraduate friendship dyads. Results revealed mean-level differences between methods, similar self-informant agreement across methods, stronger agreement for Big Five dimensions than for evaluative dimensions, and incremental validity for meta-perceptual informant rating methods. Limited power reduced the interpretability of several sparse acquaintanceship effects. We conclude that traditional informant methods are appropriate for most personality traits, but meta-perceptual methods may be more appropriate when personality questionnaire items reflect indirect indicators of the trait being measured, which is particularly likely for evaluative traits. PMID:21541262

Standardized assessment of tumor-infiltrating lymphocytes in breast cancer: an evaluation of inter-observer agreement between pathologists.

PubMed

Tramm, Trine; Di Caterino, Tina; Jylling, Anne-Marie B; Lelkaitis, Giedrius; Lænkholm, Anne-Vibeke; Ragó, Péter; Tabor, Tomasz P; Talman, Maj-Lis M; Vouza, Emmanouela

2018-01-01

In breast cancer, there is a growing body of evidence that tumor-infiltrating lymphocytes (TILs) may have clinical utility and may be able to direct clinical decisions for subgroups of patients. Clinical utility is, however, not sufficient for warranting the implementation of a new biomarker in the routine practice, and evaluation of the analytical validity is needed, including testing the reproducibility of decentralized assessment of TILs. The aim of this study was to evaluate the inter-observer agreement of TILs assessment using a standardized method, as proposed by the International TILs Working Group 2014, applied to a cohort of breast cancers reflecting an average breast cancer population. Stromal TILs were assessed using full slide sections from 124 breast cancers with varying histology, malignancy grade and ER- and HER2 status. TILs were estimated by nine dedicated breast pathologists using scanned hematoxylin-eosin stainings. TILs results were categorized using various cutoffs, and the inter-observer agreement was evaluated using the intraclass coefficient (ICC), Kappa statistics as well as individual overall agreements with the median value of TILs. Evaluation of TILs led to an ICC of 0.71 (95% CI: 0.65-0.77) corresponding to an acceptable agreement. Kappa values were in the range of 0.38-0.46 corresponding to a fair to moderate agreement. The individual agreements increased, when using only two categories ('high' vs. 'low' TILs) and a cutoff of 50-60%. The results of the present study are in accordance with previous studies, and shows that the proposed methodology for standardized evaluation of TILs renders an acceptable inter-observer agreement. The findings, however, indicate that assessment of TILs needs further refinement, and is in support of the latest St. Gallen Consensus, that routine reporting of TILs for early breast cancer is not ready for implementation in a clinical setting.

Comparative measurement of collagen bundle orientation by Fourier analysis and semiquantitative evaluation: reliability and agreement in Masson's trichrome, Picrosirius red and confocal microscopy techniques.

PubMed

Marcos-Garcés, V; Harvat, M; Molina Aguilar, P; Ferrández Izquierdo, A; Ruiz-Saurí, A

2017-08-01

Measurement of collagen bundle orientation in histopathological samples is a widely used and useful technique in many research and clinical scenarios. Fourier analysis is the preferred method for performing this measurement, but the most appropriate staining and microscopy technique remains unclear. Some authors advocate the use of Haematoxylin-Eosin (H&E) and confocal microscopy, but there are no studies comparing this technique with other classical collagen stainings. In our study, 46 human skin samples were collected, processed for histological analysis and stained with Masson's trichrome, Picrosirius red and H&E. Five microphotographs of the reticular dermis were taken with a 200× magnification with light microscopy, polarized microscopy and confocal microscopy, respectively. Two independent observers measured collagen bundle orientation with semiautomated Fourier analysis with the Image-Pro Plus 7.0 software and three independent observers performed a semiquantitative evaluation of the same parameter. The average orientation for each case was calculated with the values of the five pictures. We analyzed the interrater reliability, the consistency between Fourier analysis and average semiquantitative evaluation and the consistency between measurements in Masson's trichrome, Picrosirius red and H&E-confocal. Statistical analysis for reliability and agreement was performed with the SPSS 22.0 software and consisted of intraclass correlation coefficient (ICC), Bland-Altman plots and limits of agreement and coefficient of variation. Interrater reliability was almost perfect (ICC > 0.8) with all three histological and microscopy techniques and always superior in Fourier analysis than in average semiquantitative evaluation. Measurements were consistent between Fourier analysis by one observer and average semiquantitative evaluation by three observers, with an almost perfect agreement with Masson's trichrome and Picrosirius red techniques (ICC > 0.8) and a strong

Evaluation of high fidelity patient simulator in assessment of performance of anaesthetists.

PubMed

Weller, J M; Bloch, M; Young, S; Maze, M; Oyesola, S; Wyner, J; Dob, D; Haire, K; Durbridge, J; Walker, T; Newble, D

2003-01-01

There is increasing emphasis on performance-based assessment of clinical competence. The High Fidelity Patient Simulator (HPS) may be useful for assessment of clinical practice in anaesthesia, but needs formal evaluation of validity, reliability, feasibility and effect on learning. We set out to assess the reliability of a global rating scale for scoring simulator performance in crisis management. Using a global rating scale, three judges independently rated videotapes of anaesthetists in simulated crises in the operating theatre. Five anaesthetists then independently rated subsets of these videotapes. There was good agreement between raters for medical management, behavioural attributes and overall performance. Agreement was high for both the initial judges and the five additional raters. Using a global scale to assess simulator performance, we found good inter-rater reliability for scoring performance in a crisis. We estimate that two judges should provide a reliable assessment. High fidelity simulation should be studied further for assessing clinical performance.

Minnesota urban partnership agreement national evaluation : surveys, interviews, and focus groups test plan.

DOT National Transportation Integrated Search

2009-11-17

This report presents the test plan for developing, conducting, and analyzing surveys, interviews, and focus groups for evaluating the Minnesota Urban Partnership Agreement (UPA) under the United States Department of Transportation (U.S. DOT) UPA Prog...

Rater agreement of visual lameness assessment in horses during lungeing

PubMed Central

Hammarberg, M.; Egenvall, A.; Pfau, T.

2015-01-01

Summary Reasons for performing study Lungeing is an important part of lameness examinations as the circular path may accentuate low‐grade lameness. Movement asymmetries related to the circular path, to compensatory movements and to pain make the lameness evaluation complex. Scientific studies have shown high inter‐rater variation when assessing lameness during straight line movement. Objectives The aim was to estimate inter‐ and intra‐rater agreement of equine veterinarians evaluating lameness from videos of sound and lame horses during lungeing and to investigate the influence of veterinarians’ experience and the objective degree of movement asymmetry on rater agreement. Study design Cross‐sectional observational study. Methods Video recordings and quantitative gait analysis with inertial sensors were performed in 23 riding horses of various breeds. The horses were examined at trot on a straight line and during lungeing on soft or hard surfaces in both directions. One video sequence was recorded per condition and the horses were classified as forelimb lame, hindlimb lame or sound from objective straight line symmetry measurements. Equine veterinarians (n = 86), including 43 with >5 years of orthopaedic experience, participated in a web‐based survey and were asked to identify the lamest limb on 60 videos, including 10 repeats. The agreements between (inter‐rater) and within (intra‐rater) veterinarians were analysed with κ statistics (Fleiss, Cohen). Results Inter‐rater agreement κ was 0.31 (0.38/0.25 for experienced/less experienced) and higher for forelimb (0.33) than for hindlimb lameness (0.11) or soundness (0.08) evaluation. Median intra‐rater agreement κ was 0.57. Conclusions Inter‐rater agreement was poor for less experienced raters, and for all raters when evaluating hindlimb lameness. Since identification of the lame limb/limbs is a prerequisite for successful diagnosis, treatment and recovery, the high inter‐rater variation

Evaluation of Service Level Agreement Approaches for Portfolio Management in the Financial Industry

NASA Astrophysics Data System (ADS)

Pontz, Tobias; Grauer, Manfred; Kuebert, Roland; Tenschert, Axel; Koller, Bastian

The idea of service-oriented Grid computing seems to have the potential for fundamental paradigm change and a new architectural alignment concerning the design of IT infrastructures. There is a wide range of technical approaches from scientific communities which describe basic infrastructures and middlewares for integrating Grid resources in order that by now Grid applications are technically realizable. Hence, Grid computing needs viable business models and enhanced infrastructures to move from academic application right up to commercial application. For a commercial usage of these evolutions service level agreements are needed. The developed approaches are primary of academic interest and mostly have not been put into practice. Based on a business use case of the financial industry, five service level agreement approaches have been evaluated in this paper. Based on the evaluation, a management architecture has been designed and implemented as a prototype.

Echocardiographic agreement in the diagnostic evaluation for infective endocarditis.

PubMed

Lauridsen, Trine Kiilerich; Selton-Suty, Christine; Tong, Steven; Afonso, Luis; Cecchi, Enrico; Park, Lawrence; Yow, Eric; Barnhart, Huiman X; Paré, Carlos; Samad, Zainab; Levine, Donald; Peterson, Gail; Stancoven, Amy Butler; Johansson, Magnus Carl; Dickerman, Stuart; Tamin, Syahidah; Habib, Gilbert; Douglas, Pamela S; Bruun, Niels Eske; Crowley, Anna Lisa

2016-07-01

Echocardiography is essential for the diagnosis and management of infective endocarditis (IE). However, the reproducibility for the echocardiographic assessment of variables relevant to IE is unknown. Objectives of this study were: (1) To define the reproducibility for IE echocardiographic variables and (2) to describe a methodology for assessing quality in an observational cohort containing site-interpreted data. IE reproducibility was assessed on a subset of echocardiograms from subjects enrolled in the International Collaboration on Endocarditis registry. Specific echocardiographic case report forms were used. Intra-observer agreement was assessed from six site readers on ten randomly selected echocardiograms. Inter-observer agreement between sites and an echocardiography core laboratory was assessed on a separate random sample of 110 echocardiograms. Agreement was determined using intraclass correlation (ICC), coverage probability (CP), and limits of agreement for continuous variables and kappa statistics (κweighted) and CP for categorical variables. Intra-observer agreement for LVEF was excellent [ICC = 0.93 ± 0.1 and all pairwise differences for LVEF (CP) were within 10 %]. For IE categorical echocardiographic variables, intra-observer agreement was best for aortic abscess (κweighted = 1.0, CP = 1.0 for all readers). Highest inter-observer agreement for IE categorical echocardiographic variables was obtained for vegetation location (κweighted = 0.95; 95 % CI 0.92-0.99) and lowest agreement was found for vegetation mobility (κweighted = 0.69; 95 % CI 0.62-0.86). Moderate to excellent intra- and inter-observer agreement is observed for echocardiographic variables in the diagnostic assessment of IE. A pragmatic approach for determining echocardiographic data reproducibility in a large, multicentre, site interpreted observational cohort is feasible.

Mother-Daughter Agreement on Adolescent Adopted Chinese Girls' Academic Performance and Internalizing Symptoms

ERIC Educational Resources Information Center

Tan, Tony Xing; Marn, Travis

2014-01-01

This study reported mother-daughter agreement on adolescent adopted Chinese girls' adjustment. Data on the girls' academic performance and internalizing syndromes were collected from the adopted Chinese girls and their adoptive mothers separately. The adoptive mothers also provided data on mother-daughter relationship quality. There were 219 girls…

Sandia National Laboratories: Agreements

Science.gov Websites

Technology Partnerships Business, Industry, & Non-Profits Government Universities Center for Development Agreement (CRADA) Strategic Partnership Projects, Non-Federal Entity (SPP/NFE) Agreements New Projects, Non-Federal Entity (SPP/NFE) Agreements Sandia performs work on a reimbursable basis for a non

(Dis)agreement on Sight-Singing Assessment of Undergraduate Musicians

PubMed Central

Bortz, Graziela; Germano, Nayana G.; Cogo-Moreira, Hugo

2018-01-01

Assessment criteria for sight-singing abilities are similar to those used to judge music performances across music school programs. However, little evidence of agreement among judges has been provided in the literature. Fifty out of 152 participants were randomly selected and blindly assessed by three judges, who evaluated students based on given criteria. Participants were recorded while sight-singing 19 intervals and 10 tonal melodies. Interjudge agreement on melodic sight-singing was tested considering four items in a five-point Likert scale format as follows: (1) Intonation and pitch accuracy; (2) Tonal sense and memory; (3) Rhythmic precision, regularity of pulse and subdivisions; (4) Fluency and music direction. Intervals were scored considering a 3-point Likert scale. Agreement was conducted using weighted kappa. For melodic sight-singing considering the ten tonal melodies, on average, the weighted kappa (κw) were: κ1w = 0.296, κ2w = 0.487, κ3w = 0.224, and κ4w = 0.244, ranging from fair to moderate.. For intervals, the lowest agreement was kappa = 0.406 and the highest was kappa = 0.792 (on average, kappa = 0.637). These findings light up the discussion on the validity and reliability of models that have been taken for granted in assessing music performance in auditions and contests, and illustrate the need to better discuss evaluation criteria. PMID:29896144

Evaluating Random Error in Clinician-Administered Surveys: Theoretical Considerations and Clinical Applications of Interobserver Reliability and Agreement.

PubMed

Bennett, Rebecca J; Taljaard, Dunay S; Olaithe, Michelle; Brennan-Jones, Chris; Eikelboom, Robert H

2017-09-18

The purpose of this study is to raise awareness of interobserver concordance and the differences between interobserver reliability and agreement when evaluating the responsiveness of a clinician-administered survey and, specifically, to demonstrate the clinical implications of data types (nominal/categorical, ordinal, interval, or ratio) and statistical index selection (for example, Cohen's kappa, Krippendorff's alpha, or interclass correlation). In this prospective cohort study, 3 clinical audiologists, who were masked to each other's scores, administered the Practical Hearing Aid Skills Test-Revised to 18 adult owners of hearing aids. Interobserver concordance was examined using a range of reliability and agreement statistical indices. The importance of selecting statistical measures of concordance was demonstrated with a worked example, wherein the level of interobserver concordance achieved varied from "no agreement" to "almost perfect agreement" depending on data types and statistical index selected. This study demonstrates that the methodology used to evaluate survey score concordance can influence the statistical results obtained and thus affect clinical interpretations.

A Statistical Analysis of Reviewer Agreement and Bias in Evaluating Medical Abstracts 1

PubMed Central

Cicchetti, Domenic V.; Conn, Harold O.

1976-01-01

Observer variability affects virtually all aspects of clinical medicine and investigation. One important aspect, not previously examined, is the selection of abstracts for presentation at national medical meetings. In the present study, 109 abstracts, submitted to the American Association for the Study of Liver Disease, were evaluated by three “blind” reviewers for originality, design-execution, importance, and overall scientific merit. Of the 77 abstracts rated for all parameters by all observers, interobserver agreement ranged between 81 and 88%. However, corresponding intraclass correlations varied between 0.16 (approaching statistical significance) and 0.37 (p < 0.01). Specific tests of systematic differences in scoring revealed statistically significant levels of observer bias on most of the abstract components. Moreover, the mean differences in interobserver ratings were quite small compared to the standard deviations of these differences. These results emphasize the importance of evaluating the simple percentage of rater agreement within the broader context of observer variability and systematic bias. PMID:997596

Cross-Disciplinary and Intermode Agreement on the Description and Evaluation of Landscape Resources

ERIC Educational Resources Information Center

Zube, Ervin H.

1974-01-01

Data obtained from ground reconnaissance and from office studies employing aerial photography show that the extent of agreement between environmental designers and resource managers on the use of descriptive and evaluative landscape terms is generally high. Use of remote data sources for science resource assessments is supported. (DT)

Factors associated with agreement between self-perception and clinical evaluation of dental treatment needs in adults in Brazil and Minas Gerais.

PubMed

Nascimento, Alex Rodrigues do; Andrade, Fabíola Bof de; César, Cibele Comini

2016-11-03

This study sought to describe the agreement between self-perception and clinical evaluation of dental treatment needs in adults and analyze associated factors. The sample comprised adult individuals who took part in SBBrazil 2010 and SBMinas Gerais 2012. The study's outcome was agreement between self-perception and clinical evaluation of dental treatment needs. We used multiple Poisson regression in order to determine the factors associated with the outcome. Agreement between self-perception and clinical evaluation was 78.8% in Brazil and 73.8% in Minas Gerais. Clinical and self-reported oral health conditions that affect function and quality of life were associated with a higher agreement, while a recent visit to the dentist was associated with a lower agreement. Identifying associated factors may enable the development of questionnaires that favor correct self-perception regarding treatment needs.

Therapeutic drug monitoring of infliximab: performance evaluation of three commercial ELISA kits.

PubMed

Schmitz, Ellen M H; van de Kerkhof, Daan; Hamann, Dörte; van Dongen, Joost L J; Kuijper, Philip H M; Brunsveld, Luc; Scharnhorst, Volkher; Broeren, Maarten A C

2016-07-01

Therapeutic drug monitoring (TDM) of infliximab (IFX, Remicade®) can aid to optimize therapy efficacy. Many assays are available for this purpose. However, a reference standard is lacking. Therefore, we evaluated the analytical performance, agreement and clinically relevant differences of three commercially available IFX ELISA kits on an automated processing system. The kits of Theradiag (Lisa Tracker Infliximab), Progenika (Promonitor IFX) and apDia (Infliximab ELISA) were implemented on an automated processing system. Imprecision was determined by triplicate measurements of patient samples on five days. Agreement was evaluated by analysis of 30 patient samples and four spiked samples by the selected ELISA kits and the in-house IFX ELISA of Sanquin Diagnostics (Amsterdam, The Netherlands). Therapeutic consequences were evaluated by dividing patients into four treatment groups using cut-off levels of 1, 3 and 7 μg/mL and determining assay concordance. Within-run and between-run imprecision were acceptable (≤12% and ≤17%, respectively) within the quantification range of the selected ELISA kits. The apDia assay had the best precision and agreement to target values. Statistically significant differences were found between all assays except between Sanquin Diagnostics and the Lisa Tracker assay. The Promonitor assay measured the lowest IFX concentrations, the apDia assay the highest. When patients were classified in four treatment categories, 70% concordance was achieved. Although all assays are suitable for TDM, significant differences were observed in both imprecision and agreement. Therapeutic consequences were acceptable when patients were divided in treatment categories, but this could be improved by assay standardization.

Inter-rater agreement in evaluation of disability: systematic review of reproducibility studies.

PubMed

Barth, Jürgen; de Boer, Wout E L; Busse, Jason W; Hoving, Jan L; Kedzia, Sarah; Couban, Rachel; Fischer, Katrin; von Allmen, David Y; Spanjer, Jerry; Kunz, Regina

2017-01-25

 To explore agreement among healthcare professionals assessing eligibility for work disability benefits.  Systematic review and narrative synthesis of reproducibility studies.  Medline, Embase, and PsycINFO searched up to 16 March 2016, without language restrictions, and review of bibliographies of included studies.  Observational studies investigating reproducibility among healthcare professionals performing disability evaluations using a global rating of working capacity and reporting inter-rater reliability by a statistical measure or descriptively. Studies could be conducted in insurance settings, where decisions on ability to work include normative judgments based on legal considerations, or in research settings, where decisions on ability to work disregard normative considerations. : Teams of paired reviewers identified eligible studies, appraised their methodological quality and generalisability, and abstracted results with pretested forms. As heterogeneity of research designs and findings impeded a quantitative analysis, a descriptive synthesis stratified by setting (insurance or research) was performed.  From 4562 references, 101 full text articles were reviewed. Of these, 16 studies conducted in an insurance setting and seven in a research setting, performed in 12 countries, met the inclusion criteria. Studies in the insurance setting were conducted with medical experts assessing claimants who were actual disability claimants or played by actors, hypothetical cases, or short written scenarios. Conditions were mental (n=6, 38%), musculoskeletal (n=4, 25%), or mixed (n=6, 38%). Applicability of findings from studies conducted in an insurance setting to real life evaluations ranged from generalisable (n=7, 44%) and probably generalisable (n=3, 19%) to probably not generalisable (n=6, 37%). Median inter-rater reliability among experts was 0.45 (range intraclass correlation coefficient 0.86 to κ-0.10). Inter-rater reliability was poor in six studies (37

Inter-rater agreement in evaluation of disability: systematic review of reproducibility studies

PubMed Central

Barth, Jürgen; de Boer, Wout E L; Busse, Jason W; Hoving, Jan L; Kedzia, Sarah; Couban, Rachel; Fischer, Katrin; von Allmen, David Y; Spanjer, Jerry

2017-01-01

Objectives To explore agreement among healthcare professionals assessing eligibility for work disability benefits. Design Systematic review and narrative synthesis of reproducibility studies. Data sources Medline, Embase, and PsycINFO searched up to 16 March 2016, without language restrictions, and review of bibliographies of included studies. Eligibility criteria Observational studies investigating reproducibility among healthcare professionals performing disability evaluations using a global rating of working capacity and reporting inter-rater reliability by a statistical measure or descriptively. Studies could be conducted in insurance settings, where decisions on ability to work include normative judgments based on legal considerations, or in research settings, where decisions on ability to work disregard normative considerations.Teams of paired reviewers identified eligible studies, appraised their methodological quality and generalisability, and abstracted results with pretested forms. As heterogeneity of research designs and findings impeded a quantitative analysis, a descriptive synthesis stratified by setting (insurance or research) was performed. Results From 4562 references, 101 full text articles were reviewed. Of these, 16 studies conducted in an insurance setting and seven in a research setting, performed in 12 countries, met the inclusion criteria. Studies in the insurance setting were conducted with medical experts assessing claimants who were actual disability claimants or played by actors, hypothetical cases, or short written scenarios. Conditions were mental (n=6, 38%), musculoskeletal (n=4, 25%), or mixed (n=6, 38%). Applicability of findings from studies conducted in an insurance setting to real life evaluations ranged from generalisable (n=7, 44%) and probably generalisable (n=3, 19%) to probably not generalisable (n=6, 37%). Median inter-rater reliability among experts was 0.45 (range intraclass correlation coefficient 0.86 to κ−0

78 FR 48726 - Placement of the Georgia Agreement State Program on Probation

Federal Register 2010, 2011, 2012, 2013, 2014

2013-08-09

... the Integrated Materials Performance Evaluation Program (IMPEP). The Management Review Board (MRB), in... CONTACT: Lisa Dimmick, Office of Federal and State Materials and Environmental Management Programs, U.S... Agreement State radiation control programs, using performance indicators, to ensure that public health and...

75 FR 52010 - Land and Water Conservation Fund Description and Notification, Performance Reports, Agreements...

Federal Register 2010, 2011, 2012, 2013, 2014

2010-08-24

... Notification, Performance Reports, Agreements and Amendments, On-site Inspection Reports, and Conversion of Use... to an existing grant, or a Conversion of Use Provisions request. The DNF becomes part of the official... Time per Response: 30 minutes. Estimated Total Annual Burden Hours: 2,996. 5. Conversion of Use...

SFCHECK: a unified set of procedures for evaluating the quality of macromolecular structure-factor data and their agreement with the atomic model.

PubMed

Vaguine, A A; Richelle, J; Wodak, S J

1999-01-01

In this paper we present SFCHECK, a stand-alone software package that features a unified set of procedures for evaluating the structure-factor data obtained from X-ray diffraction experiments and for assessing the agreement of the atomic coordinates with these data. The evaluation is performed completely automatically, and produces a concise PostScript pictorial output similar to that of PROCHECK [Laskowski, MacArthur, Moss & Thornton (1993). J. Appl. Cryst. 26, 283-291], greatly facilitating visual inspection of the results. The required inputs are the structure-factor amplitudes and the atomic coordinates. Having those, the program summarizes relevant information on the deposited structure factors and evaluates their quality using criteria such as data completeness, structure-factor uncertainty and the optical resolution computed from the Patterson origin peak. The dependence of various parameters on the nominal resolution (d spacing) is also given. To evaluate the global agreement of the atomic model with the experimental data, the program recomputes the R factor, the correlation coefficient between observed and calculated structure-factor amplitudes and Rfree (when appropriate). In addition, it gives several estimates of the average error in the atomic coordinates. The local agreement between the model and the electron-density map is evaluated on a per-residue basis, considering separately the macromolecule backbone and side-chain atoms, as well as solvent atoms and heterogroups. Among the criteria are the normalized average atomic displacement, the local density correlation coefficient and the polymer chain connectivity. The possibility of computing these criteria using the omit-map procedure is also provided. The described software should be a valuable tool in monitoring the refinement procedure and in assessing structures deposited in databases.

Inter-observer agreement on a checklist to evaluate scientific publications in the field of animal reproduction.

PubMed

Simoneit, Céline; Heuwieser, Wolfgang; Arlt, Sebastian P

2012-01-01

This study's objective was to determine respondents' inter-observer agreement on a detailed checklist to evaluate three exemplars (one case report, one randomized controlled study without blinding, and one blinded, randomized controlled study) of the scientific literature in the field of bovine reproduction. Fourteen international scientists in the field of animal reproduction were provided with the three articles, three copies of the checklist, and a supplementary explanation. Overall, 13 responded to more than 90% of the items. Overall repeatability between respondents using Fleiss's κ was 0.35 (fair agreement). Combining the "strongly agree" and "agree" responses and the "strongly disagree" and "disagree" responses increased κ to 0.49 (moderate agreement). Evaluation of information given in the three articles on housing of the animals (35% identical answers) and preconditions or pretreatments (42%) varied widely. Even though the overall repeatability was fair, repeatability concerning the important categories was high (e.g., level of agreement=98%). Our data show that the checklist is a reasonable and practical supporting tool to assess the quality of publications. Therefore, it may be used in teaching and practicing evidence-based veterinary medicine. It can support training in systematic and critical appraisal of information and in clinical decision making.

Appropriate Statistics for Determining Chance-Removed Interpractitioner Agreement.

PubMed

Popplewell, Michael; Reizes, John; Zaslawski, Chris

2018-05-31

Fleiss' Kappa (FK) has been commonly, but incorrectly, employed as the "standard" for evaluating chance-removed inter-rater agreement with ordinal data. This practice may lead to misleading conclusions in inter-rater agreement research. An example is presented that demonstrates the conditions where FK produces inappropriate results, compared with Gwet's AC2, which is proposed as a more appropriate statistic. A novel format for recording a Chinese Medical (CM) diagnoses, called the Diagnostic System of Oriental Medicine (DSOM), was used to record and compare patient diagnostic data, which, unlike the contemporary CM diagnostic format, allows agreement by chance to be considered when evaluating patient data obtained with unrestricted diagnostic options available to diagnosticians. Five CM practitioners diagnosed 42 subjects drawn from an open population. Subjects' diagnoses were recorded using the DSOM format. All the available data were initially used to evaluate agreement. Then, the subjects were sorted into three groups to demonstrate the effects of differing data marginality on the calculated chance-removed agreement. Agreement between the practitioners for each subject was evaluated with linearly weighted simple agreement, FK and Gwet's AC2. In all cases, overall agreement was much lower with FK than Gwet's AC2. Larger differences occurred when the data were more free marginal. Inter-rater agreement determined with FK statistics is unlikely to be correct unless it can be shown that the data from which agreement is determined are, in fact, fixed marginal. It follows that results obtained on agreement between practitioners with FK are probably incorrect. It is shown that inter-rater agreement evaluated with AC2 statistic is an appropriate measure when fixed marginal data are neither expected nor guaranteed. The AC2 statistic should be used as the standard statistical approach for determining agreement between practitioners.

Apprentice Performance Evaluation.

ERIC Educational Resources Information Center

Gast, Clyde W.

The Granite City (Illinois) Steel apprentices are under a performance evaluation from entry to graduation. Federally approved, the program is guided by joint apprenticeship committees whose monthly meetings include performance evaluation from three information sources: journeymen, supervisors, and instructors. Journeymen's evaluations are made…

Performance, physiological, and oculometer evaluation of VTOL landing displays

NASA Technical Reports Server (NTRS)

North, R. A.; Stackhouse, S. P.; Graffunder, K.

1979-01-01

A methodological approach to measuring workload was investigated for evaluation of new concepts in VTOL aircraft displays. Physiological, visual response, and conventional flight performance measures were recorded for landing approaches performed in the NASA Visual Motion Simulator (VMS). Three displays (two computer graphic and a conventional flight director), three crosswind amplitudes, and two motion base conditions (fixed vs. moving base) were tested in a factorial design. Multivariate discriminant functions were formed from flight performance and/or visual response variables. The flight performance variable discriminant showed maximum differentation between crosswind conditions. The visual response measure discriminant maximized differences between fixed vs. motion base conditions and experimental displays. Physiological variables were used to attempt to predict the discriminant function values for each subject/condition trial. The weights of the physiological variables in these equations showed agreement with previous studies. High muscle tension, light but irregular breathing patterns, and higher heart rate with low amplitude all produced higher scores on this scale and thus represent higher workload levels.

Solar power plant performance evaluation: simulation and experimental validation

NASA Astrophysics Data System (ADS)

Natsheh, E. M.; Albarbar, A.

2012-05-01

In this work the performance of solar power plant is evaluated based on a developed model comprise photovoltaic array, battery storage, controller and converters. The model is implemented using MATLAB/SIMULINK software package. Perturb and observe (P&O) algorithm is used for maximizing the generated power based on maximum power point tracker (MPPT) implementation. The outcome of the developed model are validated and supported by a case study carried out using operational 28.8kW grid-connected solar power plant located in central Manchester. Measurements were taken over 21 month's period; using hourly average irradiance and cell temperature. It was found that system degradation could be clearly monitored by determining the residual (the difference) between the output power predicted by the model and the actual measured power parameters. It was found that the residual exceeded the healthy threshold, 1.7kW, due to heavy snow in Manchester last winter. More important, the developed performance evaluation technique could be adopted to detect any other reasons that may degrade the performance of the P V panels such as shading and dirt. Repeatability and reliability of the developed system performance were validated during this period. Good agreement was achieved between the theoretical simulation and the real time measurement taken the online grid connected solar power plant.

46 CFR 8.130 - Agreement conditions.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 46 Shipping 1 2010-10-01 2010-10-01 false Agreement conditions. 8.130 Section 8.130 Shipping COAST... ALTERNATIVES General § 8.130 Agreement conditions. (a) Delegated functions performed by, and statutory... provisions of its agreement with the Commandant. Any agreement between the Commandant and a recognized...

46 CFR 8.130 - Agreement conditions.

Code of Federal Regulations, 2012 CFR

2012-10-01

... 46 Shipping 1 2012-10-01 2012-10-01 false Agreement conditions. 8.130 Section 8.130 Shipping COAST... ALTERNATIVES General § 8.130 Agreement conditions. (a) Delegated functions performed by, and statutory... provisions of its agreement with the Commandant. Any agreement between the Commandant and a recognized...

46 CFR 8.130 - Agreement conditions.

Code of Federal Regulations, 2011 CFR

2011-10-01

... 46 Shipping 1 2011-10-01 2011-10-01 false Agreement conditions. 8.130 Section 8.130 Shipping COAST... ALTERNATIVES General § 8.130 Agreement conditions. (a) Delegated functions performed by, and statutory... provisions of its agreement with the Commandant. Any agreement between the Commandant and a recognized...

Prospective Comparison of Live Evaluation and Video Review in the Evaluation of Operator Performance in a Pediatric Emergency Airway Simulation

PubMed Central

House, Joseph B.; Dooley-Hash, Suzanne; Kowalenko, Terry; Sikavitsas, Athina; Seeyave, Desiree M.; Younger, John G.; Hamstra, Stanley J.; Nypaver, Michele M.

2012-01-01

Introduction Real-time assessment of operator performance during procedural simulation is a common practice that requires undivided attention by 1 or more reviewers, potentially over many repetitions of the same case. Objective To determine whether reviewers display better interrater agreement of procedural competency when observing recorded, rather than live, performance; and to develop an assessment tool for pediatric rapid sequence intubation (pRSI). Methods A framework of a previously established Objective Structured Assessment of Technical Skills (OSATS) tool was modified for pRSI. Emergency medicine residents (postgraduate year 1–4) were prospectively enrolled in a pRSI simulation scenario and evaluated by 2 live raters using the modified tool. Sessions were videotaped and reviewed by the same raters at least 4 months later. Raters were blinded to their initial rating. Interrater agreement was determined by using the Krippendorff generalized concordance method. Results Overall interrater agreement for live review was 0.75 (95% confidence interval [CI], 0.72–0.78) and for video was 0.79 (95% CI, 0.73–0.82). Live review was significantly superior to video review in only 1 of the OSATS domains (Preparation) and was equivalent in the other domains. Intrarater agreement between the live and video evaluation was very good, greater than 0.75 for all raters, with a mean of 0.81 (95% CI, 0.76–0.85). Conclusion The modified OSATS assessment tool demonstrated some evidence of validity in discriminating among levels of resident experience and high interreviewer reliability. With this tool, intrareviewer reliability was high between live and 4-months' delayed video review of the simulated procedure, which supports feasibility of delayed video review in resident assessment. PMID:23997874

75 FR 21349 - Solicitation for a Cooperative Agreement-Evaluation of Technical Assistance for Evidence-Based...

Federal Register 2010, 2011, 2012, 2013, 2014

2010-04-23

... DEPARTMENT OF JUSTICE National Institute of Corrections Solicitation for a Cooperative Agreement--Evaluation of Technical Assistance for Evidence-Based Decisionmaking in Local Criminal Justice Systems AGENCY: National Institute of Corrections, U.S. Department of Justice. ACTION: Solicitation for a Cooperative...

A Local Agreement Pattern Measure Based on Hazard Functions for Survival Outcomes

PubMed Central

Dai, Tian; Guo, Ying; Peng, Limin; Manatunga, Amita K.

2017-01-01

Summary Assessing agreement is often of interest in biomedical and clinical research when measurements are obtained on the same subjects by different raters or methods. Most classical agreement methods have been focused on global summary statistics, which cannot be used to describe various local agreement patterns. The objective of this work is to study the local agreement pattern between two continuous measurements subject to censoring. In this paper, we propose a new agreement measure based on bivariate hazard functions to characterize the local agreement pattern between two correlated survival outcomes. The proposed measure naturally accommodates censored observations, fully captures the dependence structure between bivariate survival times and provides detailed information on how the strength of agreement evolves over time. We develop a nonparametric estimation method for the proposed local agreement pattern measure and study theoretical properties including strong consistency and asymptotical normality. We then evaluate the performance of the estimator through simulation studies and illustrate the method using a prostate cancer data example. PMID:28724196

Application Agreement and Integration Services

NASA Technical Reports Server (NTRS)

Driscoll, Kevin R.; Hall, Brendan; Schweiker, Kevin

2013-01-01

Application agreement and integration services are required by distributed, fault-tolerant, safety critical systems to assure required performance. An analysis of distributed and hierarchical agreement strategies are developed against the backdrop of observed agreement failures in fielded systems. The documented work was performed under NASA Task Order NNL10AB32T, Validation And Verification of Safety-Critical Integrated Distributed Systems Area 2. This document is intended to satisfy the requirements for deliverable 5.2.11 under Task 4.2.2.3. This report discusses the challenges of maintaining application agreement and integration services. A literature search is presented that documents previous work in the area of replica determinism. Sources of non-deterministic behavior are identified and examples are presented where system level agreement failed to be achieved. We then explore how TTEthernet services can be extended to supply some interesting application agreement frameworks. This document assumes that the reader is familiar with the TTEthernet protocol. The reader is advised to read the TTEthernet protocol standard [1] before reading this document. This document does not re-iterate the content of the standard.

Rater agreement of visual lameness assessment in horses during lungeing.

PubMed

Hammarberg, M; Egenvall, A; Pfau, T; Rhodin, M

2016-01-01

Lungeing is an important part of lameness examinations as the circular path may accentuate low-grade lameness. Movement asymmetries related to the circular path, to compensatory movements and to pain make the lameness evaluation complex. Scientific studies have shown high inter-rater variation when assessing lameness during straight line movement. The aim was to estimate inter- and intra-rater agreement of equine veterinarians evaluating lameness from videos of sound and lame horses during lungeing and to investigate the influence of veterinarians' experience and the objective degree of movement asymmetry on rater agreement. Cross-sectional observational study. Video recordings and quantitative gait analysis with inertial sensors were performed in 23 riding horses of various breeds. The horses were examined at trot on a straight line and during lungeing on soft or hard surfaces in both directions. One video sequence was recorded per condition and the horses were classified as forelimb lame, hindlimb lame or sound from objective straight line symmetry measurements. Equine veterinarians (n = 86), including 43 with >5 years of orthopaedic experience, participated in a web-based survey and were asked to identify the lamest limb on 60 videos, including 10 repeats. The agreements between (inter-rater) and within (intra-rater) veterinarians were analysed with κ statistics (Fleiss, Cohen). Inter-rater agreement κ was 0.31 (0.38/0.25 for experienced/less experienced) and higher for forelimb (0.33) than for hindlimb lameness (0.11) or soundness (0.08) evaluation. Median intra-rater agreement κ was 0.57. Inter-rater agreement was poor for less experienced raters, and for all raters when evaluating hindlimb lameness. Since identification of the lame limb/limbs is a prerequisite for successful diagnosis, treatment and recovery, the high inter-rater variation when evaluating lameness on the lunge is likely to influence the accuracy and repeatability of lameness examinations

Agreement between T2 and haste sequences in the evaluation of thoracolumbar intervertebral disc disease in dogs.

PubMed

Mankin, Joseph M; Hecht, Silke; Thomas, William B

2012-01-01

The purpose of this study was to compare half-Fourier-acquisition single-shot turbo spin-echo (HASTE) and T2-weighted (T2-W) sequences in dogs with thoracolumbar disc extrusion. MRI studies in 60 dogs (767 individual intervertebral disc spaces) were evaluated. Agreement between T2-W and HASTE sequences was assessed for two criteria: presence of an extradural lesion and treatment recommendation. There was moderate agreement between T2-W and HASTE sequences as to presence of an extradural lesion (kappa = 0.575). HASTE was in agreement in 96.1% of the sites where no extradural lesion was identified on T2-W images, but only in 58.1% of the sites where extradural lesions were identified on T2-W images. There was also moderate agreement between T2-W and HASTE sequences as to treatment recommendations (kappa = 0.476). HASTE was in agreement in 98.4% of the sites where a lesion was considered nonsurgical on T2 but only 82.1% of sites a lesion was considered surgical on T2. In 1.0% of sites considered not surgical and in 9.8% of sites considered equivocal based on T2-W images, a surgical lesion was identified on HASTE. Acquisition of a HASTE sequence in addition to conventional sequences may be beneficial in determining the severity of spinal cord compression in some cases when evaluating the canine spine.

Multicenter accuracy and interobserver agreement of spot sign identification in acute intracerebral hemorrhage.

PubMed

Huynh, Thien J; Flaherty, Matthew L; Gladstone, David J; Broderick, Joseph P; Demchuk, Andrew M; Dowlatshahi, Dar; Meretoja, Atte; Davis, Stephen M; Mitchell, Peter J; Tomlinson, George A; Chenkin, Jordan; Chia, Tze L; Symons, Sean P; Aviv, Richard I

2014-01-01

Rapid, accurate, and reliable identification of the computed tomography angiography spot sign is required to identify patients with intracerebral hemorrhage for trials of acute hemostatic therapy. We sought to assess the accuracy and interobserver agreement for spot sign identification. A total of 131 neurology, emergency medicine, and neuroradiology staff and fellows underwent imaging certification for spot sign identification before enrolling patients in 3 trials targeting spot-positive intracerebral hemorrhage for hemostatic intervention (STOP-IT, SPOTLIGHT, STOP-AUST). Ten intracerebral hemorrhage cases (spot-positive/negative ratio, 1:1) were presented for evaluation of spot sign presence, number, and mimics. True spot positivity was determined by consensus of 2 experienced neuroradiologists. Diagnostic performance, agreement, and differences by training level were analyzed. Mean accuracy, sensitivity, and specificity for spot sign identification were 87%, 78%, and 96%, respectively. Overall sensitivity was lower than specificity (P<0.001) because of true spot signs incorrectly perceived as spot mimics. Interobserver agreement for spot sign presence was moderate (k=0.60). When true spots were correctly identified, 81% correctly identified the presence of single or multiple spots. Median time needed to evaluate the presence of a spot sign was 1.9 minutes (interquartile range, 1.2-3.1 minutes). Diagnostic performance, interobserver agreement, and time needed for spot sign evaluation were similar among staff physicians and fellows. Accuracy for spot identification is high with opportunity for improvement in spot interpretation sensitivity and interobserver agreement particularly through greater reliance on computed tomography angiography source data and awareness of limitations of multiplanar images. Further prospective study is needed.

The META score for differentiating metastatic from osteoporotic vertebral fractures: an independent agreement assessment.

PubMed

Besa, Pablo; Urrutia, Julio; Campos, Mauricio; Mobarec, Sebastián; Cruz, Juan Pablo; Cikutovic, Pablo; Diaz, Gonzalo

2018-04-27

Differentiating osteoporotic vertebral fractures (OVFs) from metastatic vertebral fractures (MVFs) is an important clinical challenge. A novel magnetic resonance imaging (MRI)-based score (the META score) was described, aiming to differentiate OVF from MVF. This score showed an almost perfect agreement by the group developing it, but an independent agreement evaluation is pending. We aimed to perform an independent inter- and intraobserver agreement evaluation of the META score and to test the score's capability of differentiating OVF from MVF. This is an agreement study of the META score. Sixty-four patients with confirmed OVF or MVF were assessed by six independent evaluators (three spine surgeons and three fellowship-trained radiologists) using the META score. We used the intraclass correlation coefficient (ICC) to determine the overall inter-and intraobserver agreement, and the kappa statistic (κ) to express the agreement for each individual score criterion. The score accuracy was determined by calculating the area under the receiver operating characteristic curve. Finally, we used κ to evaluate the agreement among raters to determine whether the fracture was OVF or MVF. The overall interobserver agreement was poor [ICC=0.10 (0.02-0.20)]; spine surgeons [ICC=0.75 (0.66-0.83)] had better agreement than radiologists did [ICC=0.05 (-0.08 to 0.21)]. The intraobserver agreement was poor [ICC=0.17 (0.01-0.32)]; both spine surgeons [ICC=0.21 (0.05-0.41)] and radiologists had a poor agreement [ICC=0.03 (-0.29 to 0.27)]. The agreement for each specific criterion varied from κ=0.24 to κ=0.60. The area under the receiver operating characteristic curve was 0.58 (0.64 for spine surgeons and 0.52 for radiologists, p<.01). The interobserver agreement using the META score was adequate for spine surgeons but not for other potential users (radiologists); the intraobserver agreement was poor. Further studies are thus necessary before the use of this score is recommended

Inter-observer agreement improves with PERCIST 1.0 as opposed to qualitative evaluation in non-small cell lung cancer patients evaluated with F-18-FDG PET/CT early in the course of chemo-radiotherapy.

PubMed

Fledelius, Joan; Khalil, Azza; Hjorthaug, Karin; Frøkiær, Jørgen

2016-12-01

The purpose of this study is to determine whether a qualitative approach or a semi-quantitative approach provides the most robust method for early response evaluation with 2'-deoxy-2'-[(18)F]fluoro-D-glucose (F-18-FDG) positron emission tomography combined with whole body computed tomography (PET/CT) in non-small cell lung cancer (NSCLC). In this study eight Nuclear Medicine consultants analyzed F-18-FDG PET/CT scans from 35 patients with locally advanced NSCLC. Scans were performed at baseline and after 2 cycles of chemotherapy. Each observer used two different methods for evaluation: (1) PET response criteria in solid tumors (PERCIST) 1.0 and (2) a qualitative approach. Both methods allocate patients into one of four response categories (complete and partial metabolic response (CMR and PMR) and stable and progressive metabolic disease (SMD and PMD)). The inter-observer agreement was evaluated using Fleiss' kappa for multiple raters, Cohens kappa for comparison of the two methods, and intraclass correlation coefficients (ICC) for comparison of lean body mass corrected standardized uptake value (SUL) peak measurements. The agreement between observers when determining the percentage change in SULpeak was "almost perfect", with ICC = 0.959. There was a strong agreement among observers allocating patients to the different response categories with a Fleiss kappa of 0.76 (0.71-0.81). In 22 of the 35 patients, complete agreement was observed with PERCIST 1.0. The agreement was lower when using the qualitative method, moderate, having a Fleiss kappa of 0.60 (0.55-0.64). Complete agreement was achieved in only 10 of the 35 patients. The difference between the two methods was statistically significant (p < 0.005) (chi-squared). Comparing the two methods for each individual observer showed Cohen's kappa values ranging from 0.64 to 0.79, translating into a strong agreement between the two methods. PERCIST 1.0 provides a higher overall agreement between observers

The Problem of Limited Inter-rater Agreement in Modelling Music Similarity

PubMed Central

Flexer, Arthur; Grill, Thomas

2016-01-01

One of the central goals of Music Information Retrieval (MIR) is the quantification of similarity between or within pieces of music. These quantitative relations should mirror the human perception of music similarity, which is however highly subjective with low inter-rater agreement. Unfortunately this principal problem has been given little attention in MIR so far. Since it is not meaningful to have computational models that go beyond the level of human agreement, these levels of inter-rater agreement present a natural upper bound for any algorithmic approach. We will illustrate this fundamental problem in the evaluation of MIR systems using results from two typical application scenarios: (i) modelling of music similarity between pieces of music; (ii) music structure analysis within pieces of music. For both applications, we derive upper bounds of performance which are due to the limited inter-rater agreement. We compare these upper bounds to the performance of state-of-the-art MIR systems and show how the upper bounds prevent further progress in developing better MIR systems. PMID:28190932

MRMC analysis of agreement studies

NASA Astrophysics Data System (ADS)

Gallas, Brandon D.; Anam, Amrita; Chen, Weijie; Wunderlich, Adam; Zhang, Zhiwei

2016-03-01

The purpose of this work is to present and evaluate methods based on U-statistics to compare intra- or inter-reader agreement across different imaging modalities. We apply these methods to multi-reader multi-case (MRMC) studies. We measure reader-averaged agreement and estimate its variance accounting for the variability from readers and cases (an MRMC analysis). In our application, pathologists (readers) evaluate patient tissue mounted on glass slides (cases) in two ways. They evaluate the slides on a microscope (reference modality) and they evaluate digital scans of the slides on a computer display (new modality). In the current work, we consider concordance as the agreement measure, but many of the concepts outlined here apply to other agreement measures. Concordance is the probability that two readers rank two cases in the same order. Concordance can be estimated with a U-statistic and thus it has some nice properties: it is unbiased, asymptotically normal, and its variance is given by an explicit formula. Another property of a U-statistic is that it is symmetric in its inputs; it doesn't matter which reader is listed first or which case is listed first, the result is the same. Using this property and a few tricks while building the U-statistic kernel for concordance, we get a mathematically tractable problem and efficient software. Simulations show that our variance and covariance estimates are unbiased.

Breast lesion shape and margin evaluation: BI-RADS based metrics understate radiologists' actual levels of agreement.

PubMed

Rawashdeh, Mohammad; Lewis, Sarah; Zaitoun, Maha; Brennan, Patrick

2018-05-01

While there is much literature describing the radiologic detection of breast cancer, there are limited data available on the agreement between experts when delineating and classifying breast lesions. The aim of this work is to measure the level of agreement between expert radiologists when delineating and classifying breast lesions as demonstrated through Breast Imaging Reporting and Data System (BI-RADS) and quantitative shape metrics. Forty mammographic images, each containing a single lesion, were presented to nine expert breast radiologists using a high specification interactive digital drawing tablet with stylus. Each reader was asked to manually delineate the breast masses using the tablet and stylus and then visually classify the lesion according to the American College of Radiology (ACR) BI-RADS lexicon. The delineated lesion compactness and elongation were computed using Matlab software. Intraclass Correlation Coefficient (ICC) and Cohen's kappa were used to assess inter-observer agreement for delineation and classification outcomes, respectively. Inter-observer agreement was fair for BI-RADS shape (kappa = 0.37) and moderate for margin (kappa = 0.58) assessments. Agreement for quantitative shape metrics was good for lesion elongation (ICC = 0.82) and excellent for compactness (ICC = 0.93). Fair to moderate levels of agreement was shown by radiologists for shape and margin classifications of cancers using the BI-RADS lexicon. When quantitative shape metrics were used to evaluate radiologists' delineation of lesions, good to excellent inter-observer agreement was found. The results suggest that qualitative descriptors such as BI-RADS lesion shape and margin understate the actual level of expert radiologist agreement. Copyright © 2018 Elsevier Ltd. All rights reserved.

A Statistical Evaluation of the Diagnostic Performance of MEDAS-The Medical Emergency Decision Assistance System

PubMed Central

Georgakis, D. Christine; Trace, David A.; Naeymi-Rad, Frank; Evens, Martha

1990-01-01

Medical expert systems require comprehensive evaluation of their diagnostic accuracy. The usefulness of these systems is limited without established evaluation methods. We propose a new methodology for evaluating the diagnostic accuracy and the predictive capacity of a medical expert system. We have adapted to the medical domain measures that have been used in the social sciences to examine the performance of human experts in the decision making process. Thus, in addition to the standard summary measures, we use measures of agreement and disagreement, and Goodman and Kruskal's λ and τ measures of predictive association. This methodology is illustrated by a detailed retrospective evaluation of the diagnostic accuracy of the MEDAS system. In a study using 270 patients admitted to the North Chicago Veterans Administration Hospital, diagnoses produced by MEDAS are compared with the discharge diagnoses of the attending physicians. The results of the analysis confirm the high diagnostic accuracy and predictive capacity of the MEDAS system. Overall, the agreement of the MEDAS system with the “gold standard” diagnosis of the attending physician has reached a 90% level.

Agreement between Computerized and Human Assessment of Performance on the Ruff Figural Fluency Test

PubMed Central

Elderson, Martin F.; Pham, Sander; van Eersel, Marlise E. A.; Wolffenbuttel, Bruce H. R.; Kok, Johan; Gansevoort, Ron T.; Tucha, Oliver; van der Klauw, Melanie M.; Slaets, Joris P. J.

2016-01-01

The Ruff Figural Fluency Test (RFFT) is a sensitive test for nonverbal fluency suitable for all age groups. However, assessment of performance on the RFFT is time-consuming and may be affected by interrater differences. Therefore, we developed computer software specifically designed to analyze performance on the RFFT by automated pattern recognition. The aim of this study was to compare assessment by the new software with conventional assessment by human raters. The software was developed using data from the Lifelines Cohort Study and validated in an independent cohort of the Prevention of Renal and Vascular End Stage Disease (PREVEND) study. The total study population included 1,761 persons: 54% men; mean age (SD), 58 (10) years. All RFFT protocols were assessed by the new software and two independent human raters (criterion standard). The mean number of unique designs (SD) was 81 (29) and the median number of perseverative errors (interquartile range) was 9 (4 to 16). The intraclass correlation coefficient (ICC) between the computerized and human assessment was 0.994 (95%CI, 0.988 to 0.996; p<0.001) and 0.991 (95%CI, 0.990 to 0.991; p<0.001) for the number of unique designs and perseverative errors, respectively. The mean difference (SD) between the computerized and human assessment was -1.42 (2.78) and +0.02 (1.94) points for the number of unique designs and perseverative errors, respectively. This was comparable to the agreement between two independent human assessments: ICC, 0.995 (0.994 to 0.995; p<0.001) and 0.985 (0.982 to 0.988; p<0.001), and mean difference (SD), -0.44 (2.98) and +0.56 (2.36) points for the number of unique designs and perseverative errors, respectively. We conclude that the agreement between the computerized and human assessment was very high and comparable to the agreement between two independent human assessments. Therefore, the software is an accurate tool for the assessment of performance on the RFFT. PMID:27661083

MR imaging of silicone breast implants: evaluation of prospective and retrospective interpretations and interobserver agreement.

PubMed

Quinn, S F; Neubauer, N M; Sheley, R C; Demlow, T A; Szumowski, J

1996-01-01

MR imaging was used to evaluate the integrity of silicone breast implants in 54 women with 108 implants. MR images were interpreted by relatively inexperienced readers who tried to reproduce the experiences reported in the literature. The study examines the interobserver agreement using different diagnostic signs and the influence of experience on interpretation errors. Prospective and retrospective interpretations were compared with surgical findings at the time of explanation. Diagnostic indicators, including the linguine sign, the inverted tear drop sign, the C sign, water droplets mixed with silicone, and extracapsular globules of silicone, were evaluated for diagnostic efficacy and interobserver agreement. The prospective sensitivity and specificity were 87% and 78%, respectively. With the retrospective interpretations, the sensitivity and specificity increased to 93% and 92%, respectively. Most of the prospective false-positive interpretations were due to misinterpreting radial folds as signs of implant rupture. Six implants interpreted retrospectively as false positives had gross amounts of silicone around the implants at surgery but there were no obvious rents in the implant shells. There was fair to excellent interobserver agreement with the individual diagnostic signs except for extracapsular globules of silicone. All of the signs had specificities of greater than 90%. The sensitivities of the individual signs were less than the overall retrospective sensitivity. With experience, the sensitivity improved from 87% to 93% and the specificity improved from 78% to 92%. This study helps substantiate the use of diagnostic signs used by other authors to detect silicone loss from breast implants by MR imaging; however, questions remain as to the clinical role of MR imaging in evaluating implants for silicone loss.

48 CFR 242.002 - Interagency agreements.

Code of Federal Regulations, 2012 CFR

2012-10-01

... 48 Federal Acquisition Regulations System 3 2012-10-01 2012-10-01 false Interagency agreements... agreements. (b)(i) DoD requires reimbursement, at a rate set by the Under Secretary of Defense (Comptroller... administration, and audit services provided under a no-charge reciprocal agreement; (B) Services performed under...

A local agreement pattern measure based on hazard functions for survival outcomes.

PubMed

Dai, Tian; Guo, Ying; Peng, Limin; Manatunga, Amita K

2018-03-01

Assessing agreement is often of interest in biomedical and clinical research when measurements are obtained on the same subjects by different raters or methods. Most classical agreement methods have been focused on global summary statistics, which cannot be used to describe various local agreement patterns. The objective of this work is to study the local agreement pattern between two continuous measurements subject to censoring. In this article, we propose a new agreement measure based on bivariate hazard functions to characterize the local agreement pattern between two correlated survival outcomes. The proposed measure naturally accommodates censored observations, fully captures the dependence structure between bivariate survival times and provides detailed information on how the strength of agreement evolves over time. We develop a nonparametric estimation method for the proposed local agreement pattern measure and study theoretical properties including strong consistency and asymptotical normality. We then evaluate the performance of the estimator through simulation studies and illustrate the method using a prostate cancer data example. © 2017, The International Biometric Society.

41 CFR 60-741.62 - Conciliation agreements.

Code of Federal Regulations, 2010 CFR

2010-07-01

... 41 Public Contracts and Property Management 1 2010-07-01 2010-07-01 true Conciliation agreements... Conciliation agreements. If a compliance evaluation, complaint investigation or other review by OFCCP finds a... consideration of formal enforcement) is appropriate, a written conciliation agreement will be required. The...

41 CFR 60-741.62 - Conciliation agreements.

Code of Federal Regulations, 2011 CFR

2011-07-01

... 41 Public Contracts and Property Management 1 2011-07-01 2009-07-01 true Conciliation agreements... Conciliation agreements. If a compliance evaluation, complaint investigation or other review by OFCCP finds a... consideration of formal enforcement) is appropriate, a written conciliation agreement will be required. The...

The Pfirrmann classification of lumbar intervertebral disc degeneration: an independent inter- and intra-observer agreement assessment.

PubMed

Urrutia, Julio; Besa, Pablo; Campos, Mauricio; Cikutovic, Pablo; Cabezon, Mario; Molina, Marcelo; Cruz, Juan Pablo

2016-09-01

Grading inter-vertebral disc degeneration (IDD) is important in the evaluation of many degenerative conditions, including patients with low back pain. Magnetic resonance imaging (MRI) is considered the best imaging instrument to evaluate IDD. The Pfirrmann classification is commonly used to grade IDD; the authors describing this classification showed an adequate agreement using it; however, there has been a paucity of independent agreement studies using this grading system. The aim of this study was to perform an independent inter- and intra-observer agreement study using the Pfirrmann classification. T2-weighted sagittal images of 79 patients consecutively studied with lumbar spine MRI were classified using the Pfirrmann grading system by six evaluators (three spine surgeons and three radiologists). After a 6-week interval, the 79 cases were presented to the same evaluators in a random sequence for repeat evaluation. The intra-class correlation coefficient (ICC) and the weighted kappa (wκ) were used to determine the inter- and intra-observer agreement. The inter-observer agreement was excellent, with an ICC = 0.94 (0.93-0.95) and wκ = 0.83 (0.74-0.91). There were no differences between spine surgeons and radiologists. Likewise, there were no differences in agreement evaluating the different lumbar discs. Most differences among observers were only of one grade. Intra-observer agreement was also excellent with ICC = 0.86 (0.83-0.89) and wκ = 0.89 (0.85-0.93). In this independent study, the Pfirrmann classification demonstrated an adequate agreement among different observers and by the same observer on separate occasions. Furthermore, it allows communication between radiologists and spine surgeons.

How to determine leg dominance: The agreement between self-reported and observed performance in healthy adults

PubMed Central

Meddeler, Bart M.; Hoogeboom, Thomas J.; Nijhuis-van der Sanden, Maria W. G.; van Cingel, Robert E. H.

2017-01-01

Context Since decades leg dominance is suggested to be important in rehabilitation and return to play in athletes with anterior cruciate ligament injuries. However, an ideal method to determine leg dominance in relation to task performance is still lacking. Objective To test the agreement between self-reported and observed leg dominance in bilateral mobilizing and unilateral stabilizing tasks, and to assess whether the dominant leg switches between bilateral mobilizing tasks and unilateral stabilizing tasks. Design Cross-sectional study. Participants Forty-one healthy adults: 21 men aged 36 ± 17 years old and 20 women aged 36 ±15 years old. Measurement and analysis Participants self-reported leg dominance in the Waterloo Footedness Questionnaire-Revised (WFQ-R), and leg dominance was observed during performance of four bilateral mobilizing tasks and two unilateral stabilizing tasks. Descriptive statistics and crosstabs were used to report the percentages of agreement. Results The leg used to kick a ball had 100% agreement between the self-reported and observed dominant leg for both men and women. The dominant leg in kicking a ball and standing on one leg was the same in 66.7% of the men and 85.0% of the women. The agreement with jumping with one leg was lower: 47.6% for men and 70.0% for women. Conclusions It is appropriate to ask healthy adults: “If you would shoot a ball on a target, which leg would you use to shoot the ball?” to determine leg dominance in bilateral mobilizing tasks. However, a considerable number of the participants switched the dominant leg in a unilateral stabilizing task. PMID:29287067

[Evaluation of agreement and correlation of three occlusal indices in an assessment of orthodontic treatment need].

PubMed

Djordjević, Jelena; Sćepan, Ivana; Glisić, Branislav

2011-02-01

Occlusal indices are quantitative diagnostic indicators of malocclusion severity, orthodontic treatment need, complexity and outcome. The aim of this study was to determine correlations and agreement among three occlusal indices: the Index of Orthodontic Treatment Need (IOTN), the Peer Assessment Rating Index (PAR) and the Index of Complexity, Outcome and Need (ICON) in evaluating orthodontic treatment need. A total eighty study models of patients referred to the Department of Orthodontics, School of Dentistry, Belgrade, were assessed in this retrospective study. Malocclusions of various types and severity in the permanent dentition were included. The Aesthetic and the Dental Health Component of IOTN determined orthodontic treatment need in 25% and 51% of the patients, respectively. PAR determined orthodontic treatment need in 59% and ICON in 53% of patients. The Aesthetic Component of IOTN and ICON had the highest correlation (Spearman's correlation coefficient 0.95, p < 0.01). Correlations between indices were 0.44 to 0.61 with statistical significance (p < 0.01). The agreement between indices, calculated using Kappa statistics, was 0.22 to 0.63. The most critical in malocclusion assessment was PAR. The Aesthetic Component of IOTN and ICON correlated highly (p < 0.01). Correlations between other pairs of indices were moderate (p < 0.01). The Aesthetic Component of IOTN and ICON had substantial agreement, whereas agreement between other indices was fair or moderate. ICON could replace PAR and IOTN. Application of occlusal indices enables objective evaluation of orthodontic treatment need and easier determination of the treatment priorities.

Prospective assessment of interobserver agreement for defecography in fecal incontinence.

PubMed

Dobben, Annette C; Wiersma, Tjeerd G; Janssen, Lucas W M; de Vos, Rien; Terra, Maaike P; Baeten, Cor G; Stoker, Jaap

2005-11-01

The primary aim of our study was to determine the interobserver agreement of defecography in diagnosing enterocele, anterior rectocele, intussusception, and anismus in fecal-incontinent patients. The subsidiary aim was to evaluate the influence of level of experience on interpreting defecography. Defecography was performed in 105 consecutive fecal-incontinent patients. Observers were classified by level of experience and their findings were compared with the findings of an expert radiologist. The quality of the expert radiologist's findings was evaluated by an intraobserver agreement procedure. Intraobserver agreement was good to very good except for anismus: incomplete evacuation after 30 sec (kappa, 0.55) and puborectalis impression (kappa, 0.54). Interobserver agreement for enterocele and rectocele was good (kappa, 0.66 for both) and for intussusception, fair (kappa, 0.29). Interobserver agreement for anismus: incomplete evacuation after 30 sec was moderate (kappa, 0.47), and for anismus: puborectalis impression was fair (kappa, 0.24). Agreement in grading of enterocele and rectocele was good (kappa, 0.64 and 0.72, respectively) and for intussusception, fair (kappa, 0.39). Agreement separated by experience level was very good for rectocele (kappa, 0.83) and grading of rectoceles (kappa, 0.83) and moderate for intussusception (kappa, 0.44) at the most experienced level. For enterocele and grading, experience level did not influence the reproducibility. Reproducibility for enterocele, anterior rectocele, and severity grading is good, but for intussusception is fair to moderate. For anismus, the diagnosis of incomplete evacuation after 30 sec is more reproducible than puborectalis impression. The level of experience seems to play a role in diagnosing anterior rectocele and its grading and in diagnosing intussusception.

Emergency Physicians Are Able to Detect Right Ventricular Dilation With Good Agreement Compared to Cardiology.

PubMed

Rutz, Matt A; Clary, Julie M; Kline, Jeffrey A; Russell, Frances M

2017-07-01

Focused cardiac ultrasound (FOCUS) is a useful tool in evaluating patients presenting to the emergency department (ED) with acute dyspnea. Prior work has shown that right ventricular (RV) dilation is associated with repeat hospitalizations and shorter life expectancy. Traditionally, RV assessment has been evaluated by cardiologist-interpreted comprehensive echocardiography. The primary goal of this study was to determine the inter-rater reliability between emergency physicians (EPs) and a cardiologist for determining RV dilation on FOCUS performed on ED patients with acute dyspnea. This was a prospective, observational study at two urban academic EDs; patients were enrolled if they had acute dyspnea and a computed tomographic pulmonary angiogram without acute disease. All patients had an EP-performed FOCUS to assess for RV dilation. RV dilation was defined as an RV to left ventricular ratio greater than 1. FOCUS interpretations were compared to a blinded cardiologist FOCUS interpretation using agreement and kappa statistics. Of 84 FOCUS examinations performed on 83 patients, 17% had RV dilation. Agreement and kappa, for EP-performed FOCUS for RV dilation were 89% (95% confidence interval [CI] 80-95%) and 0.68 (95% CI 0.48-0.88), respectively. Emergency physician sonographers are able to detect RV dilation with good agreement when compared to cardiology. These results support the wider use of EP-performed FOCUS to evaluate for RV dilation in ED patients with dyspnea. © 2017 by the Society for Academic Emergency Medicine.

Log-Linear Modeling of Agreement among Expert Exposure Assessors

PubMed Central

Hunt, Phillip R.; Friesen, Melissa C.; Sama, Susan; Ryan, Louise; Milton, Donald

2015-01-01

Background: Evaluation of expert assessment of exposure depends, in the absence of a validation measurement, upon measures of agreement among the expert raters. Agreement is typically measured using Cohen’s Kappa statistic, however, there are some well-known limitations to this approach. We demonstrate an alternate method that uses log-linear models designed to model agreement. These models contain parameters that distinguish between exact agreement (diagonals of agreement matrix) and non-exact associations (off-diagonals). In addition, they can incorporate covariates to examine whether agreement differs across strata. Methods: We applied these models to evaluate agreement among expert ratings of exposure to sensitizers (none, likely, high) in a study of occupational asthma. Results: Traditional analyses using weighted kappa suggested potential differences in agreement by blue/white collar jobs and office/non-office jobs, but not case/control status. However, the evaluation of the covariates and their interaction terms in log-linear models found no differences in agreement with these covariates and provided evidence that the differences observed using kappa were the result of marginal differences in the distribution of ratings rather than differences in agreement. Differences in agreement were predicted across the exposure scale, with the likely moderately exposed category more difficult for the experts to differentiate from the highly exposed category than from the unexposed category. Conclusions: The log-linear models provided valuable information about patterns of agreement and the structure of the data that were not revealed in analyses using kappa. The models’ lack of dependence on marginal distributions and the ease of evaluating covariates allow reliable detection of observational bias in exposure data. PMID:25748517

Multiparametric MRI of the prostate: diagnostic performance and interreader agreement of two scoring systems.

PubMed

Lin, Wei-Ching; Muglia, Valdair F; Silva, Gyl E B; Chodraui Filho, Salomão; Reis, Rodolfo B; Westphalen, Antonio C

2016-06-01

To compare the diagnostic accuracies and interreader agreements of the Prostate Imaging Reporting and Data System (PI-RADS) v. 2 and University of California San Francisco (UCSF) multiparametric prostate MRI scale for diagnosing clinically significant prostate cancer. This institutional review board-approved retrospective study included 49 males who had 1.5 T endorectal MRI and prostatectomy. Two radiologists scored suspicious lesions on MRI using PI-RADS v. 2 and the UCSF scale. Percent agreement, 2 × 2 tables and the area under the receiver operating characteristic curves (Az) were used to assess and compare the individual and overall scores of these scales. Interreader agreements were estimated with kappa statistics. Reader 1 (R1) detected 78 lesions, and Reader 2 (R2) detected 80 lesions. Both identified 52 of 65 significant cancers. The Az for PI-RADS v. 2 and UCSF scale for R1 were 0.68 and 0.69 [T2 weighted imaging (T2WI)], 0.75 and 0.68 [diffusion-weighted imaging (DWI)] and 0.64 and 0.72 (overall score), respectively, and were 0.72 and 0.75 (T2WI), 0.73 and 0.67 (DWI) and 0.66 and 0.75 (overall score) for R2. The dynamic contrast-enhanced percent agreements between scales were 100% (R1) and 95% (R2). PI-RADS v. 2 DWI of R1 performed better than UCSF DWI (Az = 0.75 vs Az = 0.68; p = 0.05); no other differences were found. The interreader agreements were higher for PI-RADS v. 2 (T2WI: 0.56 vs 0.42; DWI: 0.60 vs 0.46; overall: 0.61 vs 0.42). The UCSF approach to derive the overall PI-RADS v. 2 scores increased the Az for the identification of significant cancer (R1 to 0.76, p < 0.05; R2 to 0.71, p = 0.35). Although PI-RADS v. 2 DWI score may have a higher discriminatory performance than the UCSF scale counterpart to diagnose clinically significant cancer, the utilization of the UCSF scale weighing system for the integration of PI-RADS v. 2 individual parameter scores improved the accuracy its overall score. PI-RADS v. 2 is

CYBER WARFARE GOVERNANCE: EVALUATION OF CURRENT INTERNATIONAL AGREEMENTS ON THE OFFENSIVE USE OF CYBER

DTIC Science & Technology

2015-10-01

AIR COMMAND AND STAFF COLLEGE DISTANCE LEARNING AIR UNIVERSITY CYBER WARFARE GOVERNANCE: EVALUATION OF CURRENT INTERNATIONAL AGREEMENTS ON THE...order to prevent catastrophic second and third order effects. Rule 43 “prohibits means or methods of cyber warfare that indiscriminate by nature...Means and methods of cyber warfare are indiscriminate by nature if they cannot be: directed at a specific military objective, or limited in their

Flat-Plate Solar-Collector Performance Evaluation with a Solar Simulator as a Basis for Collector Selection and Performance Prediction

NASA Technical Reports Server (NTRS)

Simon, F. F.

1975-01-01

The use of a solar simulator for performance determination permits collector testing under standard conditions of wind, ambient temperature, flow rate and sun. The performance results determined with the simulator have been found to be in good agreement with outdoor performance results. The measured thermal efficiency and evaluation of 23 collectors are reported which differ according to absorber material (copper, aluminum, steel), absorber coating (nonselective black paint, selective copper oxide, selective black nickel, selective black chrome), type of glazing material (glass, Tedlar, Lexan, antireflection glass), the use of honeycomb material and the use of vacuum to prevent thermal convection losses. The collectors were given performance rankings based on noon-hour solar conditions and all-day solar conditions. The determination with the simulator of an all-day collector performance was made possible by tests at different incident angles. The solar performance rankings were made based on whether the collector is to be used for pool heating, hot water, absorption air conditioning, heating, or for a solar Rankine machine.

78 FR 26794 - Prospective Grant of Start-Up Exclusive Evaluation Option License Agreement: Gene Therapy and...

Federal Register 2010, 2011, 2012, 2013, 2014

2013-05-08

...-Up Exclusive Evaluation Option License Agreement: Gene Therapy and Cell-Based Therapy for Cardiac... the field of use may be limited to ``Gene therapy and cell-based therapy for cardiac arrhythmias in...\\2+\\-activated adenylyl cyclase, as well as cardiac cells or cardiac-like cells derived from...

Scales of degree of facial paralysis: analysis of agreement.

PubMed

Fonseca, Kércia Melo de Oliveira; Mourão, Aline Mansueto; Motta, Andréa Rodrigues; Vicente, Laelia Cristina Caseiro

2015-01-01

It has become common to use scales to measure the degree of involvement of facial paralysis in phonoaudiological clinics. To analyze the inter- and intra-rater agreement of the scales of degree of facial paralysis and to elicit point of view of the appraisers regarding their use. Cross-sectional observational clinical study of the Chevalier and House & Brackmann scales performed by five speech therapists with clinical experience, who analyzed the facial expression of 30 adult subjects with impaired facial movements two times, with a one week interval between evaluations. The kappa analysis was employed. There was excellent inter-rater agreement for both scales (kappa>0.80), and on the Chevalier scale a substantial intra-rater agreement in the first assessment (kappa=0.792) and an excellent agreement in the second assessment (kappa=0.928). The House & Brackmann scale showed excellent agreement at both assessments (kappa=0.850 and 0.857). As for the appraisers' point of view, one appraiser thought prior training is necessary for the Chevalier scale and, four appraisers felt that training is important for the House & Brackmann scale. Both scales have good inter- and intra-rater agreement and most of the appraisers agree on the ease and relevance of the application of these scales. Copyright © 2014 Associação Brasileira de Otorrinolaringologia e Cirurgia Cérvico-Facial. Published by Elsevier Editora Ltda. All rights reserved.

22 CFR 120.22 - Technical assistance agreement.

Code of Federal Regulations, 2014 CFR

2014-04-01

... 22 Foreign Relations 1 2014-04-01 2014-04-01 false Technical assistance agreement. 120.22 Section 120.22 Foreign Relations DEPARTMENT OF STATE INTERNATIONAL TRAFFIC IN ARMS REGULATIONS PURPOSE AND DEFINITIONS § 120.22 Technical assistance agreement. An agreement (e.g., contract) for the performance of a...

22 CFR 120.22 - Technical assistance agreement.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 22 Foreign Relations 1 2010-04-01 2010-04-01 false Technical assistance agreement. 120.22 Section 120.22 Foreign Relations DEPARTMENT OF STATE INTERNATIONAL TRAFFIC IN ARMS REGULATIONS PURPOSE AND DEFINITIONS § 120.22 Technical assistance agreement. An agreement (e.g., contract) for the performance of a...

22 CFR 120.22 - Technical assistance agreement.

Code of Federal Regulations, 2012 CFR

2012-04-01

... 22 Foreign Relations 1 2012-04-01 2012-04-01 false Technical assistance agreement. 120.22 Section 120.22 Foreign Relations DEPARTMENT OF STATE INTERNATIONAL TRAFFIC IN ARMS REGULATIONS PURPOSE AND DEFINITIONS § 120.22 Technical assistance agreement. An agreement (e.g., contract) for the performance of a...

22 CFR 120.22 - Technical assistance agreement.

Code of Federal Regulations, 2013 CFR

2013-04-01

... 22 Foreign Relations 1 2013-04-01 2013-04-01 false Technical assistance agreement. 120.22 Section 120.22 Foreign Relations DEPARTMENT OF STATE INTERNATIONAL TRAFFIC IN ARMS REGULATIONS PURPOSE AND DEFINITIONS § 120.22 Technical assistance agreement. An agreement (e.g., contract) for the performance of a...

22 CFR 120.22 - Technical assistance agreement.

Code of Federal Regulations, 2011 CFR

2011-04-01

... 22 Foreign Relations 1 2011-04-01 2011-04-01 false Technical assistance agreement. 120.22 Section 120.22 Foreign Relations DEPARTMENT OF STATE INTERNATIONAL TRAFFIC IN ARMS REGULATIONS PURPOSE AND DEFINITIONS § 120.22 Technical assistance agreement. An agreement (e.g., contract) for the performance of a...

Evaluating agreement, histological features and relevance of separating pleomorphic and florid lobular carcinoma-in-situ subtypes.

PubMed

Singh, Kamaljeet; Paquette, Cherie; Kalife, Elizabeth T; Wang, Yihong; Mangray, Shamlal; MPhil, M Ruhul Quddus Md; Steinhoff, Margaret M

2018-05-09

Morphological variants of lobular carcinoma in situ (LCIS) include classical- (CLCIS), pleomorphic- (PLCIS) and florid-type (FLCIS). Treatment guidelines suggest managing PLCIS and FLCIS like ductal carcinoma in situ (DCIS); therefore accurate identification of LCIS subtypes is critical. However significance of separating PLCIS from FLCIS is not clear. Also inter-observer agreement in identifying LCIS subtypes, using contemporary criteria, is not known. We aimed to evaluate inter-observer agreement amongst breast pathologists in diagnosing LCIS subtypes and use the agreement data to justify LCIS classification for management purposes. Six breast pathologists independently reviewed 50 hematoxylin and eosin stained slides comprised of a mix of LCIS subtypes. After reviewing published criteria participants diagnosed PLCIS, CLCIS and apocrine change in a marked region of interest and FLCIS based on entire section. PLCIS was identified in 8-37 slides with overall moderate agreement (Fleiss' κ =0.565) and pairwise κ (Cohen's) ranging from -.008 to 0.492. FLCIS was diagnosed in 15-26 slides with overall substantial agreement (Fleiss' κ =0.687) and pairwise κ ranging from -.068 to 0.706. Both FLCIS and PLCIS coexisted in 45% of slides with consensus on non-classical LCIS. Comedo-type necrosis (Odds ratio=5.5) and apoptosis (Odds ratio=1.8) predicted FLCIS. We found moderate and substantial agreement in diagnosing PLCIS and FLCIS respectively. Objective histological features linked with aggressive behavior were more frequent with FLCIS. PLCIS and FLCIS patterns frequently coexist, contain similar molecular aberrations, and are managed similarly (like DCIS); therefore combining FLCIS and PLCIS into one category (non-classical LCIS) should be considered. Copyright © 2018. Published by Elsevier Inc.

An agreement coefficient for image comparison

USGS Publications Warehouse

Ji, Lei; Gallo, Kevin

2006-01-01

Combination of datasets acquired from different sensor systems is necessary to construct a long time-series dataset for remotely sensed land-surface variables. Assessment of the agreement of the data derived from various sources is an important issue in understanding the data continuity through the time-series. Some traditional measures, including correlation coefficient, coefficient of determination, mean absolute error, and root mean square error, are not always optimal for evaluating the data agreement. For this reason, we developed a new agreement coefficient for comparing two different images. The agreement coefficient has the following properties: non-dimensional, bounded, symmetric, and distinguishable between systematic and unsystematic differences. The paper provides examples of agreement analyses for hypothetical data and actual remotely sensed data. The results demonstrate that the agreement coefficient does include the above properties, and therefore is a useful tool for image comparison.

Improving agreement in assessment of synovitis in rheumatoid arthritis.

PubMed

Cheung, Peter P; Dougados, Maxime; Andre, Vincent; Balandraud, Nathalie; Chales, Gérard; Chary-Valckenaere, Isabelle; Chatelus, Emmanuel; Dernis, Emmanuelle; Gill, Ghislaine; Gilson, Mélanie; Guis, Sandrine; Mouterde, Gael; Pavy, Stephan; Pouyol, François; Marhadour, Thierry; Richette, Pascal; Ruyssen-Witrand, Adeline; Soubrier, Martin; Gossec, Laure

2013-03-01

Synovitis assessment through evaluation of swollen joints is integral in steering treatment decisions in rheumatoid arthritis (RA). However, there is high inter-observer variation. The objective was to assess if a short collegiate consensus would improve swollen joint agreement between rheumatologists and whether this was affected by experience. Eighteen rheumatologists from French university rheumatology units participated in three 30 minutes rounds over a half day meeting evaluating joint counts of RA patients in small groups, followed by short consensus discussions. Agreement was evaluated at the end of each round as follows: (i) global agreement of swollen joints (ii) swollen joint agreement according to level of experience of the rheumatologist (iii) swollen joint count and (iv) agreement of disease activity state according to the Disease Activity Score (DAS28). Agreement was calculated using percentage agreement and kappa. Global agreement of swollen joints failed to improve (kappa 0.50 to 0.52) at the joint level. Agreement between seniors did not improve but agreement between newly qualified rheumatologists and their senior peer, which was initially poor (kappa 0.28), improved significantly (to 0.54) at the end of the consensus exercises. Concordance of DAS28 activity states improved from 71% to 87%. Consensus exercises for swollen joint assessment is worthwhile and may potentially improve agreement between clinicians in clinical synovitis and disease activity state, benefit was mostly observed in newly qualified rheumatologists. Copyright © 2012 Société française de rhumatologie. Published by Elsevier SAS. All rights reserved.

Good Agreement Between Transabdominal and Endoscopic Ultrasound of the Pancreas in Chronic Pancreatitis.

PubMed

Engjom, Trond; Pham, Khahn Do-Chong; Erchinger, Friedemann; Haldorsen, Ingfrid Salvesen; Gilja, Odd Helge; Dimcevski, Georg; Havre, Roald Flesland

2018-03-26

 We aimed to evaluate the agreement of single criteria and dedicated scores from transabdominal ultrasound of the pancreas (US) compared to standards by endoscopic ultrasound (EUS) and computed tomography (CT).  In this observational cohort study performed in a tertiary care center, US and EUS were performed in 110 patients referred for suspected CP. Based on the Mayo score, 52 patients were diagnosed with CP. The sonographic findings obtained by both methods were registered. The number of criteria was counted and scored according to the Rosemont score.  Agreement between the number of detected US and EUS criteria was substantial (ICC = 0.74 [0.61 - 0.83]. Adding Rosemont weighting improved the agreement (ICC = 0.88 [0.81 - 0.92]). Regarding individual criteria, the agreement was substantial for the detection of calcifications (κ = 0.86) and moderate for cysts and irregular or dilated pancreatic duct (κ = 0.42 - 0.58). Agreement for the other criteria was poorer (κ≤ 0.40). The diagnostic performance indices [95 % CI] of US for diagnosing CP (using Mayo score as reference standard) were for the unweighted score: Sensitivity: 0.65 [0.51 - 0.78], specificity: 0.97 [0.87 - 1.00]; and for Rosemont score: Sensitivity: 0.75 [0.61 - 0.86], specificity: 0.95 [0.83 - 0.99].  The agreement between US and EUS for the unweighted and weighted scores was substantial. For the features calcifications, cysts and main pancreatic duct (MPD) changes, agreement was moderate to substantial. For the other detected US criteria, the agreement with EUS was too poor to be clinically relevant. © Georg Thieme Verlag KG Stuttgart · New York.

48 CFR 436.604 - Performance evaluation.

Code of Federal Regulations, 2012 CFR

2012-10-01

... 48 Federal Acquisition Regulations System 4 2012-10-01 2012-10-01 false Performance evaluation... Performance evaluation. Preparation of performance evaluation reports. (a) In addition to the requirements of FAR 36.604, performance evaluation reports shall be prepared for indefinite-delivery type contracts...

48 CFR 436.604 - Performance evaluation.

Code of Federal Regulations, 2014 CFR

2014-10-01

... 48 Federal Acquisition Regulations System 4 2014-10-01 2014-10-01 false Performance evaluation... Performance evaluation. Preparation of performance evaluation reports. (a) In addition to the requirements of FAR 36.604, performance evaluation reports shall be prepared for indefinite-delivery type contracts...

48 CFR 436.604 - Performance evaluation.

Code of Federal Regulations, 2011 CFR

2011-10-01

... 48 Federal Acquisition Regulations System 4 2011-10-01 2011-10-01 false Performance evaluation... Performance evaluation. Preparation of performance evaluation reports. (a) In addition to the requirements of FAR 36.604, performance evaluation reports shall be prepared for indefinite-delivery type contracts...

48 CFR 436.604 - Performance evaluation.

Code of Federal Regulations, 2013 CFR

2013-10-01

... 48 Federal Acquisition Regulations System 4 2013-10-01 2013-10-01 false Performance evaluation... Performance evaluation. Preparation of performance evaluation reports. (a) In addition to the requirements of FAR 36.604, performance evaluation reports shall be prepared for indefinite-delivery type contracts...

48 CFR 436.604 - Performance evaluation.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 48 Federal Acquisition Regulations System 4 2010-10-01 2010-10-01 false Performance evaluation... Performance evaluation. Preparation of performance evaluation reports. (a) In addition to the requirements of FAR 36.604, performance evaluation reports shall be prepared for indefinite-delivery type contracts...

Interobserver agreement in CTG interpretation using the 2015 FIGO guidelines for intrapartum fetal monitoring.

PubMed

Rei, Mariana; Tavares, Sara; Pinto, Pedro; Machado, Ana P; Monteiro, Sofia; Costa, Antónia; Costa-Santos, Cristina; Bernardes, João; Ayres-De-Campos, Diogo

2016-10-01

Visual analysis of cardiotocographic (CTG) tracings has been shown to be prone to poor intra- and interobserver agreement when several interpretation guidelines are used, and this may have an important impact on the technology's performance. The aim of this study was to evaluate agreement in CTG interpretation using the new 2015 FIGO guidelines on intrapartum fetal monitoring. A pre-existing database of intrapartum CTG tracings was used to sequentially select 151 cases acquired with a fetal electrode, with duration exceeding 60minutes, and signal loss less than 15%. These tracings were presented to six clinicians, three with more than 5 years' experience in the labor ward, and three with 5 or less years' experience. Observers were asked to evaluate tracings independently, to assess basic CTG features: baseline, variability, accelerations, decelerations, sinusoidal pattern, tachysystole, and to classify each tracing as normal, suspicious or pathologic, according to the 2015 FIGO guidelines on intrapartum fetal monitoring. Agreement between observers was evaluated using the proportions of agreement (Pa), with 95% confidence intervals (95%CI). A good interobserver agreement was found in the evaluation of most CTG features, but not bradycardia, reduced variability, saltatory pattern, absence of accelerations and absence of decelerations. For baseline classification Pa was 0.85 [0.82-0.90], for variability 0.82 [0.78-0.85], for accelerations 0.72 [0.68-0.75], for tachysystole 0.77 [0.74-0.81], for decelerations 0.92 [0.90-0.95], for variable decelerations 0.62 [0.58-0.65], for late decelerations 0.63 [0.59-0.66], for repetitive decelerations 0.73 [0.69-0.78], and for prolonged decelerations 0.81 [0.77-0.85]. For overall CTG classification, Pa were 0.60 [0.56-0.64], for classification as normal 0.67 [0.61-0.72], for suspicious 0.54 [0.48-0.60] and for pathologic 0.59 [0.51-0.66]. No differences in agreement according to the level of expertise were observed, except in the

[Innovative medicines and market access agreements].

PubMed

Toumi, M; Zard, J; Duvillard, R; Jommi, C

2013-09-01

Market Access Agreements (MAA) for drugs have emerged in response to the need to control health expenditures, as well as to the uncertainty about the true benefit of a drug. It is possible to group MAA in two types of agreements: financial agreements and outcome-based agreements. MAA is a growing trend and is shifting towards conditional access. However, the willingness to use these contracts and their implementation differ across countries, and some are still resistant to put them in place. The MAA challenges to overcome encompass the complexity of the schemes, the administrative burden and the difficulty of evaluating MAA. It is likely that these agreements might experience further evolution in the future to become a faster pathway for therapeutic innovations, at a fair price. Copyright © 2013 Elsevier Masson SAS. All rights reserved.

AVIRIS performance during the 1987 flight season: An AVIRIS project assessment and summary of the NASA-sponsored performance evaluation

NASA Technical Reports Server (NTRS)

Vane, Gregg; Porter, Wallace M.; Reimer, John H.; Chrien, Thomas G.; Green, Robert O.

1988-01-01

Results are presented of the assessment of AVIRIS performance during the 1987 flight season by the AVIRIS project and the earth scientists who were chartered by NASA to conduct an independent data quality and sensor performance evaluation. The AVIRIS evaluation program began in late June 1987 with the sensor meeting most of its design requirements except for signal-to-noise ratio in the fourth spectrometer, which was about half of the required level. Several events related to parts failures and design flaws further reduced sensor performance over the flight season. Substantial agreement was found between the assessments by the project and the independent investigators of the effects of these various factors. A summary of the engineering work that is being done to raise AVIRIS performance to its required level is given. In spite of degrading data quality over the flight season, several exciting scientific results were obtained from the data. These include the mapping of the spatial variation of atmospheric precipitable water, detection of environmentally-induced shifts in the spectral red edge of stressed vegetation, detection of spectral features related to pigment, leaf water and ligno-cellulose absorptions in plants, and the identification of many diagnostic mineral absorption features in a variety of geological settings.

Influence of socioeconomic status on trauma center performance evaluations in a Canadian trauma system.

PubMed

Moore, Lynne; Turgeon, Alexis F; Sirois, Marie-Josée; Murat, Valérie; Lavoie, André

2011-09-01

Trauma center performance evaluations generally include adjustment for injury severity, age, and comorbidity. However, disparities across trauma centers may be due to other differences in source populations that are not accounted for, such as socioeconomic status (SES). We aimed to evaluate whether SES influences trauma center performance evaluations in an inclusive trauma system with universal access to health care. The study was based on data collected between 1999 and 2006 in a Canadian trauma system. Patient SES was quantified using an ecologic index of social and material deprivation. Performance evaluations were based on mortality adjusted using the Trauma Risk Adjustment Model. Agreement between performance results with and without additional adjustment for SES was evaluated with correlation coefficients. The study sample comprised a total of 71,784 patients from 48 trauma centers, including 3,828 deaths within 30 days (4.5%) and 5,549 deaths within 6 months (7.7%). The proportion of patients in the highest quintile of social and material deprivation varied from 3% to 43% and from 11% to 90% across hospitals, respectively. The correlation between performance results with or without adjustment for SES was almost perfect (r = 0.997; 95% CI 0.995-0.998) and the same hospital outliers were identified. We observed an important variation in SES across trauma centers but no change in risk-adjusted mortality estimates when SES was added to adjustment models. Results suggest that after adjustment for injury severity, age, comorbidity, and transfer status, disparities in SES across trauma center source populations do not influence trauma center performance evaluations in a system offering universal health coverage. Copyright © 2011 American College of Surgeons. Published by Elsevier Inc. All rights reserved.

Evaluation of compliance with the self-regulation agreement of the food and drink vending machine sector in primary schools in Madrid, Spain, in 2008.

PubMed

Royo-Bordonada, Miguel A; Martínez-Huedo, María A

2014-01-01

To evaluate compliance with the self-regulation agreement of the food and drink vending machine sector in primary schools in Madrid, Spain. Cross-sectional study of the prevalence of vending machines in 558 primary schools in 2008. Using the directory of all registered primary schools in Madrid, we identified the presence of machines by telephone interviews and evaluated compliance with the agreement by visiting the schools and assessing accessibility, type of publicity, the products offered and knowledge of the agreement. The prevalence of schools with vending machines was 5.8%. None of the schools reported knowledge of the agreement or of its nutritional guidelines, and most machines were accessible to primary school pupils (79.3%) and packed with high-calorie, low-nutrient-dense foods (58.6%). Compliance with the self-regulation agreement of the vending machines sector was low. Stricter regulation should receive priority in the battle against the obesity epidemic. Copyright © 2013 SESPAS. Published by Elsevier Espana. All rights reserved.

48 CFR 236.604 - Performance evaluation.

Code of Federal Regulations, 2014 CFR

2014-10-01

... 48 Federal Acquisition Regulations System 3 2014-10-01 2014-10-01 false Performance evaluation... Architect-Engineer Services 236.604 Performance evaluation. Prepare a separate performance evaluation after... familiar with the architect-engineer contractor's performance. [76 FR 58155, Sept. 20, 2011] ...

48 CFR 236.604 - Performance evaluation.

Code of Federal Regulations, 2013 CFR

2013-10-01

... 48 Federal Acquisition Regulations System 3 2013-10-01 2013-10-01 false Performance evaluation... Architect-Engineer Services 236.604 Performance evaluation. Prepare a separate performance evaluation after... familiar with the architect-engineer contractor's performance. [76 FR 58155, Sept. 20, 2011] ...

48 CFR 236.604 - Performance evaluation.

Code of Federal Regulations, 2012 CFR

2012-10-01

... 48 Federal Acquisition Regulations System 3 2012-10-01 2012-10-01 false Performance evaluation... Architect-Engineer Services 236.604 Performance evaluation. Prepare a separate performance evaluation after... familiar with the architect-engineer contractor's performance. [76 FR 58155, Sept. 20, 2011] ...

48 CFR 236.604 - Performance evaluation.

Code of Federal Regulations, 2011 CFR

2011-10-01

... 48 Federal Acquisition Regulations System 3 2011-10-01 2011-10-01 false Performance evaluation... Architect-Engineer Services 236.604 Performance evaluation. Prepare a separate performance evaluation after... familiar with the architect-engineer contractor's performance. [76 FR 58155, Sept. 20, 2011] ...

48 CFR 236.604 - Performance evaluation.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 48 Federal Acquisition Regulations System 3 2010-10-01 2010-10-01 false Performance evaluation... Architect-Engineer Services 236.604 Performance evaluation. (a) Preparation of performance reports. Use DD Form 2631, Performance Evaluation (Architect-Engineer), instead of SF 1421. (2) Prepare a separate...

Gas exchange and lactate anaerobic thresholds: inter- and intraevaluator agreement.

PubMed

Gladden, L B; Yates, J W; Stremel, R W; Stamford, B A

1985-06-01

Twenty-four coded graph sets of gas exchange variables and blood lactate concentration (LA) plotted against time at 15-s intervals were analyzed by nine evaluators who determined the gas exchange (ATGE) and LA (ATLA) anaerobic thresholds. In addition, ATGE and ATLA were determined by a linear regression computer program. Agreement between ATGE and ATLA was poor; the median intraclass correlation coefficient (ri) was 0.53. Among evaluators, ATLA agreement (median ri = 0.81) was better than ATGE agreement (median ri = 0.70). In general, the ability of any evaluator to choose similar values from duplicate plots for either ATGE (median ri = 0.97) or ATLA (median ri = 0.995) was good. There was better agreement between the mean ATLA of the evaluators and the computer ATLA (ComLA) (ri = 0.88) than between the mean ATGE of the evaluators and the computer ATGE (ComVE), (ri = 0.58). Agreement between ComVE and ComLA was poor (ri = 0.29). These results suggest that ATGE does not accurately predict ATLA and that different evaluators choose different thresholds from the same data. Further assessment of the validity and precision of ATGE based on breath-by-breath and minute-by-minute data is needed.

48 CFR 2936.604 - Performance evaluation.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 48 Federal Acquisition Regulations System 7 2010-10-01 2010-10-01 false Performance evaluation... Performance evaluation. (a) The HCA must establish procedures to evaluate architect-engineer contractor performance as required in FAR 36.604. Normally, the performance report must be prepared by the contracting...

48 CFR 2936.604 - Performance evaluation.

Code of Federal Regulations, 2011 CFR

2011-10-01

... 48 Federal Acquisition Regulations System 7 2011-10-01 2011-10-01 false Performance evaluation... Performance evaluation. (a) The HCA must establish procedures to evaluate architect-engineer contractor performance as required in FAR 36.604. Normally, the performance report must be prepared by the contracting...

48 CFR 2936.604 - Performance evaluation.

Code of Federal Regulations, 2014 CFR

2014-10-01

... 48 Federal Acquisition Regulations System 7 2014-10-01 2014-10-01 false Performance evaluation... Performance evaluation. (a) The HCA must establish procedures to evaluate architect-engineer contractor performance as required in FAR 36.604. Normally, the performance report must be prepared by the contracting...

48 CFR 2936.604 - Performance evaluation.

Code of Federal Regulations, 2012 CFR

2012-10-01

... 48 Federal Acquisition Regulations System 7 2012-10-01 2012-10-01 false Performance evaluation... Performance evaluation. (a) The HCA must establish procedures to evaluate architect-engineer contractor performance as required in FAR 36.604. Normally, the performance report must be prepared by the contracting...

48 CFR 2936.604 - Performance evaluation.

Code of Federal Regulations, 2013 CFR

2013-10-01

... 48 Federal Acquisition Regulations System 7 2013-10-01 2012-10-01 true Performance evaluation... Performance evaluation. (a) The HCA must establish procedures to evaluate architect-engineer contractor performance as required in FAR 36.604. Normally, the performance report must be prepared by the contracting...

Beta-Testing Agreement | FNLCR Staging

Cancer.gov

Beta-Testing Agreements are appropriate forlimited term evaluation and applications development of new software, technology, or equipment platforms by the Frederick National Labin collaboration with an external commercial partner. It may

Traveling-wave piezoelectric linear motor part II: experiment and performance evaluation.

PubMed

Ting, Yung; Li, Chun-Chung; Chen, Liang-Chiang; Yang, Chieh-Min

2007-04-01

This article continues the discussion of a traveling-wave piezoelectric linear motor. Part I of this article dealt with the design and analysis of the stator of a traveling-wave piezoelectric linear motor. In this part, the discussion focuses on the structure and modeling of the contact layer and the carriage. In addition, the performance analysis and evaluation of the linear motor also are dealt with in this study. The traveling wave is created by stator, which is constructed by a series of bimorph actuators arranged in a line and connected to form a meander-line structure. Analytical and experimental results of the performance are presented and shown to be almost in agreement. Power losses due to friction and transmission are studied and found to be significant. Compared with other types of linear motors, the motor in this study is capable of supporting heavier loads and provides a larger thrust force.

Minnesota urban partnership agreement national evaluation : telecommuting test plan.

DOT National Transportation Integrated Search

2009-11-17

This report presents the telecommuting test plan for the Minnesota Urban Partnership Agreement (UPA) under the United States Department of Transportation (U.S. DOT) UPA Program. The Minnesota UPA projects focus on reducing congestion by employing str...

Evaluation of interobserver agreement for postoperative pain and sedation assessment in cats.

PubMed

Benito, Javier; Monteiro, Beatriz P; Beauchamp, Guy; Lascelles, B Duncan X; Steagall, Paulo V

2017-09-01

OBJECTIVE To evaluate agreement between observers with different training and experience for assessment of postoperative pain and sedation in cats by use of a dynamic and interactive visual analog scale (DIVAS) and for assessment of postoperative pain in the same cats with a multidimensional composite pain scale (MCPS). DESIGN Randomized, controlled, blinded study. ANIMALS 45 adult cats undergoing ovariohysterectomy. PROCEDURES Cats received 1 of 3 preoperative treatments: bupivacaine, IP; meloxicam, SC with saline (0.9% NaCl) solution, IP, (positive control); or saline solution only, IP (negative control). All cats received premedication with buprenorphine prior to general anesthesia. An experienced observer (observer 1; male; native language, Spanish) used scales in English, and an inexperienced observer (observer 2; female; native language, French) used scales in French to assess signs of sedation and pain. Rescue analgesia was administered according to MCPS scoring by observer 1. Mean pain and sedation scores per treatment and time point, proportions of cats in each group with MCPS scores necessitating rescue analgesia, and mean MCPS scores assigned at the time of rescue analgesia were compared between observers. Agreement was assessed by intraclass correlation coefficient determination. Percentage disagreement between observers on the need for rescue analgesia was calculated. RESULTS Interobserver agreements for pain scores were good, and that for sedation scores was fair. On the basis of observer 1's MCPS scores, a greater proportion of cats in the negative control group received rescue analgesia than in the bupivacaine or positive control groups. Scores from observer 2 indicated a greater proportion of cats in the negative control group than in the positive control group required rescue analgesia but identified no significant difference between the negative control and bupivacaine groups for this variable. Overall, disagreement regarding need for rescue

48 CFR 36.604 - Performance evaluation.

Code of Federal Regulations, 2011 CFR

2011-10-01

... 48 Federal Acquisition Regulations System 1 2011-10-01 2011-10-01 false Performance evaluation. 36.604 Section 36.604 Federal Acquisition Regulations System FEDERAL ACQUISITION REGULATION SPECIAL... Performance evaluation. See 42.1502(f) for the requirements for preparing past performance evaluations for...

48 CFR 36.604 - Performance evaluation.

Code of Federal Regulations, 2012 CFR

2012-10-01

... 48 Federal Acquisition Regulations System 1 2012-10-01 2012-10-01 false Performance evaluation. 36.604 Section 36.604 Federal Acquisition Regulations System FEDERAL ACQUISITION REGULATION SPECIAL... Performance evaluation. See 42.1502(f) for the requirements for preparing past performance evaluations for...

48 CFR 36.604 - Performance evaluation.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 48 Federal Acquisition Regulations System 1 2010-10-01 2010-10-01 false Performance evaluation. 36.604 Section 36.604 Federal Acquisition Regulations System FEDERAL ACQUISITION REGULATION SPECIAL... Performance evaluation. See 42.1502(f) for the requirements for preparing past performance evaluations for...

48 CFR 36.604 - Performance evaluation.

Code of Federal Regulations, 2014 CFR

2014-10-01

... 48 Federal Acquisition Regulations System 1 2014-10-01 2014-10-01 false Performance evaluation. 36.604 Section 36.604 Federal Acquisition Regulations System FEDERAL ACQUISITION REGULATION SPECIAL... Performance evaluation. See 42.1502(f) for the requirements for preparing past performance evaluations for...

Agreement in the assessment of metastatic spine disease using scoring systems.

PubMed

Arana, Estanislao; Kovacs, Francisco M; Royuela, Ana; Asenjo, Beatriz; Pérez-Ramírez, Ursula; Zamora, Javier

2015-04-01

To assess variability in the use of Tomita and modified Bauer scores in spine metastases. Clinical data and imaging from 90 patients with biopsy-proven spinal metastases, were provided to 83 specialists from 44 hospitals. Spinal levels involved and the Tomita and modified Bauer scores for each case were determined twice by each clinician, with a minimum of 6-week interval. Clinicians were blinded to every evaluation. Kappa statistic was used to assess intra and inter-observer agreement. Subgroup analyses were performed according to clinicians' specialty (medical oncology, neurosurgery, radiology, orthopedic surgery and radiation oncology), years of experience (⩽7, 8-13, ⩾14), and type of hospital (four levels). For metastases identification, intra-observer agreement was "substantial" (0.600.80) at the other levels. Inter-observer agreement was "almost perfect" at lumbar spine, and "substantial" at the other levels. Intra-observer agreement for the Tomita and Bauer scores was almost perfect. Inter-observer agreement was almost perfect for the Tomita score and substantial for the Bauer one. Results were similar across specialties, years of experience and type of hospital. Agreement in the assessment of metastatic spine disease is high. These scoring systems can improve communication among clinicians involved in oncology care. Copyright © 2015 Elsevier Ireland Ltd. All rights reserved.

The decision to extract: Part 1--Interclinician agreement.

PubMed

Baumrind, S; Korn, E L; Boyd, R L; Maxwell, R

1996-03-01

As part of an ongoing prospective clinical trial of conventional orthodontic treatment, the decision making patterns of a representative group of orthodontic clinicians were examined. Data were available for 148 subjects (100 adolescents and 48 adults) who had presented at the University of California San Francisco Graduate Orthodontic Clinic requesting treatment for correction of a Class I or Class II malocclusion. The records for each subject were evaluated independently by each of five members of the clinical faculty, making available a total of 740 independent patient evaluations. With regard to the primary decision as to whether extraction or nonextraction treatment was to be preferred, agreement among clinicians was higher than had been anticipated. In almost two thirds of the cases, the decisions of all five clinicians were in agreement as to whether extraction or nonextraction was the preferred treatment modality. (This figure included 59 cases of complete agreement for extraction therapy (40%) and 38 cases of complete agreement for nonextraction therapy (26%)). In only 51 cases (34%), did the reviewing clinicians disagree as to whether extraction or nonextraction was the preferred modality of treatment. The clinicians were also asked to indicate their opinions as to whether orthognathic surgery was likely to be a part of the ultimate treatment course for each individual subject. Nine percent of the 740 patient evaluations contained a clinician judgement that surgery would be a probable or definite component of the orthodontic treatment plan. For 29% of the adult subjects (14 cases) and 23% of the adolescent subjects (23 cases), one or more of the five examining clinicians believed that adjunctive surgical intervention would probably or definitely be appropriate. These high values were unexpected, particularly because the sample had been prescreened by a single clinician to exclude subjects who might require orthognathic surgery. Clinician agreement of

Performance evaluation of PCA-based spike sorting algorithms.

PubMed

Adamos, Dimitrios A; Kosmidis, Efstratios K; Theophilidis, George

2008-09-01

Deciphering the electrical activity of individual neurons from multi-unit noisy recordings is critical for understanding complex neural systems. A widely used spike sorting algorithm is being evaluated for single-electrode nerve trunk recordings. The algorithm is based on principal component analysis (PCA) for spike feature extraction. In the neuroscience literature it is generally assumed that the use of the first two or most commonly three principal components is sufficient. We estimate the optimum PCA-based feature space by evaluating the algorithm's performance on simulated series of action potentials. A number of modifications are made to the open source nev2lkit software to enable systematic investigation of the parameter space. We introduce a new metric to define clustering error considering over-clustering more favorable than under-clustering as proposed by experimentalists for our data. Both the program patch and the metric are available online. Correlated and white Gaussian noise processes are superimposed to account for biological and artificial jitter in the recordings. We report that the employment of more than three principal components is in general beneficial for all noise cases considered. Finally, we apply our results to experimental data and verify that the sorting process with four principal components is in agreement with a panel of electrophysiology experts.

Minnesota urban partnership agreement national evaluation : tolling test plan.

DOT National Transportation Integrated Search

2009-11-17

This report presents the test plan for collecting and analyzing toll data for the Minnesota Urban Partnership Agreement (UPA) under the United States Department of Transportation (U.S. DOT) UPA Program. The Minnesota UPA projects focus on reducing co...

13 CFR 304.4 - Performance evaluations.

Code of Federal Regulations, 2011 CFR

2011-01-01

... 13 Business Credit and Assistance 1 2011-01-01 2011-01-01 false Performance evaluations. 304.4... ECONOMIC DEVELOPMENT DISTRICTS § 304.4 Performance evaluations. (a) EDA shall evaluate the management standards, financial accountability and program performance of each District Organization within three (3...

13 CFR 304.4 - Performance evaluations.

Code of Federal Regulations, 2014 CFR

2014-01-01

... 13 Business Credit and Assistance 1 2014-01-01 2014-01-01 false Performance evaluations. 304.4... ECONOMIC DEVELOPMENT DISTRICTS § 304.4 Performance evaluations. (a) EDA shall evaluate the management standards, financial accountability and program performance of each District Organization within three (3...

13 CFR 304.4 - Performance evaluations.

Code of Federal Regulations, 2013 CFR

2013-01-01

... 13 Business Credit and Assistance 1 2013-01-01 2013-01-01 false Performance evaluations. 304.4... ECONOMIC DEVELOPMENT DISTRICTS § 304.4 Performance evaluations. (a) EDA shall evaluate the management standards, financial accountability and program performance of each District Organization within three (3...

Multicentre evaluation of multidisciplinary team meeting agreement on diagnosis in diffuse parenchymal lung disease: a case-cohort study.

PubMed

Walsh, Simon L F; Wells, Athol U; Desai, Sujal R; Poletti, Venerino; Piciucchi, Sara; Dubini, Alessandra; Nunes, Hilario; Valeyre, Dominique; Brillet, Pierre Y; Kambouchner, Marianne; Morais, António; Pereira, José M; Moura, Conceição Souto; Grutters, Jan C; van den Heuvel, Daniel A; van Es, Hendrik W; van Oosterhout, Matthijs F; Seldenrijk, Cornelis A; Bendstrup, Elisabeth; Rasmussen, Finn; Madsen, Line B; Gooptu, Bibek; Pomplun, Sabine; Taniguchi, Hiroyuki; Fukuoka, Junya; Johkoh, Takeshi; Nicholson, Andrew G; Sayer, Charlie; Edmunds, Lilian; Jacob, Joseph; Kokosi, Maria A; Myers, Jeffrey L; Flaherty, Kevin R; Hansell, David M

2016-07-01

Diffuse parenchymal lung disease represents a diverse and challenging group of pulmonary disorders. A consistent diagnostic approach to diffuse parenchymal lung disease is crucial if clinical trial data are to be applied to individual patients. We aimed to evaluate inter-multidisciplinary team agreement for the diagnosis of diffuse parenchymal lung disease. We did a multicentre evaluation of clinical data of patients who presented to the interstitial lung disease unit of the Royal Brompton and Harefield NHS Foundation Trust (London, UK; host institution) and required multidisciplinary team meeting (MDTM) characterisation between March 1, 2010, and Aug 31, 2010. Only patients whose baseline clinical, radiological, and, if biopsy was taken, pathological data were undertaken at the host institution were included. Seven MDTMs, consisting of at least one clinician, radiologist, and pathologist, from seven countries (Denmark, France, Italy, Japan, Netherlands, Portugal, and the UK) evaluated cases of diffuse parenchymal lung disease in a two-stage process between Jan 1, and Oct 15, 2015. First, the clinician, radiologist, and pathologist (if lung biopsy was completed) independently evaluated each case, selected up to five differential diagnoses from a choice of diffuse lung diseases, and chose likelihoods (censored at 5% and summing to 100% in each case) for each of their differential diagnoses, without inter-disciplinary consultation. Second, these specialists convened at an MDTM and reviewed all data, selected up to five differential diagnoses, and chose diagnosis likelihoods. We compared inter-observer and inter-MDTM agreements on patient first-choice diagnoses using Cohen's kappa coefficient (κ). We then estimated inter-observer and inter-MDTM agreement on the probability of diagnosis using weighted kappa coefficient (κw). We compared inter-observer and inter-MDTM confidence of patient first-choice diagnosis. Finally, we evaluated the prognostic significance of a

Evaluating the performance of the two-phase flow solver interFoam

NASA Astrophysics Data System (ADS)

Deshpande, Suraj S.; Anumolu, Lakshman; Trujillo, Mario F.

2012-01-01

The performance of the open source multiphase flow solver, interFoam, is evaluated in this work. The solver is based on a modified volume of fluid (VoF) approach, which incorporates an interfacial compression flux term to mitigate the effects of numerical smearing of the interface. It forms a part of the C + + libraries and utilities of OpenFOAM and is gaining popularity in the multiphase flow research community. However, to the best of our knowledge, the evaluation of this solver is confined to the validation tests of specific interest to the users of the code and the extent of its applicability to a wide range of multiphase flow situations remains to be explored. In this work, we have performed a thorough investigation of the solver performance using a variety of verification and validation test cases, which include (i) verification tests for pure advection (kinematics), (ii) dynamics in the high Weber number limit and (iii) dynamics of surface tension-dominated flows. With respect to (i), the kinematics tests show that the performance of interFoam is generally comparable with the recent algebraic VoF algorithms; however, it is noticeably worse than the geometric reconstruction schemes. For (ii), the simulations of inertia-dominated flows with large density ratios {\\sim }\\mathscr {O}(10^3) yielded excellent agreement with analytical and experimental results. In regime (iii), where surface tension is important, consistency of pressure-surface tension formulation and accuracy of curvature are important, as established by Francois et al (2006 J. Comput. Phys. 213 141-73). Several verification tests were performed along these lines and the main findings are: (a) the algorithm of interFoam ensures a consistent formulation of pressure and surface tension; (b) the curvatures computed by the solver converge to a value slightly (10%) different from the analytical value and a scope for improvement exists in this respect. To reduce the disruptive effects of spurious

Does adding clinical data to administrative data improve agreement among hospital quality measures?

PubMed

Hanchate, Amresh D; Stolzmann, Kelly L; Rosen, Amy K; Fink, Aaron S; Shwartz, Michael; Ash, Arlene S; Abdulkerim, Hassen; Pugh, Mary Jo V; Shokeen, Priti; Borzecki, Ann

2017-09-01

Hospital performance measures based on patient mortality and readmission have indicated modest rates of agreement. We examined if combining clinical data on laboratory tests and vital signs with administrative data leads to improved agreement with each other, and with other measures of hospital performance in the nation's largest integrated health care system. We used patient-level administrative and clinical data, and hospital-level data on quality indicators, for 2007-2010 from the Veterans Health Administration (VA). For patients admitted for acute myocardial infarction (AMI), heart failure (HF) and pneumonia we examined changes in hospital performance on 30-d mortality and 30-d readmission rates as a result of adding clinical data to administrative data. We evaluated whether this enhancement yielded improved measures of hospital quality, based on concordance with other hospital quality indicators. For 30-d mortality, data enhancement improved model performance, and significantly changed hospital performance profiles; for 30-d readmission, the impact was modest. Concordance between enhanced measures of both outcomes, and with other hospital quality measures - including Joint Commission process measures, VA Surgical Quality Improvement Program (VASQIP) mortality and morbidity, and case volume - remained poor. Adding laboratory tests and vital signs to measure hospital performance on mortality and readmission did not improve the poor rates of agreement across hospital quality indicators in the VA. Efforts to improve risk adjustment models should continue; however, evidence of validation should precede their use as reliable measures of quality. Published by Elsevier Inc.

Evacuation performance evaluation tool.

PubMed

Farra, Sharon; Miller, Elaine T; Gneuhs, Matthew; Timm, Nathan; Li, Gengxin; Simon, Ashley; Brady, Whittney

2016-01-01

Hospitals conduct evacuation exercises to improve performance during emergency events. An essential aspect in this process is the creation of reliable and valid evaluation tools. The objective of this article is to describe the development and implications of a disaster evacuation performance tool that measures one portion of the very complex process of evacuation. Through the application of the Delphi technique and DeVellis's framework, disaster and neonatal experts provided input in developing this performance evaluation tool. Following development, content validity and reliability of this tool were assessed. Large pediatric hospital and medical center in the Midwest. The tool was pilot tested with an administrative, medical, and nursing leadership group and then implemented with a group of 68 healthcare workers during a disaster exercise of a neonatal intensive care unit (NICU). The tool has demonstrated high content validity with a scale validity index of 0.979 and inter-rater reliability G coefficient (0.984, 95% CI: 0.948-0.9952). The Delphi process based on the conceptual framework of DeVellis yielded a psychometrically sound evacuation performance evaluation tool for a NICU.

Low agreement between the fitnessgram criterion references for adolescents

PubMed Central

Coledam, Diogo Henrique Constantino; Batista, João Pedro; Glaner, Maria Fátima

2015-01-01

OBJECTIVE: To analyze the association and agreement of fitnessgram reference criteria (RC) for cardiorespiratory fitness, body mass index (BMI) and strength in youth. METHODS: The study included 781 youth, 386 females, aged 10 to 18 years of Londrina-PR. It were performed cardiorespiratory fitness and muscular strength tests and was calculated body mass index. The association between the tests was analyzed using Poisson regression to obtain prevalence ratio (PR) and confidence intervals of 95%, while agreement of the reference criteria was tested by Kappa index. RESULTS: Significant associations were found between cardiorespiratory fitness and BMI (PR=1,49, 1,27-1,75), muscle strength and BMI (PR=1,55, 1,17-2,08), cardiorespiratory fitness and muscle strength (PR=1,81, 1,47-2,24). The agreement between reference criteria ranged from weak to fair, 48.8% (k=0.05, p=0.10) for cardiorespiratory fitness and BMI, 52.9% (k=0.09, p=0.001) for muscle strength and BMI and 38.4% (k=0.22, p<0.001) for cardiorespiratory fitness and muscle strength. CONCLUSIONS: Although RC for cardiorespiratory fitness, muscle strength and BMI are associated, the agreement between them ranged from weak to fair. To evaluate health related physical fitness it is suggest the execution of all tests, since each test has specific characteristics. PMID:25649383

[Inter-observes agreement of Ishak and Metavir scores in histological evaluation of chronic viral hepatitis B and C].

PubMed

Rammeh, Soumaya; Khadra, Hajer Ben; Znaidi, Nadia Sabbegh; Romdhane, Neila Attia; Najjar, Taoufik; Bouzaidi, Slim; Zermani, Rachida

2014-01-01

Many classification systems are currently used for histological evaluation of the severity of chronic viral hepatitis, including the Ishak and Metavir scores, but there is not a consensus classification. The objective of this work was to study the intra and inter-observers agreement of these two scores in the histopathological analysis of liver biopsies in patients with chronic viral hepatitis B or C. Fifty nine patients were included in the study, 26 had chronic hepatitis C and 33 had chronic hepatitis B. To investigate the inter-observers agreement, the liver biopsies were analyzed separately by two pathologists without prior consensus reading. The two pathologists conducted then a consensual reading before reviewing all cases independently. Cohen's kappa coefficient was calculated and in case of asymmetry Spearman's rho coefficient. Before the consensus reading, the agreement was moderate for the analysis of histological activity with both scores (Metavir: kappa=0.41, Ishak: rho=0.58). For the analysis of fibrosis, the agreement was good with both scores (Metavir: kappa=0.61, Ishak: rho=0.86). The consensus reading has improved the reproducibility of the activity that has become good with both scores (Metavir: kappa=0.77, Ishak: rho=0.76). For fibrosis improvement was observed with the Ishak score which agreement became excellent (kappa=0.81). In conclusion, we recommend in routine practice, a combined score: Metavir for activity and Ishak for fibrosis and to make a double reading for each biopsy.

48 CFR 8.606 - Evaluating FPI performance.

Code of Federal Regulations, 2010 CFR

2010-10-01

.... 8.606 Evaluating FPI performance. Agencies shall evaluate FPI contract performance in accordance with subpart 42.15. Performance evaluations do not negate the requirements of 8.602 and 8.604, but they... 48 Federal Acquisition Regulations System 1 2010-10-01 2010-10-01 false Evaluating FPI performance...

Performance Evaluation: A Deadly Disease?

ERIC Educational Resources Information Center

Aluri, Rao; Reichel, Mary

1994-01-01

W. Edwards Deming condemned performance evaluations as a deadly disease afflicting American management. He argued that performance evaluations nourish fear, encourage short-term thinking, stifle teamwork, and are no better than lotteries. This article examines library literature from Deming's perspective. Although that literature accepts…

The supervisor's performance appraisal: evaluating the evaluator.

PubMed

McConnell, C R

1993-04-01

The focus of much performance appraisal in the coming decade or so will likely be on the level of customer satisfaction achieved through performance. Ultimately, evaluating the evaluator--that is, appraising the supervisor--will likely become a matter of assessing how well the supervisor's department meets the needs of its customers. Since meeting the needs of one's customers can well become the strongest determinant of organizational success or failure, it follows that relative success in ensuring these needs are met can become the primary indicator of one's relative success as a supervisor. This has the effect of placing the emphasis on supervisory performance exactly at the point it belongs, right on the bottom-line results of the supervisor's efforts.

75 FR 76729 - Market Access Agreement

Federal Register 2010, 2011, 2012, 2013, 2014

2010-12-09

... falls below pre-established financial performance thresholds. The draft amendment (MAA Amendment) is... System banks and the Funding Corporation that establishes certain financial performance criteria. Under... Agreement). FOR FURTHER INFORMATION CONTACT: Chris Wilson, Financial Analyst, Office of Regulatory Policy...

Evaluation of an advanced-practice physiotherapist in triaging patients with lumbar spine pain: surgeon-physiotherapist level of agreement and patient satisfaction.

PubMed

Robarts, Susan; Stratford, Paul; Kennedy, Deborah; Malcolm, Barry; Finkelstein, Joel

2017-08-01

Surgery for lumbar spine pain is indicated for specific etiologies. Given the majority of individuals referred to spine surgeons are not surgical candidates, care delivery is inefficient, with consultations being of limited value for most. Using specially trained physiotherapists in triage is a human resource strategy that may optimize surgeons' time and the patient experience. An advanced-practice physiotherapist (APP) and a surgeon assessed consecutive patients with lumbar spine pain presenting at an academic health centre's spine surgery clinic. The second assessor was blinded to the outcome of the first. We used the κ statistic to evaluate surgeon-APP level of chance-corrected agreement concerning patients' need for a surgical consultation. To assess satisfaction with the APP, patients completed a modified version of the validated Visit-specific Questionnaire. The sample included 102 participants (54 women) with a mean age of 54.3 ± 14.3 years and a mean Oswestry Disability Index score of 35.4 ± 16.6. The assessors' overall agreement was 86%. The κ coefficient for the need for a surgical consultation was 0.69 (95% confidence interval 0.54-0.84). The APP identified that 77% of patients did not require a surgical consultation. Twenty-one patients underwent surgery. Satisfaction scores for the APP were very high (mean score 92 out of 100). In triaging patients with lumbar spine pain, the APP and surgeon had a high level of agreement. An APP performing triage at a surgical centre can effectively reduce wait lists by 70%, reserving surgical consultations for those patients in whom they are indicated.

40 CFR Table 6 to Subpart Wwww of... - Basic Requirements for Performance Tests, Performance Evaluations, and Design Evaluations for New...

Code of Federal Regulations, 2013 CFR

2013-07-01

... Tests, Performance Evaluations, and Design Evaluations for New and Existing Sources Using Add-On Control... Performance Tests, Performance Evaluations, and Design Evaluations for New and Existing Sources Using Add-On Control Devices As required in § 63.5850 you must conduct performance tests, performance evaluations, and...

40 CFR Table 6 to Subpart Wwww of... - Basic Requirements for Performance Tests, Performance Evaluations, and Design Evaluations for New...

Code of Federal Regulations, 2014 CFR

2014-07-01

... Tests, Performance Evaluations, and Design Evaluations for New and Existing Sources Using Add-On Control... Performance Tests, Performance Evaluations, and Design Evaluations for New and Existing Sources Using Add-On Control Devices As required in § 63.5850 you must conduct performance tests, performance evaluations, and...

40 CFR Table 6 to Subpart Wwww of... - Basic Requirements for Performance Tests, Performance Evaluations, and Design Evaluations for New...

Code of Federal Regulations, 2012 CFR

2012-07-01

... Tests, Performance Evaluations, and Design Evaluations for New and Existing Sources Using Add-On Control... Performance Tests, Performance Evaluations, and Design Evaluations for New and Existing Sources Using Add-On Control Devices As required in § 63.5850 you must conduct performance tests, performance evaluations, and...

Performance Evaluation of the United Nations Environment ...

EPA Pesticide Factsheets

A request for technical collaboration between the UNEP and the US EPA resulted in the establishment of a MCRADA. The purpose of this agreement was to evaluate an air quality monitoring system (referred to as the UNEP pod) developed by the UNEP for use in environmental situations where more sophisticated monitoring instrumentation was not available. The US EPA has conducted numerous evaluations of other similar sensor pods at its Research Triangle Park, NC research campus and has trained staff as well as established research designs for such efforts. Under the terms of the MCRADA, the US EPA would operate the pod using UNEP provided operating procedures in a manner consistent with its planned intent of deployment. The US EPA would collect air quality monitoring data from the pod involving select environmental measures over a period of approximately one month. Reference monitoring data collected from collocated federal regulatory monitors would be used to establish a comparison between the two systems and thus establishment of performance characteristics. In addition, the US EPA would provide feedback information to the UNEP as to observed ease of use features of the pod that would be beneficial in its future evolution and deployment. The UNEP recently developed a multipollutant sensor pod called the UNEP Air Quality Monitoring Unit, herein simply defined as the UNEP pod (http://aqicn.org/faq/2015-10-28/unep-air-quality-monitoring-station/). First introduced in 20

PROSPECTIVE EVALUATION OF VISUAL ACUITY AGREEMENT BETWEEN STANDARD EARLY TREATMENT DIABETIC RETINOPATHY STUDY CHART AND A HANDHELD EQUIVALENT IN EYES WITH RETINAL PATHOLOGY.

PubMed

Rahimy, Ehsan; Reddy, Sahitya; DeCroos, Francis Char; Khan, M Ali; Boyer, David S; Gupta, Omesh P; Regillo, Carl D; Haller, Julia A

2015-08-01

To evaluate the visual acuity agreement between a standard back-illuminated Early Treatment Diabetic Retinopathy Study (ETDRS) chart and a handheld internally illuminated ETDRS chart. Two-center prospective study. Seventy patients (134 eyes) with retinal pathology were enrolled between October 2012 and August 2013. Visual acuity was measured using both the ETDRS chart and the handheld device by masked independent examiners after best protocol refraction. Examination was performed in the same room under identical illumination and testing conditions. The mean number of letters seen was 63.0 (standard deviation: 19.8 letters) and 61.2 letters (standard deviation: 19.1 letters) for the ETDRS chart and handheld device, respectively. Mean difference per eye between the ETDRS and handheld device was 1.8 letters. A correlation coefficient (r) of 0.95 demonstrated a positive linear correlation between ETDRS chart and handheld device measured acuities. Intraclass correlation coefficient was performed to assess the reproducibility of the measurements made by different observers measuring the same quantity and was calculated to be 0.95 (95% confidence interval: 0.93-0.96). Agreement was independent of retinal disease. The strong correlation between measured visual acuity using the ETDRS and handheld equivalent suggests that they may be used interchangeably, with accurate measurements. Potential benefits of this device include convenience and portability, as well as the ability to assess ETDRS visual acuity without a dedicated testing lane.

Performance evaluation of the Abbott CELL-DYN Ruby and the Sysmex XT-2000i haematology analysers.

PubMed

Leers, M P G; Goertz, H; Feller, A; Hoffmann, J J M L

2011-02-01

Two mid-range haematology analysers (Abbott CELL-DYN Ruby and Sysmex XT-2000i) were evaluated to determine their analytical performance and workflow efficiency in the haematology laboratory. In total 418 samples were processed for determining equivalence of complete blood count (CBC) measurements, and 100 for reticulocyte comparison. Blood smears served for assessing the agreement of the differential counts. Inter-instrument agreement for most parameters was good although small numbers of discrepancies were observed. Systematic biases were found for mean cell volume, reticulocytes, platelets and mean platelet volume. CELL-DYN Ruby WBC differentials were obtained with all samples while the XT-2000i suppressed differentials partially or completely in 13 samples (3.1%). WBC subpopulation counts were otherwise in good agreement with no major outliers. Following first-pass CBC/differential analysis, 88 (21%) of XT-2000i samples required further analyser processing compared to 18 (4.3%) for the CELL-DYN Ruby. Smear referrals for suspected WBC/nucleated red blood cells and platelet abnormalities were indicated for 106 (25.4%) and 95 (22.7%) of the XT-2000i and CELL-DYN Ruby samples respectively. Flagging efficiencies for both analysers were found to be similar. The Sysmex XT-2000i and Abbott CELL-DYN Ruby analysers have broadly comparable analytical performance, but the CELL-DYN Ruby showed superior first-pass efficiency. © 2010 Blackwell Publishing Ltd.

Use of Classification Agreement Analyses to Evaluate RTI Implementation

ERIC Educational Resources Information Center

VanDerHeyden, Amanda

2010-01-01

RTI as a framework for decision making has implications for the diagnosis of specific learning disabilities. Any diagnostic tool must meet certain standards to demonstrate that its use leads to predictable decisions with minimal risk. Classification agreement analyses are described as optimal for demonstrating the technical adequacy of RTI…

Monte Carlo simulation of Ray-Scan 64 PET system and performance evaluation using GATE toolkit

NASA Astrophysics Data System (ADS)

Li, Suying; Zhang, Qiushi; Vuletic, Ivan; Xie, Zhaoheng; Yang, Kun; Ren, Qiushi

2017-02-01

In this study, we aimed to develop a GATE model for the simulation of Ray-Scan 64 PET scanner and model its performance characteristics. A detailed implementation of system geometry and physical process were included in the simulation model. Then we modeled the performance characteristics of Ray-Scan 64 PET system for the first time, based on National Electrical Manufacturers Association (NEMA) NU-2 2007 protocols and validated the model against experimental measurement, including spatial resolution, sensitivity, counting rates and noise equivalent count rate (NECR). Moreover, an accurate dead time module was investigated to simulate the counting rate performance. Overall results showed reasonable agreement between simulation and experimental data. The validation results showed the reliability and feasibility of the GATE model to evaluate major performance of Ray-Scan 64 PET system. It provided a useful tool for a wide range of research applications.

;Agreement; in the IPCC Confidence measure

NASA Astrophysics Data System (ADS)

Rehg, William; Staley, Kent

2017-02-01

The Intergovernmental Panel on Climate Change (IPCC) has, in its most recent Assessment Report (AR5), articulated guidelines for evaluating and communicating uncertainty that include a qualitative scale of confidence. We examine one factor included in that scale: the "degree of agreement." Some discussions of the degree of agreement in AR5 suggest that the IPCC is employing a consensus-oriented social epistemology. We consider the application of the degree of agreement factor in practice in AR5. Our findings, though based on a limited examination, suggest that agreement attributions do not so much track the overall consensus among investigators as the degree to which relevant research findings substantively converge in offering support for IPCC claims. We articulate a principle guiding confidence attributions in AR5 that centers not on consensus but on the notion of support. In concluding, we tentatively suggest a pluralist approach to the notion of support.

12 CFR 133.5 - Related agreements considered a single agreement.

Code of Federal Regulations, 2012 CFR

2012-01-01

... agreement. 133.5 Section 133.5 Banks and Banking COMPTROLLER OF THE CURRENCY, DEPARTMENT OF THE TREASURY DISCLOSURE AND REPORTING OF CRA-RELATED AGREEMENTS § 133.5 Related agreements considered a single agreement... § 133.2 of this part. (a) Agreements entered into by same parties. All written agreements to which an...

FY2011 Progress Report: Agreement 8697 - NOx Sensor Development

DOE Office of Scientific and Technical Information (OSTI.GOV)

Woo, L Y; Glass, R S

Objectives are: (1) Develop an inexpensive, rapid-response, high-sensitivity and selective electrochemical sensor for oxides of nitrogen (NO{sub x}) for compression-ignition, direct-injection (CIDI) OBD II systems; (2) Explore and characterize novel, effective sensing methodologies based on impedance measurements and designs and manufacturing methods that are compatible with mass fabrication; and (3) Collaborate with industry in order to (ultimately) transfer the technology to a supplier for commercialization. Approach used is: (1) Use an ionic (O{sup 2-}) conducting ceramic as a solid electrolyte and metal or metal-oxide electrodes; (2) Correlate NO{sub x} concentration with changes in cell impedance; (3) Evaluate sensing mechanisms andmore » aging effects on long-term performance using electrochemical techniques; and (4) Collaborate with Ford Research Center to optimize sensor performance and perform dynamometer and on-vehicle testing. Work in FY2011 focused on using an algorithm developed in FY2010 in a simplified strategy to demonstrate how data from controlled laboratory evaluation could be applied to data from real-world engine testing. The performance of a Au wire prototype sensor was evaluated in the laboratory with controlled gas compositions and in dynamometer testing with diesel exhaust. The laboratory evaluation indicated a nonlinear dependence of the NO{sub x} and O{sub 2} sensitivity with concentration. For both NO{sub x} and O{sub 2}, the prototype sensor had higher sensitivity at concentrations less than {approx}20 ppm and {approx}7%, respectively, compared to lower NO{sub x} and O{sub 2} sensitivity at concentrations greater than {approx}50 ppm and {approx}10.5%, respectively. Results in dynamometer diesel exhaust generally agreed with the laboratory results. Diesel exhaust after-treatment systems will likely require detection levels less than {approx}20 ppm in order to meet emission regulations. The relevant mathematical expressions for

Head-camera video recordings of trauma core competency procedures can evaluate surgical resident's technical performance as well as colocated evaluators.

PubMed

Mackenzie, Colin F; Pasley, Jason; Garofalo, Evan; Shackelford, Stacy; Chen, Hegang; Longinaker, Nyaradzo; Granite, Guinevere; Pugh, Kristy; Hagegeorge, George; Tisherman, Samuel A

2017-07-01

Unbiased evaluation of trauma core competency procedures is necessary to determine if residency and predeployment training courses are useful. We tested whether a previously validated individual procedure score (IPS) for individual procedure vascular exposure and fasciotomy (FAS) performance skills could discriminate training status by comparing IPS of evaluators colocated with surgeons to blind video evaluations. Performance of axillary artery (AA), brachial artery (BA), and femoral artery (FA) vascular exposures and lower extremity FAS on fresh cadavers by 40 PGY-2 to PGY-6 residents was video-recorded from head-mounted cameras. Two colocated trained evaluators assessed IPS before and after training. One surgeon in each pretraining tertile of IPS for each procedure was randomly identified for blind video review. The same 12 surgeons were video-recorded repeating the procedures less than 4 weeks after training. Five evaluators independently reviewed all 96 randomly arranged deidentified videos. Inter-rater reliability/consistency, intraclass correlation coefficients were compared by colocated versus video review of IPS, and errors. Study methodology and bias were judged by Medical Education Research Study Quality Instrument and the Quality Assessment of Diagnostic Accuracy Studies criteria. There were no differences (p ≥ 0.5) in IPS for AA, FA, FAS, whether evaluators were colocated or reviewed video recordings. Evaluator consistency was 0.29 (BA) - 0.77 (FA). Video and colocated evaluators were in total agreement (p = 1.0) for error recognition. Intraclass correlation coefficient was 0.73 to 0.92, dependent on procedure. Correlations video versus colocated evaluations were 0.5 to 0.9. Except for BA, blinded video evaluators discriminated (p < 0.002) whether procedures were performed before training versus after training. Study methodology by Medical Education Research Study Quality Instrument criteria scored 15.5/19, Quality Assessment of Diagnostic Accuracy

Evaluation of intra-aortic CT angiography performances for the visualisation of spinal vascular malformations' angioarchitecture.

PubMed

Clarençon, Frédéric; Di Maria, Federico; Sourour, Nader-Antoine; Gabrieli, Joseph; Nouet, Aurélien; Shotar, Eimad; Cormier, Evelyne; Fahed, Robert; Cornu, Philippe; Chiras, Jacques

2016-10-01

To evaluate the performances of the CT-angiography by direct intra-aortic contrast media injection (IA-CTA) for spinal vascular malformations (SVMs)' imaging. Thirteen patients (8 males, 5 females, mean age: 56 y) with suspected SVM underwent IA-CTAs by direct intra-aortic iodinated contrast media injection (5 cc/s; 100 cc) via an arterial femoral or humeral access. Two independent observers evaluated the angioarchitecture of the SVMs and the visualisation of both the Adamkiewicz artery and the anterior spinal artery. Then a consensus was obtained between the 2 reviewers; the results of the IA-CTA were finally compared with those of the full spinal DSA evaluated in consensus. The IA-CTA was feasible in all cases and depicted the SVM in all except one case (92 %). Interrater agreement was good for the location of the SVMs' level. Intermodality (IA-CTA/DSA) agreement was excellent for the level and side of the shunt point, as well as for the SVM subtype evaluation. In 77 % of the cases, the Adamkiewicz artery was satisfactorily seen at the same time on IA-CTA. IA-CTA is a new technique that seems helpful to reach a better understanding of SMVs and may help to tailor more precisely their treatment. • IA-CTA is an accurate technique for the SVMs' angioarchitecture analysis • IA-CTA can locate, at the same time, the Adamkiewicz artery (AKA) • IA-CTA may be helpful in elderly patients with troublesome vasculature.

Evaluating performance in sweat testing in medical biochemistry laboratories in Croatia.

PubMed

Aralica, Merica; Krleza, Jasna Lenicek

2017-02-15

Sweat test has a diagnostic role in evaluation of cystic fibrosis. Its performance includes sweat stimulation, collection and analysis. All listed may be sources of inconsistencies in everyday practice. The aim of this study was an evaluation of external quality assessment (EQA) of sweat chloride measurement including sweat test performance in medical biochemistry laboratories in Croatia. EQA for sweat chloride measurement was provided by Croatian Centre for Quality Assessment in Laboratory Medicine (CROQALM) in five consecutive exercises to medical biochemistry laboratories (MBL) that offered sweat testing. A questionnaire regarding all phases of testing was mailed to involved MBL (N = 10). Survey results were compared to current guidelines for sweat test performance. Reported results of EQA in 2015 exercises showed coefficients of variation (CV) from 28.9%, 29.0% to 35.3%, respectively. An introduction of uniform sweat chloride measurement protocol resulted in CV of 15.5% and 14.7% reported in following two exercises in 2016. All MBL included in this study replied to the questionnaire. Results reported by MBL indicated: lack of patient information policy (7/10), use of unacceptable electrodes (6/9), misuse of minimum of acceptable sweat weight (6/9), lack of internal quality assessment (5/9) and recommended reference ranges (5/9 and 4/9). Agreements to guidelines were found in approach to unsuitable patients (9/10) and sweat collection (8/9). Presented results indicate major weak points of current practice in sweat test performance in Croatian MBL and stress the need for its standardization on a national level.

Evaluating performance in sweat testing in medical biochemistry laboratories in Croatia

PubMed Central

Aralica, Merica; Krleza, Jasna Lenicek

2017-01-01

Introduction Sweat test has a diagnostic role in evaluation of cystic fibrosis. Its performance includes sweat stimulation, collection and analysis. All listed may be sources of inconsistencies in everyday practice. The aim of this study was an evaluation of external quality assessment (EQA) of sweat chloride measurement including sweat test performance in medical biochemistry laboratories in Croatia. Materials and methods EQA for sweat chloride measurement was provided by Croatian Centre for Quality Assessment in Laboratory Medicine (CROQALM) in five consecutive exercises to medical biochemistry laboratories (MBL) that offered sweat testing. A questionnaire regarding all phases of testing was mailed to involved MBL (N = 10). Survey results were compared to current guidelines for sweat test performance. Results Reported results of EQA in 2015 exercises showed coefficients of variation (CV) from 28.9%, 29.0% to 35.3%, respectively. An introduction of uniform sweat chloride measurement protocol resulted in CV of 15.5% and 14.7% reported in following two exercises in 2016. All MBL included in this study replied to the questionnaire. Results reported by MBL indicated: lack of patient information policy (7/10), use of unacceptable electrodes (6/9), misuse of minimum of acceptable sweat weight (6/9), lack of internal quality assessment (5/9) and recommended reference ranges (5/9 and 4/9). Agreements to guidelines were found in approach to unsuitable patients (9/10) and sweat collection (8/9). Conclusion Presented results indicate major weak points of current practice in sweat test performance in Croatian MBL and stress the need for its standardization on a national level. PMID:28392735

40 CFR Table 6 to Subpart Wwww of... - Basic Requirements for Performance Tests, Performance Evaluations, and Design Evaluations for New...

Code of Federal Regulations, 2010 CFR

2010-07-01

... Tests, Performance Evaluations, and Design Evaluations for New and Existing Sources Using Add-On Control... Tests, Performance Evaluations, and Design Evaluations for New and Existing Sources Using Add-On Control Devices As required in § 63.5850 you must conduct performance tests, performance evaluations, and design...

40 CFR Table 6 to Subpart Wwww of... - Basic Requirements for Performance Tests, Performance Evaluations, and Design Evaluations for New...

Code of Federal Regulations, 2011 CFR

2011-07-01

... Tests, Performance Evaluations, and Design Evaluations for New and Existing Sources Using Add-On Control... Tests, Performance Evaluations, and Design Evaluations for New and Existing Sources Using Add-On Control Devices As required in § 63.5850 you must conduct performance tests, performance evaluations, and design...

Quality evaluation of moluodan concentrated pill using high-performance liquid chromatography fingerprinting coupled with chemometrics.

PubMed

Tao, Lingyan; Zhang, Qing; Wu, Yongjiang; Liu, Xuesong

2016-12-01

In this study, a fast and effective high-performance liquid chromatography method was developed to obtain a fingerprint chromatogram and quantitative analysis simultaneously of four indexes including gallic acid, chlorogenic acid, albiflorin and paeoniflorin of the traditional Chinese medicine Moluodan Concentrated Pill. The method was performed by using a Waters X-bridge C 18 reversed phase column on an Agilent 1200S high-performance liquid chromatography system coupled with diode array detection. The mobile phase of the high-performance liquid chromatography method was composed of 20 mmol/L phosphate solution and acetonitrile with a 1 mL/min eluent velocity, under a detection temperature of 30°C and a UV detection wavelength of 254 nm. After the methodology validation, 16 batches of Moluodan Concentrated Pill were analyzed by this high-performance liquid chromatography method and both qualitative and quantitative evaluation results were achieved by similarity analysis, principal component analysis and hierarchical cluster analysis. The results of these three chemometrics were in good agreement and all indicated that batch 10 and batch 16 showed significant differences with the other 14 batches. This suggested that the developed high-performance liquid chromatography method could be applied in the quality evaluation of Moluodan Concentrated Pill. © 2016 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim.

20 CFR 411.321 - Under what conditions will SSA terminate an agreement with an EN due to inadequate performance?

Code of Federal Regulations, 2010 CFR

2010-04-01

... 20 Employees' Benefits 2 2010-04-01 2010-04-01 false Under what conditions will SSA terminate an agreement with an EN due to inadequate performance? 411.321 Section 411.321 Employees' Benefits SOCIAL SECURITY ADMINISTRATION THE TICKET TO WORK AND SELF-SUFFICIENCY PROGRAM Employment Networks § 411.321 Under...

20 CFR 411.321 - Under what conditions will SSA terminate an agreement with an EN due to inadequate performance?

Code of Federal Regulations, 2011 CFR

2011-04-01

... 20 Employees' Benefits 2 2011-04-01 2011-04-01 false Under what conditions will SSA terminate an agreement with an EN due to inadequate performance? 411.321 Section 411.321 Employees' Benefits SOCIAL SECURITY ADMINISTRATION THE TICKET TO WORK AND SELF-SUFFICIENCY PROGRAM Employment Networks § 411.321 Under...

20 CFR 411.321 - Under what conditions will SSA terminate an agreement with an EN due to inadequate performance?

Code of Federal Regulations, 2014 CFR

2014-04-01

... 20 Employees' Benefits 2 2014-04-01 2014-04-01 false Under what conditions will SSA terminate an agreement with an EN due to inadequate performance? 411.321 Section 411.321 Employees' Benefits SOCIAL SECURITY ADMINISTRATION THE TICKET TO WORK AND SELF-SUFFICIENCY PROGRAM Employment Networks § 411.321 Under...

20 CFR 411.321 - Under what conditions will SSA terminate an agreement with an EN due to inadequate performance?

Code of Federal Regulations, 2013 CFR

2013-04-01

... 20 Employees' Benefits 2 2013-04-01 2013-04-01 false Under what conditions will SSA terminate an agreement with an EN due to inadequate performance? 411.321 Section 411.321 Employees' Benefits SOCIAL SECURITY ADMINISTRATION THE TICKET TO WORK AND SELF-SUFFICIENCY PROGRAM Employment Networks § 411.321 Under...

20 CFR 411.321 - Under what conditions will SSA terminate an agreement with an EN due to inadequate performance?

Code of Federal Regulations, 2012 CFR

2012-04-01

... 20 Employees' Benefits 2 2012-04-01 2012-04-01 false Under what conditions will SSA terminate an agreement with an EN due to inadequate performance? 411.321 Section 411.321 Employees' Benefits SOCIAL SECURITY ADMINISTRATION THE TICKET TO WORK AND SELF-SUFFICIENCY PROGRAM Employment Networks § 411.321 Under...

Can't get no satisfaction? Will pay for performance help?: toward an economic framework for understanding performance-based risk-sharing agreements for innovative medical products.

PubMed

Towse, Adrian; Garrison, Louis P

2010-01-01

This article examines performance-based risk-sharing agreements for pharmaceuticals from a theoretical economic perspective. We position these agreements as a form of coverage with evidence development. New performance-based risk sharing could produce a more efficient market equilibrium, achieved by adjustment of the price post-launch to reflect outcomes combined with a new approach to the post-launch costs of evidence collection. For this to happen, the party best able to manage or to bear specific risks must do so. Willingness to bear risk will depend not only on ability to manage it, but on the degree of risk aversion. We identify three related frameworks that provide relevant insights: value of information, real option theory and money-back guarantees. We identify four categories of risk sharing: budget impact, price discounting, outcomes uncertainty and subgroup uncertainty. We conclude that a value of information/real option framework is likely to be the most helpful approach for understanding the costs and benefits of risk sharing. There are a number of factors that are likely to be crucial in determining if performance-based or risk-sharing agreements are efficient and likely to become more important in the future: (i) the cost and practicality of post-launch evidence collection relative to pre-launch; (ii) the feasibility of coverage with evidence development without a pre-agreed contract as to how the evidence will be used to adjust price, revenues or use, in which uncertainty around the pay-off to additional research will reduce the incentive for the manufacturer to collect the information; (iii) the difficulty of writing and policing risk-sharing agreements; (iv) the degree of risk aversion (and therefore opportunity to trade) on the part of payers and manufacturers; and (v) the extent of transferability of data from one country setting to another to support coverage with evidence development in a risk-sharing framework. There is no doubt that

13 CFR 304.4 - Performance evaluations.

Code of Federal Regulations, 2010 CFR

2010-01-01

... 13 Business Credit and Assistance 1 2010-01-01 2010-01-01 false Performance evaluations. 304.4 Section 304.4 Business Credit and Assistance ECONOMIC DEVELOPMENT ADMINISTRATION, DEPARTMENT OF COMMERCE ECONOMIC DEVELOPMENT DISTRICTS § 304.4 Performance evaluations. (a) EDA shall evaluate the management...

Agreement between gonioscopy and ultrasound biomicroscopy in detecting iridotrabecular apposition.

PubMed

Barkana, Yaniv; Dorairaj, Syril K; Gerber, Yariv; Liebmann, Jeffrey M; Ritch, Robert

2007-10-01

To assess the agreement between findings obtained at dark-room gonioscopy and ultrasound biomicroscopy (UBM) in the diagnosis of iridotrabecular apposition in light and dark conditions. We enrolled patients with appositional angle closure at dark-room gonioscopy performed using a 1-mm slitlamp beam that did not cross the pupil. Ultrasound biomicroscopic images were acquired in normal room light and subsequently with all room lights off. Images were evaluated for the presence or absence of iris-cornea contact. The angle opening distance at 500 microm was calculated. Iridotrabecular apposition in at least 1 angle quadrant was demonstrated in all 18 eyes at dark-room gonioscopy, 17 eyes (94%) at dark-room UBM, and only 10 eyes (56%) at UBM in room light. Of 18 superior angles that were appositionally closed at dark-room gonioscopy, apposition was demonstrated on UBM images in 16 (89%) in a dark room but only 6 (33%) in room light. Angle opening distance was less during dark-room gonioscopy in all but the nasal quadrant. We found high agreement between gonioscopy and UBM when both are performed in a completely dark room. Our findings support the recommendation that, in routine clinical practice, gonioscopy be performed in a dark room to avoid misdiagnosis of treatable iridotrabecular apposition.

7 CFR 1709.16 - Performance reviews.

Code of Federal Regulations, 2014 CFR

2014-01-01

... 7 Agriculture 11 2014-01-01 2014-01-01 false Performance reviews. 1709.16 Section 1709.16... AGRICULTURE ASSISTANCE TO HIGH ENERGY COST COMMUNITIES General Requirements § 1709.16 Performance reviews. Each grant agreement shall include performance criteria and RUS will regularly evaluate the progress...

Performance Evaluation of Nano-JASMINE

NASA Astrophysics Data System (ADS)

Hatsutori, Y.; Kobayashi, Y.; Gouda, N.; Yano, T.; Murooka, J.; Niwa, Y.; Yamada, Y.

We report the results of performance evaluation of the first Japanese astrometry satellite, Nano-JASMINE. It is a very small satellite and weighs only 35 kg. It aims to carry out astrometry measurement of nearby bright stars (z ≤ 7.5 mag) with an accuracy of 3 milli-arcseconds. Nano-JASMINE will be launched by Cyclone-4 rocket in August 2011 from Brazil. The current status is in the process of evaluating the performances. A series of performance tests and numerical analysis were conducted. As a result, the engineering model (EM) of the telescope was measured to be achieving a diffraction-limited performance and confirmed that it has enough performance for scientific astrometry.

Evaluation of icing drag coefficient correlations applied to iced propeller performance prediction

NASA Technical Reports Server (NTRS)

Miller, Thomas L.; Shaw, R. J.; Korkan, K. D.

1987-01-01

Evaluation of three empirical icing drag coefficient correlations is accomplished through application to a set of propeller icing data. The various correlations represent the best means currently available for relating drag rise to various flight and atmospheric conditions for both fixed-wing and rotating airfoils, and the work presented here ilustrates and evaluates one such application of the latter case. The origins of each of the correlations are discussed, and their apparent capabilities and limitations are summarized. These correlations have been made to be an integral part of a computer code, ICEPERF, which has been designed to calculate iced propeller performance. Comparison with experimental propeller icing data shows generally good agreement, with the quality of the predicted results seen to be directly related to the radial icing extent of each case. The code's capability to properly predict thrust coefficient, power coefficient, and propeller efficiency is shown to be strongly dependent on the choice of correlation selected, as well as upon proper specificatioon of radial icing extent.

Evaluation of EIT system performance.

PubMed

Yasin, Mamatjan; Böhm, Stephan; Gaggero, Pascal O; Adler, Andy

2011-07-01

An electrical impedance tomography (EIT) system images internal conductivity from surface electrical stimulation and measurement. Such systems necessarily comprise multiple design choices from cables and hardware design to calibration and image reconstruction. In order to compare EIT systems and study the consequences of changes in system performance, this paper describes a systematic approach to evaluate the performance of the EIT systems. The system to be tested is connected to a saline phantom in which calibrated contrasting test objects are systematically positioned using a position controller. A set of evaluation parameters are proposed which characterize (i) data and image noise, (ii) data accuracy, (iii) detectability of single contrasts and distinguishability of multiple contrasts, and (iv) accuracy of reconstructed image (amplitude, resolution, position and ringing). Using this approach, we evaluate three different EIT systems and illustrate the use of these tools to evaluate and compare performance. In order to facilitate the use of this approach, all details of the phantom, test objects and position controller design are made publicly available including the source code of the evaluation and reporting software.

7 CFR 58.46 - Fees for service performed under cooperative agreement.

Code of Federal Regulations, 2010 CFR

2010-01-01

... MARKETING SERVICE (Standards, Inspections, Marketing Practices), DEPARTMENT OF AGRICULTURE (CONTINUED) REGULATIONS AND STANDARDS UNDER THE AGRICULTURAL MARKETING ACT OF 1946 AND THE EGG PRODUCTS INSPECTION ACT... provided for by such agreement. Marking, Branding, and Identifying Product ...

20 CFR 617.59 - Agreements with State agencies.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 20 Employees' Benefits 3 2010-04-01 2010-04-01 false Agreements with State agencies. 617.59 Section 617.59 Employees' Benefits EMPLOYMENT AND TRAINING ADMINISTRATION, DEPARTMENT OF LABOR TRADE... § 617.59 Agreements with State agencies. (a) Authority. Before performing any function or exercising any...

Clock Agreement Among Parallel Supercomputer Nodes

DOE Data Explorer

Jones, Terry R.; Koenig, Gregory A.

2014-04-30

This dataset presents measurements that quantify the clock synchronization time-agreement characteristics among several high performance computers including the current world's most powerful machine for open science, the U.S. Department of Energy's Titan machine sited at Oak Ridge National Laboratory. These ultra-fast machines derive much of their computational capability from extreme node counts (over 18000 nodes in the case of the Titan machine). Time-agreement is commonly utilized by parallel programming applications and tools, distributed programming application and tools, and system software. Our time-agreement measurements detail the degree of time variance between nodes and how that variance changes over time. The dataset includes empirical measurements and the accompanying spreadsheets.

7 CFR 1494.801 - Enforcement and termination of agreements with CCC.

Code of Federal Regulations, 2014 CFR

2014-01-01

... 7 Agriculture 10 2014-01-01 2014-01-01 false Enforcement and termination of agreements with CCC... Enhancement Program Operations § 1494.801 Enforcement and termination of agreements with CCC. (a) Performance in accordance with an Agreement with CCC. (1) An exporter which enters into an Agreement with CCC...

7 CFR 1494.801 - Enforcement and termination of agreements with CCC.

Code of Federal Regulations, 2013 CFR

2013-01-01

... 7 Agriculture 10 2013-01-01 2013-01-01 false Enforcement and termination of agreements with CCC... Enhancement Program Operations § 1494.801 Enforcement and termination of agreements with CCC. (a) Performance in accordance with an Agreement with CCC. (1) An exporter which enters into an Agreement with CCC...

7 CFR 1494.801 - Enforcement and termination of agreements with CCC.

Code of Federal Regulations, 2012 CFR

2012-01-01

... 7 Agriculture 10 2012-01-01 2012-01-01 false Enforcement and termination of agreements with CCC... Enhancement Program Operations § 1494.801 Enforcement and termination of agreements with CCC. (a) Performance in accordance with an Agreement with CCC. (1) An exporter which enters into an Agreement with CCC...

AERMOD performance evaluation for three coal-fired electrical generating units in Southwest Indiana.

PubMed

Frost, Kali D

2014-03-01

An evaluation of the steady-state dispersion model AERMOD was conducted to determine its accuracy at predicting hourly ground-level concentrations of sulfur dioxide (SO2) by comparing model-predicted concentrations to a full year of monitored SO2 data. The two study sites are comprised of three coal-fired electrical generating units (EGUs) located in southwest Indiana. The sites are characterized by tall, buoyant stacks,flat terrain, multiple SO2 monitors, and relatively isolated locations. AERMOD v12060 and AERMOD v12345 with BETA options were evaluated at each study site. For the six monitor-receptor pairs evaluated, AERMOD showed generally good agreement with monitor values for the hourly 99th percentile SO2 design value, with design value ratios that ranged from 0.92 to 1.99. AERMOD was within acceptable performance limits for the Robust Highest Concentration (RHC) statistic (RHC ratios ranged from 0.54 to 1.71) at all six monitors. Analysis of the top 5% of hourly concentrations at the six monitor-receptor sites, paired in time and space, indicated poor model performance in the upper concentration range. The amount of hourly model predicted data that was within a factor of 2 of observations at these higher concentrations ranged from 14 to 43% over the six sites. Analysis of subsets of data showed consistent overprediction during low wind speed and unstable meteorological conditions, and underprediction during stable, low wind conditions. Hourly paired comparisons represent a stringent measure of model performance; however given the potential for application of hourly model predictions to the SO2 NAAQS design value, this may be appropriate. At these two sites, AERMOD v12345 BETA options do not improve model performance. A regulatory evaluation of AERMOD utilizing quantile-quantile (Q-Q) plots, the RHC statistic, and 99th percentile design value concentrations indicates that model performance is acceptable according to widely accepted regulatory performance

Minnesota urban partnership agreement national evaluation : content analysis test plan.

DOT National Transportation Integrated Search

2009-11-17

This report presents the content analysis test plan for the Minnesota Urban Partnership Agreement (UPA) under the United States Department of Transportation (U.S. DOT) UPA Program. The Minnesota UPA projects focus on reducing congestion by employing ...

Development of a Walking-Safety Scale for Older Adults, Part II: Interrater and Test–Retest Agreement of the GEM Scale

PubMed Central

Rousseau, Jacqueline; Kaegi, Christine; Boudreault, Renée; Nadeau, Sylvie

2008-01-01

Purpose: The GEM scale is an objective assessment tool, specifically developed for older adults, to evaluate walking safety using standardized tasks. The purpose of this study was to estimate the interrater and test–retest agreement of the GEM scale. Method: Participants (n = 41; ≥ 65 years) were recruited from geriatric units and assessed independently and simultaneously by three raters on two occasions using the GEM scale. Kappa coefficients and percentage agreement were calculated for each item of the scale. Results: A majority of walking items (n = 22) showed fair to substantial interrater agreement (κ ≥ 0.25) and substantial to almost perfect test–retest agreement (κ ≥ 0.60). Mean percentage agreement was high for both interrater and test–retest agreement (79% ± 15% and 83% ± 16% respectively). Moreover, detailed analyses demonstrated that the relatively low agreement of some items resulted from changes in the performance of some participants and the low variability of scores. Although some walking items showed less agreement, the final decision regarding the participants’ ability to walk safely resulted in moderate to substantial interrater and test–retest agreement. Conclusion: The GEM scale is a new assessment tool that can now be used with estimated interrater and test–retest properties to allow therapists to objectively evaluate walking safety among the elderly. PMID:20145760

The polar bear management agreement for the southern Beaufort Sea: an evaluation of the first ten years of a unique conservation agreement

USGS Publications Warehouse

Brower, C.D.; Carpenter, A.; Branigan, M.L.; Calvert, W.; Evans, T.; Fischbach, Anthony S.; Nagy, J.A.; Schliebe, S.; Stirling, I.

2002-01-01

Polar bears (Ursus maritimus) of the southern Beaufort Sea population, distributed from approximately Icy Cape, west of Point Barrow, to Pearce Point, east of Paulatuk in Canada, are harvested by hunters from both countries. In Canada, quotas to control polar bear hunting have been in place, with periodic modifications, since 1968. In Alaska, passage of the United State Marine Mammal Protection Act (MMPA) of 1972 banned polar bear hunting unless done by Alaska Natives for subsistence. However, the MMPA placed no restrictions on numbers or composition of the subsistence hunt, leaving open the potential for an overharvest with no possible legal management response until the population was declared depleted. Recognizing that as a threat to the conservation of the shared polar bear population, the Inuvialuit Game Council from Canada and the North Slop Borough from Alaska negotiated and signed a user-to-user agreement, the Polar Bear Management Agreement for the Southern Beaufort Sea, in 1988. We reviewed the functioning of the agreement through its first 10 years and concluded that, overall, it has been successful because both the total harvest and the proportion of females in the harvest have been contained within sustainable limits. However, harvest monitoring needs to be improved in Alaska, and awareness of the need to prevent overharvest of females needs to be increased in both countries. This agreement is a useful model for other user-to-user conservation agreements.

Performance Evaluation of Resource Management in Cloud Computing Environments.

PubMed

Batista, Bruno Guazzelli; Estrella, Julio Cezar; Ferreira, Carlos Henrique Gomes; Filho, Dionisio Machado Leite; Nakamura, Luis Hideo Vasconcelos; Reiff-Marganiec, Stephan; Santana, Marcos José; Santana, Regina Helena Carlucci

2015-01-01

Cloud computing is a computational model in which resource providers can offer on-demand services to clients in a transparent way. However, to be able to guarantee quality of service without limiting the number of accepted requests, providers must be able to dynamically manage the available resources so that they can be optimized. This dynamic resource management is not a trivial task, since it involves meeting several challenges related to workload modeling, virtualization, performance modeling, deployment and monitoring of applications on virtualized resources. This paper carries out a performance evaluation of a module for resource management in a cloud environment that includes handling available resources during execution time and ensuring the quality of service defined in the service level agreement. An analysis was conducted of different resource configurations to define which dimension of resource scaling has a real influence on client requests. The results were used to model and implement a simulated cloud system, in which the allocated resource can be changed on-the-fly, with a corresponding change in price. In this way, the proposed module seeks to satisfy both the client by ensuring quality of service, and the provider by ensuring the best use of resources at a fair price.

Performance Evaluation of Resource Management in Cloud Computing Environments

PubMed Central

Batista, Bruno Guazzelli; Estrella, Julio Cezar; Ferreira, Carlos Henrique Gomes; Filho, Dionisio Machado Leite; Nakamura, Luis Hideo Vasconcelos; Reiff-Marganiec, Stephan; Santana, Marcos José; Santana, Regina Helena Carlucci

2015-01-01

Cloud computing is a computational model in which resource providers can offer on-demand services to clients in a transparent way. However, to be able to guarantee quality of service without limiting the number of accepted requests, providers must be able to dynamically manage the available resources so that they can be optimized. This dynamic resource management is not a trivial task, since it involves meeting several challenges related to workload modeling, virtualization, performance modeling, deployment and monitoring of applications on virtualized resources. This paper carries out a performance evaluation of a module for resource management in a cloud environment that includes handling available resources during execution time and ensuring the quality of service defined in the service level agreement. An analysis was conducted of different resource configurations to define which dimension of resource scaling has a real influence on client requests. The results were used to model and implement a simulated cloud system, in which the allocated resource can be changed on-the-fly, with a corresponding change in price. In this way, the proposed module seeks to satisfy both the client by ensuring quality of service, and the provider by ensuring the best use of resources at a fair price. PMID:26555730

Minnesota urban partnership agreement national evaluation : safety data test plan.

DOT National Transportation Integrated Search

2009-11-17

This report provides the safety data test plan for the Minnesota Urban Partnership Agreement (UPA) under the United States Department of Transportation (U.S. DOT) UPA Program. The Minnesota UPA projects focus on reducing congestion by employing strat...

Virginia Transit Performance Evaluation Package (VATPEP).

DOT National Transportation Integrated Search

1987-01-01

The Virginia Transit Performance Evaluation Package (VATPEP), a computer software package, is documented. This is the computerized version of the methodology used by the Virginia Department of Transportation to evaluate the performance of public tran...

Quality performance of laboratory testing in pharmacies: a collaborative evaluation.

PubMed

Zaninotto, Martina; Miolo, Giorgia; Guiotto, Adriano; Marton, Silvia; Plebani, Mario

2016-11-01

The quality performance and the comparability between results of pharmacies point-of-care-testing (POCT) and institutional laboratories have been evaluated. Eight pharmacies participated in the project: a capillary specimen collected by the pharmacist and, simultaneously, a lithium-heparin sample drawn by a physician of laboratory medicine for the pharmacy customers (n=106) were analyzed in the pharmacy and in the laboratory, respectively. Glucose, cholesterol, HDL-cholesterol, triglycerides, creatinine, uric acid, aspartate aminotransferase, alanine aminotransferase, were measured using: Reflotron, n=5; Samsung, n=1; Cardiocheck PA, n=1; Cholestech LDX, n=1 and Cobas 8000. The POCT analytical performance only (phase 2) were evaluated testing, in pharmacies and in the laboratory, the lithium heparin samples from a female drawn fasting daily in a week, and a control sample containing high concentrations of glucose, cholesterol and triglycerides. For all parameters, except triglycerides, the slopes showed a satisfactory correlation. For triglycerides, a median value higher in POCT in comparison to the laboratory (1.627 mmol/L vs. 0.950 mmol/L) has been observed. The agreement in the subjects classification, demonstrates that for glucose, 70% of the subjects show concentrations below the POCT recommended level (5.8-6.1 mmol/L), while 56% are according to the laboratory limit (<5.6 mmol/L). Total cholesterol exhibits a similar trend while POCT triglycerides show a greater percentage of increased values (21% vs. 9%). The reduction in triglycerides bias (phase 2) suggests that differences between POCT and central laboratory is attributable to a pre-analytical problem. The results confirm the acceptable analytical performance of POCT pharmacies and specific criticisms in the pre- and post-analytical phases.

S-NPP ATMS Instrument Prelaunch and On-Orbit Performance Evaluation

NASA Technical Reports Server (NTRS)

Kim, Edward; Lyu, Cheng-Hsuan; Anderson, Kent; Leslie, Vincent R.; Blackwell, William J.

2014-01-01

The first of a new generation of microwave sounders was launched aboard the Suomi-National Polar-Orbiting Partnership satellite in October 2011. The Advanced Technology Microwave Sounder (ATMS) combines the capabilities and channel sets of three predecessor sounders into a single package to provide information on the atmospheric vertical temperature and moisture profiles that are the most critical observations needed for numerical weather forecast models. Enhancements include size/mass/power approximately one third of the previous total, three new sounding channels, the first space-based, Nyquist-sampled cross-track microwave temperature soundings for improved fusion with infrared soundings, plus improved temperature control and reliability. This paper describes the ATMS characteristics versus its predecessor, the advanced microwave sounding unit (AMSU), and presents the first comprehensive evaluation of key prelaunch and on-orbit performance parameters. Two-year on-orbit performance shows that the ATMS has maintained very stable radiometric sensitivity, in agreement with prelaunch data, meeting requirements for all channels (with margins of 40% for channels 1-15), and improvements over AMSU-A when processed for equivalent spatial resolution. The radiometric accuracy, determined by analysis from ground test measurements, and using on-orbit instrument temperatures, also shows large margins relative to requirements (specified as <1.0K for channels 1, 2, and 16-22 and <0.75 K for channels 3-15). A thorough evaluation of the performance of ATMS is especially important for this first proto-flight model unit of what will eventually be a series of ATMS sensors providing operational sounding capability for the U.S. and its international partners well into the next decade.

Evaluation of audit-based performance measures for dental care plans.

PubMed

Bader, J D; Shugars, D A; White, B A; Rindal, D B

1999-01-01

Although a set of clinical performance measures, i.e., a report card for dental plans, has been designed for use with administrative data, most plans do not have administrative data systems containing the data needed to calculate the measures. Therefore, we evaluated the use of a set of proxy clinical performance measures calculated from data obtained through chart audits. Chart audits were conducted in seven dental programs--three public health clinics, two dental health maintenance organizations (DHMO), and two preferred provider organizations (PPO). In all instances audits were completed by clinical staff who had been trained using telephone consultation and a self-instructional audit manual. The performance measures were calculated for the seven programs, audit reliability was assessed in four programs, and for one program the audit-based proxy measures were compared to the measures calculated using administrative data. The audit-based measures were sensitive to known differences in program performance. The chart audit procedures yielded reasonably reliable data. However, missing data in patient charts rendered the calculation of some measures problematic--namely, caries and periodontal disease assessment and experience. Agreement between administrative and audit-based measures was good for most, but not all, measures in one program. The audit-based proxy measures represent a complex but feasible approach to the calculation of performance measures for those programs lacking robust administrative data systems. However, until charts contain more complete diagnostic information (i.e., periodontal charting and diagnostic codes or reason-for-treatment codes), accurate determination of these aspects of clinical performance will be difficult.

Observer Agreement for Measurements in Videolaryngostroboscopy.

PubMed

Brunings, Jan Wouter; Vanbelle, Sophie; Akkermans, Annemarie; Heemskerk, Nienke M M; Kremer, Bernd; Stokroos, Robert J; Baijens, Laura W J

2017-11-06

This study evaluated the levels of intraobserver and interobserver agreement for measurements of visuoperceptual variables in videolaryngostroboscopic examinations and compared the observers' behavior during independent versus consensus panel rating. This is a retrospective study. This study was conducted in a single-center tertiary care facility. Sixty-four patients with dysphonia of heterogeneous etiology were included. All subjects underwent a standardized videolaryngostroboscopic examination. Two experienced and trained observers scored exactly the same examinations, first independently and then on a consensus panel. Specific visuoperceptual variables and the clinical diagnosis (as recommended by the Committee on Phoniatrics and the Phonosurgery Committee of the European Laryngological Society and advised by the American Speech-Language-Hearing Association) were scored. Descriptive and kappa statistics were used. In general, intraobserver agreement was better than agreement between observers for measurements of several variables. The intrapanel observer agreement levels were slightly higher than the intraobserver agreement levels on the independent rating task. When rating on the consensus panel, the observers deviated considerably from the scores they had previously given on the independent rating task. Observer agreement in videolaryngostroboscopic assessment has important implications not only for the diagnosis and treatment of dysphonic patients but also for the interpretation of the results of scientific studies using videolaryngostroboscopic outcome parameters. The identification of factors that can influence the levels of observer agreement can provide a better understanding of the rating process and its limitations. The results of this study suggest that future research could achieve better agreement levels by rating the visuoperceptual variables in a panel setting. Copyright © 2017 The Voice Foundation. Published by Elsevier Inc. All rights reserved.

48 CFR 436.201 - Evaluation of contractor performance.

Code of Federal Regulations, 2010 CFR

2010-10-01

... Construction 436.201 Evaluation of contractor performance. Preparation of performance evaluation reports. In addition to the requirements of FAR 36.201, performance evaluation reports shall be prepared for indefinite... of services to be ordered exceeds $500,000.00. For these contracts, performance evaluation reports...

High Agreement and High Prevalence: The Paradox of Cohen's Kappa.

PubMed

Zec, Slavica; Soriani, Nicola; Comoretto, Rosanna; Baldi, Ileana

2017-01-01

Cohen's Kappa is the most used agreement statistic in literature. However, under certain conditions, it is affected by a paradox which returns biased estimates of the statistic itself. The aim of the study is to provide sufficient information which allows the reader to make an informed choice of the correct agreement measure, by underlining some optimal properties of Gwet's AC1 in comparison to Cohen's Kappa, using a real data example. During the process of literature review, we have asked a panel of three evaluators to come up with a judgment on the quality of 57 randomized controlled trials assigning a score to each trial using the Jadad scale. The quality was evaluated according to the following dimensions: adopted design, randomization unit, type of primary endpoint. With respect to each of the above described features, the agreement between the three evaluators has been calculated using Cohen's Kappa statistic and Gwet's AC1 statistic and, finally, the values have been compared with the observed agreement. The values of the Cohen's Kappa statistic would lead to believe that the agreement levels for the variables Unit, Design and Primary Endpoints are totally unsatisfactory. The AC1 statistic, on the contrary, shows plausible values which are in line with the respective values of the observed concordance. We conclude that it would always be appropriate to adopt the AC1 statistic, thus bypassing any risk of incurring the paradox and drawing wrong conclusions about the results of agreement analysis.

48 CFR 1252.216-72 - Performance evaluation plan.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 48 Federal Acquisition Regulations System 5 2010-10-01 2010-10-01 false Performance evaluation....216-72 Performance evaluation plan. As prescribed in (TAR) 48 CFR 1216.406(b), insert the following clause: Performance Evaluation Plan (OCT 1994) (a) A Performance Evaluation Plan shall be unilaterally...

A Time for Flexible Donor Agreements.

ERIC Educational Resources Information Center

Fischer, Gerald B.

2003-01-01

Discusses why volatile markets and new donor expectations make now a good time to rework payout rates and gift agreements to bolster financial and strategic performance. Suggests seven options for action. (EV)

Evaluation of Calibration Laboratories Performance

NASA Astrophysics Data System (ADS)

Filipe, Eduarda

2011-12-01

One of the main goals of interlaboratory comparisons (ILCs) is the evaluation of the laboratories performance for the routine calibrations they perform for the clients. In the frame of Accreditation of Laboratories, the national accreditation boards (NABs) in collaboration with the national metrology institutes (NMIs) organize the ILCs needed to comply with the requirements of the international accreditation organizations. In order that an ILC is a reliable tool for a laboratory to validate its best measurement capability (BMC), it is needed that the NMI (reference laboratory) provides a better traveling standard—in terms of accuracy class or uncertainty—than the laboratories BMCs. Although this is the general situation, there are cases where the NABs ask the NMIs to evaluate the performance of the accredited laboratories when calibrating industrial measuring instruments. The aim of this article is to discuss the existing approaches for the evaluation of ILCs and propose a basis for the validation of the laboratories measurement capabilities. An example is drafted with the evaluation of the results of mercury-in-glass thermometers ILC with 12 participant laboratories.

Robust ECC-based authenticated key agreement scheme with privacy protection for Telecare medicine information systems.

PubMed

Zhang, Liping; Zhu, Shaohui

2015-05-01

To protect the transmission of the sensitive medical data, a secure and efficient authenticated key agreement scheme should be deployed when the healthcare delivery session is established via Telecare Medicine Information Systems (TMIS) over the unsecure public network. Recently, Islam and Khan proposed an authenticated key agreement scheme using elliptic curve cryptography for TMIS. They claimed that their proposed scheme is provably secure against various attacks in random oracle model and enjoys some good properties such as user anonymity. In this paper, however, we point out that any legal but malicious patient can reveal other user's identity. Consequently, their scheme suffers from server spoofing attack and off-line password guessing attack. Moreover, if the malicious patient performs the same time of the registration as other users, she can further launch the impersonation attack, man-in-the-middle attack, modification attack, replay attack, and strong replay attack successfully. To eliminate these weaknesses, we propose an improved ECC-based authenticated key agreement scheme. Security analysis demonstrates that the proposed scheme can resist various attacks and enables the patient to enjoy the remote healthcare services with privacy protection. Through the performance evaluation, we show that the proposed scheme achieves a desired balance between security and performance in comparisons with other related schemes.

Agency Agreements Process Champion Support Intern

NASA Technical Reports Server (NTRS)

Miksa, Ember

2018-01-01

This document will provide information on the 2018 Spring semester NIFS Intern who represented the Office of Chief Financial Officer (OCFO) as a Reimbursable Accountant at Kennedy Space Center (KSC). This intern supported the Agency Agreements Process Champions and Team Lead, Susan Kroskey, Sandy Massey and Mecca Murphy, with major initiatives to advance the KSC OCFO's vision of creating and innovating healthy financial management practices that maximize the value of resources entrusted to NASA. These initiatives include, but are not limited to: updating the Agency Guidance and NASA Procedural Guidance 9090.1 Agreements, implementing a new budget structure to be utilized across all centers, submitting a Call Request (CRQ) to enhance non-federal customer reporting, initiating a discussion to incorporate a 3-year funding program for NASA agreements, and undertaking the Office of Inspector General (OIG) Audit. In support of these initiatives, this intern identified technical methods to enhance and reduce the workload of financial processes for reimbursable and non-reimbursable agreements, prepared reports in support of accounting functions, and performed administrative work and miscellaneous technical tasks in support of the OCFO as requested. In conclusion of the internship, the intern will become knowledgeable on reimbursable accounting, reimbursable policy, types of reimbursable agreements, the agreements process, estimated pricing reports, and the roles and responsibilities of the Financial Accounting and Financial Services offices.

48 CFR 236.201 - Evaluation of contractor performance.

Code of Federal Regulations, 2010 CFR

2010-10-01

... CONTRACTS Special Aspects of Contracting for Construction 236.201 Evaluation of contractor performance. (a) Preparation of performance evaluation reports. Use DD Form 2626, Performance Evaluation (Construction... 48 Federal Acquisition Regulations System 3 2010-10-01 2010-10-01 false Evaluation of contractor...

Conductor gestures influence evaluations of ensemble performance

PubMed Central

Morrison, Steven J.; Price, Harry E.; Smedley, Eric M.; Meals, Cory D.

2014-01-01

Previous research has found that listener evaluations of ensemble performances vary depending on the expressivity of the conductor’s gestures, even when performances are otherwise identical. It was the purpose of the present study to test whether this effect of visual information was evident in the evaluation of specific aspects of ensemble performance: articulation and dynamics. We constructed a set of 32 music performances that combined auditory and visual information and were designed to feature a high degree of contrast along one of two target characteristics: articulation and dynamics. We paired each of four music excerpts recorded by a chamber ensemble in both a high- and low-contrast condition with video of four conductors demonstrating high- and low-contrast gesture specifically appropriate to either articulation or dynamics. Using one of two equivalent test forms, college music majors and non-majors (N = 285) viewed sixteen 30 s performances and evaluated the quality of the ensemble’s articulation, dynamics, technique, and tempo along with overall expressivity. Results showed significantly higher evaluations for performances featuring high rather than low conducting expressivity regardless of the ensemble’s performance quality. Evaluations for both articulation and dynamics were strongly and positively correlated with evaluations of overall ensemble expressivity. PMID:25104944

Intra- and interobserver agreement for fetal cerebral measurements in 3D-ultrasonography.

PubMed

Albers, Maria E W A; Buisman, Erato T I A; Kahn, René S; Franx, Arie; Onland-Moret, N Charlotte; de Heus, Roel

2018-04-10

The aim of this study is to evaluate intra- and interobserver agreement for measurement of intracranial, cerebellar, and thalamic volume with the Virtual Organ Computer-aided AnaLysis (VOCAL) technique in three-dimensional ultrasound images, in comparison to two-dimensional measurements of these brain structures. Three-dimensional ultrasound images of the brains of 80 fetuses at 20-24 weeks' gestational age were obtained from YOUth, a Dutch prospective cohort study. Two observers performed offline measurement of the occipitofrontal diameter, intracranial volume, transcerebellar diameter, cerebellar volume, and thalamic width, area, and volume, independently. VOCAL was used for calculation of the volumes. The two-way random, single measures intraclass correlation coefficient (ICC) was used for analysis of agreement and Bland-Altman plots were configured. Intra- and interobserver agreement was almost perfect for occipitofrontal diameter (intra ICC 0.88, 95% CI 0.82-0.92; inter ICC 0.91, 95% CI 0.85-0.94), intracranial volume (intra ICC 0.96, 95% CI 0.91-0.98; inter ICC 0.97, 95% CI 0.96-0.98) and transcerebellar diameter (intra ICC 0.91, 95% CI 0.86-0.94; inter ICC 0.86, 95% CI 0.78-0.910). For cerebellar volume, the intraobserver agreement was almost perfect (0.85, 95% CI 0.76-0.90), whereas the interobserver agreement was substantial (0.75, 95% CI 0.44-0.88). Agreement was only moderate for thalamic measurements. Bland-Altman plots for the volume measurements are normally distributed with acceptable mean differences and 95% limits of agreement. The intra- and interobserver agreement of the measurement of intracranial and cerebellar volume with VOCAL was almost perfect. These measurements are therefore reliable, and can be used to investigate fetal brain development. Thalamic measurements are not reliable enough. © 2018 Wiley Periodicals, Inc.

What principles should govern the use of managed entry agreements?

PubMed

Klemp, Marianne; Frønsdal, Katrine B; Facey, Karen

2011-01-01

To ensure rapid access to new potentially beneficial health technologies, obtain best value for money, and ensure affordability, healthcare payers are adopting a range of innovative reimbursement approaches that may be called Managed Entry Agreements (MEAs). The Health Technology Assessment International (HTAi) Policy Forum sought to identify why MEAs might be used, issues associated with implementation and develop principles for their use. A 2-day deliberative workshop discussed key papers, members' experiences, and collectively addressed four policy questions that resulted in this study. MEAs are used to give access to new technologies where traditional reimbursement is deemed inappropriate. Three different forms of MEAs have been identified: management of budget impact, management of uncertainty relating to clinical and/or cost-effectiveness, and management of utilization to optimize performance. The rationale for using these approaches and their advantages and disadvantages differ. However, all forms of MEA should take the form of a formal written agreement among stakeholders, clearly identifying the rationale for the agreement, aspects to be assessed, methods of data collection and review, and the criteria for ending the agreement. MEAs should only be used when HTA identifies issues or concerns about key outcomes and/or costs and/or organizational/budget impacts that are material to a reimbursement decision. They provide patient access and can be useful to manage technology diffusion and optimize use. However, they are administratively complex and may be difficult to negotiate and their effectiveness has yet to be evaluated.

Multiple-Respondent Anecdotal Assessments: An Analysis of Interrater Agreement and Correspondence with Analogue Assessment Outcomes

ERIC Educational Resources Information Center

Smith, Carla M.; Smith, Richard G.; Dracobly, Joseph D.; Pace, Amy Peterson

2012-01-01

We evaluated interrater agreement across multiple respondents on anecdotal assessments and compared cases in which agreement was obtained with outcomes of functional analyses. Experiment 1 evaluated agreement among multiple respondents on the function of problem behavior for 27 individuals across 42 target behaviors using the Motivation Assessment…

Agreement among Magnetic Resonance Imaging/Magnetic Resonance Cholangiopancreatography (MRI-MRCP) and Endoscopic Ultrasound (EUS) in the evaluation of morphological features of Branch Duct Intraductal Papillary Mucinous Neoplasm (BD-IPMN).

PubMed

Uribarri-Gonzalez, Laura; Keane, Margaret G; Pereira, Stephen P; Iglesias-García, Julio; Dominguez-Muñoz, J Enrique; Lariño-Noia, Jose

2018-03-01

To evaluate the agreement between the imaging modalities MRI-MRCP and EUS in cystic lesions of the pancreas which were thought to be a BD-IPMN. Multicenter retrospective study included all patients between 2010 and 2015 with a suspected BD-IPMN who underwent an EUS and MRI-MRCP within 6 months or less of each other. Location, number, size, worrisome features and high-risk stigmata were evaluated. Interobserver agreement was evaluated by Kappa score. 173 patients were included (97 UHSC, 76 UCLH-RFH), mean age 65 (range 25-87 years), 66 males. When comparing both modalities there was good agreement for the location of the cyst. The median lesion size was larger by MRI-MRCP than EUS although it was not significant. With regards to worrisome features, there was moderate agreement for main PD of 5-9 mm and abrupt change (k = 0.45 and 0.52). Fair agreement was seen for the cyst wall thickening (k = 0.25). No agreement was seen between the presence of non-enhanced mural nodules or lymphadenopathy (k < 0). With regards to high-risk stigmata, poor agreement was obtained for the detection of an enhanced solid component (k = 0.12). No agreement was observed for main PD > 10 mm (k < 0). In this multicentre study of patients with a BD-IPMN under active surveillance, most disagreement between these modalities was seen in the proximal pancreas. There was generally only minimal concordance between the imaging findings of EUS and MRI-MRCP for the detection of high-risk stigmata and worrisome features. Copyright © 2018 IAP and EPC. All rights reserved.

48 CFR 3052.216-72 - Performance evaluation plan.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 48 Federal Acquisition Regulations System 7 2010-10-01 2010-10-01 false Performance evaluation... CONTRACT CLAUSES Text of Provisions and Clauses 3052.216-72 Performance evaluation plan. As prescribed in... Evaluation Plan (DEC 2003) (a) A Performance Evaluation Plan shall be unilaterally established by the...

Performance Evaluation Process.

ERIC Educational Resources Information Center

1998

This document contains four papers from a symposium on the performance evaluation process and human resource development (HRD). "Assessing the Effectiveness of OJT (On the Job Training): A Case Study Approach" (Julie Furst-Bowe, Debra Gates) is a case study of the effectiveness of OJT in one of a high-tech manufacturing company's product…

44 CFR 208.22 - Preparedness Cooperative Agreement process.

Code of Federal Regulations, 2010 CFR

2010-10-01

... and supporting documentation to be submitted with the application. (2) Period of performance. Absent special circumstances, the period of performance for Preparedness Cooperative Agreements will be 1 year from the date of award. The Assistance Officer may allow for an alternate period of performance with...

Indicators for evaluating cancer organizations' support services: performance and associations with empowerment.

PubMed

Lauzier, Sophie; Campbell, H Sharon; Livingston, Patricia M; Maunsell, Elizabeth

2014-10-15

Community-based cancer organizations provide services to support patients. An anticipated benefit of these services is patient empowerment. However, this outcome has not been evaluated because of the lack of validated health-related empowerment questionnaires in the cancer context. In this validation study, the authors assessed the extent to which 16 indicators used by the Canadian Cancer Society (CCS) and the Cancer Council Victoria, Australia (CCV) to evaluate their services were associated with health-related empowerment. Cancer patients/survivors who were diagnosed < 3 years earlier and who used CCS programs completed a questionnaire that included the 16 CCS-CCV indicators and 5 scales from the Health Education Impact Questionnaire (heiQ) measuring key dimensions of empowerment. To determine whether the CCS-CCV indicators captured empowerment, differences in heiQ scores were compared between 2 groups: those with higher levels of agreement (agreeing or agreeing strongly) with an indicator and those with lower levels of agreement (agreeing slightly or disagreeing to any degree). Participation was 72% (207 of 289 eligible CCS users). Compared with participants who had lower levels of agreement on CCS-CCV indicators, those who had higher levels of agreement were more likely to report higher levels of empowerment on the different heiQ scales. For 15 of 16 indicators, these differences were significant (Wilcoxon rank-sum test; P < .10) on ≥ 1 of 5 heiQ scales and for 10 of 16 indicators on ≥ 3 of 5 heiQ scales. Two indicators were associated significantly with all 5 heiQ scales (cope better and feel more in control). Using CCS-CCV indicators to evaluate community-based cancer organizations' services will help determine whether these services are reaching one of their important goals: namely empowering patients. © 2014 American Cancer Society.

The Context and Process for Performance Evaluations: Necessary Preconditions for the Use of Performance Evaluations as a Measure of Performance--A Critique of Perry

ERIC Educational Resources Information Center

McCarthy, Mary L.

2006-01-01

This article challenges Perry's research using performance evaluations to determine whether the educational background of child welfare workers is predictive of performance. Institutional theory, an understanding of street-level bureaucracies, and evaluations of field education performance measures are offered as necessary frameworks for Perry's…

The health impact of trade and investment agreements: a quantitative systematic review and network co-citation analysis.

PubMed

Barlow, Pepita; McKee, Martin; Basu, Sanjay; Stuckler, David

2017-03-08

Regional trade agreements are major international policy instruments that shape macro-economic and political systems. There is widespread debate as to whether and how these agreements pose risks to public health. Here we perform a comprehensive systematic review of quantitative studies of the health impact of trade and investment agreements. We identified studies from searches in PubMed, Web of Science, EMBASE, and Global Health Online. Research articles were eligible for inclusion if they were quantitative studies of the health impacts of trade and investment agreements or policy. We systematically reviewed study findings, evaluated quality using the Quality Assessment Tool from the Effective Public Health Practice Project, and performed network citation analysis to study disciplinary siloes. Seventeen quantitative studies met our inclusion criteria. There was consistent evidence that implementing trade agreements was associated with increased consumption of processed foods and sugar-sweetened beverages. Granting import licenses for patented drugs was associated with increased access to pharmaceuticals. Implementing trade agreements and associated policies was also correlated with higher cardiovascular disease incidence and higher Body Mass Index (BMI), whilst correlations with tobacco consumption, under-five mortality, maternal mortality, and life expectancy were inconclusive. Overall, the quality of studies is weak or moderately weak, and co-citation analysis revealed a relative isolation of public health from economics. We identified limitations in existing studies which preclude definitive conclusions of the health impacts of regional trade and investment agreements. Few address unobserved confounding, and many possible consequences and mechanisms linking trade and investment agreements to health remain poorly understood. Results from our co-citation analysis suggest scope for greater interdisciplinary collaboration. Notwithstanding these limitations, our

Statistical Validation for Clinical Measures: Repeatability and Agreement of Kinect™-Based Software

PubMed Central

Tello, Emanuel; Rodrigo, Alejandro; Valentinuzzi, Max E.

2018-01-01

Background The rehabilitation process is a fundamental stage for recovery of people's capabilities. However, the evaluation of the process is performed by physiatrists and medical doctors, mostly based on their observations, that is, a subjective appreciation of the patient's evolution. This paper proposes a tracking platform of the movement made by an individual's upper limb using Kinect sensor(s) to be applied for the patient during the rehabilitation process. The main contribution is the development of quantifying software and the statistical validation of its performance, repeatability, and clinical use in the rehabilitation process. Methods The software determines joint angles and upper limb trajectories for the construction of a specific rehabilitation protocol and quantifies the treatment evolution. In turn, the information is presented via a graphical interface that allows the recording, storage, and report of the patient's data. For clinical purposes, the software information is statistically validated with three different methodologies, comparing the measures with a goniometer in terms of agreement and repeatability. Results The agreement of joint angles measured with the proposed software and goniometer is evaluated with Bland-Altman plots; all measurements fell well within the limits of agreement, meaning interchangeability of both techniques. Additionally, the results of Bland-Altman analysis of repeatability show 95% confidence. Finally, the physiotherapists' qualitative assessment shows encouraging results for the clinical use. Conclusion The main conclusion is that the software is capable of offering a clinical history of the patient and is useful for quantification of the rehabilitation success. The simplicity, low cost, and visualization possibilities enhance the use of the software Kinect for rehabilitation and other applications, and the expert's opinion endorses the choice of our approach for clinical practice. Comparison of the new measurement

Statistical Validation for Clinical Measures: Repeatability and Agreement of Kinect™-Based Software.

PubMed

Lopez, Natalia; Perez, Elisa; Tello, Emanuel; Rodrigo, Alejandro; Valentinuzzi, Max E

2018-01-01

The rehabilitation process is a fundamental stage for recovery of people's capabilities. However, the evaluation of the process is performed by physiatrists and medical doctors, mostly based on their observations, that is, a subjective appreciation of the patient's evolution. This paper proposes a tracking platform of the movement made by an individual's upper limb using Kinect sensor(s) to be applied for the patient during the rehabilitation process. The main contribution is the development of quantifying software and the statistical validation of its performance, repeatability, and clinical use in the rehabilitation process. The software determines joint angles and upper limb trajectories for the construction of a specific rehabilitation protocol and quantifies the treatment evolution. In turn, the information is presented via a graphical interface that allows the recording, storage, and report of the patient's data. For clinical purposes, the software information is statistically validated with three different methodologies, comparing the measures with a goniometer in terms of agreement and repeatability. The agreement of joint angles measured with the proposed software and goniometer is evaluated with Bland-Altman plots; all measurements fell well within the limits of agreement, meaning interchangeability of both techniques. Additionally, the results of Bland-Altman analysis of repeatability show 95% confidence. Finally, the physiotherapists' qualitative assessment shows encouraging results for the clinical use. The main conclusion is that the software is capable of offering a clinical history of the patient and is useful for quantification of the rehabilitation success. The simplicity, low cost, and visualization possibilities enhance the use of the software Kinect for rehabilitation and other applications, and the expert's opinion endorses the choice of our approach for clinical practice. Comparison of the new measurement technique with established

Evaluation of the 95% limits of agreement of the volumes of 5-year clinically stable solid nodules for the development of a follow-up system for indeterminate solid nodules in CT lung cancer screening

PubMed Central

Muramatsu, Yukio; Yamamichi, Junta; Gomi, Shiho; Oubel, Estanislao; Moriyama, Noriyuki

2018-01-01

Background This study sought to evaluate the 95% limits of agreement of the volumes of 5-year clinically stable solid nodules for the development of a follow-up system for indeterminate solid nodules. Methods The volumes of 226 solid nodules that had been clinically stable for 5 years were measured in 186 patients (53 female never-smokers, 36 male never-smokers, 51 males with <30 pack-years, and 46 males with ≥30 pack-years) using a three-dimensional semiautomated method. Volume changes were evaluated using three methods: percent change, proportional change and growth rate. The 95% limits of agreement were evaluated using the Bland-Altman method. Results The 95% limits of agreement were as follows: range of percent change, from ±34.5% to ±37.8%; range of proportional change, from ±34.1% to ±36.8%; and range of growth rate, from ±39.2% to ±47.4%. Percent change-based, proportional change-based, and growth rate-based diagnoses of an increase or decrease in ten solid nodules were made at a mean of 302±402, 367±455, and 329±496 days, respectively, compared with a clinical diagnosis made at 809±616 days (P<0.05). Conclusions The 95% limits of agreement for volume change in 5-year stable solid nodules may enable the detection of an increase or decrease in the solid nodule at an earlier stage than that enabled by a clinical diagnosis, possibly contributing to the development of a follow-up system for reducing the number of additional Computed tomography (CT) scans performed during the follow-up period. PMID:29600047

Evaluation of the 95% limits of agreement of the volumes of 5-year clinically stable solid nodules for the development of a follow-up system for indeterminate solid nodules in CT lung cancer screening.

PubMed

Kakinuma, Ryutaro; Muramatsu, Yukio; Yamamichi, Junta; Gomi, Shiho; Oubel, Estanislao; Moriyama, Noriyuki

2018-01-01

This study sought to evaluate the 95% limits of agreement of the volumes of 5-year clinically stable solid nodules for the development of a follow-up system for indeterminate solid nodules. The volumes of 226 solid nodules that had been clinically stable for 5 years were measured in 186 patients (53 female never-smokers, 36 male never-smokers, 51 males with <30 pack-years, and 46 males with ≥30 pack-years) using a three-dimensional semiautomated method. Volume changes were evaluated using three methods: percent change, proportional change and growth rate. The 95% limits of agreement were evaluated using the Bland-Altman method. The 95% limits of agreement were as follows: range of percent change, from ±34.5% to ±37.8%; range of proportional change, from ±34.1% to ±36.8%; and range of growth rate, from ±39.2% to ±47.4%. Percent change-based, proportional change-based, and growth rate-based diagnoses of an increase or decrease in ten solid nodules were made at a mean of 302±402, 367±455, and 329±496 days, respectively, compared with a clinical diagnosis made at 809±616 days (P<0.05). The 95% limits of agreement for volume change in 5-year stable solid nodules may enable the detection of an increase or decrease in the solid nodule at an earlier stage than that enabled by a clinical diagnosis, possibly contributing to the development of a follow-up system for reducing the number of additional Computed tomography (CT) scans performed during the follow-up period.

Handbook for Improving Superintendent Performance Evaluation.

ERIC Educational Resources Information Center

Candoli, Carl; And Others

This handbook for superintendent performance evaluation contains information for boards of education as they institute or improve their evaluation system. The handbook answers questions involved in operationalizing, implementing, and evaluating a superintendent-evaluation system. The information was developed from research on superintendent…

Model Performance Evaluation and Scenario Analysis (MPESA) Tutorial

EPA Science Inventory

This tool consists of two parts: model performance evaluation and scenario analysis (MPESA). The model performance evaluation consists of two components: model performance evaluation metrics and model diagnostics. These metrics provides modelers with statistical goodness-of-fit m...

Inter-observer and intra-observer agreement on interpretation of uroflowmetry curves of kindergarten children.

PubMed

Chang, Shang-Jen; Yang, Stephen S D

2008-12-01

To evaluate the inter-observer and intra-observer agreement on the interpretation of uroflowmetry curves of children. Healthy kindergarten children were enrolled for evaluation of uroflowmetry. Uroflowmetry curves were classified as bell-shaped, tower, plateau, staccato and interrupted. Only the bell-shaped curves were regarded as normal. Two urodynamists evaluated the curves independently after reviewing the definitions of the different types of uroflowmetry curve. The senior urodynamist evaluated the curves twice 3 months apart. The final conclusion was made when consensus was reached. Agreement among observers was analyzed using kappa statistics. Of 190 uroflowmetry curves eligible for analysis, the intra-observer agreement in interpreting each type of curve and interpreting normalcy vs abnormality was good (kappa=0.71 and 0.68, respectively). Very good inter-observer agreement (kappa=0.81) on normalcy and good inter-observer agreement (kappa=0.73) on types of uroflowmetry were observed. Poor inter-observer agreement existed on the classification of specific types of abnormal uroflowmetry curves (kappa=0.07). Uroflowmetry is a good screening tool for normalcy of kindergarten children, while not a good tool to define the specific types of abnormal uroflowmetry.

A conceptual framework for evaluation of public health and primary care system performance in iran.

PubMed

Jahanmehr, Nader; Rashidian, Arash; Khosravi, Ardeshir; Farzadfar, Farshad; Shariati, Mohammad; Majdzadeh, Reza; Akbari Sari, Ali; Mesdaghinia, Alireza

2015-01-26

The main objective of this study was to design a conceptual framework, according to the policies and priorities of the ministry of health to evaluate provincial public health and primary care performance and to assess their share in the overall health impacts of the community. We used several tools and techniques, including system thinking, literature review to identify relevant attributes of health system performance framework and interview with the key stakeholders. The PubMed, Scopus, web of science, Google Scholar and two specialized databases of Persian language literature (IranMedex and SID) were searched using main terms and keywords. Following decision-making and collective agreement among the different stakeholders, 51 core indicators were chosen from among 602 obtained indicators in a four stage process, for monitoring and evaluation of Health Deputies. We proposed a conceptual framework by identifying the performance area for Health Deputies between other determinants of health, as well as introducing a chain of results, for performance, consisting of Input, Process, Output and Outcome indicators. We also proposed 5 dimensions for measuring the performance of Health Deputies, consisting of efficiency, effectiveness, equity, access and improvement of health status. The proposed Conceptual Framework illustrates clearly the Health Deputies success in achieving best results and consequences of health in the country. Having the relative commitment of the ministry of health and Health Deputies at the University of Medical Sciences is essential for full implementation of this framework and providing the annual performance report.

The effects of non-evaluative feedback on drivers' self-evaluation and performance.

PubMed

Dogan, Ebru; Steg, Linda; Delhomme, Patricia; Rothengatter, Talib

2012-03-01

Drivers' tend to overestimate their competences, which may result in risk taking behavior. Providing drivers with feedback has been suggested as one of the solutions to overcome drivers' inaccurate self-evaluations. In practice, many tests and driving simulators provide drivers with non-evaluative feedback, which conveys information on the level of performance but not on what caused the performance. Is this type of feedback indeed effective in reducing self-enhancement biases? The current study aimed to investigate the effect of non-evaluative performance feedback on drivers' self-evaluations using a computerized hazard perception test. A between-subjects design was used with one group receiving feedback on performance in the hazard perception test while the other group not receiving any feedback. The results indicated that drivers had a robust self-enhancement bias in their self-evaluations regardless of the presence of performance feedback and that they systematically estimated their performance to be higher than they actually achieved in the test. Furthermore, they devalued the credibility of the test instead of adjusting their self-evaluations in order to cope with the negative feelings following the failure feedback. We discuss the theoretical and practical implications of these counterproductive effects of non-evaluative feedback. Copyright © 2011 Elsevier Ltd. All rights reserved.

Interrater Reliability and Diagnostic Performance of Subjective Evaluation of Sublingual Microcirculation Images by Physicians and Nurses: A Multicenter Observational Study.

PubMed

Lima, Alexandre; López, Alejandra; van Genderen, Michel E; Hurtado, Francisco Javier; Angulo, Martin; Grignola, Juan C; Shono, Atsuko; van Bommel, Jasper

2015-09-01

This was a cross-sectional multicenter study to investigate the ability of physicians and nurses from three different countries to subjectively evaluate sublingual microcirculation images and thereby discriminate normal from abnormal sublingual microcirculation based on flow and density abnormalities. Forty-five physicians and 61 nurses (mean age, 36 ± 10 years; 44 males) from three different centers in The Netherlands (n = 61), Uruguay (n = 12), and Japan (n = 33) were asked to subjectively evaluate a sample of 15 microcirculation videos randomly selected from an experimental model of endotoxic shock in pigs. All videos were first analyzed offline using the A.V.A. software by an independent, experienced investigator and were categorized as good, bad, or very bad microcirculation based on the microvascular flow index, perfused capillary density, and proportion of perfused capillaries. Then, the videos were randomly assigned to the examiners, who were instructed to subjectively categorize each image as good, bad, or very bad. An interrater analysis was performed, and sensitivity and specificity tests were calculated to evaluate the proportion of A.V.A. score abnormalities that the examiners correctly identified. The κ statistics indicated moderate agreement in the evaluation of microcirculation abnormalities using three categories, i.e., good, bad, or very bad (κ = 0.48), and substantial agreement using two categories, i.e., normal (good) and abnormal (bad or very bad) (κ = 0.66). There was no significant difference between the κ three and κ two statistics. We found that the examiner's subjective evaluations had good diagnostic performance and were highly sensitive (84%; 95% confidence interval, 81%-86%) and specific (87%; 95% confidence interval, 84%-90%) for sublingual microcirculatory abnormalities as assessed using the A.V.A. software. The subjective evaluations of sublingual microcirculation by physicians and nurses agreed well with a conventional offline

Assisting allied health in performance evaluation: a systematic review.

PubMed

Lizarondo, Lucylynn; Grimmer, Karen; Kumar, Saravana

2014-11-14

Performance evaluation raises several challenges to allied health practitioners and there is no agreed approach to measuring or monitoring allied health service performance. The aim of this review was to examine the literature on performance evaluation in healthcare to assist in the establishment of a framework that can guide the measurement and evaluation of allied health clinical service performance. This review determined the core elements of a performance evaluation system, tools for evaluating performance, and barriers to the implementation of performance evaluation. A systematic review of the literature was undertaken. Five electronic databases were used to search for relevant articles: MEDLINE, Embase, CINAHL, PsychInfo, and Academic Search Premier. Articles which focussed on any allied health performance evaluation or those which examined performance in health care in general were considered in the review. Content analysis was used to synthesise the findings from individual articles. A total of 37 articles were included in the review. The literature suggests there are core elements involved in performance evaluation which include prioritising clinical areas for measurement, setting goals, selecting performance measures, identifying sources of feedback, undertaking performance measurement, and reporting the results to relevant stakeholders. The literature describes performance evaluation as multi-dimensional, requiring information or data from more than one perspective to provide a rich assessment of performance. A range of tools or instruments are available to capture various perspectives and gather a comprehensive picture of health care quality. Every allied health care delivery system has different performance needs and will therefore require different approaches. However, there are core processes that can be used as a framework to evaluate allied health performance. A careful examination of barriers to performance evaluation and subsequent tailoring of

48 CFR 2936.201 - Evaluation of contractor performance.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 48 Federal Acquisition Regulations System 7 2010-10-01 2010-10-01 false Evaluation of contractor... Construction 2936.201 Evaluation of contractor performance. The HCA must establish procedures to evaluate construction contractor performance and prepare performance reports as required by FAR 36.201. ...

7 CFR 1494.801 - Enforcement and termination of agreements with CCC.

Code of Federal Regulations, 2011 CFR

2011-01-01

... 7 Agriculture 10 2011-01-01 2011-01-01 false Enforcement and termination of agreements with CCC... CCC. (a) Performance in accordance with an Agreement with CCC. (1) An exporter which enters into an Agreement with CCC must ensure that the eligible commodity is exported from the U.S. and enters the eligible...

San Francisco urban partnership agreement, national evaluation : content analysis test plan.

DOT National Transportation Integrated Search

2011-06-01

This report presents the test plan for collecting and analyzing information on outreach activities, media coverage, : and reactions of the public, policy makers, and other groups to the UPA projects for the San Francisco Urban : Partnership Agreement...

48 CFR 2452.216-73 - Performance evaluation plan.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 48 Federal Acquisition Regulations System 6 2010-10-01 2010-10-01 true Performance evaluation plan... 2452.216-73 Performance evaluation plan. As prescribed in 2416.406(e)(3), insert the following clause in all award fee contracts: Performance Evaluation Plan (AUG 1987) (a) The Government shall...

Minnesota urban partnership agreement national evaluation : cost benefit analysis test plan.

DOT National Transportation Integrated Search

2009-11-17

This report presents the cost benefit analysis test plan for the Minnesota Urban Partnership Agreement (UPA) under the United States Department of Transportation (U.S. DOT) UPA Program. The Minnesota UPA projects focus on reducing congestion by emplo...

Minnesota urban partnership agreement national evaluation : traffic system data test plan.

DOT National Transportation Integrated Search

2009-11-17

This report presents the traffic system data test plan for the Minnesota Urban Partnership Agreement (UPA) under the United States Department of Transportation (U.S. DOT) UPA Program. The Minnesota UPA projects focus on reducing congestion by employi...

San Francisco urban partnership agreement, national evaluation : exogenous factors test plan.

DOT National Transportation Integrated Search

2011-06-01

This report presents the test plan for collecting and analyzing exogenous factors data for the San Francisco Urban Partnership Agreement (UPA) under the United States Department of Transportation (U.S. DOT) UPA Program. The San Francisco UPA projects...

San Francisco urban partnership agreement, national evaluation : environmental data test plan.

DOT National Transportation Integrated Search

2011-06-01

This report presents the test plan for collecting and analyzing environmental data for the San Francisco Urban Partnership Agreement (UPA) under the United States Department of Transportation (U.S. DOT) UPA Program. The San Francisco UPA projects foc...

A rater training protocol to assess team performance.

PubMed

Eppich, Walter; Nannicelli, Anna P; Seivert, Nicholas P; Sohn, Min-Woong; Rozenfeld, Ranna; Woods, Donna M; Holl, Jane L

2015-01-01

Simulation-based methodologies are increasingly used to assess teamwork and communication skills and provide team training. Formative feedback regarding team performance is an essential component. While effective use of simulation for assessment or training requires accurate rating of team performance, examples of rater-training programs in health care are scarce. We describe our rater training program and report interrater reliability during phases of training and independent rating. We selected an assessment tool shown to yield valid and reliable results and developed a rater training protocol with an accompanying rater training handbook. The rater training program was modeled after previously described high-stakes assessments in the setting of 3 facilitated training sessions. Adjacent agreement was used to measure interrater reliability between raters. Nine raters with a background in health care and/or patient safety evaluated team performance of 42 in-situ simulations using post-hoc video review. Adjacent agreement increased from the second training session (83.6%) to the third training session (85.6%) when evaluating the same video segments. Adjacent agreement for the rating of overall team performance was 78.3%, which was added for the third training session. Adjacent agreement was 97% 4 weeks posttraining and 90.6% at the end of independent rating of all simulation videos. Rater training is an important element in team performance assessment, and providing examples of rater training programs is essential. Articulating key rating anchors promotes adequate interrater reliability. In addition, using adjacent agreement as a measure allows differentiation between high- and low-performing teams on video review. © 2015 The Alliance for Continuing Education in the Health Professions, the Society for Academic Continuing Medical Education, and the Council on Continuing Medical Education, Association for Hospital Medical Education.

Performance Evaluation Model for Application Layer Firewalls.

PubMed

Xuan, Shichang; Yang, Wu; Dong, Hui; Zhang, Jiangchuan

2016-01-01

Application layer firewalls protect the trusted area network against information security risks. However, firewall performance may affect user experience. Therefore, performance analysis plays a significant role in the evaluation of application layer firewalls. This paper presents an analytic model of the application layer firewall, based on a system analysis to evaluate the capability of the firewall. In order to enable users to improve the performance of the application layer firewall with limited resources, resource allocation was evaluated to obtain the optimal resource allocation scheme in terms of throughput, delay, and packet loss rate. The proposed model employs the Erlangian queuing model to analyze the performance parameters of the system with regard to the three layers (network, transport, and application layers). Then, the analysis results of all the layers are combined to obtain the overall system performance indicators. A discrete event simulation method was used to evaluate the proposed model. Finally, limited service desk resources were allocated to obtain the values of the performance indicators under different resource allocation scenarios in order to determine the optimal allocation scheme. Under limited resource allocation, this scheme enables users to maximize the performance of the application layer firewall.

24 CFR 570.491 - Performance and evaluation report.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 24 Housing and Urban Development 3 2010-04-01 2010-04-01 false Performance and evaluation report... Development Block Grant Program § 570.491 Performance and evaluation report. The annual performance and evaluation report shall be submitted in accordance with 24 CFR part 91. (Approved by the Office of Management...

Performability evaluation of the SIFT computer

NASA Technical Reports Server (NTRS)

Meyer, J. F.; Furchtgott, D. G.; Wu, L. T.

1979-01-01

Performability modeling and evaluation techniques are applied to the SIFT computer as it might operate in the computational evironment of an air transport mission. User-visible performance of the total system (SIFT plus its environment) is modeled as a random variable taking values in a set of levels of accomplishment. These levels are defined in terms of four attributes of total system behavior: safety, no change in mission profile, no operational penalties, and no economic process whose states describe the internal structure of SIFT as well as relavant conditions of the environment. Base model state trajectories are related to accomplishment levels via a capability function which is formulated in terms of a 3-level model hierarchy. Performability evaluation algorithms are then applied to determine the performability of the total system for various choices of computer and environment parameter values. Numerical results of those evaluations are presented and, in conclusion, some implications of this effort are discussed.

Interobserver Agreement and Disagreement in Continuous Recording Exemplified by Measurement of Behavior State.

ERIC Educational Resources Information Center

Mudford, Oliver C.; Hogg, James; Roberts, Jessica

1997-01-01

Continuous observational recording over 57 hours evaluated behavior states of three adults with profound and multiple disabilities. Two independent observers also recorded for 22 hours. Although overall percentage agreement was satisfactory (above 80%), agreement on occurrence was unsatisfactory (mean of 65%). Agreement data were superimposed on…

Franchise Agreements and Clean Energy: Issues in Illinois

EPA Pesticide Factsheets

This project evaluates the impact on energy efficiency of municipal franchise agreements that supply electricity or gas service without a direct charge (unbilled energy) for certain municipal government facilities in Illinois.)

The performance evaluation of WinOSPM model for urban street canyons of Nantes in France.

PubMed

Gokhale, Sharad B; Rebours, Arnaud; Pavageau, Michel

2005-01-01

was done to evaluate the models performance using index of agreement and correlation coefficient. The index of agreement (d) and correlation coefficient (r) for all the streets show that estimated concentration levels are in good agreement with that of measurements. From the index of agreements, it can be inferred that model has very less potential for errors. The model's sensitivity to building-exceptions was also tested for the rue de Strasbourg. Results did not reflect this feature very well. It is perceived that the influence of this feature might have been suppressed in averaging the annual hourly values. This influence is apparently seen in hourly average time series variations. Finally, WinOSPM model was found a simple but very useful model. It could very well represent the detailed flow and dispersion conditions in urban streets.

Performance Evaluation and Community Application of Low-Cost Sensors for Ozone and Nitrogen Dioxide.

PubMed

Duvall, Rachelle M; Long, Russell W; Beaver, Melinda R; Kronmiller, Keith G; Wheeler, Michael L; Szykman, James J

2016-10-13

This study reports on the performance of electrochemical-based low-cost sensors and their use in a community application. CairClip sensors were collocated with federal reference and equivalent methods and operated in a network of sites by citizen scientists (community members) in Houston, Texas and Denver, Colorado, under the umbrella of the NASA-led DISCOVER-AQ Earth Venture Mission. Measurements were focused on ozone (O₃) and nitrogen dioxide (NO₂). The performance evaluation showed that the CairClip O₃/NO₂ sensor provided a consistent measurement response to that of reference monitors (r² = 0.79 in Houston; r² = 0.72 in Denver) whereas the CairClip NO₂ sensor measurements showed no agreement to reference measurements. The CairClip O₃/NO₂ sensor data from the citizen science sites compared favorably to measurements at nearby reference monitoring sites. This study provides important information on data quality from low-cost sensor technologies and is one of few studies that reports sensor data collected directly by citizen scientists.

A joint computational and experimental study to evaluate Inconel-sheathed thermocouple performance in flames.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Brundage, Aaron L.; Nicolette, Vernon F.; Donaldson, A. Burl

2005-09-01

A joint experimental and computational study was performed to evaluate the capability of the Sandia Fire Code VULCAN to predict thermocouple response temperature. Thermocouple temperatures recorded by an Inconel-sheathed thermocouple inserted into a near-adiabatic flat flame were predicted by companion VULCAN simulations. The predicted thermocouple temperatures were within 6% of the measured values, with the error primarily attributable to uncertainty in Inconel 600 emissivity and axial conduction losses along the length of the thermocouple assembly. Hence, it is recommended that future thermocouple models (for Inconel-sheathed designs) include a correction for axial conduction. Given the remarkable agreement between experiment and simulation,more » it is recommended that the analysis be repeated for thermocouples in flames with pollutants such as soot.« less

Performance evaluation of the Aptima HSV-1 and 2 assay for the detection of HSV in cutaneous and mucocutaneous lesion specimens.

PubMed

Sam, Soya S; Caliendo, Angela M; Ingersoll, Jessica; Abdul-Ali, Deborah; Kraft, Colleen S

Timely and precise laboratory diagnosis of Herpes simplex viruses (HSV) is required to guide clinical management. The study evaluated limit of detection (LOD) and performance characteristics of the Aptima HSV 1 & 2 assay in comparison to four assays. The multi-center study compared qualitative detection of HSV-1 and 2 by the Aptima HSV-1 and 2 assay (Hologic) to ELVIS culture, Lyra Direct (Quidel), AmpliVue (Quidel) and a laboratory developed test (LDT). LOD was performed using VTM and STM diluted viral concentrations and clinical performance was evaluated using 505 swab specimens. The Aptima LOD studies performed showed a lower detection limit for STM specimens as 1450 copies/mL and 430 copies/mL for HSV1 and HSV-2 respectively; the LOD for VTM specimens was 9370 copies/mL and 8045 copies/mL for HSV-1 and HSV-2 respectively. When the assays were analyzed based on the positive consensus result established the Aptima had 95% of percent positive agreement (PPA) and 100% negative percent agreement (NPA) for the HSV-1. For the HSV-2, the PPA and NPA for Aptima were 96% and 100% respectively. AmpliVue had 1.8% invalid rate, while Lyra had no invalid results but an inhibition rate of 0.8%. Aptima and LDT did not have any invalid or inhibited results. The results indicate that the Aptima HSV-1 & 2 assay is sensitive and the performance characteristics of the Aptima assay is comparable to the assays analyzed for the detection and differentiation of HSV-1 and 2 from cutaneous and mucocutaneous lesions. Copyright © 2017 Elsevier B.V. All rights reserved.

A Conceptual Framework for Evaluation of Public Health and Primary Care System Performance in Iran

PubMed Central

Jahanmehr, Nader; Rashidian, Arash; Khosravi, Ardeshir; Farzadfar, Farshad; Shariati, Mohammad; Majdzadeh, Reza; Sari, Ali Akbari; Mesdaghinia, Alireza

2015-01-01

Introduction: The main objective of this study was to design a conceptual framework, according to the policies and priorities of the ministry of health to evaluate provincial public health and primary care performance and to assess their share in the overall health impacts of the community. Methods: We used several tools and techniques, including system thinking, literature review to identify relevant attributes of health system performance framework and interview with the key stakeholders. The PubMed, Scopus, web of science, Google Scholar and two specialized databases of Persian language literature (IranMedex and SID) were searched using main terms and keywords. Following decision-making and collective agreement among the different stakeholders, 51 core indicators were chosen from among 602 obtained indicators in a four stage process, for monitoring and evaluation of Health Deputies. Results: We proposed a conceptual framework by identifying the performance area for Health Deputies between other determinants of health, as well as introducing a chain of results, for performance, consisting of Input, Process, Output and Outcome indicators. We also proposed 5 dimensions for measuring the performance of Health Deputies, consisting of efficiency, effectiveness, equity, access and improvement of health status. Conclusion: The proposed Conceptual Framework illustrates clearly the Health Deputies success in achieving best results and consequences of health in the country. Having the relative commitment of the ministry of health and Health Deputies at the University of Medical Sciences is essential for full implementation of this framework and providing the annual performance report. PMID:25946937

Measuring agreement of multivariate discrete survival times using a modified weighted kappa coefficient.

PubMed

Guo, Ying; Manatunga, Amita K

2009-03-01

Assessing agreement is often of interest in clinical studies to evaluate the similarity of measurements produced by different raters or methods on the same subjects. We present a modified weighted kappa coefficient to measure agreement between bivariate discrete survival times. The proposed kappa coefficient accommodates censoring by redistributing the mass of censored observations within the grid where the unobserved events may potentially happen. A generalized modified weighted kappa is proposed for multivariate discrete survival times. We estimate the modified kappa coefficients nonparametrically through a multivariate survival function estimator. The asymptotic properties of the kappa estimators are established and the performance of the estimators are examined through simulation studies of bivariate and trivariate survival times. We illustrate the application of the modified kappa coefficient in the presence of censored observations with data from a prostate cancer study.

Steps to a HealthierUS Cooperative Agreement Program: foundational elements for program evaluation planning, implementation, and use of findings.

PubMed

MacDonald, Goldie; Garcia, Danyael; Zaza, Stephanie; Schooley, Michael; Compton, Don; Bryant, Terry; Bagnol, Lulu; Edgerly, Cathy; Haverkate, Rick

2006-01-01

The Steps to a HealthierUS Cooperative Agreement Program (Steps Program) enables funded communities to implement chronic disease prevention and health promotion efforts to reduce the burden of diabetes, obesity, asthma, and related risk factors. At both the national and community levels, investment in surveillance and program evaluation is substantial. Public health practitioners engaged in program evaluation planning often identify desired outcomes, related indicators, and data collection methods but may pay only limited attention to an overarching vision for program evaluation among participating sites. We developed a set of foundational elements to provide a vision of program evaluation that informs the technical decisions made throughout the evaluation process. Given the diversity of activities across the Steps Program and the need for coordination between national- and community-level evaluation efforts, our recommendations to guide program evaluation practice are explicit yet leave room for site-specific context and needs. Staff across the Steps Program must consider these foundational elements to prepare a formal plan for program evaluation. Attention to each element moves the Steps Program closer to well-designed and complementary plans for program evaluation at the national, state, and community levels.

Formative and Summative Evaluation: Related Issues in Performance Measurement.

ERIC Educational Resources Information Center

Wholey, Joseph S.

1996-01-01

Performance measurement can serve both formative and summative evaluation functions. Formative evaluation is typically more useful for government purposes whereas performance measurement is more useful than one-shot evaluations of either formative or summative nature. Evaluators should study performance measurement through case studies and…

Gender and Agreement Processing in Children with Developmental Language Disorder

ERIC Educational Resources Information Center

Rakhlin, Natalia; Kornilov, Sergey A.; Grigorenko, Elena L.

2014-01-01

Two experiments tested whether Russian-speaking children with Developmental Language Disorder (DLD) are sensitive to gender agreement when performing a gender decision task. In Experiment 1, the presence of overt gender agreement between verbs and/or adjectival modifiers and postverbal subject nouns memory was varied. In Experiment 2, agreement…

Performance evaluation of Louisiana superpave mixtures.

DOT National Transportation Integrated Search

2008-12-01

This report documents the performance of Louisiana Superpave mixtures through laboratory mechanistic tests, mixture : volumetric properties, gradation analysis, and early field performance. Thirty Superpave mixtures were evaluated in this : study. Fo...

Prediction of pump cavitation performance

NASA Technical Reports Server (NTRS)

Moore, R. D.

1974-01-01

A method for predicting pump cavitation performance with various liquids, liquid temperatures, and rotative speeds is presented. Use of the method requires that two sets of test data be available for the pump of interest. Good agreement between predicted and experimental results of cavitation performance was obtained for several pumps operated in liquids which exhibit a wide range of properties. Two cavitation parameters which qualitatively evaluate pump cavitation performance are also presented.

Performance Evaluation of the United Nations Environment Programme Air Quality Monitoring Unit

EPA Science Inventory

A request for technical collaboration between the UNEP and the US EPA resulted in the establishment of a MCRADA. The purpose of this agreement was to evaluate an air quality monitoring system (referred to as the UNEP pod) developed by the UNEP for use in environmental situations ...

46 CFR 298.38 - Partnership agreements and limited liability company agreements.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 46 Shipping 8 2010-10-01 2010-10-01 false Partnership agreements and limited liability company... liability company agreements. Partnership and limited liability company agreements must be in form and...) Duration of the entity; (b) Adequate partnership or limited liability company funding requirements and...

Reliability and agreement in student ratings of the class environment.

PubMed

Nelson, Peter M; Christ, Theodore J

2016-09-01

The current study estimated the reliability and agreement of student ratings of the classroom environment obtained using the Responsive Environmental Assessment for Classroom Teaching (REACT; Christ, Nelson, & Demers, 2012; Nelson, Demers, & Christ, 2014). Coefficient alpha, class-level reliability, and class agreement indices were evaluated as each index provides important information for different interpretations and uses of student rating scale data. Data for 84 classes across 29 teachers in a suburban middle school were sampled to derive reliability and agreement indices for the REACT subscales across 4 class sizes: 25, 20, 15, and 10. All participating teachers were White and a larger number of 6th-grade classes were included (42%) relative to 7th- (33%) or 8th- (23%) grade classes. Teachers were responsible for a variety of content areas, including language arts (26%), science (26%), math (20%), social studies (19%), communications (6%), and Spanish (3%). Coefficient alpha estimates were generally high across all subscales and class sizes (α = .70-.95); class-mean estimates were greatly impacted by the number of students sampled from each class, with class-level reliability values generally falling below .70 when class size was reduced from 25 to 20. Further, within-class student agreement varied widely across the REACT subscales (mean agreement = .41-.80). Although coefficient alpha and test-retest reliability are commonly reported in research with student rating scales, class-level reliability and agreement are not. The observed differences across coefficient alpha, class-level reliability, and agreement indices provide evidence for evaluating students' ratings of the class environment according to their intended use (e.g., differentiating between classes, class-level instructional decisions). (PsycINFO Database Record (c) 2016 APA, all rights reserved).

Computed Tomography Assessment of Hepatic Metastases of Breast Cancer with Revised Response Evaluation Criteria in Solid Tumors (RECIST) Criteria (Version 1.1): Inter-Observer Agreement.

PubMed

Ghobrial, Fady Emil Ibrahim; Eldin, Manal Salah; Razek, Ahmed Abdel Khalek Abdel; Atwan, Nadia Ibrahim; Shamaa, Sameh Sayed Ahmed

2017-01-01

To assess inter-observer agreement of revised RECIST criteria (version 1.1) for computed tomography assessment of hepatic metastases of breast cancer. A prospective study was conducted in 28 female patients with breast cancer and with at least one measurable metastatic lesion in the liver that was treated with 3 cycles of anthracycline-based chemotherapy. All patients underwent computed tomography of the abdomen with 64-row multi- detector CT at baseline and after 3 cycles of chemotherapy for response assessment. Image analysis was performed by 2 observers, based on the RECIST criteria (version 1.1). Computed tomography revealed partial response of hepatic metastases in 7 patients (25%) by one observer and in 10 patients (35.7%) by the other observer, with good inter-observer agreement (k=0.75, percent agreement of 89.29%). Stable disease was detected in 19 patients (67.8%) by one observer and in 16 patients (57.1%) by the other observer, with good agreement (k=0.774, percent agreement of 89.29%). Progressive disease was detected in 2 patients (7.2%) by both observers, with perfect agreement (k=1, percent agreement of 100%). The overall inter-observer agreement in the CT-based response assessment of hepatic metastasis between the two observers was good ( k =0.793, percent agreement of 89.29%). We concluded that computed tomography is a reliable and reproducible imaging modality for response assessment of hepatic metastases of breast cancer according to the RECIST criteria (version 1.1).

Inter- and intra-observer agreement of BI-RADS-based subjective visual estimation of amount of fibroglandular breast tissue with magnetic resonance imaging: comparison to automated quantitative assessment.

PubMed

Wengert, G J; Helbich, T H; Woitek, R; Kapetas, P; Clauser, P; Baltzer, P A; Vogl, W-D; Weber, M; Meyer-Baese, A; Pinker, Katja

2016-11-01

To evaluate the inter-/intra-observer agreement of BI-RADS-based subjective visual estimation of the amount of fibroglandular tissue (FGT) with magnetic resonance imaging (MRI), and to investigate whether FGT assessment benefits from an automated, observer-independent, quantitative MRI measurement by comparing both approaches. Eighty women with no imaging abnormalities (BI-RADS 1 and 2) were included in this institutional review board (IRB)-approved prospective study. All women underwent un-enhanced breast MRI. Four radiologists independently assessed FGT with MRI by subjective visual estimation according to BI-RADS. Automated observer-independent quantitative measurement of FGT with MRI was performed using a previously described measurement system. Inter-/intra-observer agreements of qualitative and quantitative FGT measurements were assessed using Cohen's kappa (k). Inexperienced readers achieved moderate inter-/intra-observer agreement and experienced readers a substantial inter- and perfect intra-observer agreement for subjective visual estimation of FGT. Practice and experience reduced observer-dependency. Automated observer-independent quantitative measurement of FGT was successfully performed and revealed only fair to moderate agreement (k = 0.209-0.497) with subjective visual estimations of FGT. Subjective visual estimation of FGT with MRI shows moderate intra-/inter-observer agreement, which can be improved by practice and experience. Automated observer-independent quantitative measurements of FGT are necessary to allow a standardized risk evaluation. • Subjective FGT estimation with MRI shows moderate intra-/inter-observer agreement in inexperienced readers. • Inter-observer agreement can be improved by practice and experience. • Automated observer-independent quantitative measurements can provide reliable and standardized assessment of FGT with MRI.

Model Performance Evaluation and Scenario Analysis ...

EPA Pesticide Factsheets

This tool consists of two parts: model performance evaluation and scenario analysis (MPESA). The model performance evaluation consists of two components: model performance evaluation metrics and model diagnostics. These metrics provides modelers with statistical goodness-of-fit measures that capture magnitude only, sequence only, and combined magnitude and sequence errors. The performance measures include error analysis, coefficient of determination, Nash-Sutcliffe efficiency, and a new weighted rank method. These performance metrics only provide useful information about the overall model performance. Note that MPESA is based on the separation of observed and simulated time series into magnitude and sequence components. The separation of time series into magnitude and sequence components and the reconstruction back to time series provides diagnostic insights to modelers. For example, traditional approaches lack the capability to identify if the source of uncertainty in the simulated data is due to the quality of the input data or the way the analyst adjusted the model parameters. This report presents a suite of model diagnostics that identify if mismatches between observed and simulated data result from magnitude or sequence related errors. MPESA offers graphical and statistical options that allow HSPF users to compare observed and simulated time series and identify the parameter values to adjust or the input data to modify. The scenario analysis part of the too

Grading of Total Mesorectal Excision Specimens: Assessment of Interrater Agreement.

PubMed

Goebel, Emily A; Stegmaier, Melissa; Gorassini, Donald R; Kubica, Matthew; Parfitt, Jeremy R; Driman, David K

2018-06-01

Total mesorectal excision is the standard of care for patients with rectal cancer. Pathological evaluation of the quality of the total mesorectal excision specimen is an important prognostic factor that correlates with local recurrence, but is potentially subjective. This study aimed to determine the degree of variation in grading, both between assessors and between fresh and formalin-fixed specimens. Raters included surgeons, pathologists, pathology residents, pathologists' assistants, and pathologists' assistant trainees. Specimens were assessed by up to 6 raters in the fresh state and by 2 raters postfixation. Four parameters were evaluated: mesorectal bulk, surface regularity, defects, and coning. Interrater agreement was measured using ordinal α-values. The study was conducted at a single academic center. The primary outcome was agreement between individuals when grading total mesorectal excision specimens. A total of 37 total mesorectal excision specimens were assessed. Reliability between all raters for fresh specimens for mesorectal bulk, surface regularity, defects, coning, and overall grade were 0.85, 0.85, 0.92, 0.84, and 0.91. When compared with all raters, pathologists and residents had higher agreement and pathologists and surgeons had lower agreement. Ordinal α-values comparing pathologist and pathologist's assistant agreement for overall grade were similar pre- and postfixation (0.78 vs 0.80), but agreement for assessing defects decreased postfixation. Among pathologists' assistants, agreement was higher when grading specimens postfixation than when grading fresh specimens. Assessment bias may have occurred because of the greater number of pathologists' assistants participating than the number of residents and pathologists. The results indicate good interrater agreement for the assessment of overall grade, with defects showing the best interrater agreement in fresh specimens. Although total mesorectal excision specimens may be consistently graded

Evaluation of agreement between temporal series obtained from electrocardiogram and pulse wave.

NASA Astrophysics Data System (ADS)

Leikan, GM; Rossi, E.; Sanz, MCuadra; Delisle Rodríguez, D.; Mántaras, MC; Nicolet, J.; Zapata, D.; Lapyckyj, I.; Siri, L. Nicola; Perrone, MS

2016-04-01

Heart rate variability allows to study the cardiovascular autonomic nervous system modulation. Usually, this signal is obtained from the electrocardiogram (ECG). A simpler method for recording the pulse wave (PW) is by means of finger photoplethysmography (PPG), which also provides information about the duration of the cardiac cycle. In this study, the correlation and agreement between the time series of the intervals between heartbeats obtained from the ECG with those obtained from the PPG, were studied. Signals analyzed were obtained from young, healthy and resting subjects. For statistical analysis, the Pearson correlation coefficient and the Bland and Altman limits of agreement were used. Results show that the time series constructed from the PW would not replace the ones obtained from ECG.

Surviving annual performance reviews.

PubMed

Lazarus, Arthur

2008-01-01

Physicians who work in organizational settings can expect to be evaluated at least twice a year. Yet physicians are accustomed to functioning autonomously, and they may resist having their performance measured or become anxious at the thought of it. Several recommendations are made to help physicians survive the ordeal: (1) establish measurable goals and objectives for the year; (2) perform at your very best at all times; (3) obtain feedback about your performance from your colleagues; (4) ask for a mentor if you lack experience; (5) learn to manage upward; (6) let your boss know when other people have praised your work; (7) insist on face-to-face evaluations; and (8) sign your annual performance review and indicate agreement or disagreement.

Performance evaluation of a tailor-made passive sampler for monitoring of tropospheric ozone.

PubMed

Ozden, Ozlem; Döğeroğlu, Tuncay

2012-09-01

This study presents the performance evaluation of a tailor-made passive sampler developed for the monitoring of tropospheric ozone. The performance of the passive sampler was tested in the field conditions in terms of accuracy, precision, blank values, detection limit, effects of some parameters such as sampling site characteristics and sampling period on the field blanks, self-consistency, experimental and theoretical uptake rates, shelf life and comparison with commercial passive samplers. There was an agreement (R (2) = 0.84) between the responses of passive sampler and the continuous automatic analyser. The accuracy of the sampler, expressed as percent relative error, was obtained lower than 15%. Method precision in terms of coefficient of variance for three simultaneously applied passive samplers was 12%. Sampler detection limit was 2.42 μg m(-3) for an exposure period of 1 week, and the sampler can be stored safely for a period of up to 8 weeks before exposure. Satisfactory self-consistency results showed that extended periods gave the same integrated response as a series of short-term samplers run side by side. The uptake rate of ozone was found to be 10.21 mL min(-1) in a very good agreement with the theoretical uptake rate (10.32 mL min(-1)). The results of the comparison study conducted against a commercially available diffusion tube (Gradko diffusion tube) showed a good linear relationship (R (2) = 0.93) between two passive samplers. The sampler seems suitable to be used in large-scale measurements of ozone where no data are available or the number of existing automated monitors is not sufficient.

The Impact of Self-Evaluation Instruction on Student Self-Evaluation, Music Performance, and Self-Evaluation Accuracy

ERIC Educational Resources Information Center

Hewitt, Michael P.

2011-01-01

The author sought to determine whether self-evaluation instruction had an impact on student self-evaluation, music performance, and self-evaluation accuracy of music performance among middle school instrumentalists. Participants (N = 211) were students at a private middle school located in a metropolitan area of a mid-Atlantic state. Students in…

48 CFR 1552.209-76 - Contractor performance evaluations.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 48 Federal Acquisition Regulations System 6 2010-10-01 2010-10-01 true Contractor performance... 1552.209-76 Contractor performance evaluations. As prescribed in section 1509.170-1, insert the following clause in all applicable solicitations and contracts. Contractor Performance Evaluations (OCT 2002...

48 CFR 8.406-7 - Contractor Performance Evaluation.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 48 Federal Acquisition Regulations System 1 2010-10-01 2010-10-01 false Contractor Performance... ACQUISITION PLANNING REQUIRED SOURCES OF SUPPLIES AND SERVICES Federal Supply Schedules 8.406-7 Contractor Performance Evaluation. Ordering activities must prepare an evaluation of contractor performance for each...

48 CFR 1536.201 - Evaluation of contracting performance.

Code of Federal Regulations, 2010 CFR

2010-10-01

... performance. 1536.201 Section 1536.201 Federal Acquisition Regulations System ENVIRONMENTAL PROTECTION AGENCY... Contracting for Construction 1536.201 Evaluation of contracting performance. (a) The Contracting Officer will... will file the form in the contractor performance evaluation files which it maintains. (e) The Quality...

Combining agreement and frequency rating scales to optimize psychometrics in measuring behavioral health functioning.

PubMed

Marfeo, Elizabeth E; Ni, Pengsheng; Chan, Leighton; Rasch, Elizabeth K; Jette, Alan M

2014-07-01

The goal of this article was to investigate optimal functioning of using frequency vs. agreement rating scales in two subdomains of the newly developed Work Disability Functional Assessment Battery: the Mood & Emotions and Behavioral Control scales. A psychometric study comparing rating scale performance embedded in a cross-sectional survey used for developing a new instrument to measure behavioral health functioning among adults applying for disability benefits in the United States was performed. Within the sample of 1,017 respondents, the range of response category endorsement was similar for both frequency and agreement item types for both scales. There were fewer missing values in the frequency items than the agreement items. Both frequency and agreement items showed acceptable reliability. The frequency items demonstrated optimal effectiveness around the mean ± 1-2 standard deviation score range; the agreement items performed better at the extreme score ranges. Findings suggest an optimal response format requires a mix of both agreement-based and frequency-based items. Frequency items perform better in the normal range of responses, capturing specific behaviors, reactions, or situations that may elicit a specific response. Agreement items do better for those whose scores are more extreme and capture subjective content related to general attitudes, behaviors, or feelings of work-related behavioral health functioning. Copyright © 2014 Elsevier Inc. All rights reserved.

Expressing analytical performance from multi-sample evaluation in laboratory EQA.

PubMed

Thelen, Marc H M; Jansen, Rob T P; Weykamp, Cas W; Steigstra, Herman; Meijer, Ron; Cobbaert, Christa M

2017-08-28

To provide its participants with an external quality assessment system (EQAS) that can be used to check trueness, the Dutch EQAS organizer, Organization for Quality Assessment of Laboratory Diagnostics (SKML), has innovated its general chemistry scheme over the last decade by introducing fresh frozen commutable samples whose values were assigned by Joint Committee for Traceability in Laboratory Medicine (JCTLM)-listed reference laboratories using reference methods where possible. Here we present some important innovations in our feedback reports that allow participants to judge whether their trueness and imprecision meet predefined analytical performance specifications. Sigma metrics are used to calculate performance indicators named 'sigma values'. Tolerance intervals are based on both Total Error allowable (TEa) according to biological variation data and state of the art (SA) in line with the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Milan consensus. The existing SKML feedback reports that express trueness as the agreement between the regression line through the results of the last 12 months and the values obtained from reference laboratories and calculate imprecision from the residuals of the regression line are now enriched with sigma values calculated from the degree to which the combination of trueness and imprecision are within tolerance limits. The information and its conclusion to a simple two-point scoring system are also graphically represented in addition to the existing difference plot. By adding sigma metrics-based performance evaluation in relation to both TEa and SA tolerance intervals to its EQAS schemes, SKML provides its participants with a powerful and actionable check on accuracy.

Interrater and intrarater agreements of magnetic resonance imaging findings in the lumbar spine: significant variability across degenerative conditions.

PubMed

Fu, Michael C; Buerba, Rafael A; Long, William D; Blizzard, Daniel J; Lischuk, Andrew W; Haims, Andrew H; Grauer, Jonathan N

2014-10-01

Magnetic resonance imaging (MRI) is frequently used in the evaluation of degenerative conditions in the lumbar spine. The relative interrater and intrarater agreements of MRI findings across different pathologic conditions are underexplored, as most studies are focused on specific findings. The purpose of this study was to characterize the interrater and intrarater agreements of MRI findings used to assess the degenerative lumbar spine. A retrospective diagnostic study at a large academic medical center was undertaken with a panel of orthopedic surgeons and musculoskeletal radiologists to assess lumbar MRIs using standardized criteria. Seventy-five subjects who underwent routine lumbar spine MRI at our institution were included. Each MRI study was assessed for 10 lumbar degenerative findings using standardized criteria. Lumbar vertebral levels were assessed independently, where applicable, for a total of 52 data points collected per study. T2-weighted axial and sagittal MRI sequences were presented in random order to the four reviewers (two orthopedic spine surgeons and two musculoskeletal radiologists) independently to determine interrater agreement. The first 10 studies were reevaluated at the end to determine intrarater agreement. Images were assessed using standardized and pilot-tested criteria to assess disc degeneration, stenosis, and other degenerative changes. Interrater and intrarater absolute percent agreements were calculated. To highlight the most clinically important MRI disagreements, a modified agreement analysis was also performed (in which disagreements between the lowest two severity grades for applicable conditions were ignored). Fleiss kappa coefficients for interrater agreement were determined. The overall absolute and modified interrater agreements were 76.9% and 93.5%, respectively. The absolute and modified intrarater agreements were 81.3% and 92.7%, respectively. Average Fleiss kappa coefficient was 0.431, suggesting moderate overall

Reproducibility and diagnostic performance of shear wave elastography in evaluating breast solid mass.

PubMed

Hong, Sun; Woo, Ok Hee; Shin, Hye Seon; Hwang, Soon-Young; Cho, Kyu Ran; Seo, Bo Kyoung

Shear wave elastography (SWE) was performed independently by two radiologists in 264 solid breast masses. The images were reviewed for color overlay pattern (COP) classification by the two radiologists, double blinded to any information. The interobserver agreement of the COP was almost perfect (κ=0.908) and high in E max (ICC=0.89). The AUC value of the COP (0.954) was significantly higher than that of E max (0.915) (p=0.002) but not significantly different from that of E max combined with COP (0.957) (p=0.098). The SWE color overlay pattern and E max of breast masses were highly reproducible. The COP had better diagnostic ability than E max , suggesting that COP may be a more reliable parameter for solid breast mass evaluation. Copyright © 2017 Elsevier Inc. All rights reserved.

Retroperitoneal Sarcoma Target Volume and Organ at Risk Contour Delineation Agreement Among NRG Sarcoma Radiation Oncologists

DOE Office of Scientific and Technical Information (OSTI.GOV)

Baldini, Elizabeth H., E-mail: ebaldini@partners.org; Abrams, Ross A.; Bosch, Walter

Purpose: The purpose of this study was to evaluate the variability in target volume and organ at risk (OAR) contour delineation for retroperitoneal sarcoma (RPS) among 12 sarcoma radiation oncologists. Methods and Materials: Radiation planning computed tomography (CT) scans for 2 cases of RPS were distributed among 12 sarcoma radiation oncologists with instructions for contouring gross tumor volume (GTV), clinical target volume (CTV), high-risk CTV (HR CTV: area judged to be at high risk of resulting in positive margins after resection), and OARs: bowel bag, small bowel, colon, stomach, and duodenum. Analysis of contour agreement was performed using the simultaneousmore » truth and performance level estimation (STAPLE) algorithm and kappa statistics. Results: Ten radiation oncologists contoured both RPS cases, 1 contoured only RPS1, and 1 contoured only RPS2 such that each case was contoured by 11 radiation oncologists. The first case (RPS 1) was a patient with a de-differentiated (DD) liposarcoma (LPS) with a predominant well-differentiated (WD) component, and the second case (RPS 2) was a patient with DD LPS made up almost entirely of a DD component. Contouring agreement for GTV and CTV contours was high. However, the agreement for HR CTVs was only moderate. For OARs, agreement for stomach, bowel bag, small bowel, and colon was high, but agreement for duodenum (distorted by tumor in one of these cases) was fair to moderate. Conclusions: For preoperative treatment of RPS, sarcoma radiation oncologists contoured GTV, CTV, and most OARs with a high level of agreement. HR CTV contours were more variable. Further clarification of this volume with the help of sarcoma surgical oncologists is necessary to reach consensus. More attention to delineation of the duodenum is also needed.« less

Retroperitoneal Sarcoma Target Volume and Organ at Risk Contour Delineation Agreement Among NRG Sarcoma Radiation Oncologists

PubMed Central

Baldini, Elizabeth H.; Abrams, Ross A.; Bosch, Walter; Roberge, David; Haas, Rick L.M.; Catton, Charles N.; Indelicato, Daniel J.; Olsen, Jeffrey R.; Deville, Curtiland; Chen, Yen-Lin; Finkelstein, Steven E.; DeLaney, Thomas F.; Wang, Dian

2015-01-01

Purpose The purpose of this study was to evaluate the variability in target volume and organ at risk (OAR) contour delineation for retroperitoneal sarcoma (RPS) among 12 sarcoma radiation oncologists. Methods and Materials Radiation planning computed tomography (CT) scans for 2 cases of RPS were distributed among 12 sarcoma radiation oncologists with instructions for contouring gross tumor volume (GTV), clinical target volume (CTV), high-risk CTV (HR CTV: area judged to be at high risk of resulting in positive margins after resection), and OARs: bowel bag, small bowel, colon, stomach, and duodenum. Analysis of contour agreement was performed using the simultaneous truth and performance level estimation (STAPLE) algorithm and kappa statistics. Results Ten radiation oncologists contoured both RPS cases, 1 contoured only RPS1, and 1 contoured only RPS2 such that each case was contoured by 11 radiation oncologists. The first case (RPS 1) was a patient with a de-differentiated (DD) liposarcoma (LPS) with a predominant well-differentiated (WD) component, and the second case (RPS 2) was a patient with DD LPS made up almost entirely of a DD component. Contouring agreement for GTV and CTV contours was high. However, the agreement for HR CTVs was only moderate. For OARs, agreement for stomach, bowel bag, small bowel, and colon was high, but agreement for duodenum (distorted by tumor in one of these cases) was fair to moderate. Conclusions For preoperative treatment of RPS, sarcoma radiation oncologists contoured GTV, CTV, and most OARs with a high level of agreement. HR CTV contours were more variable. Further clarification of this volume with the help of sarcoma surgical oncologists is necessary to reach consensus. More attention to delineation of the duodenum is also needed. PMID:26194680

Prospective cohort study of ultrasound-ultrasound and ultrasound-MR enterography agreement in the evaluation of pediatric small bowel Crohn disease.

PubMed

Dillman, Jonathan R; Smith, Ethan A; Sanchez, Ramon; DiPietro, Michael A; Dehkordy, Soudabeh Fazeli; Adler, Jeremy; DeMatos-Maillard, Vera; Khalatbari, Shokoufeh; Davenport, Matthew S

2016-04-01

There is a paucity of published literature describing ultrasound (US)-US and US-MR enterography (MRE) inter-radiologist agreement in pediatric small bowel Crohn disease. To prospectively assess US-US and US-MRE inter-radiologist agreement in pediatric small bowel Crohn disease. Institutional Review Board approval and informed consent/assent were obtained for this HIPAA-compliant prospective cohort study of children with newly diagnosed distal small bowel Crohn disease (July 2012 to December 2014). Enrolled subjects (n = 29) underwent two small bowel US examinations performed by blinded independent radiologists both before and at multiple time points after initiation of medical therapy (231 unique US examinations, in total); 134 US examinations were associated with concurrent MRE. The MRE examination was interpreted by a third blinded radiologist. The following was documented on each examination: involved length of ileum (cm); maximum bowel wall thickness (mm); amount of bowel wall and mesenteric Doppler signal, and presence of stricture, penetrating disease and/or abscess. Inter-radiologist agreement was assessed with single-measure, three-way, mixed-model intra-class correlation coefficients (ICC) and prevalence-adjusted, bias-adjusted kappa statistics (κ). Numbers in brackets are 95% confidence intervals. Ultrasound-US agreement was moderate for involved length (ICC: 0.41 [0.35-0.49]); substantial for maximum bowel wall thickness (ICC: 0.67 [0.64-0.70]); moderate for bowel wall Doppler signal (ICC: 0.53 [0.48-0.59]); slight for mesenteric Doppler signal (ICC: 0.25 [0.18-0.42]), and moderate to almost perfect for stricture (κ: 0.54), penetrating disease (κ: 0.80), and abscess (κ: 0.96). US-MRE agreement was moderate for involved length (ICC: 0.42 [0.37-0.49]); substantial for maximum bowel wall thickness (ICC: 0.66 [0.65-0.69]), and substantial to almost perfect for stricture (κ: 0.61), penetrating disease (κ: 0.72) and abscess (κ: 0.88). Ultrasound

Healthcare system intervention for prevention of birth injuries - process evaluation of self-assessment, peer review, feedback and agreement for change.

PubMed

Nyström, Monica E; Westerlund, Anna; Höög, Elisabet; Millde-Luthander, Charlotte; Högberg, Ulf; Grunewald, Charlotta

2012-08-24

Patient safety is fundamental in high quality healthcare systems but despite an excellent record of perinatal care in Sweden some children still suffer from substandard care and unnecessary birth injuries. Sustainable patient safety improvements assume changes in key actors' mental models, norms and culture as well as in the tools, design and organisation of work. Interventions positively affecting team mental models on safety issues are a first step to enhancing change. Our purpose was to study a national intervention programme for the prevention of birth injuries with the aim to elucidate how the main interventions of self-assessment, peer review, feedback and written agreement for change affected the teams and their mental model of patient safety, and thereby their readiness for change. Knowledge of relevant considerations before implementing this type of patient safety intervention series could thereby be increased. Eighty participants in twenty-seven maternity units were interviewed after the first intervention sequence of the programme. A content analysis using a priori coding was performed in order to relate results to the anticipated outcomes of three basic interventions: self-assessment, peer review and written feedback, and agreement for change. The self-assessment procedure was valuable and served as a useful tool for elucidating strengths and weaknesses and identifying areas for improvement for a safer delivery in maternity units. The peer-review intervention was appreciated, despite it being of less value when considering the contribution to explicit outcome effects (i.e. new input to team mental models and new suggestions for actions). The feedback report and the mutual agreement on measures for improvements reached when signing the contract seemed exert positive pressures for change. Our findings are in line with several studies stressing the importance of self-evaluation by encouraging a thorough review of objectives, practices and outcomes for the

Effects of Performers' External Characteristics on Performance Evaluations.

ERIC Educational Resources Information Center

Bermingham, Gudrun A.

2000-01-01

States that fairness has been a major concern in the field of music adjudication. Reviews the research literature to reveal information about three external characteristics (race, gender, and physical attractiveness) that may affect judges' performance evaluations and influence fairness of music adjudication. Includes references. (CMK)

Alternative performance measures for evaluating congestion.

DOT National Transportation Integrated Search

2004-04-01

This report summarizes the results of the work performed under the project Alternative Performance Measures for Evaluating : Congestion. The study first outlines existing approaches to looking at congestion. It then builds on the previous work in the...

Technology Partnership Agreements | NREL

Science.gov Websites

Partnership Agreements Technology Partnership Agreements Looking for Funding? We do not fund any projects under a technology partnership agreement. The partner provides the necessary resources and, in using technology partnership agreements. See a summary of our Fiscal Year 2017 technology partnership

Predicting phonetic transcription agreement: Insights from research in infant vocalizations

PubMed Central

RAMSDELL, HEATHER L.; OLLER, D. KIMBROUGH; ETHINGTON, CORINNA A.

2010-01-01

The purpose of this study is to provide new perspectives on correlates of phonetic transcription agreement. Our research focuses on phonetic transcription and coding of infant vocalizations. The findings are presumed to be broadly applicable to other difficult cases of transcription, such as found in severe disorders of speech, which similarly result in low reliability for a variety of reasons. We evaluated the predictiveness of two factors not previously documented in the literature as influencing transcription agreement: canonicity and coder confidence. Transcribers coded samples of infant vocalizations, judging both canonicity and confidence. Correlation results showed that canonicity and confidence were strongly related to agreement levels, and regression results showed that canonicity and confidence both contributed significantly to explanation of variance. Specifically, the results suggest that canonicity plays a major role in transcription agreement when utterances involve supraglottal articulation, with coder confidence offering additional power in predicting transcription agreement. PMID:17882695

22 CFR 124.6 - Termination of manufacturing license agreements and technical assistance agreements.

Code of Federal Regulations, 2010 CFR

2010-04-01

... and technical assistance agreements. 124.6 Section 124.6 Foreign Relations DEPARTMENT OF STATE... Termination of manufacturing license agreements and technical assistance agreements. The U.S. party to a manufacturing license or a technical assistance agreement must inform the Directorate of Defense Trade Controls...

Intraobserver and Interobserver Agreement of Structural and Functional Software Programs for Measuring Glaucoma Progression.

PubMed

Moreno-Montañés, Javier; Antón, Vanesa; Antón, Alfonso; Larrosa, José M; Martinez-de-la-Casa, José María; Rebolleda, Gema; Ussa, Fernando; García-Granero, Marta

2017-04-01

It is important to evaluate intraobserver and interobserver agreement using visual field (VF) testing and optical coherence tomography (OCT) software in order to understand whether the use of this software is sufficient to detect glaucoma progression and to make decisions regarding its treatment. To evaluate agreement in VF and OCT software among 5 glaucoma specialists. The printout pages from VF progression software and OCT progression software from 100 patients were randomized, and the 5 glaucoma specialists subjectively and independently evaluated them for glaucoma. Each image was classified as having no progression, questionable progression, or progression. The principal investigator classified the patients previously as without variability (normal) or with high variability among tests (difficult). Using both software, the specialists also evaluated whether the glaucoma damage had progressed and if treatment change was needed. One month later, the same observers reevaluated the patients in a different order to determine intraobserver reproducibility. Intraobserver and interobserver agreement was estimated using κ statistics and Gwet second-order agreement coefficient. The agreement was compared with other factors. Of the 100 observed patients, half were male and all were white; the mean (SD) age was 69.7 (14.1) years. Intraobserver agreement was substantial to almost perfect for VF software (overall κ [95% CI], 0.59 [0.46-0.72] to 0.87 [0.79-0.96]) and similar for OCT software (overall κ [95% CI], 0.59 [0.46-0.71] to 0.85 [0.76-0.94]). Interobserver agreement among the 5 glaucoma specialists with the VF progression software was moderate (κ, 0.48; 95% CI, 0.41-0.55) and similar to OCT progression software (κ, 0.52; 95% CI, 0.44-0.59). Interobserver agreement was substantial in images classified as having no progression but only fair in those classified as having questionable glaucoma progression or glaucoma progression. Interobserver agreement was fair

Building Leadership Talent through Performance Evaluation

ERIC Educational Resources Information Center

Clifford, Matthew

2015-01-01

Most states and districts scramble to provide professional development to support principals, but "principal evaluation" is often lost amid competing priorities. Evaluation is an important method for supporting principal growth, communicating performance expectations to principals, and improving leadership practice. It provides leaders…

Performance Evaluation and Benchmarking of Intelligent Systems

DOE Office of Scientific and Technical Information (OSTI.GOV)

Madhavan, Raj; Messina, Elena; Tunstel, Edward

To design and develop capable, dependable, and affordable intelligent systems, their performance must be measurable. Scientific methodologies for standardization and benchmarking are crucial for quantitatively evaluating the performance of emerging robotic and intelligent systems technologies. There is currently no accepted standard for quantitatively measuring the performance of these systems against user-defined requirements; and furthermore, there is no consensus on what objective evaluation procedures need to be followed to understand the performance of these systems. The lack of reproducible and repeatable test methods has precluded researchers working towards a common goal from exchanging and communicating results, inter-comparing system performance, and leveragingmore » previous work that could otherwise avoid duplication and expedite technology transfer. Currently, this lack of cohesion in the community hinders progress in many domains, such as manufacturing, service, healthcare, and security. By providing the research community with access to standardized tools, reference data sets, and open source libraries of solutions, researchers and consumers will be able to evaluate the cost and benefits associated with intelligent systems and associated technologies. In this vein, the edited book volume addresses performance evaluation and metrics for intelligent systems, in general, while emphasizing the need and solutions for standardized methods. To the knowledge of the editors, there is not a single book on the market that is solely dedicated to the subject of performance evaluation and benchmarking of intelligent systems. Even books that address this topic do so only marginally or are out of date. The research work presented in this volume fills this void by drawing from the experiences and insights of experts gained both through theoretical development and practical implementation of intelligent systems in a variety of diverse application domains. The book

Reliability and Agreement of Neck Functional Capacity Evaluation Tests in Patients With Chronic Multifactorial Neck Pain.

PubMed

Reneman, M F; Roelofs, M; Schiphorst Preuper, H R

2017-07-01

To analyze test-retest reliability and agreement, and to explore the safety of neck functional capacity evaluation (Neck-FCE) tests in patients with chronic multifactorial neck pain. Test-retest; 2 FCE sessions were held with a 2-week interval. University-based outpatient rehabilitation center. Individuals (N=18; 14 women) with a mean age of 34 years. Not applicable. The Neck-FCE protocol consists of 6 tests: lifting waist to overhead (kg), 2-handed carrying (kg), overhead working (s), bending and overhead reaching (s), and repetitive side reaching (left and right) (s). Intraclass correlation coefficients (ICCs) and limits of agreement (LoA) were calculated. ICC point estimates between .75 and .90 were considered as good, and >.90 were considered as excellent reliability. ICC point estimates ranged between .39 and .96. Ratios of the LoA ranged between 32.0% and 56.5%. Mean ± SD numeric rating scale pain scores in the neck and shoulder 24 hours after the test were 6.7±2.6 and 6.3±3.0, respectively. Based on ICC point estimates and 95% confidence intervals, 3 tests had excellent reliability and 3 had poor reliability. LoA were substantial in all 6 tests. Safety was confirmed. Copyright © 2016 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.

Inter- and intra-rater reliability and agreement in determining subcutaneous tumour margins in dogs.

PubMed

Ranganathan, B; Milovancev, M; Leeper, H; Townsend, K L; Bracha, S; Curran, K

2018-03-01

The objective of this prospective study was to evaluate agreement and reliability of calliper-based measurements of locally invasive subcutaneous malignant tumours in dogs. Four raters measured the longest diameter of 12 subcutaneous tumours (7 soft tissue sarcomas and 5 mast cell tumours) from 11 client-owned dogs during 3 randomized, blinded measurement trials, both pre- and post-sedation. Inter- and intra-rater reliability was evaluated using intra-class correlation coefficient (ICC) and agreement was evaluated using Bland-Altman plots. Inter- and intra-rater reliability was good (ICC range of 0.8694-0.89520) and excellent (ICC range of 0.9720-0.9966), respectively. For agreement calculations, an a priori clinically relevant limit of agreement of 10 mm was set. Inter- and intra-rater agreement was unacceptable with inter-rater limits of agreement ranging from 15.9 to 55.6 mm and intra-rater limit of agreement ranging from 11.9 to 28.1 mm. Review of the measurement trial photographs revealed that calliper orientation changes were frequent, occurring in 9/12 (75%) and 8/12 (67%) pre- and post-sedation cases. No significant correlation was found between inter-rater measurement standard deviations and calliper orientation changes or dog body condition score. These findings suggest veterinarians may have poor agreement in determining the gross edge of tumours, which is expected to introduce bias and inconsistency in tumour staging, assessing response to therapy, and surgical margin planning. Due to the potential consequences for veterinary cancer patients, future studies are needed to validate the present findings. © 2018 John Wiley & Sons Ltd.

25 CFR 163.70 - Purpose of agreements.

Code of Federal Regulations, 2012 CFR

2012-04-01

... programs; (2) Develop and publish cooperative environmental education and natural resource planning materials; and (3) Perform land and facility improvements, including forestry and other natural resources... related to land and natural resource management. (b) The Secretary may enter into such agreements when he...

25 CFR 163.70 - Purpose of agreements.

Code of Federal Regulations, 2013 CFR

2013-04-01

... programs; (2) Develop and publish cooperative environmental education and natural resource planning materials; and (3) Perform land and facility improvements, including forestry and other natural resources... related to land and natural resource management. (b) The Secretary may enter into such agreements when he...

25 CFR 163.70 - Purpose of agreements.

Code of Federal Regulations, 2014 CFR

2014-04-01

... programs; (2) Develop and publish cooperative environmental education and natural resource planning materials; and (3) Perform land and facility improvements, including forestry and other natural resources... related to land and natural resource management. (b) The Secretary may enter into such agreements when he...

25 CFR 163.70 - Purpose of agreements.

Code of Federal Regulations, 2010 CFR

2010-04-01

... programs; (2) Develop and publish cooperative environmental education and natural resource planning materials; and (3) Perform land and facility improvements, including forestry and other natural resources... related to land and natural resource management. (b) The Secretary may enter into such agreements when he...

25 CFR 163.70 - Purpose of agreements.

Code of Federal Regulations, 2011 CFR

2011-04-01

... programs; (2) Develop and publish cooperative environmental education and natural resource planning materials; and (3) Perform land and facility improvements, including forestry and other natural resources... related to land and natural resource management. (b) The Secretary may enter into such agreements when he...

Comparison and Agreement Between the Richmond Agitation-Sedation Scale and the Riker Sedation-Agitation Scale in Evaluating Patients’ Eligibility for Delirium Assessment in the ICU

PubMed Central

Guzman, Oscar; Campbell, Noll L.; Walroth, Todd; Tricker, Jason L.; Hui, Siu L.; Perkins, Anthony; Zawahiri, Mohammed; Buckley, John D.; Farber, Mark O.; Ely, E. Wesley; Boustani, Malaz A.

2012-01-01

Background: Delirium evaluation in patients in the ICU requires the use of an arousal/sedation assessment tool prior to assessing consciousness. The Richmond Agitation-Sedation Scale (RASS) and the Riker Sedation-Agitation Scale (SAS) are well-validated arousal/sedation tools. We sought to assess the concordance of RASS and SAS assessments in determining eligibility of patients in the ICU for delirium screening using the confusion assessment method for the ICU (CAM-ICU). Methods: We performed a prospective cohort study in the adult medical, surgical, and progressive (step-down) ICUs of a tertiary care, university-affiliated, urban hospital in Indianapolis, Indiana. The cohort included 975 admissions to the ICU between January and October 2009. Results: The outcome measures of interest were the correlation and agreement between RASS and SAS measurements. In 2,469 RASS and SAS paired screens, the rank correlation using the Spearman correlation coefficient was 0.91, and the agreement between the two screening tools for assessing CAM-ICU eligibility as estimated by the κ coefficient was 0.93. Analysis showed that 70.1% of screens were eligible for CAM-ICU assessment using RASS (7.1% sedated [RASS −3 to −1]; 62.6% calm [0]; and 0.4% restless, agitated [+1 to +3]), compared with 72.1% using SAS (5% sedated [SAS 3]; 66.5% calm [4]; and 0.6% anxious, agitated [5, 6]). In the mechanically ventilated subgroup, RASS identified 19.1% CAM-ICU eligible patients compared with 24.6% by SAS. The correlation coefficient in this subgroup was 0.70 and the agreement was 0.81. Conclusion: Both SAS and RASS led to similar rates of delirium assessment using the CAM-ICU. PMID:22539644

San Francisco urban partnership agreement, national evaluation : traveler information data test plan.

DOT National Transportation Integrated Search

2011-06-01

This report presents the test plan for collecting and analyzing traveler information data for the San Francisco Urban Partnership Agreement (UPA) under the United States Department of Transportation (U.S. DOT) UPA Program. The San Francisco UPA proje...

Advanced Video Analysis Needs for Human Performance Evaluation

NASA Technical Reports Server (NTRS)

Campbell, Paul D.

1994-01-01

Evaluators of human task performance in space missions make use of video as a primary source of data. Extraction of relevant human performance information from video is often a labor-intensive process requiring a large amount of time on the part of the evaluator. Based on the experiences of several human performance evaluators, needs were defined for advanced tools which could aid in the analysis of video data from space missions. Such tools should increase the efficiency with which useful information is retrieved from large quantities of raw video. They should also provide the evaluator with new analytical functions which are not present in currently used methods. Video analysis tools based on the needs defined by this study would also have uses in U.S. industry and education. Evaluation of human performance from video data can be a valuable technique in many industrial and institutional settings where humans are involved in operational systems and processes.

Evaluating supplier quality performance using analytical hierarchy process

NASA Astrophysics Data System (ADS)

Kalimuthu Rajoo, Shanmugam Sundram; Kasim, Maznah Mat; Ahmad, Nazihah

2013-09-01

This paper elaborates the importance of evaluating supplier quality performance to an organization. Supplier quality performance evaluation reflects the actual performance of the supplier exhibited at customer's end. It is critical in enabling the organization to determine the area of improvement and thereafter works with supplier to close the gaps. Success of the customer partly depends on supplier's quality performance. Key criteria as quality, cost, delivery, technology support and customer service are categorized as main factors in contributing to supplier's quality performance. 18 suppliers' who were manufacturing automotive application parts evaluated in year 2010 using weight point system. There were few suppliers with common rating which led to common ranking observed by few suppliers'. Analytical Hierarchy Process (AHP), a user friendly decision making tool for complex and multi criteria problems was used to evaluate the supplier's quality performance challenging the weight point system that was used for 18 suppliers'. The consistency ratio was checked for criteria and sub-criteria. Final results of AHP obtained with no overlap ratings, therefore yielded a better decision making methodology as compared to weight point rating system.

San Francisco urban partnership agreement, national evaluation : cost benefit analysis test plan.

DOT National Transportation Integrated Search

2011-06-01

This report presents the test plan for collecting and analyzing cost and benefit data for the San Francisco Urban Partnership Agreement (UPA) under the United States Department of Transportation (U.S. DOT) UPA Program. The San Francisco UPA projects ...

San Francisco urban partnership agreement, national evaluation : traffic system data test plan.

DOT National Transportation Integrated Search

2011-06-01

This report presents the test plan for collecting and analyzing traffic system data for the San Francisco Urban Partnership Agreement (UPA) under the United States Department of Transportation (U.S. DOT) UPA Program. The San Francisco UPA projects fo...

San Francisco urban partnership agreement, national evaluation : transit system data test plan.

DOT National Transportation Integrated Search

2011-06-01

This report presents the test plan for collecting and analyzing the transit system data for the San Francisco Urban Partnership Agreement (UPA) under the United States Department of Transportation (U.S. DOT) UPA Program. The San Francisco UPA project...

Miami urban partnership agreement (UPA) Pines Boulevard transit signal priority evaluation .report.

DOT National Transportation Integrated Search

2011-09-01

The Miami Urban Partnership Agreement included the conversion of high occupancy vehicle (HOV) lanes on I-95 to high occupancy toll : (HOT) lanes and additional express bus service. It also included funding for the installation of transit signal prior...

Beta-Testing Agreement | Frederick National Laboratory for Cancer Research

Cancer.gov

Beta-Testing Agreements are appropriate forlimited term evaluation and applications development of new software, technology, or equipment platforms by the Frederick National Laboratory in collaboration with an external commercial partner. It ma

Interjudge agreement in videofluoroscopic studies of swallowing.

PubMed

Wilcox, F; Liss, J M; Siegel, G M

1996-02-01

Videofluoroscopic swallowing examinations of 3 patients with dysphagia were reviewed independently by 10 speech-language pathologists. Prior to viewing each video, clinicians were provided with information about the patient's history, the results of a bedside swallow examination, and oral-facial and oral motor control examinations. Clinicians completed a swallowing observation protocol as they viewed each video. They then recommended, from a list of treatment strategies, intervention techniques that would be most appropriate for each patient. Interjudge agreement was calculated by determining how many clinicians observed a given swallowing event or deficit, and how many recommended a given treatment strategy. Results suggest that the level of interjudge agreement for videofluoroscopic evaluations is not encouragingly high.

A pilot study of clinical agreement in cardiovascular preparticipation examinations: how good is the standard of care?

PubMed

O'Connor, Francis G; Johnson, Jeremy D; Chapin, Mark; Oriscello, Ralph G; Taylor, Dean C

2005-05-01

To evaluate the interobserver agreement between physicians regarding a abnormal cardiovascular assessment on athletic preparticipation examinations. Cross-sectional clinical survey. Outpatient Clinic, United States Military Academy, West Point, NY. We randomly selected 101 out of 539 cadet-athletes presenting for a preparticipation examination. Two primary care sports medicine fellows and a cardiologist examined the cadets. After obtaining informed consent from all participants, all 3 physicians separately evaluated all 101 cadets. The physicians recorded their clinical findings and whether they thought further cardiovascular evaluation (echocardiography) was indicated. Rate of referral for further cardiovascular evaluation, clinical agreement between sports medicine fellows, and clinical agreement between sports medicine fellows and the cardiologist. Each fellow referred 6 of the 101 evaluated cadets (5.9%). The cardiologist referred none. Although each fellow referred 6 cadets, only 1 cadet was referred by both. The kappa statistic for clinical agreement between fellows is 0.114 (95% CI, -0.182 to 0.411). There was no clinical agreement between the fellows and the cardiologist. This pilot study reveals a low level of agreement between physicians regarding which athletes with an abnormal examination deserved further testing. It challenges the standard of care and questions whether there is a need for improved technologies or improved training in cardiovascular clinical assessment.

Identifying and processing the gap between perceived and actual agreement in breast pathology interpretation.

PubMed

Carney, Patricia A; Allison, Kimberly H; Oster, Natalia V; Frederick, Paul D; Morgan, Thomas R; Geller, Berta M; Weaver, Donald L; Elmore, Joann G

2016-07-01

We examined how pathologists' process their perceptions of how their interpretations on diagnoses for breast pathology cases agree with a reference standard. To accomplish this, we created an individualized self-directed continuing medical education program that showed pathologists interpreting breast specimens how their interpretations on a test set compared with a reference diagnosis developed by a consensus panel of experienced breast pathologists. After interpreting a test set of 60 cases, 92 participating pathologists were asked to estimate how their interpretations compared with the standard for benign without atypia, atypia, ductal carcinoma in situ and invasive cancer. We then asked pathologists their thoughts about learning about differences in their perceptions compared with actual agreement. Overall, participants tended to overestimate their agreement with the reference standard, with a mean difference of 5.5% (75.9% actual agreement; 81.4% estimated agreement), especially for atypia and were least likely to overestimate it for invasive breast cancer. Non-academic affiliated pathologists were more likely to more closely estimate their performance relative to academic affiliated pathologists (77.6 vs 48%; P=0.001), whereas participants affiliated with an academic medical center were more likely to underestimate agreement with their diagnoses compared with non-academic affiliated pathologists (40 vs 6%). Before the continuing medical education program, nearly 55% (54.9%) of participants could not estimate whether they would overinterpret the cases or underinterpret them relative to the reference diagnosis. Nearly 80% (79.8%) reported learning new information from this individualized web-based continuing medical education program, and 23.9% of pathologists identified strategies they would change their practice to improve. In conclusion, when evaluating breast pathology specimens, pathologists do a good job of estimating their diagnostic agreement with a

Evaluation of pavement marking performance.

DOT National Transportation Integrated Search

2008-06-01

The objective of the investigation was to evaluate the useful life of pavement markings. The Manual on Uniform Traffic Control Devices (MUTCD) provides general guidelines for the application and installation of pavement markings. However, performance...

Suboptimal Agreement Among Cytopathologists in Diagnosis of Malignancy Based on Endoscopic Ultrasound Needle Aspirates of Solid Pancreatic Lesions: A Validation Study.

PubMed

Marshall, Carrie; Mounzer, Rawad; Hall, Matt; Simon, Violette; Centeno, Barbara; Dennis, Katie; Dhillon, Jasreman; Fan, Fang; Khazai, Laila; Klapman, Jason; Komanduri, Srinadh; Lin, Xiaoqi; Lu, David; Mehrotra, Sanjana; Muthusamy, V Raman; Nayar, Ritu; Paintal, Ajit; Rao, Jianyu; Sams, Sharon; Shah, Janak; Watson, Rabindra; Rastogi, Amit; Wani, Sachin

2018-07-01

Despite the widespread use of endoscopic ultrasound-guided fine-needle aspiration (EUS-FNA) to sample pancreatic lesions and the standardization of pancreaticobiliary cytopathologic nomenclature, there are few data on inter-observer agreement among cytopathologists evaluating pancreatic cytologic specimens obtained by EUS-FNA. We developed a scoring system to assess agreement among cytopathologists in overall diagnosis and quantitative and qualitative parameters, and evaluated factors associated with agreement. We performed a prospective study to validate results from our pilot study that demonstrated moderate to substantial inter-observer agreement among cytopathologists for the final cytologic diagnosis. In the first phase, 3 cytopathologists refined criteria for assessment of quantity and quality measures. During phase 2, EUS-FNA specimens of solid pancreatic lesions from 46 patients were evaluated by 11 cytopathologists at 5 tertiary care centers using a standardized scoring tool. Individual quantitative and qualitative measures were scored and an overall cytologic diagnosis was determined. Clinical and EUS parameters were assessed as predictors of unanimous agreement. Inter-observer agreement (IOA) was calculated using multi-rater kappa (κ) statistics and a logistic regression model was created to identify factors associated with unanimous agreement. The IOA for final diagnoses, based on cytologic analysis, was moderate (κ = 0.56; 95% CI, 0.43-0.70). Kappa values did not increase when categories of suspicious for malignancy, malignant, and neoplasm were combined. IOA was slight to moderate for individual quantitative (κ = 0.007; 95% CI, -0.03 to -0.04) and qualitative parameters (κ = 0.5; 95% CI, 0.47-0.53). Jaundice was the only factor associated with agreement among all cytopathologists on multivariate analysis (odds ratio for unanimous agreement, 5.3; 95% CI, 1.1-26.89). There is a suboptimal level of agreement among cytopathologists in the diagnosis

Agreement between microscopic examination and bacterial culture of bile samples for detection of bactibilia in dogs and cats with hepatobiliary disease.

PubMed

Pashmakova, Medora B; Piccione, Julie; Bishop, Micah A; Nelson, Whitney R; Lawhon, Sara D

2017-05-01

OBJECTIVE To evaluate the agreement between results of microscopic examination and bacterial culture of bile samples from dogs and cats with hepatobiliary disease for detection of bactibilia. DESIGN Cross-sectional study. ANIMALS 31 dogs and 21 cats with hepatobiliary disease for which subsequent microscopic examination and bacterial culture of bile samples was performed from 2004 through 2014. PROCEDURES Electronic medical records of included dogs and cats were reviewed to extract data regarding diagnosis, antimicrobials administered, and results of microscopic examination and bacterial culture of bile samples. Agreement between these 2 diagnostic tests was assessed by calculation of the Cohen κ value. RESULTS 17 (33%) dogs and cats had bactibilia identified by microscopic examination of bile samples, and 11 (21%) had bactibilia identified via bacterial culture. Agreement between these 2 tests was substantial (percentage agreement [positive and negative results], 85%; κ = 0.62; 95% confidence interval, 0.38 to 0.89) and improved to almost perfect when calculated for only animals that received no antimicrobials within 24 hours prior to sample collection (percentage agreement, 94%; κ = 0.84; 95% confidence interval, 0.61 to 1.00). CONCLUSIONS AND CLINICAL RELEVANCE Results indicated that agreement between microscopic examination and bacterial culture of bile samples for detection of bactibilia is optimized when dogs and cats are not receiving antimicrobials at the time of sample collection. Concurrent bacterial culture and microscopic examination of bile samples are recommended for all cats and dogs evaluated for hepatobiliary disease.

Performance evaluation and model analysis of BTEX contaminated air in corn-cob biofilter system.

PubMed

Rahul; Mathur, Anil Kumar; Balomajumder, Chandrajit

2013-04-01

Biofiltration of BTEX with corn-cob packing material have been performed for a period of 68 days in five distinct phases. The overall performance of a biofilter has been evaluated in terms of its elimination capacity by using 3-D mesh techniques. Maximum removal efficiency was found more than 99.85% of all four compounds at an EBRT of 3.06 min in phase I for an inlet BTEX concentration of 0.0970, 0.0978, 0.0971 and 0.0968 g m(-3), respectively. Nearly 100% removal achieved at average BTEX loadings of 20.257 g m(-3) h(-1) to biofilter. A maximum elimination capacity (EC) of 20.239 g m(-3) h(-1) of the biofilter was obtained at inlet BTEX load of 20.391 g m(-3) h(-1). Moreover, using convection-diffusion reaction (CDR) model for biofilter depth shows good agreement with the experimental values for benzene, toluene and ethyl benzene, but for o-xylene the model results deviated from the experimental. Copyright © 2013 Elsevier Ltd. All rights reserved.

Performance Evaluation and Community Application of Low-Cost Sensors for Ozone and Nitrogen Dioxide

PubMed Central

Duvall, Rachelle M.; Long, Russell W.; Beaver, Melinda R.; Kronmiller, Keith G.; Wheeler, Michael L.; Szykman, James J.

2016-01-01

This study reports on the performance of electrochemical-based low-cost sensors and their use in a community application. CairClip sensors were collocated with federal reference and equivalent methods and operated in a network of sites by citizen scientists (community members) in Houston, Texas and Denver, Colorado, under the umbrella of the NASA-led DISCOVER-AQ Earth Venture Mission. Measurements were focused on ozone (O3) and nitrogen dioxide (NO2). The performance evaluation showed that the CairClip O3/NO2 sensor provided a consistent measurement response to that of reference monitors (r2 = 0.79 in Houston; r2 = 0.72 in Denver) whereas the CairClip NO2 sensor measurements showed no agreement to reference measurements. The CairClip O3/NO2 sensor data from the citizen science sites compared favorably to measurements at nearby reference monitoring sites. This study provides important information on data quality from low-cost sensor technologies and is one of few studies that reports sensor data collected directly by citizen scientists. PMID:27754370

An evaluation of agreement between pectoral spines and otoliths for estimating ages of catfishes

USGS Publications Warehouse

Olive, J.A.; Schramm, Harold; Gerard, Patrick D.; Irwin, E.

2011-01-01

Otoliths have been shown to provide more accurate ages than pectoral spine sections for several catfish populations; but sampling otoliths requires euthanizing the specimen, whereas spines can be sampled non-lethally. To evaluate whether, and under what conditions, spines provide the same or similar age estimates as otoliths, we examined data sets of individual fish aged from pectoral spines and otoliths for six blue catfish Ictalurus furcatus populations (n=420), 14 channel catfish Ictalurus punctatus populations (n=997), and 10 flathead catfish Pylodictus olivaris populations (n=947) from lotic and lentic waters throughout the central and eastern U.S. Logistic regression determined that agreement between ages estimated from otoliths and spines was consistently related to age, but inconsistently related to growth rate. When modeled at mean growth rate, we found at least 80% probability of no difference in spine- and otolith-assigned ages up to ages 4 and 5 for blue and channel catfish, respectively. For flathead catfish, an 80% probability of agreement between spine- and otolith-assigned ages did not occur at any age due to high incidence of differences in assigned ages even for age-1 fish. Logistic regression models predicted at least 80% probability that spine and otolith ages differed by ≤1 year up to ages 13, 16, and 9 for blue, channel, and flathead catfish, respectively. Age-bias assessment found mean spine-assigned age differed by less than 1 year from otolith-assigned age up to ages 19, 9, and 17 for blue catfish, channel catfish, and flathead catfish, respectively. These results can be used to help guide decisions about which structure is most appropriate for estimating catfish ages for particular populations and management objectives.

Tethered swimming can be used to evaluate force contribution for short-distance swimming performance.

PubMed

Morouço, Pedro G; Marinho, Daniel A; Keskinen, Kari L; Badillo, Juan J; Marques, Mário C

2014-11-01

The purpose of this study was two-fold: (a) to compare stroke and the physiological responses between maximal tethered and free front crawl swimming and (b) to evaluate the contribution of force exertion for swimming performance over short distances. A total of 34 male swimmers, representing various levels of competitive performance, participated in this study. Each participant was tested in both a 30-second maximal tethered swimming test and a 50-m free swimming test. The tethered force parameters, the swimming speed, stroke (stroke rate [SR]), and the physiological responses (increase in blood lactate concentration [ΔBLa], heart rate, and rate of perceived exertion) were recorded and calculated. The results showed no differences in stroke and the physiological responses between tethered and free swimming, with a high level of agreement for the SR and ΔBLa. A strong correlation was obtained between the maximum impulse of force per stroke and the speed (r = 0.91; p < 0.001). Multiple regression analysis revealed that the maximum impulse and SR in the tethered condition explained 84% of the free swimming performance. The relationship between the swimming speed and maximum force tended to be nonlinear, whereas linear relationships were observed with the maximum impulse. This study demonstrates that tethered swimming does not significantly alter stroke and the physiological responses compared with free swimming, and that the maximum impulse per stroke should be used to evaluate the balance between force and the ability to effectively apply force during sprint swimming. Consequently, coaches can rely on tethered forces to identify strength deficits and improve swimming performance over short distances.

An hierarchical approach to performance evaluation of expert systems

NASA Technical Reports Server (NTRS)

Dominick, Wayne D. (Editor); Kavi, Srinu

1985-01-01

The number and size of expert systems is growing rapidly. Formal evaluation of these systems - which is not performed for many systems - increases the acceptability by the user community and hence their success. Hierarchical evaluation that had been conducted for computer systems is applied for expert system performance evaluation. Expert systems are also evaluated by treating them as software systems (or programs). This paper reports many of the basic concepts and ideas in the Performance Evaluation of Expert Systems Study being conducted at the University of Southwestern Louisiana.

[Evaluation of an educational website on First Aid].

PubMed

Mori, Satomi; Whitaker, Iveth Yamaguchi; Marin, Heimar de Fátima

2013-08-01

The aim of this study was to evaluate the structure, quality of information and usability of a website on First Aid. The evaluation was performed by information technology (IT) and health care professionals and by students, using specific and validated instruments. The kappa method was used to evaluate the agreement of the answers, and Cronbach's α coefficient was used to assess the reliability of the instrument. There was no agreement (0.047) among the answers obtained from the IT professionals, indicating that the structure of the website must be reviewed. There was also no agreement in the evaluation by the health care professionals (-0.062); however, the overall positive scores suggest that the quality of the information of the website is adequate. The assessment of reliability of the instrument to evaluate the navigability rendered a value of α=0.974. Although improvement of the website structure is recommended, the quality of the information is good, and its use has contributed to the apprenticeship of students.

Inter- and Intra-Observer Agreement in Ultrasound BI-RADS Classification and Real-Time Elastography Tsukuba Score Assessment of Breast Lesions.

PubMed

Schwab, Fabienne; Redling, Katharina; Siebert, Matthias; Schötzau, Andy; Schoenenberger, Cora-Ann; Zanetti-Dällenbach, Rosanna

2016-11-01

Our aim was to prospectively evaluate inter- and intra-observer agreement between Breast Imaging Reporting and Data System (BI-RADS) classifications and Tsukuba elasticity scores (TSs) of breast lesions. The study included 164 breast lesions (63 malignant, 101 benign). The BI-RADS classification and TS of each breast lesion was assessed by the examiner and twice by three reviewers at an interval of 2 months. Weighted κ values for inter-observer agreement ranged from moderate to substantial for BI-RADS classification (κ = 0.585-0.738) and was substantial for TS (κ = 0.608-0.779). Intra-observer agreement was almost perfect for ultrasound (US) BI-RADS (κ = 0.847-0.872) and TS (κ = 0.879-0.914). Overall, individual reviewers are highly self-consistent (almost perfect intra-observer agreement) with respect to BI-RADS classification and TS, whereas inter-observer agreement was moderate to substantial. Comprehensive training is essential for achieving high agreement and minimizing the impact of subjectivity. Our results indicate that breast US and real-time elastography can achieve high diagnostic performance. Copyright © 2016 The Authors. Published by Elsevier Inc. All rights reserved.

International Space Station ECLSS Technical Task Agreement Summary Report

NASA Technical Reports Server (NTRS)

Minton-Summers, S.; Ray, C. D.

1996-01-01

A summary of work accomplished under Technical Task Agreement by the Marshall Space Flight Center (MSFC) documents activities regarding the Environmental Control and Life Support Systems (ECLSS) of the International Space Station (ISS) program. These MSFC activities were in-line to the designing, the development, the testing, and the flight of ECLSS equipment. MSFC's unique capabilities for performing integrated system testing and analyses, and its ability to perform some tasks cheaper and faster to support ISS program needs are the basis for the Technical Task Agreement activities. Tasks were completed in the Water Recovery Systems, Air Revitalization Systems, and microbiology areas. The results of each task is described in this summary report.

Operator performance evaluation using multi criteria decision making methods

NASA Astrophysics Data System (ADS)

Rani, Ruzanita Mat; Ismail, Wan Rosmanira; Razali, Siti Fatihah

2014-06-01

Operator performance evaluation is a very important operation in labor-intensive manufacturing industry because the company's productivity depends on the performance of its operators. The aims of operator performance evaluation are to give feedback to operators on their performance, to increase company's productivity and to identify strengths and weaknesses of each operator. In this paper, six multi criteria decision making methods; Analytical Hierarchy Process (AHP), fuzzy AHP (FAHP), ELECTRE, PROMETHEE II, Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) and VlseKriterijumska Optimizacija I Kompromisno Resenje (VIKOR) are used to evaluate the operators' performance and to rank the operators. The performance evaluation is based on six main criteria; competency, experience and skill, teamwork and time punctuality, personal characteristics, capability and outcome. The study was conducted at one of the SME food manufacturing companies in Selangor. From the study, it is found that AHP and FAHP yielded the "outcome" criteria as the most important criteria. The results of operator performance evaluation showed that the same operator is ranked the first using all six methods.

New FIGO and Swedish intrapartum cardiotocography classification systems incorporated in the fetal ECG ST analysis (STAN) interpretation algorithm: agreements and discrepancies in cardiotocography classification and evaluation of significant ST events.

PubMed

Olofsson, Per; Norén, Håkan; Carlsson, Ann

2018-02-01

The updated intrapartum cardiotocography (CTG) classification system by FIGO in 2015 (FIGO2015) and the FIGO2015-approached classification by the Swedish Society of Obstetricians and Gynecologist in 2017 (SSOG2017) are not harmonized with the fetal ECG ST analysis (STAN) algorithm from 2007 (STAN2007). The study aimed to reveal homogeneity and agreement between the systems in classifying CTG and ST events, and relate them to maternal and perinatal outcomes. Among CTG traces with ST events, 100 traces originally classified as normal, 100 as suspicious and 100 as pathological were randomly selected from a STAN database and classified by two experts in consensus. Homogeneity and agreement statistics between the CTG classifications were performed. Maternal and perinatal outcomes were evaluated in cases with clinically hidden ST data (n = 151). A two-tailed p < 0.05 was regarded as significant. For CTG classes, the heterogeneity was significant between the old and new systems, and agreements were moderate to strong (proportion of agreement, kappa index 0.70-0.86). Between the new classifications, heterogeneity was significant and agreements strong (0.90, 0.92). For significant ST events, heterogeneities were significant and agreements moderate to almost perfect (STAN2007 vs. FIGO2015 0.86, 0.72; STAN2007 vs. SSOG2017 0.92, 0.84; FIGO2015 vs. SSOG2017 0.94, 0.87). Significant ST events occurred more often combined with STAN2007 than with FIGO2015 classification, but not with SSOG2017; correct identification of adverse outcomes was not significantly different between the systems. There are discrepancies in the classification of CTG patterns and significant ST events between the old and new systems. The clinical relevance of the findings remains to be shown. © 2017 The Authors. Acta Obstetricia et Gynecologica Scandinavica published by John Wiley & Sons Ltd on behalf of Nordic Federation of Societies of Obstetrics and Gynecology (NFOG).

Legal Aspects of Evaluating Teacher Performance.

ERIC Educational Resources Information Center

Beckham, Joseph C.

Chapter 14 in a book on school law concerns the legal aspects of evaluating teacher performance. Careful analysis of recent decisions makes it clear the courts will compel uniform standards and unprecedented rigor in teacher evaluation practices. Particularly in the consideration of equitable standards, state and federal courts are relying on…

40 CFR 35.115 - Evaluation of performance.

Code of Federal Regulations, 2010 CFR

2010-07-01

... requirements for progress reporting under 40 CFR 31.40(b). (b) Elements of the evaluation process. The.... The recipient may request review of the Regional Administrator's decision under the dispute processes... Evaluation of performance. (a) Joint evaluation process. The applicant and the Regional Administrator will...

22 CFR 124.8 - Clauses required both in manufacturing license agreements and technical assistance agreements.

Code of Federal Regulations, 2010 CFR

2010-04-01

... agreements and technical assistance agreements. 124.8 Section 124.8 Foreign Relations DEPARTMENT OF STATE... Clauses required both in manufacturing license agreements and technical assistance agreements. The following statements must be included both in manufacturing license agreements and in technical assistance...

Agreement in cardiovascular risk rating based on anthropometric parameters

PubMed Central

Dantas, Endilly Maria da Silva; Pinto, Cristiane Jordânia; Freitas, Rodrigo Pegado de Abreu; de Medeiros, Anna Cecília Queiroz

2015-01-01

Objective To investigate the agreement in evaluation of risk of developing cardiovascular diseases based on anthropometric parameters in young adults. Methods The study included 406 students, measuring weight, height, and waist and neck circumferences. Waist-to-height ratio and the conicity index. The kappa coefficient was used to assess agreement in risk classification for cardiovascular diseases. The positive and negative specific agreement values were calculated as well. The Pearson chi-square (χ2) test was used to assess associations between categorical variables (p<0.05). Results The majority of the parameters assessed (44%) showed slight (k=0.21 to 0.40) and/or poor agreement (k<0.20), with low values of negative specific agreement. The best agreement was observed between waist circumference and waist-to-height ratio both for the general population (k=0.88) and between sexes (k=0.93 to 0.86). There was a significant association (p<0.001) between the risk of cardiovascular diseases and females when using waist circumference and conicity index, and with males when using neck circumference. This resulted in a wide variation in the prevalence of cardiovascular disease risk (5.5%-36.5%), depending on the parameter and the sex that was assessed. Conclusion The results indicate variability in agreement in assessing risk for cardiovascular diseases, based on anthropometric parameters, and which also seems to be influenced by sex. Further studies in the Brazilian population are required to better understand this issue. PMID:26466060

Healthcare system intervention for prevention of birth injuries – process evaluation of self-assessment, peer review, feedback and agreement for change

PubMed Central

2012-01-01

Background Patient safety is fundamental in high quality healthcare systems but despite an excellent record of perinatal care in Sweden some children still suffer from substandard care and unnecessary birth injuries. Sustainable patient safety improvements assume changes in key actors’ mental models, norms and culture as well as in the tools, design and organisation of work. Interventions positively affecting team mental models on safety issues are a first step to enhancing change. Our purpose was to study a national intervention programme for the prevention of birth injuries with the aim to elucidate how the main interventions of self-assessment, peer review, feedback and written agreement for change affected the teams and their mental model of patient safety, and thereby their readiness for change. Knowledge of relevant considerations before implementing this type of patient safety intervention series could thereby be increased. Methods Eighty participants in twenty-seven maternity units were interviewed after the first intervention sequence of the programme. A content analysis using a priori coding was performed in order to relate results to the anticipated outcomes of three basic interventions: self-assessment, peer review and written feedback, and agreement for change. Results The self-assessment procedure was valuable and served as a useful tool for elucidating strengths and weaknesses and identifying areas for improvement for a safer delivery in maternity units. The peer-review intervention was appreciated, despite it being of less value when considering the contribution to explicit outcome effects (i.e. new input to team mental models and new suggestions for actions). The feedback report and the mutual agreement on measures for improvements reached when signing the contract seemed exert positive pressures for change. Conclusions Our findings are in line with several studies stressing the importance of self-evaluation by encouraging a thorough review of

Nondisclosure Agreements | NREL

Science.gov Websites

the agreement. 5. Share Information and Maintain Records Once the nondisclosure agreement has been executed, appropriately labeled information may be shared. NREL and the other party then maintain records

Interobserver agreement in analysis of cardiotocograms recorded during trial of labor after cesarean.

PubMed

Caning, M M; Thisted, D L A; Amer-Wählin, I; Laier, G H; Krebs, L

2018-05-17

To examine interobserver agreement in intrapartum cardiotocography (CTG) classification in women undergoing trial of labor after a cesarean section (TOLAC) at term with or without complete uterine rupture. Nineteen blinded and independent Danish obstetricians assessed CTG tracings from 47 women (174 individual pages) with a complete uterine rupture during TOLAC and 37 women (133 individual pages) with no uterine rupture during TOLAC. Individual pages with CTG tracings lasting at least 20 min were evaluated by three different assessors and counted as an individual case. The tracings were analyzed according to the modified version of the Federation of Gynaecology and Obstetrics (FIGO) guidelines elaborated for the use of STAN (ST-analysis). Occurrence of defined abnormalities was recorded and the tracings were classified as normal, suspicious, pathological, or preterminal. The interobserver agreement was evaluated using Fleiss' kappa. Agreement on classification of a preterminal CTG was almost perfect. The interobserver agreement on normal, suspicious or pathological CTG was moderate to substantial. Regarding the presence of severe variable decelerations, the agreement was moderate. No statistical difference was found in the interobserver agreement between classification of tracings from women undergoing TOLAC with and without complete uterine rupture. The interobserver agreement on classification of CTG tracings from high-risk deliveries during TOLAC is best for assessment of a preterminal CTG and the poorest for the identification of severe variable decelerations.

An urban energy performance evaluation system and its computer implementation.

PubMed

Wang, Lei; Yuan, Guan; Long, Ruyin; Chen, Hong

2017-12-15

To improve the urban environment and effectively reflect and promote urban energy performance, an urban energy performance evaluation system was constructed, thereby strengthening urban environmental management capabilities. From the perspectives of internalization and externalization, a framework of evaluation indicators and key factors that determine urban energy performance and explore the reasons for differences in performance was proposed according to established theory and previous studies. Using the improved stochastic frontier analysis method, an urban energy performance evaluation and factor analysis model was built that brings performance evaluation and factor analysis into the same stage for study. According to data obtained for the Chinese provincial capitals from 2004 to 2013, the coefficients of the evaluation indicators and key factors were calculated by the urban energy performance evaluation and factor analysis model. These coefficients were then used to compile the program file. The urban energy performance evaluation system developed in this study was designed in three parts: a database, a distributed component server, and a human-machine interface. Its functions were designed as login, addition, edit, input, calculation, analysis, comparison, inquiry, and export. On the basis of these contents, an urban energy performance evaluation system was developed using Microsoft Visual Studio .NET 2015. The system can effectively reflect the status of and any changes in urban energy performance. Beijing was considered as an example to conduct an empirical study, which further verified the applicability and convenience of this evaluation system. Copyright © 2017 Elsevier Ltd. All rights reserved.

Provider Agreement in the Assessment of Glaucoma Progression within a Team Model.

PubMed

Shah, Saumya; Choo, Clara; Odden, Jamie; Zhao, Bingying; Fang, Chengbo; Schornack, Muriel; Stalboerger, Gina; Bennett, Jeffrey R; Khanna, Cheryl L

2018-06-04

Glaucoma specialists and optometrists who work in a team model at a single institution utilize a common definition of glaucoma progression and treatment algorithm. The purpose of this study was to assess the consistency of agreement in identifying glaucoma progression among glaucoma specialists and optometrists of one team. 399 eyes of 200 patients age 18 or older with glaucoma were enrolled over two years. Clinical data, disc photographs, OCT, and visual fields were independently reviewed by two masked optometrists and two masked fellowship-trained glaucoma specialists. Each eye was judged as progression or no-progression of glaucomatous disease. The following were assessed: (1) agreement among optometrists; (2) agreement among glaucoma specialists; and (3) agreement among optometrists and glaucoma specialists. The frequency of use of testing modality to determine progression was also studied. Kappa statistics were used to evaluate agreements. Optometrists agreed with each other for 74.2% of the eyes assessed (κ=0.42), whereas glaucoma specialists agreed with each other for 78.7% of eyes (κ=0.39). All four providers agreed with each other for 54.4% of the eyes evaluated (κ=0.37). Providers had the highest agreement when the progression decision was based on disc hemorrhage (92%) and the lowest agreement when based on OCT progression analysis (36%). Compared to optometrists, glaucoma specialists used OCT (P= <0.01) more frequently to determine disease progression. Fair to moderate agreement levels were found among providers in their assessment of glaucoma progression, suggesting that a team approach to glaucoma management may be effective. Further work is needed to investigate ways to optimize consistency within the glaucoma team.

Performance-Based Evaluation and School Librarians

ERIC Educational Resources Information Center

Church, Audrey P.

2015-01-01

Evaluation of instructional personnel is standard procedure in our Pre-K-12 public schools, and its purpose is to document educator effectiveness. With Race to the Top and No Child Left Behind waivers, states are required to implement performance-based evaluations that demonstrate student academic progress. This three-year study describes the…

Theory and Practice on Teacher Performance Evaluation

ERIC Educational Resources Information Center

Yonghong, Cai; Chongde, Lin

2006-01-01

Teacher performance evaluation plays a key role in educational personnel reform, so it has been an important yet difficult issue in educational reform. Previous evaluations on teachers failed to make strict distinction among the three dominant types of evaluation, namely, capability, achievement, and effectiveness. Moreover, teacher performance…

48 CFR 36.201 - Evaluation of contractor performance.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 48 Federal Acquisition Regulations System 1 2010-10-01 2010-10-01 false Evaluation of contractor performance. 36.201 Section 36.201 Federal Acquisition Regulations System FEDERAL ACQUISITION REGULATION... Contracting for Construction 36.201 Evaluation of contractor performance. See 42.1502(e) for the requirements...

12 CFR 1291.9 - Agreements.

Code of Federal Regulations, 2013 CFR

2013-01-01

... 12 Banks and Banking 9 2013-01-01 2013-01-01 false Agreements. 1291.9 Section 1291.9 Banks and... HOUSING PROGRAM § 1291.9 Agreements. (a) Agreements between Banks and members. A Bank shall have in place with each member receiving an AHP subsidized advance or AHP direct subsidy an agreement or agreements...

12 CFR 1291.9 - Agreements.

Code of Federal Regulations, 2014 CFR

2014-01-01

... 12 Banks and Banking 10 2014-01-01 2014-01-01 false Agreements. 1291.9 Section 1291.9 Banks and... HOUSING PROGRAM § 1291.9 Agreements. (a) Agreements between Banks and members. A Bank shall have in place with each member receiving an AHP subsidized advance or AHP direct subsidy an agreement or agreements...

12 CFR 1291.9 - Agreements.

Code of Federal Regulations, 2011 CFR

2011-01-01

... 12 Banks and Banking 7 2011-01-01 2011-01-01 false Agreements. 1291.9 Section 1291.9 Banks and... HOUSING PROGRAM § 1291.9 Agreements. (a) Agreements between Banks and members. A Bank shall have in place with each member receiving an AHP subsidized advance or AHP direct subsidy an agreement or agreements...

12 CFR 1291.9 - Agreements.

Code of Federal Regulations, 2010 CFR

2010-01-01

... 12 Banks and Banking 7 2010-01-01 2010-01-01 false Agreements. 1291.9 Section 1291.9 Banks and... HOUSING PROGRAM § 1291.9 Agreements. (a) Agreements between Banks and members. A Bank shall have in place with each member receiving an AHP subsidized advance or AHP direct subsidy an agreement or agreements...

12 CFR 1291.9 - Agreements.

Code of Federal Regulations, 2012 CFR

2012-01-01

... 12 Banks and Banking 9 2012-01-01 2012-01-01 false Agreements. 1291.9 Section 1291.9 Banks and... HOUSING PROGRAM § 1291.9 Agreements. (a) Agreements between Banks and members. A Bank shall have in place with each member receiving an AHP subsidized advance or AHP direct subsidy an agreement or agreements...

Blinded evaluation of interrater reliability of an operative competency assessment tool for direct laryngoscopy and rigid bronchoscopy.

PubMed

Ishman, Stacey L; Benke, James R; Johnson, Kaalan Erik; Zur, Karen B; Jacobs, Ian N; Thorne, Marc C; Brown, David J; Lin, Sandra Y; Bhatti, Nasir; Deutsch, Ellen S

2012-10-01

OBJECTIVES To confirm interrater reliability using blinded evaluation of a skills-assessment instrument to assess the surgical performance of resident and fellow trainees performing pediatric direct laryngoscopy and rigid bronchoscopy in simulated models. DESIGN Prospective, paired, blinded observational validation study. SUBJECTS Paired observers from multiple institutions simultaneously evaluated residents and fellows who were performing surgery in an animal laboratory or using high-fidelity manikins. The evaluators had no previous affiliation with the residents and fellows and did not know their year of training. INTERVENTIONS One- and 2-page versions of an objective structured assessment of technical skills (OSATS) assessment instrument composed of global and a task-specific surgical items were used to evaluate surgical performance. RESULTS Fifty-two evaluations were completed by 17 attending evaluators. The instrument agreement for the 2-page assessment was 71.4% when measured as a binary variable (ie, competent vs not competent) (κ = 0.38; P = .08). Evaluation as a continuous variable revealed a 42.9% percentage agreement (κ = 0.18; P = .14). The intraclass correlation was 0.53, considered substantial/good interrater reliability (69% reliable). For the 1-page instrument, agreement was 77.4% when measured as a binary variable (κ = 0.53, P = .0015). Agreement when evaluated as a continuous measure was 71.0% (κ = 0.54, P < .001). The intraclass correlation was 0.73, considered high interrater reliability (85% reliable). CONCLUSIONS The OSATS assessment instrument is an effective tool for evaluating surgical performance among trainees with acceptable interrater reliability in a simulator setting. Reliability was good for both the 1- and 2-page OSATS checklists, and both serve as excellent tools to provide immediate formative feedback on operational competency.

40 CFR 35.515 - Evaluation of performance.

Code of Federal Regulations, 2010 CFR

2010-07-01

....515 Evaluation of performance. (a) Joint evaluation process. The applicant and the Regional... work plan (see section 35.507(b)(2)(iv)). A description of the evaluation process and reporting... annually and must satisfy the requirements for progress reporting under 40 CFR 31.40(b). (b) Elements of...

Agreement between the Turkey Guidelines and the Fracture Risk Assessment Tool®-based Intervention Threshold

PubMed Central

Aydogan, Nevres Hurriyet; Tosun, Kursad

2018-01-01

Background The aim of this study was to evaluate the agreement between the fracture-risk assessment tool (FRAX®)-based intervention strategy in Turkey and the recommendations published in the Healthcare Practices Statement (HPS). Methods This descriptive cross-sectional study included individuals aged 40 to 90 years who were previously diagnosed as having osteoporosis but had not received any treatment. The intervention thresholds recommended by the National Osteoporosis Foundation for treatment were used. The criteria necessary for the start of administration of pharmacological agents in osteoporosis treatment were evaluated on the basis of the HPS guidelines. Results Of the 1,255 patients evaluated, 161 (12.8%) were male and 1,094 (87.2%) were female. In the evaluation, according to HPS, treatment was recommended for 783 patients (62.4%; HPS+) and not recommended for 472 (37.6%; HPS−). Of the 783 HPS+ patients, 391 (49.9%) were FRAX+, and of the 472 HPS− patients, 449 (95.1%) were FRAX−. A statistically significant difference was observed between the treatment recommendations of HPS and FRAX® (P<0.001). In the age group of 75 to 90 years, excellent agreement was found between the two strategies (Gwet's agreement coefficient 1=0.94). As age increased, the agreement between the two treatment strategies also increased. Conclusions The FRAX® model has different treatment recommendation rates from the HPS. The agreement between the two is at a minimal level. However, as age increased, so did the agreement between the FRAX® and the HPS treatment recommendations. In the recommendation to start pharmacological treatment primarily based on age, non-medical interventions that preserve bone density should be evaluated. PMID:29900157

Error Reduction Program. [combustor performance evaluation codes

NASA Technical Reports Server (NTRS)

Syed, S. A.; Chiappetta, L. M.; Gosman, A. D.

1985-01-01

The details of a study to select, incorporate and evaluate the best available finite difference scheme to reduce numerical error in combustor performance evaluation codes are described. The combustor performance computer programs chosen were the two dimensional and three dimensional versions of Pratt & Whitney's TEACH code. The criteria used to select schemes required that the difference equations mirror the properties of the governing differential equation, be more accurate than the current hybrid difference scheme, be stable and economical, be compatible with TEACH codes, use only modest amounts of additional storage, and be relatively simple. The methods of assessment used in the selection process consisted of examination of the difference equation, evaluation of the properties of the coefficient matrix, Taylor series analysis, and performance on model problems. Five schemes from the literature and three schemes developed during the course of the study were evaluated. This effort resulted in the incorporation of a scheme in 3D-TEACH which is usuallly more accurate than the hybrid differencing method and never less accurate.

21 CFR 26.78 - Agreements with other countries.

Code of Federal Regulations, 2011 CFR

2011-04-01

... 21 Food and Drugs 1 2011-04-01 2011-04-01 false Agreements with other countries. 26.78 Section 26... MUTUAL RECOGNITION OF PHARMACEUTICAL GOOD MANUFACTURING PRACTICE REPORTS, MEDICAL DEVICE QUALITY SYSTEM AUDIT REPORTS, AND CERTAIN MEDICAL DEVICE PRODUCT EVALUATION REPORTS: UNITED STATES AND THE EUROPEAN...

Interobserver Agreement Among Uveitis Experts on Uveitic Diagnoses: The Standardization of Uveitis Nomenclature Experience.

PubMed

Jabs, Douglas A; Dick, Andrew; Doucette, John T; Gupta, Amod; Lightman, Susan; McCluskey, Peter; Okada, Annabelle A; Palestine, Alan G; Rosenbaum, James T; Saleem, Sophia M; Thorne, Jennifer; Trusko, Brett

2018-02-01

To evaluate the interobserver agreement among uveitis experts on the diagnosis of the specific uveitic disease. Interobserver agreement analysis. Five committees, each comprised of 9 individuals and working in parallel, reviewed cases from a preliminary database of 25 uveitic diseases, collected by disease, and voted independently online whether the case was the disease in question or not. The agreement statistic, κ, was calculated for the 36 pairwise comparisons for each disease, and a mean κ was calculated for each disease. After the independent online voting, committee consensus conference calls, using nominal group techniques, reviewed all cases not achieving supermajority agreement (>75%) on the diagnosis in the online voting to attempt to arrive at a supermajority agreement. A total of 5766 cases for the 25 diseases were evaluated. The overall mean κ for the entire project was 0.39, with disease-specific variation ranging from 0.23 to 0.79. After the formalized consensus conference calls to address cases that did not achieve supermajority agreement in the online voting, supermajority agreement overall was reached on approximately 99% of cases, with disease-specific variation ranging from 96% to 100%. Agreement among uveitis experts on diagnosis is moderate at best but can be improved by discussion among them. These data suggest the need for validated and widely used classification criteria in the field of uveitis. Copyright © 2017 Elsevier Inc. All rights reserved.

Are distal radius fracture classifications reproducible? Intra and interobserver agreement.

PubMed

Belloti, João Carlos; Tamaoki, Marcel Jun Sugawara; Franciozi, Carlos Eduardo da Silveira; Santos, João Baptista Gomes dos; Balbachevsky, Daniel; Chap Chap, Eduardo; Albertoni, Walter Manna; Faloppa, Flávio

2008-05-01

Various classification systems have been proposed for fractures of the distal radius, but the reliability of these classifications is seldom addressed. For a fracture classification to be useful, it must provide prognostic significance, interobserver reliability and intraobserver reproducibility. The aim here was to evaluate the intraobserver and interobserver agreement of distal radius fracture classifications. This was a validation study on interobserver and intraobserver reliability. It was developed in the Department of Orthopedics and Traumatology, Universidade Federal de São Paulo - Escola Paulista de Medicina. X-rays from 98 cases of displaced distal radius fracture were evaluated by five observers: one third-year orthopedic resident (R3), one sixth-year undergraduate medical student (UG6), one radiologist physician (XRP), one orthopedic trauma specialist (OT) and one orthopedic hand surgery specialist (OHS). The radiographs were classified on three different occasions (times T1, T2 and T3) using the Universal (Cooney), Arbeitsgemeinschaft für Osteosynthesefragen/Association for the Study of Internal Fixation (AO/ASIF), Frykman and Fernández classifications. The kappa coefficient (kappa) was applied to assess the degree of agreement. Among the three occasions, the highest mean intraobserver k was observed in the Universal classification (0.61), followed by Fernández (0.59), Frykman (0.55) and AO/ASIF (0.49). The interobserver agreement was unsatisfactory in all classifications. The Fernández classification showed the best agreement (0.44) and the worst was the Frykman classification (0.26). The low agreement levels observed in this study suggest that there is still no classification method with high reproducibility.

Experimental Evaluation of High Performance Integrated Heat Pump

DOE Office of Scientific and Technical Information (OSTI.GOV)

Miller, William A; Berry, Robert; Durfee, Neal

2016-01-01

Integrated heat pump (IHP) technology provides significant potential for energy savings and comfort improvement for residential buildings. In this study, we evaluate the performance of a high performance IHP that provides space heating, cooling, and water heating services. Experiments were conducted according to the ASHRAE Standard 206-2013 where 24 test conditions were identified in order to evaluate the IHP performance indices based on the airside performance. Empirical curve fits of the unit s compressor maps are used in conjunction with saturated condensing and evaporating refrigerant conditions to deduce the refrigerant mass flowrate, which, in turn was used to evaluate themore » refrigerant side performance as a check on the airside performance. Heat pump (compressor, fans, and controls) and water pump power were measured separately per requirements of Standard 206. The system was charged per the system manufacturer s specifications. System test results are presented for each operating mode. The overall IHP performance metrics are determined from the test results per the Standard 206 calculation procedures.« less

Medical Evidence Influence on Inpatients and Nurses Pain Ratings Agreement

PubMed Central

Samolsky Dekel, Boaz Gedaliahu; Gori, Alberto; Vasarri, Alessio; Sorella, Maria Cristina; Di Nino, Gianfranco; Melotti, Rita Maria

2016-01-01

Biased pain evaluation due to automated heuristics driven by symptom uncertainty may undermine pain treatment; medical evidence moderators are thought to play a role in such circumstances. We explored, in this cross-sectional survey, the effect of such moderators (e.g., nurse awareness of patients' pain experience and treatment) on the agreement between n = 862 inpatients' self-reported pain and n = 115 nurses' pain ratings using a numerical rating scale. We assessed the mean of absolute difference, agreement (κ-statistics), and correlation (Spearman rank) of inpatients and nurses' pain ratings and analyzed congruence categories' (CCs: underestimation, congruence, and overestimation) proportions and dependence upon pain categories for each medical evidence moderator (χ 2 analysis). Pain ratings agreement and correlation were limited; the CCs proportions were further modulated by the studied moderators. Medical evidence promoted in nurses overestimation of low and underestimation of high inpatients' self-reported pain. Knowledge of the negative influence of automated heuristics driven by symptoms uncertainty and medical-evidence moderators on pain evaluation may render pain assessment more accurate. PMID:27445633

Interobserver Agreement in Clinical Grading of Vitreous Haze Using Alternative Grading Scales

PubMed Central

Hornbeak, Dana M; Payal, Abhishek; Pistilli, Maxwell; Biswas, Jyotirmay; Ganesh, Sudha K; Gupta, Vishali; Rathinam, Sivakumar R; Davis, Janet L; Kempen, John H

2014-01-01

Purpose To evaluate the reliability of clinical grading of vitreous haze using a new 9-step ordinal scale vs. the existing 6-step ordinal scale. Design Evaluation of Diagnostic Test (interobserver agreement study). Participants 119 consecutive patients (204 uveitic eyes) presenting for uveitis subspecialty care on the study day at one of three large uveitis centers. Methods Five pairs of uveitis specialists clinically graded vitreous haze in the same eyes, one after the other using the same equipment, using the 6- and 9-step scales. Main Outcome Measures Agreement in vitreous haze grade between each pair of specialists was evaluated by the κ statistic (exact agreement and agreement within one or two grades). Results The scales correlated well (Spearman’s ρ=0.84). Exact agreement was modest using both the 6-step and 9-step scales: average κ=0.46 (range 0.28–0.81) and κ=0.40 (range 0.15–0.63), respectively. Within-1-grade agreement was slightly more favorable for the scale with fewer steps, but values were excellent for both scales: κ=0.75 (range 0.66–0.96) and κ=0.62 (range 0.38–0.87), respectively. Within-2-grade agreement for the 9-step scale also was excellent [κ=0.85 (range 0.79–0.92)]. Two-fold more cases were potentially clinical trial eligible based on the 9- than the 6-step scale (p<0.001). Conclusions Both scales are sufficiently reproducible using clinical grading for clinical and research use with the appropriate threshold (a ≥2 and ≥3 step differences for the 6-step and 9-step scales respectively). The results suggest that more eyes are likely to meet eligibility criteria for trials using the 9-step scale. The 9-step scale appears to have higher reproducibility with Reading Center grading than clinical grading, suggesting Reading Center grading may be preferable for clinical trials. PMID:24697913

Building China's municipal healthcare performance evaluation system: a Tuscan perspective.

PubMed

Li, Hao; Barsanti, Sara; Bonini, Anna

2012-08-01

Regional healthcare performance evaluation systems can help optimize healthcare resources on regional basis and improve the performance of healthcare services provided. The Tuscany region in Italy is a good example of an institution which meets these requirements. China has yet to build such a system based on international experience. In this paper, based on comparative studies between Tuscany and China, we propose that the managing institutions in China's experimental cities can select and commission a third-party agency to, respectively, evaluate the performance of their affiliated hospitals and community health service centers. Following some features of the Tuscan experience, the Chinese municipal healthcare performance evaluation system can be built by focusing on the selection of an appropriate performance evaluation agency, the design of an adequate performance evaluation mechanism and the formulation of a complete set of laws, rules and regulations. When a performance evaluation system at city level is formed, the provincial government can extend the successful experience to other cities.

Diagnostic Performance of Ultrafast Brain MRI for Evaluation of Abusive Head Trauma.

PubMed

Kralik, S F; Yasrebi, M; Supakul, N; Lin, C; Netter, L G; Hicks, R A; Hibbard, R A; Ackerman, L L; Harris, M L; Ho, C Y

2017-04-01

MR imaging with sedation is commonly used to detect intracranial traumatic pathology in the pediatric population. Our purpose was to compare nonsedated ultrafast MR imaging, noncontrast head CT, and standard MR imaging for the detection of intracranial trauma in patients with potential abusive head trauma. A prospective study was performed in 24 pediatric patients who were evaluated for potential abusive head trauma. All patients received noncontrast head CT, ultrafast brain MR imaging without sedation, and standard MR imaging with general anesthesia or an immobilizer, sequentially. Two pediatric neuroradiologists independently reviewed each technique blinded to other modalities for intracranial trauma. We performed interreader agreement and consensus interpretation for standard MR imaging as the criterion standard. Diagnostic accuracy was calculated for ultrafast MR imaging, noncontrast head CT, and combined ultrafast MR imaging and noncontrast head CT. Interreader agreement was moderate for ultrafast MR imaging (κ = 0.42), substantial for noncontrast head CT (κ = 0.63), and nearly perfect for standard MR imaging (κ = 0.86). Forty-two percent of patients had discrepancies between ultrafast MR imaging and standard MR imaging, which included detection of subarachnoid hemorrhage and subdural hemorrhage. Sensitivity, specificity, and positive and negative predictive values were obtained for any traumatic pathology for each examination: ultrafast MR imaging (50%, 100%, 100%, 31%), noncontrast head CT (25%, 100%, 100%, 21%), and a combination of ultrafast MR imaging and noncontrast head CT (60%, 100%, 100%, 33%). Ultrafast MR imaging was more sensitive than noncontrast head CT for the detection of intraparenchymal hemorrhage ( P = .03), and the combination of ultrafast MR imaging and noncontrast head CT was more sensitive than noncontrast head CT alone for intracranial trauma ( P = .02). In abusive head trauma, ultrafast MR imaging, even combined with noncontrast

Surrogate Agreement in Tzotzil.

ERIC Educational Resources Information Center

Aissen, Judith L.

This study investigates whether other relationships in sentence structure besides the "brother-in-law" relation sanction surrogate agreement in Zinacanteco Tzotzil (Mayan). Surrogate agreement refers to cases in which an element that lies outside the class of regular agreement controllers in a language (the surrogate) controls…

23 CFR 645.113 - Agreements and authorizations.

Code of Federal Regulations, 2010 CFR

2010-04-01

... work. When Federal participation is requested, the agreement shall incorporate this regulation by reference and designate the method to be used for performing the work (by contract or force account) and for... utility is on the basis of actual direct and related indirect costs accumulated in accordance with a work...

Model Performance Evaluation and Scenario Analysis (MPESA)

EPA Pesticide Factsheets

Model Performance Evaluation and Scenario Analysis (MPESA) assesses the performance with which models predict time series data. The tool was developed Hydrological Simulation Program-Fortran (HSPF) and the Stormwater Management Model (SWMM)

An experimental study on CHVE's performance evaluation.

PubMed

Paiva, Paulo V F; Machado, Liliane S; Oliveira, Jauvane C

2012-01-01

Virtual reality-based training simulators, with collaborative capabilities, are known to improve the way users interact with one another while learning or improving skills on a given medical procedure. Performance evaluation of Collaborative Haptic Virtual Environments (CHVE) allows us to understand how such systems can work in the Internet, as well as the requirements for multisensorial and real-time data. This work discloses new performance evaluation results for the collaborative module of the CyberMed VR framework.

Evaluating performance of limestone prone to polishing.

DOT National Transportation Integrated Search

2009-12-01

This research project evaluated the effect of blending Vanport limestone and other aggregates on the frictional surface characteristic properties of constructed trial road surfaces. The study undertook the evaluation of the performance of different m...

Performance evaluation methodology for historical document image binarization.

PubMed

Ntirogiannis, Konstantinos; Gatos, Basilis; Pratikakis, Ioannis

2013-02-01

Document image binarization is of great importance in the document image analysis and recognition pipeline since it affects further stages of the recognition process. The evaluation of a binarization method aids in studying its algorithmic behavior, as well as verifying its effectiveness, by providing qualitative and quantitative indication of its performance. This paper addresses a pixel-based binarization evaluation methodology for historical handwritten/machine-printed document images. In the proposed evaluation scheme, the recall and precision evaluation measures are properly modified using a weighting scheme that diminishes any potential evaluation bias. Additional performance metrics of the proposed evaluation scheme consist of the percentage rates of broken and missed text, false alarms, background noise, character enlargement, and merging. Several experiments conducted in comparison with other pixel-based evaluation measures demonstrate the validity of the proposed evaluation scheme.

Clinical Performance Evaluations of Third-Year Medical Students and Association With Student and Evaluator Gender.

PubMed

Riese, Alison; Rappaport, Leah; Alverson, Brian; Park, Sangshin; Rockney, Randal M

2017-06-01

Clinical performance evaluations are major components of medical school clerkship grades. But are they sufficiently objective? This study aimed to determine whether student and evaluator gender is associated with assessment of overall clinical performance. This was a retrospective analysis of 4,272 core clerkship clinical performance evaluations by 829 evaluators of 155 third-year students, within the Alpert Medical School grading database for the 2013-2014 academic year. Overall clinical performance, assessed on a three-point scale (meets expectations, above expectations, exceptional), was extracted from each evaluation, as well as evaluator gender, age, training level, department, student gender and age, and length of observation time. Hierarchical ordinal regression modeling was conducted to account for clustering of evaluations. Female students were more likely to receive a better grade than males (adjusted odds ratio [AOR] 1.30, 95% confidence interval [CI] 1.13-1.50), and female evaluators awarded lower grades than males (AOR 0.72, 95% CI 0.55-0.93), adjusting for department, observation time, and student and evaluator age. The interaction between student and evaluator gender was significant (P = .03), with female evaluators assigning higher grades to female students, while male evaluators' grading did not differ by student gender. Students who spent a short time with evaluators were also more likely to get a lower grade. A one-year examination of all third-year clerkship clinical performance evaluations at a single institution revealed that male and female evaluators rated male and female students differently, even when accounting for other measured variables.

EasyDIAg: A tool for easy determination of interrater agreement.

PubMed

Holle, Henning; Rein, Robert

2015-09-01

Reliable measurements are fundamental for the empirical sciences. In observational research, measurements often consist of observers categorizing behavior into nominal-scaled units. Since the categorization is the outcome of a complex judgment process, it is important to evaluate the extent to which these judgments are reproducible, by having multiple observers independently rate the same behavior. A challenge in determining interrater agreement for timed-event sequential data is to develop clear objective criteria to determine whether two raters' judgments relate to the same event (the linking problem). Furthermore, many studies presently report only raw agreement indices, without considering the degree to which agreement can occur by chance alone. Here, we present a novel, free, and open-source toolbox (EasyDIAg) designed to assist researchers with the linking problem, while also providing chance-corrected estimates of interrater agreement. Additional tools are included to facilitate the development of coding schemes and rater training.

Italy-Japan agreement and discrepancies in diagnosis of superficial gastric lesions.

PubMed

Vindigni, Carla; Marini, Mario; Cevenini, Gabriele; Raffaella Ambrosio, Maria; Onorati, Monica; Frosini, Giorgio; Gotoda, Takuji; Taniguchi, Hirokazu; Tosi, Piero

2010-01-01

The agreement between Italian and Japanese endoscopists and pathologists on endoscopic and histopathological diagnoses of superficial gastric lesions is verified with the use of Paris and Vienna classifications. The correlations between Paris endoscopic types and Vienna histopathological categories is high in both the independent Italian and Japanese evaluations. However, the agreement between Italian and Japanese endoscopists is moderate due to the difficult evaluation of the height of the lesions, in particular when they are mixed. The agreement on the size of the lesions is fairly good. The probability of the same allocation to the Vienna categories of a single case is 87 per cent, disagreements remaining in dysplasia grading, between dysplasia, not only high-grade but also low-grade, and in situ carcinoma, and on cancer invasion of the lamina propria. The results indicate that use of the Paris and Vienna classifications has reduced the discrepancies between Western and Japanese endoscopists and pathologists in the diagnosis of these lesions.

Who Should Evaluate Teachers' Performance at Schools?

ERIC Educational Resources Information Center

Tabancali, Erkan

2017-01-01

Correct determination of whether or not the objectives are achieved or of the level of achievement of the objectives is vital in educational organizations. In this context, one of the indicators of how teachers serve the organizational objectives is performance evaluation. In Turkish Educational System, the evaluation of the performance of…

Evaluating Models of Human Performance: Safety-Critical Systems Applications

NASA Technical Reports Server (NTRS)

Feary, Michael S.

2012-01-01

This presentation is part of panel discussion on Evaluating Models of Human Performance. The purpose of this panel is to discuss the increasing use of models in the world today and specifically focus on how to describe and evaluate models of human performance. My presentation will focus on discussions of generating distributions of performance, and the evaluation of different strategies for humans performing tasks with mixed initiative (Human-Automation) systems. I will also discuss issues with how to provide Human Performance modeling data to support decisions on acceptability and tradeoffs in the design of safety critical systems. I will conclude with challenges for the future.

How the Brain Converts Negative Evaluation into Performance Facilitation.

PubMed

Prévost, Charlotte; Lau, Hakwan; Mobbs, Dean

2018-02-01

Surpassing negative evaluation is a recurrent theme of success stories. Yet, there is little evidence supporting the counterintuitive idea that negative evaluation might not only motivate people, but also enhance performance. To address this question, we designed a task that required participants to decide whether taking up a risky challenge after receiving positive or negative evaluations from independent judges. Participants believed that these evaluations were based on their prior performance on a related task. Results showed that negative evaluation caused a facilitation in performance. Concurrent functional magnetic resonance imaging revealed that the motivating effect of negative evaluation was represented in the insula and striatum, while the performance boost was associated with functional positive connectivity between the insula and a set of brain regions involved in goal-directed behavior and the orienting of attention. These findings provide new insight into the neural representation of negative evaluation-induced facilitation. © The Author 2017. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.

Performance evaluation of the RITG148+ set of TomoTherapy quality assurance tools using RTQA2 radiochromic film.

PubMed

Lobb, Eric C

2016-07-08

Version 6.3 of the RITG148+ software package offers eight automated analysis routines for quality assurance of the TomoTherapy platform. A performance evaluation of each routine was performed in order to compare RITG148+ results with traditionally accepted analysis techniques and verify that simulated changes in machine parameters are correctly identified by the software. Reference films were exposed according to AAPM TG-148 methodology for each routine and the RITG148+ results were compared with either alternative software analysis techniques or manual analysis techniques in order to assess baseline agreement. Changes in machine performance were simulated through translational and rotational adjustments to subsequently irradiated films, and these films were analyzed to verify that the applied changes were accurately detected by each of the RITG148+ routines. For the Hounsfield unit routine, an assessment of the "Frame Averaging" functionality and the effects of phantom roll on the routine results are presented. All RITG148+ routines reported acceptable baseline results consistent with alternative analysis techniques, with 9 of the 11 baseline test results showing agreement of 0.1mm/0.1° or better. Simulated changes were correctly identified by the RITG148+ routines within approximately 0.2 mm/0.2° with the exception of the Field Centervs. Jaw Setting routine, which was found to have limited accuracy in cases where field centers were not aligned for all jaw settings due to inaccurate autorotation of the film during analysis. The performance of the RITG148+ software package was found to be acceptable for introduction into our clinical environment as an automated alternative to traditional analysis techniques for routine TomoTherapy quality assurance testing.

NREL Evaluates Advanced Solar Inverter Performance for Hawaiian Electric

Science.gov Websites

Companies | Energy Systems Integration Facility | NREL NREL Evaluates Advanced Solar Inverter Performance for Hawaiian Electric Companies NREL Evaluates Advanced Solar Inverter Performance for Hawaiian performance and impacts of today's advanced solar inverters, as well as proprietary feedback to the inverter

Direct comparison of PI-RADS version 2 and version 1 regarding interreader agreement and diagnostic accuracy for the detection of clinically significant prostate cancer.

PubMed

Becker, Anton S; Cornelius, Alexander; Reiner, Cäcilia S; Stocker, Daniel; Ulbrich, Erika J; Barth, Borna K; Mortezavi, Ashkan; Eberli, Daniel; Donati, Olivio F

2017-09-01

to simultaneously evaluate interreader agreement and diagnostic accuracy in the of PI-RADS v2 and compare it to v1. A total of 67 patients (median age 65.3 y, range 51.2-78.2 y; PSA 6.8μg/L, 0.2-33μg/L) undergoing MRI of the prostate and subsequent transperineal template biopsy within ≤6 months from MRI were included. Four readers from two institutions evaluated the likelihood of prostate cancer using PI-RADS v1 and v2 in two separate reading sessions ≥3 months apart. Interreader agreement was assessed for each pulse-sequence and for total PI-RADS scores using the intraclass correlation coefficient (ICC). Differences were considered significant for non-overlapping 95%-confidence intervals. Diagnostic accuracy was assessed with the area under the receiver operating characteristic curve (A Z ). A p-value <0.05 was considered statistically significant. Interreader agreement for DCE-scores was good in v2 (ICC 2 =0.70; 95% CI: 0.66-0.74) and slightly lower in v1 (ICC 1 =0.64, 0.59-0.69). Agreement for DWI scores (ICC 1 =0.77, ICC 2 =0.76) as well as final PI-RADS scores per quadrant were nearly identical (ICC 1 =ICC 2 =0.71). Diagnostic accuracy showed no significant differences (p=0.09-0.93) between v1 and v2 in any of the readers (range: A Z =0.78-0.88). PI-RADS scores show similar interreader agreement in v2 and v1 at comparable diagnostic performance. The simplification of the DCE interpretation in v2 might slightly improve agreement while not negatively affecting diagnostic performance. Copyright © 2017 Elsevier B.V. All rights reserved.

Parent-teacher agreement on children's problems in 21 societies.

PubMed

Rescorla, Leslie A; Bochicchio, Lauren; Achenbach, Thomas M; Ivanova, Masha Y; Almqvist, Fredrik; Begovac, Ivan; Bilenberg, Niels; Bird, Hector; Dobrean, Anca; Erol, Nese; Fombonne, Eric; Fonseca, Antonio; Frigerio, Alessandra; Fung, Daniel S S; Lambert, Michael C; Leung, Patrick W L; Liu, Xianchen; Marković, Ivica; Markovic, Jasminka; Minaei, Asghar; Ooi, Yoon Phaik; Roussos, Alexandra; Rudan, Vlasta; Simsek, Zeynep; van der Ende, Jan; Weintraub, Sheila; Wolanczyk, Tomasz; Woo, Bernardine; Weiss, Bahr; Weisz, John; Zukauskiene, Rita; Verhulst, Frank C

2014-01-01

Parent-teacher cross-informant agreement, although usually modest, may provide important clinical information. Using data for 27,962 children from 21 societies, we asked the following: (a) Do parents report more problems than teachers, and does this vary by society, age, gender, or type of problem? (b) Does parent-teacher agreement vary across different problem scales or across societies? (c) How well do parents and teachers in different societies agree on problem item ratings? (d) How much do parent-teacher dyads in different societies vary in within-dyad agreement on problem items? (e) How well do parents and teachers in 21 societies agree on whether the child's problem level exceeds a deviance threshold? We used five methods to test agreement for Child Behavior Checklist (CBCL) and Teacher's Report Form (TRF) ratings. CBCL scores were higher than TRF scores on most scales, but the informant differences varied in magnitude across the societies studied. Cross-informant correlations for problem scale scores varied moderately across societies studied and were significantly higher for Externalizing than Internalizing problems. Parents and teachers tended to rate the same items as low, medium, or high, but within-dyad item agreement varied widely in every society studied. In all societies studied, both parental noncorroboration of teacher-reported deviance and teacher noncorroboration of parent-reported deviance were common. Our findings underscore the importance of obtaining information from parents and teachers when evaluating and treating children, highlight the need to use multiple methods of quantifying cross-informant agreement, and provide comprehensive baselines for patterns of parent-teacher agreement across 21 societies.

Objective Situation Awareness Measurement Based on Performance Self-Evaluation

NASA Technical Reports Server (NTRS)

DeMaio, Joe

1998-01-01

The research was conducted in support of the NASA Safe All-Weather Flight Operations for Rotorcraft (SAFOR) program. The purpose of the work was to investigate the utility of two measurement tools developed by the British Defense Evaluation Research Agency. These tools were a subjective workload assessment scale, the DRA Workload Scale and a situation awareness measurement tool. The situation awareness tool uses a comparison of the crew's self-evaluation of performance against actual performance in order to determine what information the crew attended to during the performance. These two measurement tools were evaluated in the context of a test of innovative approach to alerting the crew by way of a helmet mounted display. The situation assessment data are reported here. The performance self-evaluation metric of situation awareness was found to be highly effective. It was used to evaluate situation awareness on a tank reconnaissance task, a tactical navigation task, and a stylized task used to evaluated handling qualities. Using the self-evaluation metric, it was possible to evaluate situation awareness, without exact knowledge the relevant information in some cases and to identify information to which the crew attended or failed to attend in others.

Looking for Agreement among Criteria Used to Determine Teacher Effectiveness in Two Different Evaluation Models

ERIC Educational Resources Information Center

McGair, Charles D.

2012-01-01

Many theories, methods, and practices are utilized to evaluate teachers with the intention of determining teacher effectiveness to better inform decisions about retention, tenure, certification and performance-based pay. In the 21st century there has been a renewed emphasis on teacher evaluation in public schools, largely due to federal "Race…

A juridical review of partnership agreements that have the elements of work agreements in Indonesia

NASA Astrophysics Data System (ADS)

Nugroho, A.; Sulistyowati, E.; Hikmah, N.

2018-01-01

The Partnership Agreements place the parties in an equal position each party has something as the bargaining power. In some cases, employers prefer to use Partnership Agreements to some individuals to complete the work in their company than Work agreements. Practicality and the absence of obligations to fulfil workers’ rights such as the right to join a Union and to get social security are some of the reasons why employers use the Partnership Agreements. Sometimes Partnership Agreement contains jobs, wages and orders which is the characteristic of work agreement. Based on the fact above, the legal issues arise whether the Partnership Agreement can be considered as the Work Agreement or not and which court is authorized to hear in the event of a dispute. To analyze the above legal issues, this research uses normative legal research type with the statute approach. The technique of legal material analysis uses prescriptive techniques to assess the issue and make recommendations. Based on the analysis, it can be concluded that the Partnership Agreement, of which the elements are: wages and orders can be categorized as Work Agreement and therefore in the event of a dispute, the authorized court is Industrial Relations Court.

Conversations, Not Evaluations: An Alternative Model of Performance Management

ERIC Educational Resources Information Center

Lee, Christopher D.

2003-01-01

Traditional appraisal and evaluation systems focus almost exclusively on an employee's past performance. The desired result in each of these systems is better work performance. The very nature of most appraisals or evaluations, however, may inhibit performance unintentionally by focusing energy, attention and effort on past shortcomings rather…

Predictors of Parent-Teacher Agreement in Youth with Autism Spectrum Disorder and Their Typically Developing Siblings

ERIC Educational Resources Information Center

Stratis, Elizabeth A.; Lecavalier, Luc

2017-01-01

This study evaluated the magnitude of informant agreement and predictors of agreement on behavior and emotional problems and autism symptoms in 403 children with autism and their typically developing siblings. Parent-teacher agreement was investigated on the "Child Behavior Checklist" (CBCL) and "Social Responsiveness Scale"…

Imaging acquisition display performance: an evaluation and discussion of performance metrics and procedures.

PubMed

Silosky, Michael S; Marsh, Rebecca M; Scherzinger, Ann L

2016-07-08

When The Joint Commission updated its Requirements for Diagnostic Imaging Services for hospitals and ambulatory care facilities on July 1, 2015, among the new requirements was an annual performance evaluation for acquisition workstation displays. The purpose of this work was to evaluate a large cohort of acquisition displays used in a clinical environment and compare the results with existing performance standards provided by the American College of Radiology (ACR) and the American Association of Physicists in Medicine (AAPM). Measurements of the minimum luminance, maximum luminance, and luminance uniformity, were performed on 42 acquisition displays across multiple imaging modalities. The mean values, standard deviations, and ranges were calculated for these metrics. Additionally, visual evaluations of contrast, spatial resolution, and distortion were performed using either the Society of Motion Pictures and Television Engineers test pattern or the TG-18-QC test pattern. Finally, an evaluation of local nonuniformities was performed using either a uniform white display or the TG-18-UN80 test pattern. Displays tested were flat panel, liquid crystal displays that ranged from less than 1 to up to 10 years of use and had been built by a wide variety of manufacturers. The mean values for Lmin and Lmax for the displays tested were 0.28 ± 0.13 cd/m2 and 135.07 ± 33.35 cd/m2, respectively. The mean maximum luminance deviation for both ultrasound and non-ultrasound displays was 12.61% ± 4.85% and 14.47% ± 5.36%, respectively. Visual evaluation of display performance varied depending on several factors including brightness and contrast settings and the test pattern used for image quality assessment. This work provides a snapshot of the performance of 42 acquisition displays across several imaging modalities in clinical use at a large medical center. Comparison with existing performance standards reveals that changes in display technology and the move from cathode ray

Transuranic Waste Program Framework Agreement - December Deliverable July 2012

DOE Office of Scientific and Technical Information (OSTI.GOV)

Jones, Patricia

Framework agreement deliverables are: (1) 'DOE/NNSA commits to complete removal of all non-cemented above-ground EM Legacy TRU and newly generated TRU currently-stored at Area G as of October 1, 2011, by no later than June 30, 2014. This inventory of above-ground TRU is defined as 3706 cubic meters of material.' (2) 'DOE commits to the complete removal of all newly generated TRU received in Area G during FY 2012 and 2013 by no later than December 31, 2014.' (3) 'Based on projected funding profiles, DOE/NNSA will develop by December 31, 2012, a schedule, including pacing milestones, for disposition of themore » below-ground TRU requiring retrieval at Area G.' Objectives are to: (1) restore the 'Core Team' to develop the December, 2012 deliverable; (2) obtain agreement on the strategy for below ground water disposition; and (3) establish timeline for completion of the deliverable. Below Grade Waste Strategy is to: (1) Perform an evaluation on below grade waste currently considered retrievable TRU; (2) Only commit to retrieve waste that must be retrieved; (3) Develop the Deliverable including Pacing Milestones based on planned commitments; (4) Align all Regulatory Documents for Consistency; and (5) answer these 3 primary questions, is the waste TRU; is the waste retrievable, can retrieval cause more harm than benefit?« less

Subsolid Lung Nodule Classification: A CT Criterion for Improving Interobserver Agreement.

PubMed

Revel, Marie-Pierre; Mannes, Inès; Benzakoun, Joseph; Guinet, Claude; Léger, Thomas; Grenier, Philippe; Lupo, Audrey; Fournel, Ludovic; Chassagnon, Guillaume; Bommart, Sébastien

2018-01-01

Purpose To evaluate an objective computed tomographic (CT) criterion for distinguishing between part-solid (PS) and nonsolid (NS) lung nodules. Materials and Methods This study received institutional review board approval, and patients gave informed consent. Preoperative CT studies in all patients who underwent surgery for subsolid nodules between 2008 and 2015 were first reviewed by two senior radiologists, who subjectively classified the nodules as PS or NS. A second reading performed 1 month later used predefined classification criteria and involved a third senior radiologist as well as three junior radiologists. Subsolid nodules were classified as PS if a solid portion was detectable in the mediastinal window setting (nonmeasurable, < 50%, or > 50% of the entire nodule) and were otherwise classified as NS (subclassified as pure or heterogeneous). Interreader agreement was assessed with κ statistics and the intraclass correlation coefficient (ICC). Results A total of 99 nodules measuring a median of 20 mm (range, 5-47 mm) in lung window CT images were analyzed. Senior radiologist agreement on the PS/NS distinction increased from moderate (κ = 0.54; 95% confidence interval [CI]: 0.37, 0.71) to excellent (κ = 0.89; 95% CI: 0.80, 0.98) between the first and second readings. At the second readings, agreement among senior and junior radiologists was excellent for PS/NS distinction (ICC = 0.87; 95% CI: 0.83, 0.90) and for subcategorization (ICC = 0.82; 95% CI: 0.77, 0.87). When a solid portion was measurable in the mediastinal window, the specificity for adenocarcinoma invasiveness ranged from 86% to 96%. Conclusion Detection of a solid portion in the mediastinal window setting allows subsolid nodules to be classified as PS with excellent interreader agreement. If the solid portion is measurable, the specificity for adenocarcinoma invasiveness is high. © RSNA, 2017 Online supplemental material is available for this article.

Behavioral patterns of environmental performance evaluation programs.

PubMed

Li, Wanxin; Mauerhofer, Volker

2016-11-01

During the past decades numerous environmental performance evaluation programs have been developed and implemented on different geographic scales. This paper develops a taxonomy of environmental management behavioral patterns in order to provide a practical comparison tool for environmental performance evaluation programs. Ten such programs purposively selected are mapped against the identified four behavioral patterns in the form of diagnosis, negotiation, learning, and socialization and learning. Overall, we found that schemes which serve to diagnose environmental abnormalities are mainly externally imposed and have been developed as a result of technical debates concerning data sources, methodology and ranking criteria. Learning oriented scheme is featured by processes through which free exchange of ideas, mutual and adaptive learning can occur. Scheme developed by higher authority for influencing behaviors of lower levels of government has been adopted by the evaluated to signal their excellent environmental performance. The socializing and learning classified evaluation schemes have incorporated dialogue, participation, and capacity building in program design. In conclusion we consider the 'fitness for purpose' of the various schemes, the merits of our analytical model and the future possibilities of fostering capacity building in the realm of wicked environmental challenges. Copyright © 2016 Elsevier Ltd. All rights reserved.

49 CFR 212.105 - Agreements.

Code of Federal Regulations, 2011 CFR

2011-10-01

... 49 Transportation 4 2011-10-01 2011-10-01 false Agreements. 212.105 Section 212.105 Transportation... TRANSPORTATION STATE SAFETY PARTICIPATION REGULATIONS State/Federal Roles § 212.105 Agreements. (a) Scope. The... agreement with FRA. An agreement may delegate investigative and surveillance authority with respect to all...

7 CFR 247.4 - Agreements.

Code of Federal Regulations, 2011 CFR

2011-01-01

... 7 Agriculture 4 2011-01-01 2011-01-01 false Agreements. 247.4 Section 247.4 Agriculture... CHILD NUTRITION PROGRAMS COMMODITY SUPPLEMENTAL FOOD PROGRAM § 247.4 Agreements. (a) What agreements are necessary for agencies to administer CSFP? The following agreements are necessary for agencies to administer...

49 CFR 212.105 - Agreements.

Code of Federal Regulations, 2013 CFR

2013-10-01

... 49 Transportation 4 2013-10-01 2013-10-01 false Agreements. 212.105 Section 212.105 Transportation... TRANSPORTATION STATE SAFETY PARTICIPATION REGULATIONS State/Federal Roles § 212.105 Agreements. (a) Scope. The... agreement with FRA. An agreement may delegate investigative and surveillance authority with respect to all...

7 CFR 247.4 - Agreements.

Code of Federal Regulations, 2014 CFR

2014-01-01

... 7 Agriculture 4 2014-01-01 2014-01-01 false Agreements. 247.4 Section 247.4 Agriculture... CHILD NUTRITION PROGRAMS COMMODITY SUPPLEMENTAL FOOD PROGRAM § 247.4 Agreements. (a) What agreements are necessary for agencies to administer CSFP? The following agreements are necessary for agencies to administer...

7 CFR 247.4 - Agreements.

Code of Federal Regulations, 2010 CFR

2010-01-01

... 7 Agriculture 4 2010-01-01 2010-01-01 false Agreements. 247.4 Section 247.4 Agriculture... CHILD NUTRITION PROGRAMS COMMODITY SUPPLEMENTAL FOOD PROGRAM § 247.4 Agreements. (a) What agreements are necessary for agencies to administer CSFP? The following agreements are necessary for agencies to administer...

49 CFR 212.105 - Agreements.

Code of Federal Regulations, 2014 CFR

2014-10-01

... 49 Transportation 4 2014-10-01 2014-10-01 false Agreements. 212.105 Section 212.105 Transportation... TRANSPORTATION STATE SAFETY PARTICIPATION REGULATIONS State/Federal Roles § 212.105 Agreements. (a) Scope. The... agreement with FRA. An agreement may delegate investigative and surveillance authority with respect to all...

49 CFR 212.105 - Agreements.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 49 Transportation 4 2010-10-01 2010-10-01 false Agreements. 212.105 Section 212.105 Transportation... TRANSPORTATION STATE SAFETY PARTICIPATION REGULATIONS State/Federal Roles § 212.105 Agreements. (a) Scope. The... agreement with FRA. An agreement may delegate investigative and surveillance authority with respect to all...

7 CFR 247.4 - Agreements.

Code of Federal Regulations, 2012 CFR

2012-01-01

... 7 Agriculture 4 2012-01-01 2012-01-01 false Agreements. 247.4 Section 247.4 Agriculture... CHILD NUTRITION PROGRAMS COMMODITY SUPPLEMENTAL FOOD PROGRAM § 247.4 Agreements. (a) What agreements are necessary for agencies to administer CSFP? The following agreements are necessary for agencies to administer...

7 CFR 247.4 - Agreements.

Code of Federal Regulations, 2013 CFR

2013-01-01

... 7 Agriculture 4 2013-01-01 2013-01-01 false Agreements. 247.4 Section 247.4 Agriculture... CHILD NUTRITION PROGRAMS COMMODITY SUPPLEMENTAL FOOD PROGRAM § 247.4 Agreements. (a) What agreements are necessary for agencies to administer CSFP? The following agreements are necessary for agencies to administer...

49 CFR 212.105 - Agreements.

Code of Federal Regulations, 2012 CFR

2012-10-01

... 49 Transportation 4 2012-10-01 2012-10-01 false Agreements. 212.105 Section 212.105 Transportation... TRANSPORTATION STATE SAFETY PARTICIPATION REGULATIONS State/Federal Roles § 212.105 Agreements. (a) Scope. The... agreement with FRA. An agreement may delegate investigative and surveillance authority with respect to all...

23 CFR 505.15 - Full funding grant agreement.

Code of Federal Regulations, 2010 CFR

2010-04-01

... MANAGEMENT PROJECTS OF NATIONAL AND REGIONAL SIGNIFICANCE EVALUATION AND RATING § 505.15 Full funding grant agreement. (a) A proposed project may not be funded under this program unless the Secretary finds that the project meets the requirements of this part and there is a reasonable likelihood that the project will...

Ultrasonographic assessment of tendon thickness, Doppler activity and bony spurs of the elbow in patients with lateral epicondylitis and healthy subjects: a reliability and agreement study.

PubMed

Krogh, T P; Fredberg, U; Christensen, R; Stengaard-Pedersen, K; Ellingsen, T

2013-10-01

Tennis elbow, also known as lateral epicondylitis (LE), is a common disorder often assessed by ultrasound. The aim of this study was to evaluate the ultrasonographic outcomes and methods used in LE research and clinical practice. This study was designed as an intra- and interobserver reliability and agreement study. Ultrasonographic examination of the common extensor tendon of the elbow was performed. The intraobserver study examined tendon thickness twice in 20 right elbows from 20 healthy individuals at an interval of 7 to 12 days. The interobserver study examined tendon thickness, color Doppler activity, and bony spurs in 18 right elbows in 9 healthy individuals and 9 patients with LE. Two trained rheumatologists performed the interobserver examinations with the same scanner on the same day. The main outcomes were intra- and interclass correlation (ICC) and agreement. In the intraobserver study, the ICC with regard to tendon thickness ranged from 0.76 to 0.81, depending on the measurement techniques used. The agreement ranged from 0.06 to 0.13 mm. In the interobserver study, the tendon thickness ICC ranged from 0.45 to 0.65 and the agreement ranged from -0.17 to 0.13 mm. The ICC for color Doppler activity was 0.93, with agreement in 14/18 (78 %) of the cases. A perfect reliability was demonstrated for bony spurs, with an ICC of 1 and exact agreement in 18/18 (100 %) of the cases. Good to excellent reliability was obtained for all measurements. The ultrasonographic techniques evaluated in this trial can be recommended for use in both research and clinical practice. © Georg Thieme Verlag KG Stuttgart · New York.

24 CFR 968.330 - PHA performance and evaluation report.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 24 Housing and Urban Development 4 2010-04-01 2010-04-01 false PHA performance and evaluation... 250 or More Public Housing Units) § 968.330 PHA performance and evaluation report. For any FFY in which a PHA has received assistance under this subpart, the PHA shall submit a Performance and...

7 CFR 1499.5 - Agreements.

Code of Federal Regulations, 2013 CFR

2013-01-01

... 7 Agriculture 10 2013-01-01 2013-01-01 false Agreements. 1499.5 Section 1499.5 Agriculture... AGRICULTURE EXPORT PROGRAMS FOOD FOR PROGRESS PROGRAM § 1499.5 Agreements. (a) After FAS approves an applicant's proposal, FAS will develop an agreement in consultation with the applicant. The agreement will set...

7 CFR 1499.5 - Agreements.

Code of Federal Regulations, 2012 CFR

2012-01-01

... 7 Agriculture 10 2012-01-01 2012-01-01 false Agreements. 1499.5 Section 1499.5 Agriculture... AGRICULTURE EXPORT PROGRAMS FOOD FOR PROGRESS PROGRAM § 1499.5 Agreements. (a) After FAS approves an applicant's proposal, FAS will develop an agreement in consultation with the applicant. The agreement will set...

7 CFR 1499.5 - Agreements.

Code of Federal Regulations, 2014 CFR

2014-01-01

... 7 Agriculture 10 2014-01-01 2014-01-01 false Agreements. 1499.5 Section 1499.5 Agriculture... AGRICULTURE EXPORT PROGRAMS FOOD FOR PROGRESS PROGRAM § 1499.5 Agreements. (a) After FAS approves an applicant's proposal, FAS will develop an agreement in consultation with the applicant. The agreement will set...

Finding Culture Change in the Second Factor: Stability and Change in Cultural Consensus and Residual Agreement

ERIC Educational Resources Information Center

Dressler, William W.; Balieiro, Mauro C.; dos Santos, José Ernesto

2015-01-01

This article reports the replication after 10 years of cultural consensus analyses in four cultural domains in the city of Ribeirão Preto, Brazil. Additionally, two methods for evaluating residual agreement are applied to the data, and a new technique for evaluating how cultural knowledge is represented by residual agreement is introduced. We…

2 CFR 3001.661 - Reimbursable Agreement.

Code of Federal Regulations, 2013 CFR

2013-01-01

... 2 Grants and Agreements 1 2013-01-01 2013-01-01 false Reimbursable Agreement. 3001.661 Section 3001.661 Grants and Agreements Federal Agency Regulations for Grants and Agreements DEPARTMENT OF... Reimbursable Agreement. Reimbursable Agreement means an award in which the recipient is reimbursed for...

Current and Developing Conceptions of Use: Evaluation Use TIG Survey Results.

ERIC Educational Resources Information Center

Preskill, Hallie; Caracelli, Valerie

1997-01-01

A survey was sent to members of the Evaluation Use Topical Interest Group (TIG) to determine their perceptions about and experiences with evaluation use. Responses from 282 members show agreement on the major purposes of evaluation and an increased use of performance-results oriented and formative evaluations. (SLD)

Assessing agreement among alternative climate change projections to inform conservation recommendations in the contiguous United States.

PubMed

Belote, R Travis; Carroll, Carlos; Martinuzzi, Sebastián; Michalak, Julia; Williams, John W; Williamson, Matthew A; Aplet, Gregory H

2018-06-21

Addressing uncertainties in climate vulnerability remains a challenge for conservation planning. We evaluate how confidence in conservation recommendations may change with agreement among alternative climate projections and metrics of climate exposure. We assessed agreement among three multivariate estimates of climate exposure (forward velocity, backward velocity, and climate dissimilarity) using 18 alternative climate projections for the contiguous United States. For each metric, we classified maps into quartiles for each alternative climate projections, and calculated the frequency of quartiles assigned for each gridded location (high quartile frequency = more agreement among climate projections). We evaluated recommendations using a recent climate adaptation heuristic framework that recommends emphasizing various conservation strategies to land based on current conservation value and expected climate exposure. We found that areas where conservation strategies would be confidently assigned based on high agreement among climate projections varied substantially across regions. In general, there was more agreement in forward and backward velocity estimates among alternative projections than agreement in estimates of local dissimilarity. Consensus of climate predictions resulted in the same conservation recommendation assignments in a few areas, but patterns varied by climate exposure metric. This work demonstrates an approach for explicitly evaluating alternative predictions in geographic patterns of climate change.

Sustainability performance evaluation: Literature review and future directions.

PubMed

Büyüközkan, Gülçin; Karabulut, Yağmur

2018-07-01

Current global economic activities are increasingly being perceived as unsustainable. Despite the high number of publications, sustainability science remains highly dispersed over diverse approaches and topics. This article aims to provide a structured overview of sustainability performance evaluation related publications and to document the current state of literature, categorize publications, analyze and link trends, as well as highlight gaps and provide research recommendations. 128 articles between 2007 and 2018 are identified. The results suggest that sustainability performance evaluation models shall be more balanced, suitable criteria and their interrelations shall be well defined and subjectivity of qualitative criteria inherent to sustainability indicators shall be considered. To address this subjectivity, group decision-making techniques and other analytical methods that can deal with uncertainty, conflicting indicators, and linguistic evaluations can be used in future works. By presenting research gaps, this review stimulates researchers to establish practically applicable sustainability performance evaluation frameworks to help assess and compare the degree of sustainability, leading to more sustainable business practices. The review is unique in defining corporate sustainability performance evaluation for the first time, exploring the gap between sustainability accounting and sustainability assessment, and coming up with a structured overview of innovative research recommendations about integrating analytical assessment methods into conceptual sustainability frameworks. Copyright © 2018 Elsevier Ltd. All rights reserved.

34 CFR 675.35 - Agreement.

Code of Federal Regulations, 2013 CFR

2013-07-01

... 34 Education 3 2013-07-01 2013-07-01 false Agreement. 675.35 Section 675.35 Education Regulations..., DEPARTMENT OF EDUCATION FEDERAL WORK-STUDY PROGRAMS Job Location and Development Program § 675.35 Agreement... agreement with the Secretary. (b) The agreement must provide— (1) That the institution will administer the...

34 CFR 675.35 - Agreement.

Code of Federal Regulations, 2014 CFR

2014-07-01

... 34 Education 3 2014-07-01 2014-07-01 false Agreement. 675.35 Section 675.35 Education Regulations..., DEPARTMENT OF EDUCATION FEDERAL WORK-STUDY PROGRAMS Job Location and Development Program § 675.35 Agreement... agreement with the Secretary. (b) The agreement must provide— (1) That the institution will administer the...

16 CFR 2.32 - Agreement.

Code of Federal Regulations, 2013 CFR

2013-01-01

... 16 Commercial Practices 1 2013-01-01 2013-01-01 false Agreement. 2.32 Section 2.32 Commercial... Consent Order Procedure § 2.32 Agreement. Every agreement in settlement of a Commission complaint shall... of findings of fact and conclusions of law. Every agreement also shall waive further procedural steps...

34 CFR 675.35 - Agreement.

Code of Federal Regulations, 2012 CFR

2012-07-01

... 34 Education 3 2012-07-01 2012-07-01 false Agreement. 675.35 Section 675.35 Education Regulations..., DEPARTMENT OF EDUCATION FEDERAL WORK-STUDY PROGRAMS Job Location and Development Program § 675.35 Agreement... agreement with the Secretary. (b) The agreement must provide— (1) That the institution will administer the...

34 CFR 675.35 - Agreement.

Code of Federal Regulations, 2011 CFR

2011-07-01

... 34 Education 3 2011-07-01 2011-07-01 false Agreement. 675.35 Section 675.35 Education Regulations..., DEPARTMENT OF EDUCATION FEDERAL WORK-STUDY PROGRAMS Job Location and Development Program § 675.35 Agreement... agreement with the Secretary. (b) The agreement must provide— (1) That the institution will administer the...

16 CFR 2.32 - Agreement.

Code of Federal Regulations, 2010 CFR

2010-01-01

... 16 Commercial Practices 1 2010-01-01 2010-01-01 false Agreement. 2.32 Section 2.32 Commercial... Consent Order Procedure § 2.32 Agreement. Every agreement in settlement of a Commission complaint shall... of findings of fact and conclusions of law. Every agreement also shall waive further procedural steps...

16 CFR 2.32 - Agreement.

Code of Federal Regulations, 2012 CFR

2012-01-01

... 16 Commercial Practices 1 2012-01-01 2012-01-01 false Agreement. 2.32 Section 2.32 Commercial... Consent Order Procedure § 2.32 Agreement. Every agreement in settlement of a Commission complaint shall... of findings of fact and conclusions of law. Every agreement also shall waive further procedural steps...

16 CFR 2.32 - Agreement.

Code of Federal Regulations, 2011 CFR

2011-01-01

... 16 Commercial Practices 1 2011-01-01 2011-01-01 false Agreement. 2.32 Section 2.32 Commercial... Consent Order Procedure § 2.32 Agreement. Every agreement in settlement of a Commission complaint shall... of findings of fact and conclusions of law. Every agreement also shall waive further procedural steps...

16 CFR 2.32 - Agreement.

Code of Federal Regulations, 2014 CFR

2014-01-01

... 16 Commercial Practices 1 2014-01-01 2014-01-01 false Agreement. 2.32 Section 2.32 Commercial... Consent Order Procedure § 2.32 Agreement. Every agreement in settlement of a Commission complaint shall... of findings of fact and conclusions of law. Every agreement also shall waive further procedural steps...

34 CFR 675.35 - Agreement.

Code of Federal Regulations, 2010 CFR

2010-07-01

... 34 Education 3 2010-07-01 2010-07-01 false Agreement. 675.35 Section 675.35 Education Regulations..., DEPARTMENT OF EDUCATION FEDERAL WORK-STUDY PROGRAMS Job Location and Development Program § 675.35 Agreement... agreement with the Secretary. (b) The agreement must provide— (1) That the institution will administer the...

Multi-center Performance Evaluations of Tacrolimus and Cyclosporine Electrochemiluminescence Immunoassays in the Asia-Pacific Region

PubMed Central

Qin, Xuzhen; Rui, Jianzhong; Xia, Yong; Mu, Hong; Song, Sang Hoon; Raja Aziddin, Raja Elina; Miles, Gabrielle; Sun, Yuli

2018-01-01

Background The immunosuppressant drugs (ISDs), tacrolimus and cyclosporine, are vital for solid organ transplant patients to prevent rejection. However, toxicity is a concern, and absorption is highly variable across patients; therefore, ISD levels need to be precisely monitored. In the Asia-Pacific (APAC) region, tacrolimus and cyclosporine concentrations are typically measured using immunoassays. The objective of this study was to assess the analytical performance of Roche Elecsystacrolimus and cyclosporinee electrochemiluminescence immunoassays (ECLIAs). Methods This evaluation was performed in seven centers across China, South Korea, and Malaysia. Imprecision (repeatability and reproducibility), assay accuracy, and lot-to-lot reagent variability were tested. The Elecsys ECLIAs were compared with commercially available immunoassays (Architect, Dimension, and Viva-E systems) using whole blood samples from patients with various transplant types (kidney, liver, heart, and bone marrow). Results Coefficients of variation for repeatability and reproducibility were ≤5.4% and ≤12.4%, respectively, for the tacrolimus ECLIA, and ≤5.1% and ≤7.3%, respectively, for the cyclosporine ECLIA. Method comparisons of the tacrolimus ECLIA with Architect, Dimension, and Viva-E systems yielded slope values of 1.01, 1.14, and 0.897, respectively. The cyclosporine ECLIA showed even closer agreements with the Architect, Dimension, and Viva-E systems (slope values of 1.04, 1.04, and 1.09, respectively). No major differences were observed among the different transplant types. Conclusions The tacrolimus and cyclosporine ECLIAs demonstrated excellent precision and close agreement with other immunoassays tested. These results show that both assays are suitable for ISD monitoring in an APAC population across a range of different transplant types. PMID:29214751

2 CFR 1401.220 - Cooperative agreement.

Code of Federal Regulations, 2013 CFR

2013-01-01

... 2 Grants and Agreements 1 2013-01-01 2013-01-01 false Cooperative agreement. 1401.220 Section 1401.220 Grants and Agreements Federal Agency Regulations for Grants and Agreements DEPARTMENT OF THE... agreement. Cooperative agreement means an award of financial assistance that, consistent with 31 U.S.C. 6305...

49 CFR 605.14 - Agreement.

Code of Federal Regulations, 2012 CFR

2012-10-01

... 49 Transportation 7 2012-10-01 2012-10-01 false Agreement. 605.14 Section 605.14 Transportation... TRANSPORTATION SCHOOL BUS OPERATIONS School Bus Agreements § 605.14 Agreement. Except as provided in § 605.11 no... entered into a written agreement that the applicant will not engage in school bus operations exclusively...

49 CFR 605.14 - Agreement.

Code of Federal Regulations, 2014 CFR

2014-10-01

... 49 Transportation 7 2014-10-01 2014-10-01 false Agreement. 605.14 Section 605.14 Transportation... TRANSPORTATION SCHOOL BUS OPERATIONS School Bus Agreements § 605.14 Agreement. Except as provided in § 605.11 no... entered into a written agreement that the applicant will not engage in school bus operations exclusively...

23 CFR 660.111 - Agreements.

Code of Federal Regulations, 2014 CFR

2014-04-01

... 23 Highways 1 2014-04-01 2014-04-01 false Agreements. 660.111 Section 660.111 Highways FEDERAL... (DIRECT FEDERAL) Forest Highways § 660.111 Agreements. (a) A statewide FH agreement shall be executed among the FHWA, the FS, and each SHA. This agreement shall set forth the responsibilities of each party...

23 CFR 660.111 - Agreements.

Code of Federal Regulations, 2011 CFR

2011-04-01

... 23 Highways 1 2011-04-01 2011-04-01 false Agreements. 660.111 Section 660.111 Highways FEDERAL... (DIRECT FEDERAL) Forest Highways § 660.111 Agreements. (a) A statewide FH agreement shall be executed among the FHWA, the FS, and each SHA. This agreement shall set forth the responsibilities of each party...